How to Use Firecrawl to Extract Data from Websites with a Single Prompt

Firecrawl is an AI-powered web scraping tool that allows users to extract structured data from websites with minimal effort. This tutorial will guide you through the process of using Firecrawl to extract data from websites using a simple Python script.

Step 1: Install Firecrawl

Before using Firecrawl, ensure that you have installed the firecrawl-py package. You can do this by running:

!pip install firecrawl-py

Step 2: Initialize Firecrawl with Your API Key

To use Firecrawl, you need an API key. If you don’t have one, you can get it from Firecrawl’s website.

Initialize the Firecrawl application with your API key:

from firecrawl import FirecrawlApp

# Replace with your actual API key
key = 'your-api-key-here'

# Initialize Firecrawl
app = FirecrawlApp(api_key=key)

Step 3: Extract Data Using a Single Prompt

Firecrawl allows you to specify a prompt describing what data you want to extract. For example, if you want to extract all links from a given webpage, you can use:

data = app.extract(
    ['https://qwenlm.github.io/blog/qwen2.5-max/'],  # List of URLs to scrape
    {
        'prompt': "Extract all the links from the given URL"
    }
)

# Print the extracted data
print(data)