Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://crawlee.dev/python/docs/introduction/setting-up below:

Setting up | Crawlee for Python · Fast, reliable Python web crawlers.

This guide will help you get started with Crawlee by setting it up on your computer. Follow the steps below to ensure a smooth installation process.

Before installing Crawlee itself, make sure that your system meets the following requirements:

• Python 3.10 or higher: Crawlee requires Python 3.10 or a newer version. You can download Python from the official website.
• Python package manager: While this guide uses pip (the most common package manager), you can also use any package manager you want. You can download pip from the official website.

To check if Python and pip are installed, run the following commands:

If these commands return the respective versions, you're ready to continue.

Crawlee is available as crawlee package on PyPI. This package includes the core functionality, while additional features are available as optional extras to keep dependencies and package size minimal.

To install the core package, run:

python -m pip install crawlee

After installation, verify that Crawlee is installed correctly by checking its version:

python -c 'import crawlee; print(crawlee.__version__)'

If you do not mind the package size, you can run the following command to install Crawlee with all optional features:

python -m pip install 'crawlee[all]'

Depending on your use case, you may want to install specific extras to enable additional functionality:

For using the BeautifulSoupCrawler, install the beautifulsoup extra:

python -m pip install 'crawlee[beautifulsoup]'

For using the ParselCrawler, install the parsel extra:

python -m pip install 'crawlee[parsel]'

For using the CurlImpersonateHttpClient, install the curl-impersonate extra:

python -m pip install 'crawlee[curl-impersonate]'

If you plan to use a (headless) browser with PlaywrightCrawler, install Crawlee with the playwright extra:

python -m pip install 'crawlee[playwright]'

After installing the playwright extra, install the necessary Playwright dependencies:

You can install multiple extras at once by using a comma as a separator:

python -m pip install 'crawlee[beautifulsoup,curl-impersonate]'

The quickest way to get started with Crawlee is by using the Crawlee CLI and selecting one of the prepared templates. The CLI helps you set up a new project in seconds.

First, ensure you have uv installed. You can check if it is installed by running:

If uv is not installed, follow the official installation guide.

Then, run the Crawlee CLI using uvx and choose from the available templates:

uvx 'crawlee[cli]' create my-crawler

If you already have crawlee installed, you can spin it up by running:

crawlee create my_crawler

Follow the interactive prompts in the CLI to choose a crawler type and set up your new project.

To run your newly created project, navigate to the project directory, activate the virtual environment, and execute the Python interpreter with the project module:

• Linux
• Windows

source .venv/bin/activate

Congratulations! You have successfully set up and executed your first Crawlee project.

Next, you will learn how to create a very simple crawler and Crawlee components while building it.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4