Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://docs.apify.com/sdk/python/reference/class/ApifyHttpProxyMiddleware below:

ApifyHttpProxyMiddleware | API | SDK for Python

Apify HTTP proxy middleware for Scrapy.

This middleware enhances request processing by adding a 'proxy' field to the request's meta and an authentication header. It draws inspiration from the HttpProxyMiddleware included by default in Scrapy projects. The proxy URL is sourced from the settings under the APIFY_PROXY_SETTINGS key. The value of this key, a dictionary, should be provided by the Actor input. An example of the proxy settings:

proxy_settings = {'useApifyProxy': true, 'apifyProxyGroups': []}

• __init__(proxy_settings): None

• Parameters
  • proxy_settings: dict

    Dictionary containing proxy settings, provided by the Actor input.

  Returns None

• Create an instance of ApifyHttpProxyMiddleware from a Scrapy Crawler.

  Parameters
  • crawler: Crawler
  Returns ApifyHttpProxyMiddleware

• process_exception(request, exception, spider): None

• Process an exception that occurs during request processing.

  Parameters
  • request: Request
  • exception: Exception
  • spider: Spider
  Returns None

• async process_request(request, spider): None

• Process a Scrapy request by assigning a new proxy.

  Parameters
  • request: Request
  • spider: Spider
  Returns None

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4