Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://crawlee.dev/python/api/class/ContextPipeline below:

Website Navigation

• Previous

  BrowserforgeFingerprintGenerator

• Next

  CrawlerInstrumentor

ContextPipeline | API | Crawlee for Python · Fast, reliable Python web crawlers.

Encapsulates the logic of gradually enhancing the crawling context with additional information and utilities.

The enhancement is done by a chain of middlewares that are added to the pipeline after it's creation.

• async __call__(crawling_context, final_context_consumer): None

• Run a crawling context through the middleware chain and pipe it into a consumer function.

  Exceptions from the consumer function are wrapped together with the final crawling context.

  Parameters
  • crawling_context: BasicCrawlingContext
  • final_context_consumer: Callable[[TCrawlingContext], Awaitable[None]]
  Returns None

• __init__(*, _middleware, _parent): None

• Add a middleware to the pipeline.

  The middleware should yield exactly once, and it should yield an (optionally) extended crawling context object. The part before the yield can be used for initialization and the part after it for cleanup.

  Parameters
  • middleware: Callable[[TCrawlingContext], AsyncGenerator[TMiddlewareCrawlingContext, None]]
  Returns ContextPipeline[TMiddlewareCrawlingContext]

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4