Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://docs.hyperbrowser.ai/reference/sdks/python/scrape below:

Scrape | Hyperbrowser

1. reference
2. SDKs
3. Python

Starts a scrape job for a given URL.

Method: client.scrape.start(params: StartScrapeJobParams): StartScrapeJobResponse

Endpoint: POST /api/scrape

Parameters:

• StartScrapeJobParams:

  • url: string - URL to scrape

Response: StartScrapeJobResponse

Example:

response = client.scrape.start(StartScrapeJobParams(url="https://example.com"))
print(response.jobId)

Retrieves details of a specific scrape job.

Method: client.scrape.get(id: str): ScrapeJobResponse

Endpoint: GET /api/scrape/{id}

Parameters:

• id: string - Scrape job ID

Response: ScrapeJobResponse

Example:

response = client.scrape.get(
  "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e"
)
print(response.status)

Start a scrape job and wait for it to complete

Method: client.scrape.start_and_wait(params: StartScrapeJobParams): ScrapeJobResponse

Parameters:

• StartScrapeJobParams:

  • url: string - URL to scrape

Response: ScrapeJobResponse

Example:

response = client.scrape.start_and_wait(StartScrapeJobParams(url="https://example.com"))
print(response.status)

ScrapeFormat = Literal["markdown", "html", "links", "screenshot"]

ScrapeJobStatus = Literal["pending", "running", "completed", "failed"]

class ScrapeOptions(BaseModel):
    formats: Optional[List[ScrapeFormat]] = None
    include_tags: Optional[List[str]] = Field(
        default=None, serialization_alias="includeTags"
    )
    exclude_tags: Optional[List[str]] = Field(
        default=None, serialization_alias="excludeTags"
    )
    only_main_content: Optional[bool] = Field(
        default=None, serialization_alias="onlyMainContent"
    )
    wait_for: Optional[int] = Field(default=None, serialization_alias="waitFor")
    timeout: Optional[int] = Field(default=None, serialization_alias="timeout")

class StartScrapeJobResponse(BaseModel):
    job_id: str = Field(alias="jobId")

class ScrapeJobData(BaseModel):
    metadata: Optional[dict[str, Union[str, list[str]]]] = None
    html: Optional[str] = None
    markdown: Optional[str] = None
    links: Optional[List[str]] = None

class ScrapeJobResponse(BaseModel):
    job_id: str = Field(alias="jobId")
    status: ScrapeJobStatus
    error: Optional[str] = None
    data: Optional[ScrapeJobData] = None

Last updated 7 months ago

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4