Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://paddlepaddle.github.io/PaddleOCR/latest/en/version3.x/pipeline_usage/PP-DocTranslation.html below:

PP-DocTranslation产线 - PaddleOCR Documentation

PP-DocTranslation is a document intelligent translation solution provided by PaddlePaddle. It integrates advanced general layout analysis technology and large language model (LLM) capabilities to offer you efficient document intelligent translation services. This solution can accurately identify and extract various elements within documents, including text blocks, headings, paragraphs, images, tables, and other complex layout structures, and on this basis, achieve high-quality multilingual translation. PP-DocTranslation supports mutual translation among multiple mainstream languages, particularly excelling in handling documents with complex layouts and strong contextual dependencies, striving to deliver precise, natural, fluent, and professional translation results. This pipeline also provides flexible serving options, supporting the use of multiple programming languages on various hardware. Moreover, it offers the capability for secondary development, allowing you to train and fine-tune models on your own datasets based on this pipeline, and the trained models can also be seamlessly integrated.

The PP-DocTranslation pipeline uses the PP-StructureV3 sub-pipeline, and thus has all the functions of the PP-StructureV3 pipeline. For more information on the functions and usage details of the PP-StructureV3 pipeline, you can click on the PP-StructureV3 Pipeline Documentation page.

In this pipeline, you can select the model to use based on the benchmark data below.

Document Image Orientation Classification Module:

Text Image Unwarping Module:

Layout Detection Module Models:

Table Structure Recognition Module:

Table Classification Module Models:

Table Cell Detection Module Models:

Text Detection Module:

Text Recognition Module Models:

Text Line Orientation Classification Module (Optional):

Formula Recognition Module:

Seal Text Recognition Module:

• Performance Testing Environment
  • Test Datasets:
    • Document Image Orientation Classification Model: A dataset built by PaddleX, covering multiple scenarios such as IDs and documents, containing 1,000 images.
    • Text Image Unwarping Model: DocUNet.
    • Layout Detection Model: A layout analysis dataset built by PaddleOCR, containing 10,000 common document-type images such as Chinese and English papers, magazines, and reports.
    • PP-DocLayout_plus-L: A layout detection dataset built by PaddleOCR, containing 1,300 document-type images such as Chinese and English papers, magazines, newspapers, reports, PPTs, exams, and textbooks.
    • Table Structure Recognition Model: An internal English table recognition dataset built by PaddleX.
    • Text Detection Model: A Chinese dataset built by PaddleOCR, covering street views, web images, documents, and handwriting, with 500 images for detection.
    • Chinese Recognition Model: A Chinese dataset built by PaddleOCR, covering street views, web images, documents, and handwriting, with 11,000 images for text recognition.
    • ch_SVTRv2_rec: PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition A榜 evaluation set.
    • ch_RepSVTR_rec: PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition B榜 evaluation set.
    • English Recognition Model: An English dataset built by PaddleX.
    • Multilingual Recognition Model: A multilingual dataset built by PaddleX.
    • Text Line Orientation Classification Model: A dataset built by PaddleX, covering multiple scenarios such as IDs and documents, containing 1,000 images.
    • Seal Text Recognition Model: A dataset built by PaddleX, containing 500 circular seal images.
  • Hardware Configuration:
    • GPU: NVIDIA Tesla T4
    • CPU: Intel Xeon Gold 6271C @ 2.60GHz
  • Software Environment:
    • Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6
    • paddlepaddle 3.0.0 / paddleocr 3.0.3
• Inference Mode Description

Before using the PP-DocTranslation pipeline locally, please ensure that you have completed the installation of the wheel package according to the Installation Tutorial.

Please note: If you encounter issues such as the program becoming unresponsive, unexpected program termination, running out of memory resources, or extremely slow inference during execution, please try adjusting the configuration according to the documentation, such as disabling unnecessary features or using lighter-weight models.

Before use, you need to prepare the API key for a large language model, which supports the Baidu Cloud Qianfan Platform or local large model services that comply with the OpenAI interface standards.

You can download the test file and quickly experience the pipeline effect with a single command:

paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --qianfan_api_key your_api_key

• large: when set to large, among overlapping boxes, only the largest outer box is kept and the overlapping inner boxes are deleted;
• small: when set to small, among overlapping boxes, only the smaller inner boxes are kept and the overlapping outer boxes are deleted;
• union: no box filtering, both inner and outer boxes are kept;

• CPU: e.g. cpu means using CPU for inference;
• GPU: e.g. gpu:0 means using GPU #1 for inference;
• NPU: e.g. npu:0 means using NPU #1 for inference;
• XPU: e.g. xpu:0 means using XPU #1 for inference;
• MLU: e.g. mlu:0 means using MLU #1 for inference;
• DCU: e.g. dcu:0 means using DCU #1 for inference;

The execution results will be printed to the terminal.

The command-line method is for quickly experiencing and viewing the results. Generally, in projects, integration via code is often required. You can download the test file and use the following sample code for inference:

from paddleocr import PPDocTranslation

# Create a translation pipeline
pipeline = PPDocTranslation()

# Document path
input_path = "document_sample.pdf"

# Output directory
output_path = "./output"

# Large model configuration
chat_bot_config = {
    "module_name": "chat_bot",
    "model_name": "ernie-3.5-8k",
    "base_url": "https://qianfan.baidubce.com/v2",
    "api_type": "openai",
    "api_key": "api_key",  # your api_key
}

if input_path.lower().endswith(".md"):
    # Read markdown documents, supporting passing in directories and url links with the .md suffix
    ori_md_info_list = pipeline.load_from_markdown(input_path)
else:
    # Use PP-StructureV3 to perform layout parsing on PDF/image documents to obtain markdown information
    visual_predict_res = pipeline.visual_predict(
        input_path,
        use_doc_orientation_classify=False,
        use_doc_unwarping=False,
        use_common_ocr=True,
        use_seal_recognition=True,
use_table_recognition=True,
    )

    ori_md_info_list = []
    for res in visual_predict_res:
        layout_parsing_result = res["layout_parsing_result"]
        ori_md_info_list.append(layout_parsing_result.markdown)
        layout_parsing_result.save_to_img(output_path)
        layout_parsing_result.save_to_markdown(output_path)

    # Concatenate the markdown information of multi-page documents into a single markdown file, and save the merged original markdown text
    if input_path.lower().endswith(".pdf"):
        ori_md_info = pipeline.concatenate_markdown_pages(ori_md_info_list)
        ori_md_info.save_to_markdown(output_path)

# Perform document translation (target language: English)
tgt_md_info_list = pipeline.translate(
    ori_md_info_list=ori_md_info_list,
    target_language="en",
    chunk_size=5000,
    chat_bot_config=chat_bot_config,
)
# Save the translation results
for tgt_md_info in tgt_md_info_list:
    tgt_md_info.save_to_markdown(output_path)

After executing the above code, you will obtain the parsed results of the original document to be translated, the Markdown file of the original text to be translated, and the Markdown file of the translated document, all saved in the output directory.

The process, API description, and output description of PP-DocTranslation prediction are as follows:

• float: Any float between 0-1;
• dict: {0:0.1}, where the key is the class ID and the value is the threshold for that class;
• None: If set to None, the pipeline's initialized value will be used, defaulting to 0.5.

• float: Any float greater than 0;
• Tuple[float,float]: Expansion coefficients in horizontal and vertical directions respectively;
• dict: Keys are int representing cls_id, values are tuple, e.g. {0: (1.1, 2.0)}, meaning for class 0 detection boxes, center remains unchanged, width expanded by 1.1 times, height expanded by 2.0 times;
• None: If set to None, the pipeline's initialized value will be used, defaulting to 1.0.

• str: large, small, union, indicating whether to keep the larger box, smaller box, or both during overlap filtering;
• dict: Keys are int cls_id, values are str, e.g. {0: "large", 2: "small"}, meaning use "large" mode for class 0 boxes and "small" mode for class 2 boxes;
• None: If set to None, the pipeline's initialized value will be used, defaulting to large.

• int: Any integer greater than 0;
• None: If set to None, the pipeline's initialized value will be used, defaulting to 960.

• str: Supports min and max, where min means ensuring the shortest side of the image is not less than det_limit_side_len, and max means ensuring the longest side is not greater than limit_side_len;
• None: If set to None, the pipeline's initialized value will be used, defaulting to max.

• float: Any float greater than 0;
• None: If set to None, the pipeline's initialized value of 0.3 will be used.

• float: Any float greater than 0;
• None: If set to None, the pipeline's initialized value of 0.6 will be used.

• float: Any float greater than 0;
• None: If set to None, the pipeline's initialized value of 2.0 will be used.

• float: Any float greater than 0;
• None: If set to None, the pipeline's initialized value of 0.0 (no threshold) will be used.

• int: any integer greater than 0;
• None: if set to None, the parameter value initialized by the pipeline will be used, with a default initialization of 736.

• str: supports min and max, where min ensures the shortest image side is not less than det_limit_side_len, and max ensures the longest image side is not greater than limit_side_len;
• None: if set to None, the parameter value initialized by the pipeline will be used, with a default initialization of min.

• float: any floating number greater than 0;
• None: if set to None, the pipeline default parameter value 0.2 will be used.

• float: any floating number greater than 0;
• None: if set to None, the pipeline default parameter value 0.6 will be used.

• float: any floating number greater than 0;
• None: if set to None, the pipeline default parameter value 0.5 will be used.

• float: any floating number greater than 0;
• None: if set to None, the pipeline default parameter value 0.0 will be used, meaning no threshold is set.

{
"module_name": "chat_bot",
"model_name": "ernie-3.5-8k",
"base_url": "https://qianfan.baidubce.com/v2",
"api_type": "openai",
"api_key": "api_key"  # Please set this to the actual API key
}

• CPU: e.g. cpu means using CPU for inference;
• GPU: e.g. gpu:0 means using the first GPU for inference;
• NPU: e.g. npu:0 means using the first NPU for inference;
• XPU: e.g. xpu:0 means using the first XPU for inference;
• MLU: e.g. mlu:0 means using the first MLU for inference;
• DCU: e.g. dcu:0 means using the first DCU for inference;
• None: if set to None, initialization will prioritize using the local GPU device 0; if unavailable, CPU will be used.

• Python Var: image data such as numpy.ndarray;
• str: local path of image or PDF files, e.g. /root/data/img.jpg; URL link: network URL of image or PDF files, e.g. example; local directory: directory containing images to be predicted, e.g. /root/data/ (currently does not support PDFs in directories, PDF files need to specify exact file path);
• list: list elements must be one of the above types, e.g. [numpy.ndarray, numpy.ndarray], ["/root/data/img1.jpg", "/root/data/img2.jpg"], ["/root/data1", "/root/data2"].

If the pipeline can meet your requirements for inference speed and accuracy, you can proceed directly with development integration/deployment.

If you need to directly apply the pipeline in your Python project, you can refer to the sample code in 2.2 Python Script Approach.

In addition, PaddleOCR also offers two other deployment methods, detailed as follows:

🚀 High-Performance Inference: In real-world production environments, many applications have stringent performance criteria (especially response speed) for deployment strategies to ensure efficient system operation and a smooth user experience. To this end, PaddleOCR provides high-performance inference capabilities, aiming to deeply optimize model inference and pre/post-processing, achieving significant acceleration in the end-to-end process. For detailed information on the high-performance inference process, please refer to High-Performance Inference.

☁️ Serving: Serving is a common deployment form in real-world production environments. By encapsulating inference functions as services, clients can access these services through network requests to obtain inference results. For detailed information on the pipeline serving process, please refer to Serving.

Below are the API references for basic serving and examples of multi-language service invocation:

Main operations provided by the serving:

• HTTP request method is POST.
• Both request body and response body are JSON data (JSON objects).
• When the request is processed successfully, the response status code is 200, and the response body has the following properties:

• When the request is not successful, the response body has the following properties:

Main operations provided by the serving are as follows:

• analyzeImages

Use computer vision models to analyze images, obtaining OCR, table recognition results, etc.

POST /doctrans-visual

• Request body properties are as follows:

Serving:
  extra:
    max_num_input_imgs: null

• If true is passed: return images.
• If false is passed: do not return images.
• If this parameter is not provided in the request body or null is passed: follow the pipeline config file setting Serving.visualize.

Serving:
  visualize: False

• When the request is processed successfully, the response body's result has the following properties:

Each element in layoutParsingResults is an object with the following properties:

markdown is an object with the following properties:

• translate

Use a large model to translate documents.

POST /doctrans-translate

• Request body properties are as follows:

• When the request is successfully processed, the result in the response body has the following attributes:

Each element in translationResults is an object with the following attributes:

import base64
import pathlib
import pprint
import sys

import requests


API_BASE_URL = "http://127.0.0.1:8080"

file_path = "./demo.jpg"
target_language = "en"

with open(file_path, "rb") as file:
    file_bytes = file.read()
    file_data = base64.b64encode(file_bytes).decode("ascii")

payload = {
    "file": file_data,
    "fileType": 1,
}
resp_visual = requests.post(url=f"{API_BASE_URL}/doctrans-visual", json=payload)
if resp_visual.status_code != 200:
    print(
        f"Request to doctrans-visual failed with status code {resp_visual.status_code}."
    )
    pprint.pp(resp_visual.json())
    sys.exit(1)
result_visual = resp_visual.json()["result"]

markdown_list = []
for i, res in enumerate(result_visual["layoutParsingResults"]):
    md_dir = pathlib.Path(f"markdown_{i}")
    md_dir.mkdir(exist_ok=True)
    (md_dir / "doc.md")
write_text(res["markdown"]["text"])
    for img_path, img in res["markdown"]["images"].items():
        img_path = md_dir / img_path
        img_path.parent.mkdir(parents=True, exist_ok=True)
        img_path.write_bytes(base64.b64decode(img))
    print(f"The Markdown document to be translated is saved at {md_dir / 'doc.md'}")
    del res["markdown"]["images"]
    markdown_list.append(res["markdown"])
    for img_name, img in res["outputImages"].items():
        img_path = f"{img_name}_{i}.jpg"
        with open(img_path, "wb") as f:
            f.write(base64.b64decode(img))
        print(f"Output image saved at {img_path}")

payload = {
    "markdownList": markdown_list,
"targetLanguage": target_language,
}
resp_translate = requests.post(url=f"{API_BASE_URL}/doctrans-translate", json=payload)
if resp_translate.status_code != 200:
    print(
        f"Request to doctrans-translate failed with status code {resp_translate.status_code}."
    )
    pprint.pprint(resp_translate.json())  # Corrected 'pp' to 'pprint' for proper function call
    sys.exit(1)
result_translate = resp_translate.json()["result"]

for i, res in enumerate(result_translate["translationResults"]):
    md_dir = pathlib.Path(f"markdown_{i}")
    (md_dir / "doc_translated.md").write_text(res["markdown"]["text"])
    print(f"Translated markdown document saved at {md_dir / 'doc_translated.md'}")

If the default model weights provided by the PP-DocTranslation pipeline do not meet your accuracy or speed requirements in your scenario, you can try to useyour own data from specific domains or application scenariosto furtherfine-tunethe existing model to improve the recognition effect in your scenario.

Since the PP-DocTranslation pipeline contains several modules, if the performance of the model pipeline does not meet expectations, the issue may originate from any one of these modules. You can analyze cases with poor extraction results, use visualized images to determine which module has the problem, and refer to the corresponding fine-tuning tutorial links in the following table to fine-tune the model.

After completing fine-tuning training with your private dataset, you can obtain a local model weight file. Then, you can use the fine-tuned model weights by customizing the pipeline configuration file.

1. Obtain the pipeline configuration file

You can call the export_paddlex_config_to_yaml method of the PP-DocTranslation pipeline object in PaddleOCR to export the current pipeline configuration to a YAML file:

from paddleocr import PPDocTranslation

pipeline = PPDocTranslation()
pipeline.export_paddlex_config_to_yaml("PP-DocTranslation.yaml")

1. Modify the configuration file

After obtaining the default pipeline configuration file, replace the local path of the fine-tuned model weights with the corresponding location in the pipeline configuration file. For example,

......
SubModules:
    TextDetection:
    module_name: text_detection
    model_name: PP-OCRv5_server_det
    model_dir: null # Replace with the path to the weights of the fine-tuned text detection model
    limit_side_len: 960
    limit_type: max
    thresh: 0.3
    box_thresh: 0.6
    unclip_ratio: 1.5

    TextRecognition:
    module_name: text_recognition
    model_name: PP-OCRv5_server_rec
    model_dir: null # Replace with the path to the weights of the fine-tuned text recognition model
    batch_size: 1
            score_thresh: 0
......

The pipeline configuration file not only includes parameters supported by PaddleOCR CLI and Python API but also allows for more advanced configurations. Detailed information can be found in the corresponding pipeline usage tutorial in the Overview of PaddleX Model Pipeline Usage. Refer to the detailed instructions therein and adjust the configurations according to your needs.

1. Load the pipeline configuration file in CLI

After modifying the configuration file, specify the path to the modified pipeline configuration file using the --paddlex_config parameter in the command line. PaddleOCR will then read its contents as the pipeline configuration. Here is an example:

paddleocr pp_doctranslation --paddlex_config PP-DocTranslation.yaml ...

1. Load the pipeline configuration file in the Python API

When initializing the pipeline object, you can pass the path of the PaddleX pipeline configuration file or a configuration dict through the paddlex_config parameter, and PaddleOCR will read its content as the pipeline configuration. The example is as follows:

from paddleocr import PPDocTranslation

pipeline = PPDocTranslation(paddlex_config="PP-DocTranslation.yaml")

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4