Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://python.langchain.com/docs/integrations/llms/deepsparse below:

Website Navigation

• Previous

  DeepInfra

• Next

  Eden AI

DeepSparse | 🦜️🔗 LangChain

This page covers how to use the DeepSparse inference runtime within LangChain. It is broken into two parts: installation and setup, and then examples of DeepSparse usage.

• Install the Python package with pip install deepsparse
• Choose a SparseZoo model or export a support model to ONNX using Optimum

There exists a DeepSparse LLM wrapper, that provides a unified interface for all models:

from langchain_community.llms import DeepSparse

llm = DeepSparse(
    model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none"
)

print(llm.invoke("def fib():"))

Additional parameters can be passed using the config parameter:

config = {"max_generated_tokens": 256}

llm = DeepSparse(
    model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base-none",
    config=config,
)

• LLM conceptual guide
• LLM how-to guides

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4