RetroSearch Browse

Showing content from https://python.langchain.com/docs/integrations/llms/openllm/ below:

Our new LangChain Academy Course Deep Research with LangGraph is now live!

OpenLLM

🦾 OpenLLM lets developers run any open-source LLMs as OpenAI-compatible API endpoints with a single command.

🔬 Build for fast and production usages
🚂 Support llama3, qwen2, gemma, etc, and many quantized versions full list
⛓️ OpenAI-compatible API
💬 Built-in ChatGPT like UI
🔥 Accelerated LLM decoding with state-of-the-art inference backends
🌥️ Ready for enterprise-grade cloud deployment (Kubernetes, Docker and BentoCloud)

Installation

Install openllm through PyPI

%pip install --upgrade --quiet  openllm

Launch OpenLLM server locally

To start an LLM server, use openllm hello command:

Wrapper

from langchain_community.llms import OpenLLM

server_url = "http://localhost:3000"  
llm = OpenLLM(base_url=server_url, api_key="na")

llm("To build a LLM from scratch, the following are the steps:")

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4