RetroSearch Browse

Showing content from https://python.langchain.com/docs/integrations/llms/petals below:

Petals

Petals runs 100B+ language models at home, BitTorrent-style.

This notebook goes over how to use Langchain with Petals.

Install petals

The petals package is required to use the Petals API. Install petals using pip3 install petals.

Imports

import os

from langchain.chains import LLMChain
from langchain_community.llms import Petals
from langchain_core.prompts import PromptTemplate

Set the Environment API Key

Make sure to get your API key from Huggingface.

from getpass import getpass

HUGGINGFACE_API_KEY = getpass()

os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY

Create the Petals instance

You can specify different parameters such as the model name, max new tokens, temperature, etc.



llm = Petals(model_name="bigscience/bloom-petals")

Downloading:   1%|▏                        | 40.8M/7.19G [00:24<15:44, 7.57MB/s]

Create a Prompt Template

We will create a prompt template for Question and Answer.

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate.from_template(template)

Initiate the LLMChain

llm_chain = LLMChain(prompt=prompt, llm=llm)

Run the LLMChain

Provide a question and run the LLMChain.

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

llm_chain.run(question)

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4