Calls the Beam API wrapper to deploy and make subsequent calls to an instance of the gpt2 LLM in a cloud deployment. Requires installation of the Beam library and registration of Beam Client ID and Client Secret. By calling the wrapper an instance of the model is created and run, with returned text relating to the prompt. Additional calls can then be made by directly calling the Beam API.
import os
beam_client_id = "<Your beam client id>"
beam_client_secret = "<Your beam client secret>"
os.environ["BEAM_CLIENT_ID"] = beam_client_id
os.environ["BEAM_CLIENT_SECRET"] = beam_client_secret
!beam configure --clientId={beam_client_id} --clientSecret={beam_client_secret}
Note that a cold start might take a couple of minutes to return the response, but subsequent calls will be faster!
from langchain_community.llms.beam import Beam
llm = Beam(
model_name="gpt2",
name="langchain-gpt2-test",
cpu=8,
memory="32Gi",
gpu="A10G",
python_version="python3.8",
python_packages=[
"diffusers[torch]>=0.10",
"transformers",
"torch",
"pillow",
"accelerate",
"safetensors",
"xformers",
],
max_length="50",
verbose=False,
)
llm._deploy()
response = llm._call("Running machine learning on a remote GPU")
print(response)
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4