The C Transformers library provides Python bindings for GGML models.
This example goes over how to use LangChain to interact with C Transformers models.
Install
%pip install --upgrade --quiet ctransformers
Load Model
from langchain_community.llms import CTransformers
llm = CTransformers(model="marella/gpt-2-ggml")
Generate Text
print(llm.invoke("AI is going to"))
Streaming
from langchain_core.callbacks import StreamingStdOutCallbackHandler
llm = CTransformers(
model="marella/gpt-2-ggml", callbacks=[StreamingStdOutCallbackHandler()]
)
response = llm.invoke("AI is going to")
LLMChain
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
template = """Question: {question}
Answer:"""
prompt = PromptTemplate.from_template(template)
llm_chain = LLMChain(prompt=prompt, llm=llm)
response = llm_chain.run("What is AI?")
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4