LangChain provides a callback system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.
You can subscribe to these events by using the callbacks argument available throughout the API. This argument is a list of handler objects, which are expected to implement one or more of the methods described below in more detail.
on_chat_model_start LLM start When a llm starts on_llm_start LLM new token When an llm OR chat model emits a new token on_llm_new_token LLM ends When an llm OR chat model ends on_llm_end LLM errors When an llm OR chat model errors on_llm_error Chain start When a chain starts running on_chain_start Chain end When a chain ends on_chain_end Chain error When a chain errors on_chain_error Tool start When a tool starts running on_tool_start Tool end When a tool ends on_tool_end Tool error When a tool errors on_tool_error Agent action When an agent takes an action on_agent_action Agent finish When an agent ends on_agent_finish Retriever start When a retriever starts on_retriever_start Retriever end When a retriever ends on_retriever_end Retriever error When a retriever errors on_retriever_error Text When arbitrary text is run on_text Retry When a retry event is run on_retry Callback handlers
Callback handlers can either be sync or async:
During run-time LangChain configures an appropriate callback manager (e.g., CallbackManager or AsyncCallbackManager which will be responsible for calling the appropriate method on each "registered" callback handler when the event is triggered.
Passing callbacksThe callbacks property is available on most objects throughout the API (Models, Tools, Agents, etc.) in two different places:
Runnable objects. These callbacks are INHERITED by all children of the object they are defined on. For example, chain.invoke({"number": 25}, {"callbacks": [handler]}).chain = TheNameOfSomeChain(callbacks=[handler]). These callbacks are passed as arguments to the constructor of the object. The callbacks are scoped only to the object they are defined on, and are not inherited by any children of the object.warning
Constructor callbacks are scoped only to the object they are defined on. They are not inherited by children of the object.
If you're creating a custom chain or runnable, you need to remember to propagate request time callbacks to any child objects.
Async in Python<=3.10
Any RunnableLambda, a RunnableGenerator, or Tool that invokes other runnables and is running async in python<=3.10, will have to propagate callbacks to child objects manually. This is because LangChain cannot automatically propagate callbacks to child objects in this case.
This is a common reason why you may fail to see events being emitted from custom runnables or tools.
For specifics on how to use callbacks, see the relevant how-to guides here.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4