Human-in-the-loop (HIL) interactions are crucial for agentic systems. Waiting for human input is a common HIL interaction pattern, allowing the agent to ask the user clarifying questions and await input before proceeding.
We can implement these in LangGraph using the interrupt() function. interrupt allows us to stop graph execution to collect input from a user and continue execution with collected input.
Optionally, we can set API key for LangSmith tracing, which will give us best-in-class observability.
Let's explore a basic example of using human feedback. A straightforward approach is to create a node, human_feedback, designed specifically to collect user input. This allows us to gather feedback at a specific, chosen point in our graph.
---Step 1---
{}
--- humanFeedback ---
{
__interrupt__: [
{
value: 'Please provide feedback',
when: 'during',
resumable: true,
ns: [Array]
}
]
}
--- GRAPH INTERRUPTED ---
--- humanFeedback ---
{ humanFeedback: { userFeedback: 'go to step 3! ' } }
====
---Step 3---
{}
====
{ input: 'hello world', userFeedback: 'go to step 3! ' }
In the context of agents, waiting for user feedback is useful to ask clarifying questions.
To show this, we will build a relatively simple ReAct-style agent that does tool calling.
We will use Anthropic's models and a mock tool (purely for demonstration purposes).
We can now interact with the agent. Let's ask it to ask the user where they are, then tell them the weather.
This should make it use the askHuman tool first, then use the normal tool. Note that we switch to use streamMode: "values" to just return the update messages at each point:
================================ human Message (1) =================================
Use the search tool to ask the user where they are, then look up the weather there
--- ASKING HUMAN ---
================================ ai Message (1) =================================
[
{
type: 'text',
text: "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
type: 'tool_use',
id: 'toolu_015UDVFoXcMV7KjRqPY78Umk',
name: 'askHuman',
input: {
input: 'Where are you currently located? Please provide a city and country or region.'
}
}
]
{
messages: [
HumanMessage {
"id": "cfb461cb-0da1-48b4-acef-e3bf0d3a4e6c",
"content": "Use the search tool to ask the user where they are, then look up the weather there",
"additional_kwargs": {},
"response_metadata": {}
},
AIMessage {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"content": [
{
"type": "text",
"text": "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
"type": "tool_use",
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"name": "askHuman",
"input": {
"input": "Where are you currently located? Please provide a city and country or region."
}
}
],
"additional_kwargs": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
}
},
"response_metadata": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "askHuman",
"args": {
"input": "Where are you currently located? Please provide a city and country or region."
},
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"type": "tool_call"
}
],
"invalid_tool_calls": []
}
]
}
====
{
messages: [
HumanMessage {
"id": "cfb461cb-0da1-48b4-acef-e3bf0d3a4e6c",
"content": "Use the search tool to ask the user where they are, then look up the weather there",
"additional_kwargs": {},
"response_metadata": {}
},
AIMessage {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"content": [
{
"type": "text",
"text": "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
"type": "tool_use",
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"name": "askHuman",
"input": {
"input": "Where are you currently located? Please provide a city and country or region."
}
}
],
"additional_kwargs": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
}
},
"response_metadata": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "askHuman",
"args": {
"input": "Where are you currently located? Please provide a city and country or region."
},
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"type": "tool_call"
}
],
"invalid_tool_calls": []
},
ToolMessage {
"id": "0225d971-1756-468e-996d-9af93e608a95",
"content": "San Francisco",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_015UDVFoXcMV7KjRqPY78Umk"
}
]
}
====
{
messages: [
HumanMessage {
"id": "cfb461cb-0da1-48b4-acef-e3bf0d3a4e6c",
"content": "Use the search tool to ask the user where they are, then look up the weather there",
"additional_kwargs": {},
"response_metadata": {}
},
AIMessage {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"content": [
{
"type": "text",
"text": "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
"type": "tool_use",
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"name": "askHuman",
"input": {
"input": "Where are you currently located? Please provide a city and country or region."
}
}
],
"additional_kwargs": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
}
},
"response_metadata": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "askHuman",
"args": {
"input": "Where are you currently located? Please provide a city and country or region."
},
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"type": "tool_call"
}
],
"invalid_tool_calls": []
},
ToolMessage {
"id": "0225d971-1756-468e-996d-9af93e608a95",
"content": "San Francisco",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_015UDVFoXcMV7KjRqPY78Umk"
},
AIMessage {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"content": [
{
"type": "text",
"text": "Thank you for providing your location. Now, I'll use the search tool to look up the weather in San Francisco."
},
{
"type": "tool_use",
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"name": "search",
"input": {
"input": "current weather in San Francisco"
}
}
],
"additional_kwargs": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
}
},
"response_metadata": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "search",
"args": {
"input": "current weather in San Francisco"
},
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 590,
"output_tokens": 81,
"total_tokens": 671
}
}
]
}
====
{
messages: [
HumanMessage {
"id": "cfb461cb-0da1-48b4-acef-e3bf0d3a4e6c",
"content": "Use the search tool to ask the user where they are, then look up the weather there",
"additional_kwargs": {},
"response_metadata": {}
},
AIMessage {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"content": [
{
"type": "text",
"text": "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
"type": "tool_use",
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"name": "askHuman",
"input": {
"input": "Where are you currently located? Please provide a city and country or region."
}
}
],
"additional_kwargs": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
}
},
"response_metadata": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "askHuman",
"args": {
"input": "Where are you currently located? Please provide a city and country or region."
},
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"type": "tool_call"
}
],
"invalid_tool_calls": []
},
ToolMessage {
"id": "0225d971-1756-468e-996d-9af93e608a95",
"content": "San Francisco",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_015UDVFoXcMV7KjRqPY78Umk"
},
AIMessage {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"content": [
{
"type": "text",
"text": "Thank you for providing your location. Now, I'll use the search tool to look up the weather in San Francisco."
},
{
"type": "tool_use",
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"name": "search",
"input": {
"input": "current weather in San Francisco"
}
}
],
"additional_kwargs": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
}
},
"response_metadata": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "search",
"args": {
"input": "current weather in San Francisco"
},
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 590,
"output_tokens": 81,
"total_tokens": 671
}
},
ToolMessage {
"id": "1c52e9b5-1d0b-4889-abd0-10572053e1d8",
"content": "It's sunny in San Francisco, but you better look out if you're a Gemini 😈.",
"name": "search",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_01AzffiwYncLArNb9SbKHK2a"
}
]
}
====
{
messages: [
HumanMessage {
"id": "cfb461cb-0da1-48b4-acef-e3bf0d3a4e6c",
"content": "Use the search tool to ask the user where they are, then look up the weather there",
"additional_kwargs": {},
"response_metadata": {}
},
AIMessage {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"content": [
{
"type": "text",
"text": "Certainly! I'll use the askHuman tool to ask the user about their location, and then use the search tool to look up the weather for that location. Let's start by asking the user where they are."
},
{
"type": "tool_use",
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"name": "askHuman",
"input": {
"input": "Where are you currently located? Please provide a city and country or region."
}
}
],
"additional_kwargs": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
}
},
"response_metadata": {
"id": "msg_01TA2zHbbrenm7KXSUdcFdXD",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 465,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 112
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "askHuman",
"args": {
"input": "Where are you currently located? Please provide a city and country or region."
},
"id": "toolu_015UDVFoXcMV7KjRqPY78Umk",
"type": "tool_call"
}
],
"invalid_tool_calls": []
},
ToolMessage {
"id": "0225d971-1756-468e-996d-9af93e608a95",
"content": "San Francisco",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_015UDVFoXcMV7KjRqPY78Umk"
},
AIMessage {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"content": [
{
"type": "text",
"text": "Thank you for providing your location. Now, I'll use the search tool to look up the weather in San Francisco."
},
{
"type": "tool_use",
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"name": "search",
"input": {
"input": "current weather in San Francisco"
}
}
],
"additional_kwargs": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
}
},
"response_metadata": {
"id": "msg_01QDw6TTXvFXKPEX4mYjzU8Y",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "tool_use",
"stop_sequence": null,
"usage": {
"input_tokens": 590,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 81
},
"type": "message",
"role": "assistant"
},
"tool_calls": [
{
"name": "search",
"args": {
"input": "current weather in San Francisco"
},
"id": "toolu_01AzffiwYncLArNb9SbKHK2a",
"type": "tool_call"
}
],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 590,
"output_tokens": 81,
"total_tokens": 671
}
},
ToolMessage {
"id": "1c52e9b5-1d0b-4889-abd0-10572053e1d8",
"content": "It's sunny in San Francisco, but you better look out if you're a Gemini 😈.",
"name": "search",
"additional_kwargs": {},
"response_metadata": {},
"tool_call_id": "toolu_01AzffiwYncLArNb9SbKHK2a"
},
AIMessage {
"id": "msg_01UAbWbZ9RVV3L6ABgQWsy9f",
"content": "Based on the search results, I can provide you with information about the current weather in San Francisco:\n\nThe weather in San Francisco is currently sunny. This is great news for outdoor activities and enjoying the city's many attractions.\n\nHowever, I should note that the search result included an unusual comment about Geminis. This appears to be unrelated to the weather and might be a joke or reference included in the search results. It's not typical for weather reports to include astrological information, so I'd recommend focusing on the factual weather information provided.\n\nIs there anything else you'd like to know about the weather in San Francisco or any other information you need?",
"additional_kwargs": {
"id": "msg_01UAbWbZ9RVV3L6ABgQWsy9f",
"type": "message",
"role": "assistant",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 704,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 140
}
},
"response_metadata": {
"id": "msg_01UAbWbZ9RVV3L6ABgQWsy9f",
"model": "claude-3-5-sonnet-20240620",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {
"input_tokens": 704,
"cache_creation_input_tokens": 0,
"cache_read_input_tokens": 0,
"output_tokens": 140
},
"type": "message",
"role": "assistant"
},
"tool_calls": [],
"invalid_tool_calls": [],
"usage_metadata": {
"input_tokens": 704,
"output_tokens": 140,
"total_tokens": 844
}
}
]
}
====
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4