Add tools support in langchain#37
Conversation
…on-genai into tools-langchain
Assisted-by: Claude Opus 4.6
lmolkova
left a comment
lmolkova
left a comment
There was a problem hiding this comment.
Looks good, let's just update langchain to use new InferenceInvocation instead of deprecated LLMInvocation
|
This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment. |
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds LangChain tool-calling instrumentation support by creating dedicated tool execution spans and capturing tool definitions/tool-call requests in inference spans, along with tests, conformance coverage, and a runnable example.
Changes:
- Add tool span lifecycle handling (
on_tool_start/on_tool_end/on_tool_error) and record tool-call requests ontool_callsfinish reason. - Capture tool definitions from chat model invocation params (
tools/functions) into inference spans. - Add extensive unit/integration tests, a new conformance scenario + VCR cassette, and a tool-calling example.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py | Adds helpers to normalize tool definitions into semantic-convention types. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py | Implements tool span callbacks and emits tool-call request parts on tool-calling responses. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_tools.py | Adds unit + callback-handler integration tests for tools and tool-calling outputs. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_conformance.py | Registers the new tool-calling conformance scenario. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/conformance/tool_calling.py | Adds a tool-calling conformance scenario using ChatOpenAI.bind_tools. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/cassettes/tool_calling_conformance.yaml | Adds recorded OpenAI HTTP interactions for the conformance scenario. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/examples/tools/main.py | Adds a runnable tool-calling example. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/examples/tools/requirements.txt | Adds example dependencies. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/37.added | Records the feature in the changelog. |
| raw_arguments: Any = inputs if inputs is not None else input_str | ||
| arguments: str | None | ||
| if isinstance(raw_arguments, dict): | ||
| arguments = json.dumps(raw_arguments) |
There was a problem hiding this comment.
is this necessary? can we pass Any ?
There was a problem hiding this comment.
Yes, it's necessary. invocation.arguments is typed AttributeValue | None, which doesn't accept dict. The block converts the structured inputs: dict to a JSON string, falling back to the
plain input_str: str. Passing Any directly would be a type violation and would break span attribute setting at runtime when inputs is a dict.
| tool_invocation = self._invocation_manager.get_invocation(run_id) | ||
| if not isinstance(tool_invocation, ToolInvocation): | ||
| return | ||
| tool_invocation.tool_call_id = getattr(output, "tool_call_id", None) |
There was a problem hiding this comment.
It is defensive pattern here given the output:Any - typed parameter
examples/tools/main.py
Tools span:
Inference spans with tool details:
