Skip to content

Commit 3783675

Browse files
committed
feat: Add parallel tool calling support for Meta/Llama models
Add support for the parallel_tool_calls parameter to enable parallel function calling in Meta/Llama models, improving performance for multi-tool workflows. ## Changes - Add parallel_tool_calls class parameter to OCIGenAIBase (default: False) - Add parallel_tool_calls parameter to bind_tools() method - Support hybrid approach: class-level default + per-binding override - Pass is_parallel_tool_calls to OCI API in MetaProvider - Add validation for Cohere models (raises error if attempted) ## Testing - 9 comprehensive unit tests (all passing) - 4 integration tests with live OCI API (all passing) - No regression in existing tests ## Usage Class-level default: llm = ChatOCIGenAI( model_id="meta.llama-3.3-70b-instruct", parallel_tool_calls=True ) Per-binding override: llm_with_tools = llm.bind_tools( [tool1, tool2, tool3], parallel_tool_calls=True ) ## Benefits - Up to N× speedup for N independent tool calls - Backward compatible (default: False) - Clear error messages for unsupported models - Follows existing parameter patterns
1 parent 8b18374 commit 3783675

File tree

5 files changed

+563
-1
lines changed

5 files changed

+563
-1
lines changed

libs/oci/README.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ embeddings.embed_query("What is the meaning of life?")
6262
```
6363

6464
### 4. Use Structured Output
65-
`ChatOCIGenAI` supports structured output.
65+
`ChatOCIGenAI` supports structured output.
6666

6767
<sub>**Note:** The default method is `function_calling`. If default method returns `None` (e.g. for Gemini models), try `json_schema` or `json_mode`.</sub>
6868

@@ -79,6 +79,30 @@ structured_llm = llm.with_structured_output(Joke)
7979
structured_llm.invoke("Tell me a joke about programming")
8080
```
8181

82+
### 5. Use Parallel Tool Calling (Meta/Llama models only)
83+
Enable parallel tool calling to execute multiple tools simultaneously, improving performance for multi-tool workflows.
84+
85+
```python
86+
from langchain_oci import ChatOCIGenAI
87+
88+
# Option 1: Set at class level for all tool bindings
89+
llm = ChatOCIGenAI(
90+
model_id="meta.llama-3.3-70b-instruct",
91+
service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
92+
compartment_id="MY_COMPARTMENT_ID",
93+
parallel_tool_calls=True # Enable parallel tool calling
94+
)
95+
96+
# Option 2: Set per-binding
97+
llm = ChatOCIGenAI(model_id="meta.llama-3.3-70b-instruct")
98+
llm_with_tools = llm.bind_tools(
99+
[get_weather, calculate_tip, get_population],
100+
parallel_tool_calls=True # Tools can execute simultaneously
101+
)
102+
```
103+
104+
<sub>**Note:** Parallel tool calling is only supported for Meta/Llama models. Cohere models will raise an error if this parameter is used.</sub>
105+
82106

83107
## OCI Data Science Model Deployment Examples
84108

libs/oci/langchain_oci/chat_models/oci_generative_ai.py

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -342,6 +342,13 @@ def messages_to_oci_params(
342342
343343
This includes conversion of chat history and tool call results.
344344
"""
345+
# Cohere models don't support parallel tool calls
346+
if kwargs.get("is_parallel_tool_calls"):
347+
raise ValueError(
348+
"Parallel tool calls are not supported for Cohere models. "
349+
"This feature is only available for Meta/Llama models using GenericChatRequest."
350+
)
351+
345352
is_force_single_step = kwargs.get("is_force_single_step", False)
346353
oci_chat_history = []
347354

@@ -829,6 +836,10 @@ def _should_allow_more_tool_calls(
829836
result["tool_choice"] = self.oci_tool_choice_none()
830837
# else: Allow model to decide (default behavior)
831838

839+
# Add parallel tool calls support for Meta/Llama models
840+
if "is_parallel_tool_calls" in kwargs:
841+
result["is_parallel_tool_calls"] = kwargs["is_parallel_tool_calls"]
842+
832843
return result
833844

834845
def _process_message_content(
@@ -1186,6 +1197,7 @@ def bind_tools(
11861197
tool_choice: Optional[
11871198
Union[dict, str, Literal["auto", "none", "required", "any"], bool]
11881199
] = None,
1200+
parallel_tool_calls: Optional[bool] = None,
11891201
**kwargs: Any,
11901202
) -> Runnable[LanguageModelInput, BaseMessage]:
11911203
"""Bind tool-like objects to this chat model.
@@ -1206,6 +1218,11 @@ def bind_tools(
12061218
{"type": "function", "function": {"name": <<tool_name>>}}:
12071219
calls <<tool_name>> tool.
12081220
- False or None: no effect, default Meta behavior.
1221+
parallel_tool_calls: Whether to enable parallel function calling.
1222+
If True, the model can call multiple tools simultaneously.
1223+
If False, tools are called sequentially.
1224+
If None (default), uses the class-level parallel_tool_calls setting.
1225+
Only supported for Meta/Llama models using GenericChatRequest.
12091226
kwargs: Any additional parameters are passed directly to
12101227
:meth:`~langchain_oci.chat_models.oci_generative_ai.ChatOCIGenAI.bind`.
12111228
"""
@@ -1215,6 +1232,12 @@ def bind_tools(
12151232
if tool_choice is not None:
12161233
kwargs["tool_choice"] = self._provider.process_tool_choice(tool_choice)
12171234

1235+
# Add parallel tool calls support
1236+
# Use bind-time parameter if provided, else fall back to class default
1237+
use_parallel = parallel_tool_calls if parallel_tool_calls is not None else self.parallel_tool_calls
1238+
if use_parallel:
1239+
kwargs["is_parallel_tool_calls"] = True
1240+
12181241
return super().bind(tools=formatted_tools, **kwargs)
12191242

12201243
def with_structured_output(

libs/oci/langchain_oci/llms/oci_generative_ai.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,12 @@ class OCIGenAIBase(BaseModel, ABC):
120120
"""Maximum tool calls before forcing final answer.
121121
Prevents infinite loops while allowing multi-step orchestration."""
122122

123+
parallel_tool_calls: bool = False
124+
"""Whether to enable parallel function calling during tool use.
125+
If True, the model can call multiple tools simultaneously.
126+
Only supported for Meta/Llama models using GenericChatRequest.
127+
Default: False for backward compatibility."""
128+
123129
model_config = ConfigDict(
124130
extra="forbid", arbitrary_types_allowed=True, protected_namespaces=()
125131
)

0 commit comments

Comments
 (0)