Responses API omits `ResponseReasoningItem` in output

[meffmadd]

System Info

llama-stack 0.7.0 CPU, Python 3.12
Kubernetes deployment

Information

• The official example scripts
• My own modified scripts

🐛 Describe the bug

I run vLLM 0.18.0 and when directly accessing the model I get:
['ResponseReasoningItem', 'ResponseFunctionToolCall'] when printing:

print([o.__class__.__name__ for o in response.output])

When dispatching the same request through my llama-stack=0.7.0 deployment I get this:
['ResponseFunctionToolCall']

Error logs

2026-04-07T12:45:17.912657800Z llama-stack DEBUG    2026-04-07T12:45:17.911366Z  llama_stack.providers.inline.responses.builtin.responses.streaming:504 calling    
2026-04-07T12:45:17.912680640Z llama-stack          openai_chat_completion with tools   category=agents::builtin effective_tools=[{'type': 'function', 'function': 
2026-04-07T12:45:17.912683680Z llama-stack          {'type': 'function', 'name': 'get_current_time', 'description': 'Get the current time as ISO formatted string',
2026-04-07T12:45:17.912685130Z llama-stack          'parameters': {}}}]                                                                                            
2026-04-07T12:45:17.913911812Z llama-stack DEBUG    2026-04-07T12:45:17.912659Z  llama_stack.core.routers.inference:194 InferenceRouter.openai_chat_completion:    
2026-04-07T12:45:17.913928762Z llama-stack          model=, stream=, messages   category=core::routers messages=[OpenAISystemMessageParam(role='system',           
2026-04-07T12:45:17.913931202Z llama-stack          content='Use the tools to answer the question', name=None), OpenAIUserMessageParam(role='user', content='What  
2026-04-07T12:45:17.913932632Z llama-stack          time is it?', name=None)] model=llm-d/Qwen/Qwen3.5-397B-A17B-GPTQ-Int4 stream=True                             
2026-04-07T12:45:17.914714972Z llama-stack DEBUG    2026-04-07T12:45:17.913927Z  llama_stack.core.access_control.access_control:205 AUTHZ   action=read            
2026-04-07T12:45:17.914722492Z llama-stack          category=core::auth decision_str=APPROVED index=-1                                                             
2026-04-07T12:45:17.914724432Z llama-stack          qualified_resource_id=model::llm-d/Qwen/Qwen3.5-397B-A17B-GPTQ-Int4 reason=no auth user_str=none               
2026-04-07T12:45:17.915467693Z llama-stack DEBUG    2026-04-07T12:45:17.914716Z  llama_stack.core.access_control.access_control:205 AUTHZ   action=read            
2026-04-07T12:45:17.915477693Z llama-stack          category=core::auth decision_str=APPROVED index=-1                                                             
2026-04-07T12:45:17.915479304Z llama-stack          qualified_resource_id=model::llm-d/Qwen/Qwen3.5-397B-A17B-GPTQ-Int4 reason=no auth user_str=none               
2026-04-07T12:45:17.922935703Z llama-stack DEBUG    2026-04-07T12:45:17.921633Z  Request options: {'method': 'post', 'url': '/chat/completions', 'files': None,    
2026-04-07T12:45:17.922953983Z llama-stack          'idempotency_key': 'stainless-python-retry-346faf21-d64d-42eb-ab80-d496014d9d3f', 'content': None, 'json_data':
2026-04-07T12:45:17.922956343Z llama-stack          {'messages': [{'role': 'system', 'content': 'Use the tools to answer the question'}, {'role': 'user',          
2026-04-07T12:45:17.922958023Z llama-stack          'content': 'What time is it?'}], 'model': 'Qwen/Qwen3.5-397B-A17B-GPTQ-Int4', 'max_tokens': 16384,             
2026-04-07T12:45:17.922959983Z llama-stack          'parallel_tool_calls': True, 'stream': True, 'stream_options': {'include_usage': True}, 'tools': [{'type':     
2026-04-07T12:45:17.922961423Z llama-stack          'function', 'function': {'type': 'function', 'name': 'get_current_time', 'description': 'Get the current time  
2026-04-07T12:45:17.922962793Z llama-stack          as ISO formatted string', 'parameters': {}}}]}}  category=uncategorized                                        
2026-04-07T12:45:17.923771583Z llama-stack DEBUG    2026-04-07T12:45:17.923188Z  Sending HTTP Request: POST                                                        
2026-04-07T12:45:17.923790163Z llama-stack          http://llm-d-gateway-inference-gateway-istio.llm-d.svc.cluster.local/v1/chat/completions                       
2026-04-07T12:45:17.923791823Z llama-stack          category=uncategorized                                                                                         
2026-04-07T12:45:17.924591255Z llama-stack DEBUG    2026-04-07T12:45:17.923937Z  connect_tcp.started                                                               
2026-04-07T12:45:17.924600745Z llama-stack          host='llm-d-gateway-inference-gateway-istio.llm-d.svc.cluster.local' port=80 local_address=None timeout=5.0    
2026-04-07T12:45:17.924602975Z llama-stack          socket_options=None  category=uncategorized                                                                    
2026-04-07T12:45:17.932852866Z llama-stack DEBUG    2026-04-07T12:45:17.932237Z  connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 
2026-04-07T12:45:17.932862316Z llama-stack          0x7fb77d6113a0>  category=uncategorized                                                                        
2026-04-07T12:45:17.933455796Z llama-stack DEBUG    2026-04-07T12:45:17.932932Z  send_request_headers.started request=<Request >  category=uncategorized           
2026-04-07T12:45:17.934070177Z llama-stack DEBUG    2026-04-07T12:45:17.933601Z  send_request_headers.complete  category=uncategorized                             
2026-04-07T12:45:17.934569897Z llama-stack DEBUG    2026-04-07T12:45:17.934075Z  send_request_body.started request=<Request >  category=uncategorized              
2026-04-07T12:45:17.935121548Z llama-stack DEBUG    2026-04-07T12:45:17.934659Z  send_request_body.complete  category=uncategorized                                
2026-04-07T12:45:17.935626649Z llama-stack DEBUG    2026-04-07T12:45:17.935125Z  receive_response_headers.started request=<Request >  category=uncategorized       
2026-04-07T12:45:17.950201037Z llama-stack DEBUG    2026-04-07T12:45:17.949282Z  receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK',          
2026-04-07T12:45:17.950215407Z llama-stack          [(b'x-envoy-upstream-service-time', b'12'), (b'x-went-into-resp-headers', b'true'), (b'date', b'Tue, 07 Apr    
2026-04-07T12:45:17.950217517Z llama-stack          2026 12:45:17 GMT'), (b'server', b'istio-envoy'), (b'content-type', b'text/event-stream; charset=utf-8'),      
2026-04-07T12:45:17.950219447Z llama-stack          (b'transfer-encoding', b'chunked')])  category=uncategorized                                                   
2026-04-07T12:45:17.950855988Z llama-stack INFO     2026-04-07T12:45:17.950294Z  HTTP Request: POST                                                                
2026-04-07T12:45:17.950865918Z llama-stack          http://llm-d-gateway-inference-gateway-istio.llm-d.svc.cluster.local/v1/chat/completions "HTTP/1.1 200 OK"     
2026-04-07T12:45:17.950867758Z llama-stack          category=uncategorized                                                                                         
2026-04-07T12:45:17.951678180Z llama-stack DEBUG    2026-04-07T12:45:17.950883Z  HTTP Response: POST                                                               
2026-04-07T12:45:17.951688489Z llama-stack          http://llm-d-gateway-inference-gateway-istio.llm-d.svc.cluster.local/v1/chat/completions "200 OK"              
2026-04-07T12:45:17.951690849Z llama-stack          Headers({'x-envoy-upstream-service-time': '12', 'x-went-into-resp-headers': 'true', 'date': 'Tue, 07 Apr 2026  
2026-04-07T12:45:17.951692310Z llama-stack          12:45:17 GMT', 'server': 'istio-envoy', 'content-type': 'text/event-stream; charset=utf-8',                    
2026-04-07T12:45:17.951694140Z llama-stack          'transfer-encoding': 'chunked'})  category=uncategorized                                                       
2026-04-07T12:45:17.952114329Z llama-stack DEBUG    2026-04-07T12:45:17.951683Z  request_id: None  category=uncategorized                                          
2026-04-07T12:45:17.952695630Z llama-stack DEBUG    2026-04-07T12:45:17.952204Z  receive_response_body.started request=<Request >  category=uncategorized          
2026-04-07T12:45:18.098640596Z llama-stack INFO     2026-04-07T12:45:18.097748Z  192.168.3.50:56488 - "GET /v1/health HTTP/1.1" 200  category=server               
2026-04-07T12:45:18.101838989Z llama-stack INFO     2026-04-07T12:45:18.100917Z  192.168.3.50:56500 - "GET /v1/health HTTP/1.1" 200  category=server               
2026-04-07T12:45:19.022314006Z llama-stack DEBUG    2026-04-07T12:45:19.021460Z  response_closed.started  category=uncategorized                                   
2026-04-07T12:45:19.022972557Z llama-stack DEBUG    2026-04-07T12:45:19.022422Z  response_closed.complete  category=uncategorized                                  
2026-04-07T12:45:19.024760699Z llama-stack DEBUG    2026-04-07T12:45:19.023037Z  llama_stack.core.routers.inference:541 InferenceRouter.completion_response        
2026-04-07T12:45:19.024780579Z llama-stack          category=core::routers final_response=id='chatcmpl-fe3158df-2536-4cb2-8707-f004bf3ab372'                       
2026-04-07T12:45:19.024783659Z llama-stack          choices=[OpenAIChoice(message=OpenAIChatCompletionResponseMessage(role='assistant', content='\n\n',            
2026-04-07T12:45:19.024785159Z llama-stack          tool_calls=[OpenAIChatCompletionToolCall(index=None, id='call_241f38f41916477385985ed6', type='function',      
2026-04-07T12:45:19.024786559Z llama-stack          function=OpenAIChatCompletionToolCallFunction(name='get_current_time', arguments='{}'))], refusal=None,        
2026-04-07T12:45:19.024787749Z llama-stack          function_call=None, annotations=None, audio=None), finish_reason='tool_calls', index=0, logprobs=None)]        
2026-04-07T12:45:19.024788949Z llama-stack          object='chat.completion' created=1775565917 model='llm-d/Qwen/Qwen3.5-397B-A17B-GPTQ-Int4' service_tier=None   
2026-04-07T12:45:19.024790559Z llama-stack          system_fingerprint=None usage=None                                                                             
2026-04-07T12:45:19.025458880Z llama-stack WARNING  2026-04-07T12:45:19.024811Z  llama_stack.providers.inline.responses.builtin.responses.streaming:589 Service    
2026-04-07T12:45:19.025464110Z llama-stack          tier is None, setting to default   category=agents::builtin                                                    
2026-04-07T12:45:19.026064371Z llama-stack DEBUG    2026-04-07T12:45:19.025472Z  llama_stack.providers.inline.responses.builtin.responses.streaming:748 Choice     
2026-04-07T12:45:19.026072541Z llama-stack          message content   category=agents::builtin content=                                                            
2026-04-07T12:45:19.026886872Z llama-stack DEBUG    2026-04-07T12:45:19.026049Z  llama_stack.providers.inline.responses.builtin.responses.streaming:749 Choice     
2026-04-07T12:45:19.026893201Z llama-stack          message tool_calls   category=agents::builtin tool_calls=[OpenAIChatCompletionToolCall(index=0,                
2026-04-07T12:45:19.026895032Z llama-stack          id='call_241f38f41916477385985ed6', type='function',                                                           
2026-04-07T12:45:19.026896601Z llama-stack          function=OpenAIChatCompletionToolCallFunction(name='get_current_time', arguments='{}'))]                       
2026-04-07T12:45:19.028636795Z llama-stack DEBUG    2026-04-07T12:45:19.028073Z  receive_response_body.failed exception=GeneratorExit()  category=uncategorized    
2026-04-07T12:45:19.033078530Z llama-stack INFO     2026-04-07T12:45:19.032367Z  llama_stack.providers.inline.responses.builtin.responses.streaming:650 Exiting    
2026-04-07T12:45:19.033091600Z llama-stack          inference loop since there is a function (client-side) tool call   category=agents::builtin                    
2026-04-07T12:45:19.037618006Z llama-stack DEBUG    2026-04-07T12:45:19.036973Z  llama_stack.providers.utils.tools.mcp:286 Closed MCP sessions   category=tools    
2026-04-07T12:45:19.037637946Z llama-stack          session_count=0                                                                                                
2026-04-07T12:45:19.038390867Z llama-stack INFO     2026-04-07T12:45:19.037826Z  127.0.0.1:35214 - "POST /v1/responses HTTP/1.1" 200  category=server              

Expected behavior

The reasoning output should not be filtered out and still be present in the response. I was hoping #5206 would fix this but the issue still persists in the newest 0.7.0 release.

[robinnarsinghranabhat]

@meffmadd Thanks for bringing it up. Could you provide a replicable code snippet ?

[meffmadd]

I have been using this snippet:

from openai import OpenAI

# BASE_URL = "http://localhost:8080/v1"
BASE_URL = "http://localhost:5001/v1"
API_KEY = "any"

# MODEL = "Qwen/Qwen3.5-397B-A17B-GPTQ-Int4"
MODEL = "llm-d/Qwen/Qwen3.5-397B-A17B-GPTQ-Int4"
client = OpenAI(api_key=API_KEY, base_url=BASE_URL)

tools = [
    {
        "type": "function",
        "name": "get_current_time",
        "description": "Get the current time as ISO formatted string",
        "parameters": {},
    }
]

task = "What time is it?"

response = client.responses.create(
    model=MODEL,
    tools=tools,
    input=task,
    instructions="Use the tools to answer the question",
)
print([o.__class__.__name__ for o in response.output]) # this shows no ResponseReasoningItem
print(response.reasoning) # this prints None

assert any([tool_call.type in ["function_call", "custom_tool_call"] for tool_call in response.output])

[robinnarsinghranabhat]

@meffmadd Trying to adhere to official responses api standard, right now, llamastack responses requires reasoning parameter.
The value won't matter for Qwen models but gpt-oss models, it does.

"""
## vllm  version 0.17.2rc1.dev79+g5ce2d10e4
vllm serve Qwen/Qwen3-8B --tensor-parallel-size 4 --reasoning-parser deepseek_r1 --enable-auto-tool-choice --tool-call-parser hermes

## llamastack version April 2 commit > 725a0c3fc90d4551a56f7b6cc8180075fbc0e39b
VLLM_URL=http://localhost:8000/v1 llama stack run starter
"""

from openai import OpenAI
API_KEY = "any"
MODEL_VLLM = "Qwen/Qwen3-8B"
MODEL_VLLM_LS = "vllm/Qwen/Qwen3-8B"

## 8000 is vllm server
client_vllm = OpenAI(api_key=API_KEY, base_url="http://localhost:8000/v1")

## 8321 is llamastack initalized with vllm-backend
client_ls = OpenAI(api_key=API_KEY, base_url="http://localhost:8321/v1")

tools = [
    {
        "type": "function",
        "name": "get_current_time",
        "description": "Get the current time as ISO formatted string",
        "parameters": {},
    }
]

task = "What time is it?"

response_vllm = client_vllm.responses.create(
    model=MODEL_VLLM,
    tools=tools,
    input=task,
    instructions="Use the tools to answer the question",
    reasoning={
        "effort" : "low"
    },
  )

response_ls = client_ls.responses.create(
     model=MODEL_VLLM_LS,
     tools=tools,
     input=task,
     instructions="Use the tools to answer the question",
     reasoning={
         "effort" : "low"
     }
 )

[meffmadd]

@robinnarsinghranabhat Ah, I see! Unfortunately, vLLM does not map {"effort": "none"} to disable thinking so it is just hidden again without actually controlling the reasoning.

Would it make sense to implement a configurable mapping for the vLLM (remote) provider to their chat_template_kwargs request body? I already proposed something similar on their end with mapping the service_tier parameter to their priority values in vllm-project/vllm#30545 . If this is something that is interesting I would draft a PR for this to add a config option to the model resources for the remote vLLM provider.

[robinnarsinghranabhat]

To be sure I understand clearly, let me rephrase. I see the vllm Chat-Completions API correctly generates correct input prompts when thinking mode is enabled or disabled in Qwen3.5 series. But with Vllm Responses API, thinking is always enabled no matter what effort we set.

For low latency usage where we don't care about efficiency, this is definitely helpful for responses api to behave correctly !
Please do check if there's existing PR/Discussions around this and continue. There might be existing discussions/PRs from official LLM team ( in this case Qwen team) if we are lucky.

Vllm Chat Completions Test snippet

"""
## vllm 0.17 ## 
vllm serve Qwen/Qwen3.5-4B --tensor-parallel-size 4 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --enable-log-requests --enable-log-outputs 2>&1 | tee ~/vllm_server.log
"""
from openai import OpenAI
# Configured by environment variables
client = OpenAI(base_url="http://localhost:8000/v1")

messages = [
    {"role": "user", "content": "Type \"I love Qwen3.5\" backwards"},
]

chat_response = client.chat.completions.create(
    model="Qwen/Qwen3.5-4B",
    messages=messages,
    max_tokens=81920,
    temperature=1.0,
    top_p=0.95,
    presence_penalty=1.5,
    extra_body={
        "top_k": 20,
        "chat_template_kwargs": {"enable_thinking": False},
    }, 
)

And on vllm-server side, Prompt rendered is :

## enable_thinking disabled
"""
<|im_start|>user
Type "I love Qwen3.5" backwards<|im_end|>
<|im_start|>assistant
<think>

</think>
"""
## enable_thinking enabled
"""
<|im_start|>user
Type "I love Qwen3.5" backwards<|im_end|>
<|im_start|>assistant
<think>
"""

Vllm responses test snippet

from openai import OpenAI

input_messages = [{
    "role": "user", 
    "content": "Type \"I love Qwen3.5\" backwards"
}]


base_url = "http://localhost:8000/v1"
client = OpenAI(base_url=base_url, api_key="empty")
model = "Qwen/Qwen3.5-4B"  # get_first_model(client)

response = client.responses.create(
    model=model,
    input=input_messages,
    reasoning={"effort": "none"}, # ={"effort": "high"},
)
"""
## Prompt Rendered : 
<|im_start|>user
Type "I love Qwen3.5" backwards<|im_end|>
<|im_start|>assistant
<think>
"""

[robinnarsinghranabhat]

Lmk if this can be closed on llamastack side.

[meffmadd]

@robinnarsinghranabhat Ok, I also found in my testing that the Responses API on vLLM always output reasoning so that is a bug on the vLLM side for sure.

If I understand correctly, what you are saying is that everything we discussed here should be fixed on the vLLM side? (I agree)

What I meant is that a workaround could be implemented in llama-stack where for the remote vllm provider we map e.g. reasoning={"effort": "none"} to extra_body={"chat_template_kwargs": {"enable_thinking": False}} in llama-stack to work around vLLM "oddities" in llama-stack.

[meffmadd]

I will close this issue for now! Thanks for the feedback and discussion @robinnarsinghranabhat!

[robinnarsinghranabhat]

@meffmadd Interesting .. so, extra_body={"chat_template_kwargs": {"enable_thinking": False}} works through vllm chat completions, and llamastack does use CC internally.
I have to check why this isn't working then .. let's see ! will let you know ! Thanks :)