How to turn thinking on in IBM Granite 3.2

[PaulBlenkhorn]

IBM have recently released Granite 3.2 (https://www.ibm.com/new/announcements/ibm-granite-3-2-open-source-reasoning-and-vision). they say <IBM Granite 3.2 Instruct models allow their extended thought process to be toggled on or off by simply adding the parameter "thinking":true or "thinking":false to the API endpoint>.

I can't find a way to start a chat and set the parameter. I'm sure this is very straightforward and I'm missing something really obvious. Can anyone tell me how to do this?

[awaescher]

I don't know much about this model but I found a convo here where they say you'll have to add an additional message:

• IBM Granite 3.2 ollama/ollama#8955 (comment)

[sensoryapphouse]

Thanks.

[Ganesh-403]

@PaulBlenkhorn If you're using Ollama through OllamaSharp, I don't think there's currently a dedicated API for toggling Granite 3.2's thinking mode via a request parameter.

IBM's announcement mentions a "thinking": true/false parameter, but when running Granite 3.2 through Ollama, the model's integration is different. According to the Granite model page on Ollama, thinking is enabled by adding a control message to the conversation rather than by passing a top-level request parameter. :contentReference[oaicite:0]{index=0}

For example:

{
  "messages": [
    {
      "role": "control",
      "content": "thinking"
    },
    {
      "role": "user",
      "content": "Why is the sky blue?"
    }
  ]
}

If Microsoft.Extensions.AI or OllamaSharp doesn't currently expose a way to send a message with role = "control", then it may simply be a limitation of the abstraction rather than the model itself.

It might be worth checking whether ChatRequest.Messages can accept arbitrary roles (instead of restricting them to system, user, assistant, etc.). If not, supporting Granite's control role could be a useful enhancement for the library.

If this solves your problem, feel free to mark it as the accepted answer so others can find it easily.