Fix autothink and thinkdeeper for transformers>=5

[jacquerie]

Since version 5 of the transformers library, method apply_chat_template returns a BatchEncoding object, which wraps the output tensor in a dict-like object: https://github.com/huggingface/transformers/blob/7ea2320c76117e6742364808a666ef6f2fb40a67/MIGRATION_GUIDE_V5.md#4-apply_chat_template-returns-batchencoding

The code in autothink and thinkdeeper did not account for this, which made the following code throw:

import torch
from optillm.thinkdeeper import thinkdeeper_decode
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Qwen/Qwen2.5-0.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": "In a dance class of 20 students, 20% enrolled in contemporary dance, 25% of the remaining enrolled in jazz dance, and the rest enrolled in hip-hop dance. What percentage of the entire students enrolled in hip-hop dance?",
    },
]

thinkdeeper_decode(
    model,
    tokenizer,
    messages,
    {
        "do_sample": True,
        "temperature": 0.1,
        "max_new_tokens": 1024,
    }
)

This PR fixes this issue by noticing that, since OptiLLM explicitly depends on transformers at version 5 and above (

optillm/pyproject.toml

Line 29 in c258858

"transformers>=5.0.0,<5.13.0",

), it's safe to unwrap the BatchEncoding object and query its input_ids property.

[CLAassistant]

All committers have signed the CLA.