Align KTO collator keys with the DPO convention#6182

qgallouedec wants to merge 3 commits into

mainhuggingface/trl:mainfrom

align-kto-collator-keyshuggingface/trl:align-kto-collator-keysCopy head branch name to clipboard