New technique helps LLMs rein in CoT lengths, optimizing reasoning without exploding compute costs

TRANSING philosophy (COT) – The response of the models has become an integral part of the most recent generation of the process, which is the problem of “thoughts”, before the desires.

However, the costs of distrust of models models can be collected quickly as soon as they have created many tokens. One New paperResearchers at the University of Carnegie Mellon offers a LLM training technique that supervises the length of bed length.

The technique called LCPO managed policy optimization (LCPO) conditioning the model of the model to ensure the correct answers as the predefined miracle budget within a predetermined miracle budget. Experiments show that the teaching models in LCPo provide a smooth trade between accuracy and expenses and can surprise larger models in an equal thinking length. LCPo can help reduce the costs in enterprise practices by protecting thousands of verses in each round of conversation with an LLM.

LLM performance causes longer strokes

Openai O1 and the models of thinking like DeepSeek-r1 taught through reinforcement learning (RL) to use Test time scale and create cot prints before answering. Empirical evidence shows that the models are better performed for a longer term “thinking” and thinking about their way of thinking.

For example, R1 first trained in a clean RL without human labeled samples. One of the concepts was that as the model’s performance was improved, he also learned to create longer riding traces.

In general, long cot chains result in more accurate answers, and create a computing glass to apply justification models on a scale. The test has very few control over the calculation budget in time and sequences stretch easily on tens of thousands of feet without gaining significant profits. There were some efforts to monitor the length of the supply chains, but usually worse the model of the model.

Length-controlled policy optimization (LCPO) explained

The classic RL method is only LLMS to get the correct answer. LCPo This paradigmar changes two training purposes: 1) Get the correct result and hold the COT chain in a certain sign in length. Therefore, if the model answers correctly, if a large number of beds create tokens, it will be punished and will be forced to come to the same answer, but with a smaller token budget.

“LCPo teaching models learn to provide longitude restrictions, while optimizing justification, rather than trusting the grounded heuristics,” researchers write.

LCPO offers two flavors: (1) LCPo-accurate, the target requires LCPo-max that requires you to be completely equal to target length and (2) LCPO-MAX, the target is not more than target length.

To test techniques, researchers are finely adjusted to 1.5B parameter justification model (1.5B) in two proposed LCPO schemes to create accurate models of L1-MAX and L1. Training was based on mathematical problems with different and verifiable results. However, unexpected assignments include assessments, math problems, as well as a multitask language concept of size (Mmlu) Techniques and Graduate Level Google-proof Q & A Benchmark (Gypgu).

Their finds shows that L1 models can associate with interpoling, more accurately, with more accurate thinking, with short, effective thinking and longer, more accurate thinking with different lengths, more accurate thinking. In some positions, L1 models are multiplied in a lower sign of the performance of the original model of thinking.

Compared to S1 – the only method that limits the length of the COT length – L1 models show 150% performance earnings in various signs.

“This capital difference can be attributed to two main amidments,” researchers write. “(1) L1 often deficits the average justification of S1; and (2) L1 is clearly taught to create high-quality substantial chains of different lengths, to further distillate those who are shorter than longer chains.”

L1 increases the unpredictable colleague by 5% and 2% to GPT-4O in equal generation length. “According to our best knowledge, this is the first demonstration that the 1.5B model can take the frontal models like GPT-4O,” the researchers are writing.

Interestingly, the model cot, it shows that his thinking process is based on its remarkable budget. Məsələn, daha uzun büdcələrdə, modelin özünü düzəldilməsi və yoxlanılması (bu, “gözləmə” və “gözləyən” və “buna görə” və “buna görə” və “belə”) ilə əlaqəli tokenlər yarada bilər.

Outside of the developed length, L1 models, GPGA and MMLU, including the standard math justification, summarize a surprisingly good way.

This new research line, which can regulate the budget, has the ability to measure scale models without running expenses. It is a strong alternative to placing large, more expensive models – and can be an important factor to make the EU more economically important for high-volume, real world applications.

Researchers have an open source lcpo code and Weights for L1 models.