Cactus团队通过后训练技术对Gemma 4模型进行改进，使其能够识别自身的错误并输出0至1之间的置信度分数 [1]。
该方案的核心是一个仅包含68k参数的探针层，通过读取模型的隐藏状态来预测错误概率 [1]。开发者可以根据返回的置信度决定是否将查询路由到更大的云模型（Gemini 3.1 Flash-Lite），从而在准确性和成本之间进行权衡 [1]。根据统计，该方法仅需将15%至35%的查询路由到大模型 [1]。
在12个基准测试中，探针层的平均AUROC（受试者工作特征曲线下的面积）达到0.814，相比传统的token熵启发式方法的0.549有显著提升 [1]。在音频基准测试中，探针在未经音频数据训练的情况下仍然表现出色，AUROC达到0.79至0.88，而token熵方法在同一任务上的AUROC仅为0.32至0.52 [1]。
该项目采用MIT许可证开源 [1]，目前支持Transformers、MLX、Llama.cpp和Cactus等平台，后续计划支持Ollama、vLLM和SGLang [1]。不过该方案存在限制，仅支持单序列解码，最多生成1024个token [1]。
The Cactus team has developed a post-training approach for Gemma 4 that enables the model to identify when it is likely to make mistakes and output a confidence score between 0 and 1. [1] Using a lightweight probe layer with just 68,000 parameters that reads hidden states to predict error probability, developers can decide whether to route queries to a larger cloud model based on this confidence score. [1]
The method demonstrates strong performance across benchmarks, achieving an average AUROC of 0.814 across 12 tests, substantially outperforming token entropy heuristics which reached only 0.549. [1] In audio benchmark testing, the probe achieved AUROC scores between 0.79 and 0.88 despite receiving zero audio data during training, while token entropy scored just 0.32 to 0.52 on the same tests. [1]
The practical implementation routes only 15 to 35 percent of queries to Gemini 3.1 Flash-Lite based on confidence thresholds. [1] The solution is released under an MIT license and supports multiple platforms including Transformers, MLX, Llama.cpp, and Cactus, with planned support for Ollama, vLLM, and SGLang. [1] The implementation is currently limited to single-sequence decoding with a maximum of 1,024 generated tokens. [1]