一项名为 SimpleBench 的多选题文本基准测试显示,当前前沿大语言模型(LLM)在时空推理、社会智能及语言对抗鲁棒性方面仍落后于普通人类表现 1。该测试包含超过 200 个问题,旨在评估模型在上述特定领域的能力 1。结果显示,基于 9 名参与者小样本得出的非专业人类平均得分为 83.7%,这一分数超过了包括 Claude Fable 在内的所有受测顶级 LLM 1。
作为对比,顶级模型 Claude Fable 在 SimpleBench 中的得分为 81.9%,低于人类基线水平 1。结论指出,目前 LLM 依赖的记忆知识和近似推理检索机制,尚不足以完全超越人类在处理某些基础问题时的表现 1。
A new multiple-choice text benchmark known as SimpleBench has revealed that current top-tier large language models struggle with basic reasoning tasks, falling short of the average performance recorded by non-expert human participants 1. The benchmark evaluates capabilities in spatiotemporal reasoning, social intelligence, and linguistic adversarial robustness, comprising more than 200 questions 1.
In the results presented, a small sample group of nine human participants achieved an average score of 83.7% on the test 1. By comparison, Claude Fable, identified as one of the leading models evaluated, scored 81.9%, placing it below the human baseline 1. These findings suggest that the memorized knowledge and approximate reasoning retrieval mechanisms currently relied upon by frontier LLMs are insufficient for fully surpassing human performance on these foundational queries 1.
评论
还没有评论,欢迎留下第一条。