AI驱动的软件工厂模式面临严重的质量问题。虽然StrongDM等企业已实现了"无灯工厂"式的全自动代码生成，但这些系统缺乏人类代码审查环节 [1]。Faros AI的数据显示，自今年1月至2月以来，采用此类自动化模式的团队面临拉取请求审查质量下降、事件频率增加，以及单个开发者的bug数量上升 [1]。
问题的核心在于现有AI编码模型无法维护代码库的长期质量。虽然Claude Code通过在特定工具集内进行强化学习训练相比之前的CLI代理实现了显著改进 [1]，但SWE-bench Multilingual等主流基准测试采用二元评分机制（FAIL_TO_PASS和PASS_TO_PASS），无法真实评估代码架构质量 [1]。模型缺乏对可维护性的评估能力，因为"好的设计成本在数周、数月甚至数年内才会显现" [1]。仅通过自动化代码审查和测试无法弥补这一根本缺陷。
为解决这一问题，业界已识别出四个关键规划阶段：产品需求审查、系统架构、程序设计和垂直切片 [1]。HumanLayer等公司提出的方案是重新引入人类代码审查，目标是"在保持人类水平代码质量的同时实现2-3倍速度提升" [1]。这表明真正有效的自动化不是消除人类审查，而是通过前期规划优化流程，让AI和人类专业知识相互配合。
Automated software factories powered by artificial intelligence have demonstrated significant limitations that cannot be overcome through engineering optimization alone, according to analysis focused on the gap between automated code generation and long-term code quality maintenance.
StrongDM's "lights-off software factory" achieved full automation of code generation while eliminating human code review [1]. However, data from Faros AI revealed a concerning trend: since January and February, pull request review quality has declined, incidents have increased, and the number of bugs per developer has risen [1].
The core problem lies in how AI coding models are currently evaluated and trained. Standard benchmarks like SWE-bench Multilingual employ binary scoring systems that measure only whether code transitions from failing to passing tests (FAIL_TO_PASS) or remains passing (PASS_TO_PASS) [1]. These metrics cannot assess code architecture quality, a deficiency with real consequences—good design decisions typically manifest their value over weeks, months, or even years [1].
While Claude Code has achieved notable improvements through reinforcement learning training within specific toolsets compared to earlier command-line agents [1], this advancement does not address the fundamental evaluation gap. Current benchmarks lack mechanisms to measure code maintainability.
Rather than relying solely on automated processes, the proposed solution reintroduces human code review as a necessary safeguard. A structured development workflow is recommended, encompassing four critical stages: product requirements review, system architecture, program design, and vertical slicing [1]. This framework aims to optimize the development process by moving planning and deliberation upstream.
HumanLayer, a company offering such a framework, targets achieving 2-3x speed improvements while maintaining human-level code quality [1].