Petals是一个开源项目，使用户能够在家中以类似BitTorrent的分布式方式运行大型语言模型 [1]。该平台通过连接网络中其他用户共享模型部分来完成推理任务 [1]。
项目支持多款主流大型语言模型，包括Llama 3.1（405B）、Mixtral（8x22B）、Falcon（40B+）和BLOOM（176B） [1]。在推理速度方面，Llama 2（70B）可达到6 tokens/sec，Falcon（180B）可达到4 tokens/sec [1]。
硬件要求相对易达，用户可使用消费级GPU或Google Colab进行运行 [1]。该项目提供了包括文本生成、微调、自定义采样方法和查看隐藏状态在内的多项功能特性 [1]。
Petals是BigScience研究工作坊的一部分 [1]。
Petals is an open-source project that enables users to run large language models at home through a distributed, peer-to-peer approach similar to BitTorrent [1]. The platform allows users to perform inference by connecting to other participants in the network who share different portions of the model [1].
The project supports several state-of-the-art models, including Llama 3.1 (405B), Mixtral (8x22B), Falcon (40B+), and BLOOM (176B) [1]. Inference speeds vary depending on the model: Llama 2 (70B) achieves approximately 6 tokens per second, while Falcon (180B) reaches around 4 tokens per second [1].
Hardware requirements are modest, as the system can run on consumer-grade GPUs or Google Colab [1]. Beyond text generation, Petals offers additional capabilities including model fine-tuning, custom sampling methods, and the ability to inspect hidden states [1]. The project emerged from the BigScience research workshop [1].