一名开发者在Hacker News上展示了slotstream项目，该项目实现了在仅有48GB内存的Mac上本地运行104GB的Qwen3.8-Flash-Next模型，推理速度约12token/秒1。这一方案通过流式从SSD读取权重、动态内存管理等技术优化，使普通消费级硬件也能运行大型语言模型1。
该项目对硬件的基本要求为搭载Apple Silicon芯片的Mac、macOS 14及以上系统、以及512GB硬盘空间1。经过优化，模型在33GB内存分配点达到最佳性能表现，最低可支持8.1GB内存环境1。权重文件一次性下载需104GB，在高速网络环境下不到1小时即可完成1。预填充阶段可达到约125token/秒的速度1。该项目兼容Ollama和OpenAI SDK的chat与generate端点1。
A developer has unveiled slotstream, a project enabling the execution of Qwen3.8-Flash-Next on consumer-grade Mac computers despite a significant memory mismatch 1. The model, quantized to 104GB, runs on hardware with just 48GB of RAM through optimized streaming techniques that load weights dynamically from the SSD 1. The inference speed achieved is approximately 12 tokens per second under these constraints, while the prefill phase operates at roughly 125 tokens per second 1.
The project targets Apple Silicon machines running macOS 14 or later with a minimum of 512GB of storage 1. Optimal performance is attained with 33GB of memory allocation, though the system can function with as little as 8.1GB in minimal configurations 1. Initial weight download requires a one-time transfer of the full 104GB model, which takes under one hour on high-speed connections 1. The implementation provides compatibility with both Ollama and OpenAI SDK endpoints for chat and generation tasks 1.
评论
还没有评论，欢迎留下第一条。