WebLLM是一个支持在网页浏览器中直接运行大语言模型推理的高性能引擎1。该项目通过WebGPU实现硬件加速，无需依赖后端服务器即可完成推理任务1。
WebLLM原生支持Llama 3、Phi 3、Gemma、Mistral、通义千问等多个开源模型1，并完全兼容OpenAI API规范，提供流式输出、JSON模式、函数调用等功能1。开发者可通过NPM、Yarn或CDN等方式集成该引擎，同时项目还支持Web Worker和Service Worker优化界面性能，并可作为Chrome扩展使用1。此外，WebLLM提供JSON Schema结构化生成和完整性验证（SRI哈希）等高级功能1。
该项目由来自CMU Catalyst、华盛顿大学SAMPL实验室、上海交通大学、OctoML等机构的成员联合发起，与MLC LLM项目配套推出1。
WebLLM is a high-performance language model inference engine that enables direct execution of LLM inference within web browsers without requiring server support 1. The platform leverages WebGPU technology to deliver hardware-accelerated processing capabilities 1.
The engine maintains full compatibility with OpenAI's API, supporting advanced features including streaming responses, JSON mode, and function calling 1. It natively supports multiple open-source models including Llama 3, Phi 3, Gemma, Mistral, and Qwen 1. The project offers optimization through Web Worker and Service Worker support to enhance UI performance 1, and can be integrated via npm, Yarn, CDN, or deployed as a Chrome extension 1. Advanced capabilities include JSON Schema-based structured generation and integrity verification through SRI hashing 1.
WebLLM was initiated by members from CMU Catalyst, UW SAMPL, SJTU, OctoML, and the broader MLC community 1, working in conjunction with the MLC LLM framework 1.
评论
还没有评论，欢迎留下第一条。