DeepInfer
by deepseek-ai
DeepInfer is a high-performance inference framework for DeepSeek models, designed for large-scale production services.
New and promisingUnder two months old. Expect rough edges and fast changes.
- Stars
- 23k +2186 a day, est.
- Licence
- Apache-2.0
- Runs on
- A small server, with a GPU
- Setup
- Technical
- Language
- Python
- Age
- 10 days
What you can do with it
- Run offline inference for DeepSeek models
- Optimize model and sampler runtime
- Configure asynchronous scheduling and batch-invariant inference
Ways a business could use it
- Improve the performance of AI models in production environments
- Reduce the computational overhead of large-scale inference tasks
Can you use it commercially?
Free to use, change and sell, including in paid products. Keep the copyright notice.
Not legal advice. Read the project's licence file before building a product on it.
Getting it running
Build from source with specific prerequisites and environment setup
Follow the official instructions in the project README.
Things to check
- Early alpha
- Technical setup required
Who's talking about it
- DeepInfer: High-performance inference framework for DeepSeek models Hacker News, 2351 points
Checked 26 Oct 2026. Summary drafted by AI from the project's README.
