AI models and chat

ollama ollama

Worth a look

Ollama is a tool that downloads and runs open AI language models on your own computer, or in the cloud, and gives you a command line and a local web API to talk to them. It is for trying out models and connecting them to other apps without sending your data to a third-party AI service.

vllm vllm-project

Worth a look

vLLM is a Python library and server for running large language models quickly and with less memory. It lets you host open models behind an OpenAI-compatible API so your own apps can use them.