I built an Ollama alternative with TurboQuant, model groups, and multi-GPU support

The problem I run multi-model architectures — 3 LLMs receiving the same prompt, deliberating, and producing a consensus response. Think of it as a voting system where individual model biases cancel out. Ollama swaps models sequentially. vLLM is cloud-oriented. llama.cpp server handles one model at a time. None of them could do what I needed: load 3+ models simultaneously, send them the same prompt in parallel, collect all responses, and handle failures gracefully. So I built EIE. What EIE does EIE (Elyne Inference Engine) is a local inference server for GGUF models. It loads models, serves them via an OpenAI-compatible REST API, and manages GPU memory. It does one thing : serve completions. No agents, no RAG, no UI. Everything else runs on top. Model Groups This is the core idea. Instead of thinking in individual models, EIE thinks in groups : groups : - name : core models : [ mistral-7b , granite-3b , exaone-2.4b ] required_responses : 3 type : parallel pinned : true fallback : partia