An HTTP Gateway with Spring Boot
An Ollama-style HTTP layer in front of llama.cpp and vLLM with Spring Boot: an OpenAI-compatible API, streaming proxy, model loading and scheduling, security, metrics, and the loop gateway workshop.
An Ollama-style HTTP layer in front of llama.cpp and vLLM with Spring Boot: an OpenAI-compatible API, streaming proxy, model loading and scheduling, security, metrics, and the loop gateway workshop.