Your LLM Gateway is a Python Package. Here's Why That Should Worry You.

Two days ago, LiteLLM got backdoored . Two malicious versions published to PyPI. Credentials stolen. Kubernetes clusters compromised. 3.4 million daily downloads exposed. But this post is not just about LiteLLM. LiteLLM was the target this time. Next time it could be any Python package sitting in your AI infrastructure's critical path. If you're routing LLM requests through a Python-based gateway, here's what you need to understand about the risk you're carrying and what your options look like. What Your LLM Gateway Actually Has Access To Think about what your LLM gateway touches. If you're using LiteLLM, Portkey's open-source proxy, or any similar Python-based routing layer, it typically has: API keys for every LLM provider you route through (OpenAI, Anthropic, Google, AWS Bedrock, Azure, Mistral, Cohere) Environment variables loaded at startup Network access to your LLM providers and potentially your internal services Kubernetes service account tokens if you're running in K8s CI/CD s