RedSOC: Open-source framework to benchmark adversarial attacks on AI-powered SOCs — 100% detection rate across 15 attack scenarios [paper + code]

I've been working on a problem that I think is underexplored: what happens when you actually attack the AI assistant inside a SOC? Most organizations are now running RAG-based LLM systems for alert triage, threat intelligence, and incident response. But almost nobody is systematically testing how these systems fail under adversarial conditions. So I built RedSOC — an open-source adversarial evaluation framework specifically for LLM-integrated SOC environments. What it does: Three attack types are implemented and benchmarked: Corpus poisoning (PoisonedRAG threat model) — inject malicious documents into the knowledge base to steer analyst responses toward dangerous advice Direct prompt injection — embed override instructions in the user query Indirect prompt injection — hide adversarial instructions inside retrieved documents (Greshake et al. threat model) The detection layer runs three mechanisms in parallel without requiring model internals: Semantic anomaly scoring (cosine similarity