When swarm intelligence meets electricity data — and what goes wrong

A report on a hobby project that turned out to be more interesting than planned. I actually just wanted to see what you could do with real German electricity data. The Federal Network Agency’s SMARD platform provides hourly time-series data free of charge: wind power, photovoltaics, natural gas, pumped storage, biomass, load. It’s all publicly available and completely free. A dataset just begging to be experimented with. What followed was a classic journey of discovery: every answer led to a new question. This article sets out what I learnt along the way — for myself, and for anyone who thinks along similar lines. Step 1: The naive idea — simply setting thresholds My first approach was the obvious one: If natural gas exceeds X MW, then trigger an alarm. Classic threshold-based anomaly detection. The problem with this: you only detect what you already know. Whoever sets the threshold also determines what counts as ‘normal’. Unknown anomalies — precisely the ones of interest — fall throu