I Tested My AI Ad Generator on 3 Completely Different Ad Formats — Here's What Actually Happened

I recently open-sourced AdVideo Creator , a CLI tool that lets Claude generate complete video ads — script, images, voiceover, music, and final video — through a single prompt. In my first post , I walked through the architecture: 45 MCP tools, 5 quality gates, and a 15-step pipeline. The response was great. But one comment stuck with me: "Would love to see a follow-up post benchmarking output quality across different ad formats." Fair point. Architecture posts are nice, but what actually comes out the other end? So I picked 3 very different ad scenarios, ran them through the full pipeline, and recorded everything — scores, retries, failures, and the final videos. Here's what happened. The 3 Tests I deliberately chose formats that stress different parts of the pipeline: Product Demo Storytelling CTA / Urgency Product HydroSync (smart water bottle) Ember & Oak (coffee roastery) SkillSprint (online courses) Template Product Demo (5 scenes) Storytelling (5 scenes) Countdown/Urgency (4 sce