5 Things I Learned Wrapping a GUI-First API for AI Agents

Most AI agent tutorials assume you control both ends of the stack. You define the tools, you define the responses, everything is designed for programmatic access from the start. Real-world integrations are messier. The useful APIs — the ones that actually do something valuable — were usually built for humans with browsers. They return things like "click the Export button" and "drag the clip to the timeline." They assume someone is watching a screen. I spent the last few months building nemo-video , an OpenClaw skill that wraps a video editing backend originally designed for a web UI. Here's what I learned. Lesson 1: You Need an Interception Layer, Not Just a Wrapper The naive approach is to forward everything from the user to the backend and return whatever comes back. This breaks immediately when the backend responds with GUI instructions. The backend I was working with would say things like: "Your video is ready! Click the Export button in the top right corner to download it." If you