Building a macOS Desktop Agent with Claude - How AI Wrote Most of Its Own Code

Building a macOS Desktop Agent with Claude Here is something that sounds circular but actually works: using an AI coding assistant to build an AI desktop agent. Fazm is a macOS app that can see and control your screen. It uses ScreenCaptureKit to grab frames, accessibility APIs to click and type things, and Whisper for voice input. The interesting part is that Claude wrote most of the Swift code itself. How It Works in Practice The key was getting the architecture figured out first. Once we had clear CLAUDE.md files describing the project structure, the component boundaries, and the conventions, Claude got surprisingly good at writing native Mac code. A typical development session looks like: Describe the feature in plain language Claude reads the existing codebase and writes the implementation Build, test, iterate For something like adding a new accessibility API interaction - say, reading the contents of a specific text field in a specific app - Claude can look at how existing intera