You're staring at a government portal. It's asking for the same information you've entered six times across four different forms. Your deadline is in two hours.
You're copying. Pasting. Clicking. Scrolling. Copying again. And somewhere in the back of your mind, you're thinking — isn't this exactly what AI was supposed to fix?
Good news: both Anthropic and OpenAI now offer tools that promise to do this for you. Claude and Codex can control your computer — moving your cursor, clicking your buttons, typing in your text fields. But before you hand over the keys, you need to know what actually works.
What Computer Use Actually Means (It's Not What You Think)
Traditional AI assistants work through APIs and integrations. They plug into your calendar, your email, your project management tools — directly, through the back door.
Computer use is fundamentally different. The AI actually looks at your screen — like you do — and interacts with it using your mouse and keyboard. It's not connecting to a backend. It's seeing what you see.
Which means it can work with software that has no API at all. That government portal with no developer access? The ancient accounting software your company has used since 1998? The PDF form that requires manual entry? This is where computer use shines — not replacing API integrations, but filling the gaps where APIs don't exist.
As of March 2026, both major AI labs offer this capability. Claude's computer use is available to Pro and Max plan subscribers through the Claude Desktop app on macOS and Windows. OpenAI's Codex app works on Mac and can run desktop tasks in the background while you actively use your computer for something else.
The Hierarchy You Need to Understand
Here's what surprised me when I started testing: Claude doesn't jump straight to controlling your screen. It has a priority system.
Claude first tries direct connectors — integrations with Gmail, Slack, and Google Drive. These are faster and more reliable. If a connector isn't available, Claude tries Chrome browser navigation. Full computer use — where Claude controls your entire desktop — is the last resort.
This hierarchy exists for good reason. Direct integrations are precise. Browser automation is pretty reliable. Full desktop control? That's where things get interesting.
Codex takes a different approach. OpenAI designed it to run multiple agents in parallel, each handling separate tasks without interfering with each other. Their blog claims agents can complete "weeks of work in days." Bold claim. Let's see how it holds up.
Real Workflows, Real Results
I tested both systems on actual tasks I do every week — not demo scenarios.
First test: a vendor registration form on a state procurement website. No API. No integration. Just a web form requiring clicks through twelve pages. Claude handled it in about three minutes (versus my usual eight). It got every field right and even caught a dropdown menu I usually miss.
Second test: a PDF expense report that needed recreation in our company's ancient internal system. Copy from PDF, paste into legacy app, repeat forty times. Claude switched between windows, copied values, pasted them into the right fields. Forty line items in about six minutes.
Not perfect speed. But I wasn't doing the work. I answered emails while it ran. That's the trade-off that actually matters — you're trading your attention, not necessarily your time.
The Critical Caveats You Can't Ignore
Anthropic explicitly warns against using computer use with sensitive data — healthcare records, financial information, personal documents. This isn't legal boilerplate buried in terms of service. It's front and center on their help center. They're telling you: don't trust this with your most important stuff yet.
Claude's system actively scans for signs of prompt injection when using your computer and asks explicit permission before accessing any new application. This matters because computer use creates a new attack surface. If you're browsing a malicious website while Claude has control, things could go wrong in new ways.
And yes, failure modes exist. Sometimes Claude clicks the wrong button. Sometimes Codex misreads what's on screen. I had Claude accidentally close a window I needed — minor inconvenience, but imagine if that window was an unsaved document. Oversight still matters.
Where Computer Use Makes Sense Today
Use it for: - Repetitive data entry across applications that don't talk to each other - Form filling on web portals — especially government or healthcare systems with no modern APIs - Legacy software that your IT department can't or won't replace
Hold off on: - Anything where an API integration exists (direct integrations are faster and more reliable) - Tasks involving passwords, financial accounts, or medical records - Corporate environments with IT policies about third-party access — talk to your security team first
Your First Test Run
If you want to try computer use yourself, start with a low-stakes task. Batch file renaming is perfect — have Claude rename fifty files following a pattern. If it messes up, you can undo it. Low risk, clear success criteria.
Next level up: form filling on a non-sensitive website. Maybe a newsletter signup or test account creation. See how it handles navigating web interfaces.
One approach that's worked well: create a separate user account on your computer with limited permissions. Test computer use there before bringing it to your main environment.
We're moving from AI that answers questions to AI that takes actions. From assistants you consult to assistants that execute. Right now, you're responsible for supervising — these are powerful tools, but they're tools you're wielding. The AI isn't replacing your judgment.
Both companies are iterating fast. This is version one territory. If you're technically curious and comfortable troubleshooting, computer use is worth exploring now. If you want tools that work reliably out of the box, waiting six months might be the smarter play.
Either way, pick one repetitive task you did this week. Just one. Something annoying but not critical. Try automating it. If it works, you've saved future time. If it fails, you've learned where the limits are. Both outcomes have value.
