Broken Windows theory.

What if you didn’t have to computer as much?

This is the basic pitch for “agentic” artificial intelligence—AI products that purport to go beyond chatbots, to autonomously “go do stuff” on behalf of the user. Microsoft has been touting “agentic” capabilities coming to Windows, meaning that large language model–based AIs such as their own Copilot will be able to take actions in the operating system on the user’s behalf. The company is so excited about this new agentic development that it has decided to turn the feature off by default and is actively warning users about the security risks.

That was sarcasm, but Microsoft, to its credit, has also put numerous safeguards in place. For example, the agent won’t have access to all of the user’s files. It will be restricted to folders such as Documents, Downloads, Desktop, Music, Pictures, and Videos. In other words, there is a safety fence around Copilot, only allowing it to access EVERY PLACE A NORMAL WINDOWS USER WOULD CONCEIVABLY SAVE A FILE.

You might reasonably ask why you would want to give a chatbot access to your computer, given that chatbots already hallucinate not only facts about the world but also about their own capabilities. They even lie about what they have already done—see Evan Ratliff’s recent account in Wired of his own agents fabricating software development progress and user testing. Is that the entity I would trust to operate my devices and interact with my data?

Microsoft says that agent decisions will still need approval by a human, and any steps taken will need to be presented in advance and logged carefully. Which sounds smart to me, but I also wonder: even if the safeguards actually work, will they render the agents less useful? I had a recent project that involved testing ChatGPT’s ability to produce alt text to make images more accessible; overall, it was reasonably good at the task, but frequently produced text that was missing critical context because of safety features. OpenAI’s machine vision has come far enough that it could reliably recognize celebrities and public figures, but I found it frequently declined to do so because it ran up against some prime directive against doxxing people found in photographs. (For example, when captioning a still of Liam Neeson taken from his famous speech in Taken, ChatGPT would not identify him. Even if I provided Neeson’s identity and asked ChatGPT to incorporate it into the text, it refused.)

Suppose we do trust the guardrails, though, and the guardrails do the job, and the guardrails themselves don’t send the agent off the rails. I’m still not sold. Have you used an “agentic” AI? I have. My first experience with one was ChatGPT’s Agent Mode. (As an editorial aside, I will refer to it hereafter as Agent Mode, capitalized not out of respect for OpenAI’s product name, but because I’m imagining a cartoon government agent named Vanessa Mode).

I was, at the time, also testing an “AI”-based product for automating job applications, so that was one of the first things I tested the agent on. I gave it my resume, some demographic and background information (NOT my real email address or my SSN), and cut it loose on an online job application.

Agent Mode produced a small window that appeared to show its desktop environment and web browser, and also provided narration of what it was doing. I have no idea how much either of these corresponded to what was really going on; for all I know, the narrative was being generated by a separate model, prone to its own hallucinations. In any case, Agent Mode narrated, sometimes in minute, excruciating detail:

OK, I am clicking on the dropdown menu for state.

I am selecting Kansas.

I selected Kentucky. That’s wrong, I need Kansas.

OK, I am clicking on the dropdown menu for state again. This time I will use care and select the correct state.

I am scrolling the list of states.

Oops, I have selected Louisiana…

Again, I have no idea if I was watching an actual play-by-play, but if the narration was accurate, just one dropdown took Agent Mode more than a dozen attempts and nearly four minutes.

By the time Agent Mode wrapped up its assignment, it hadn’t wrapped up anything at all. The job application was only about 50% done, and my job history was incorrect. Agent Mode had entered a mix of incorrect job titles (these were on the resume I provided), job duties (also on the resume), companies (see the resume), cities and states (resume!), and tenure (AGENT MODE! MY OFFICE, NOW! I’LL HAVE YOUR BADGE FOR THIS!).

Which brings me back to Agent Copilot and its forthcoming ability to, you know, go do stuff in Windows. Despite the steps Microsoft says it will be taking to contain and limit the damage it can do, they acknowledge risks, including data exfiltration. Based on my agentic experience, I would also worry about the integrity of my files, even if they never leave my machine. And perhaps worst of all, “The new agentic OS features that allow AI agents to operate also open the door to malware,” as Windows Central puts it.

Two years ago this month, Microsoft announced the Secure Future Initiative—a multi-year, company-wide effort to change how Microsoft designs, builds, tests and operates its ecosystem. The company’s goal is “security above all else.”

So how did Microsoft mark the second anniversary of this new “secure by design” paradigm?

By admitting that they plan to ship a backdoor for malware, on purpose, in the form of agentic AI.