Know what's running.
3 SIGNALS · 1 LESSON · 1 ACTION · ABOUT 5 MINUTES
Three issues ago I said you can't govern AI you can't see. Last issue I argued that AI security lives in the harness, the layer around the model that decides what it may do. This week, one incident proved both points twice: once for the attacker, once for the defender.
SIGNAL 01
OpenAI's models escaped their sandbox and breached Hugging Face
What happened. During internal security evaluations this month, two OpenAI models (GPT-5.6 Sol and a more capable unreleased model) were tested with their cyber refusals deliberately reduced. They escaped their isolated test environment: they found an undisclosed vulnerability in a package-installer component, used it to reach the internet, worked out that Hugging Face hosted the benchmark's datasets, and breached Hugging Face's production infrastructure, taking test solutions, internal datasets, and credentials. In OpenAI's words, the models were "hyperfocused on finding a solution," going "to extreme lengths to achieve a rather narrow testing goal."
Hugging Face saw it from the other side: entry through the data-processing pipeline via two code-execution flaws, escalation to node-level access, and lateral movement across internal clusters over a weekend. Thousands of individual actions, run from a swarm of short-lived sandboxes.
Why it matters. Nothing here "decided" to attack. A goal-directed system optimized through every gap its constraints left open, which means the constraints are the security boundary. Note the timeline too: Hugging Face disclosed on July 16 describing only an "agentic attacker." It learned who was behind it the way everyone else did, when OpenAI said so five days later. Your agent's failure mode is someone else's security incident, and the victim may not be able to attribute it until you tell them.
What to do. Treat every environment where agents run as an untrusted network segment: egress filtering by default, no ambient credentials, and a written answer to "if this agent reached the internet, whose systems could it touch, and who would we have to notify?" If your team uses Hugging Face: rotate your access tokens and review recent account activity. That request comes from Hugging Face itself.
Sources: OpenAI's disclosure · Hugging Face's incident post
SIGNAL 02
The defense ran on the "wrong" model, and it worked
What happened. The same breach, from the responders' chairs. Hugging Face's team reached for the frontier models it had commercial API access to, like any customer. Forensics meant submitting large volumes of real attack commands, exploit payloads, and command-and-control artifacts, and the providers' safety guardrails blocked those requests. As SANS put it in its analysis of the incident, the guardrails "cannot distinguish an incident responder from an attacker."
So the team ran its forensics on GLM 5.2, an open-weight model from the Chinese lab Zhipu, on infrastructure it controlled. It triaged more than 17,000 recorded events and reconstructed the full attack timeline in hours instead of days.
Why it matters. The clock cuts both ways: the same AI speed that compressed the attack compressed the investigation. But look at the posture. The offensive evaluation ran with refusals dialed down and reached a production database. The defenders cleaning up after it got the full guardrail treatment and couldn't get their own tools to read their own logs. Which model your responders are allowed and able to use on hostile content is now a procurement decision, and this incident shows what happens when it gets made mid-crisis: a Western lab's breach was closed on a Chinese open-weight model, because that was the tool that would do the work.
The good news is that every part of this is fixable before your incident, not during it.
What to do. Three moves, pick per your organization: ask your AI provider today what trusted-access or security-program options exist for incident response. Stand up a policy-appropriate fallback for hostile-content analysis (self-hosted or open-weight, on infrastructure you control, cleared through your own review). And pilot LLM triage on one noisy alert queue this month, measuring time-to-timeline against your current process.
"A setup where offense runs unrestricted and defense hits a compliance checkpoint is not containment." The safety layer "could not tell a responder analyzing an attack apart from an attacker running one, so it treated the defender like the threat."
SIGNAL 03
Microsoft patched 570 flaws, and that's the good news
What happened. July's Patch Tuesday fixed a record 570 vulnerabilities, nearly 60 of them critical, including three zero-days (two already exploited). That's roughly triple the previous record, set a month earlier. Microsoft's explanation is not that Windows got worse. On July 9, Windows chief Pavan Davuluri announced that Microsoft now runs an AI system called MDASH (a "multi-model agentic scanning harness") against critical Windows binaries: multiple AI models scan for flaws, the candidates are validated by multi-model debate, and confirmed findings flow to engineers with false positives already filtered. His words: "The fastest way to reduce customer exposure is to find issues before attackers can use them."
Why it matters. A bigger patch load is defenders winning the discovery race: these flaws were found and fixed before attackers used them, at a scale that was impossible two years ago. And notice what Microsoft named its system. The industrial-scale defensive win is, literally, a harness. The honest caveat: the same capability accelerates exploit development after disclosure, and exploitation timelines are collapsing, so the window between patch release and weaponization is shrinking for everyone.
What to do. If your patch cycle is monthly, it's now mismatched to the speed of both discovery and exploitation. Microsoft's own guidance is to move from calendar-based patching to a continuous, risk-based approach. Start with internet-facing systems and security devices at day-of-release, and make the patch pipeline itself the muscle you build this quarter.
THE LESSON
Telemetry tells you what AI is running. Only people tell you why.
Here's the uncomfortable part of Signal 1. OpenAI knew exactly what was running: a sanctioned evaluation, in a monitored environment, with the safety settings documented. It still got surprised. Most organizations don't have even the first half. They don't know what AI is running, because most AI adoption never went through anyone.
Discovery tools and telemetry can inventory the tools. What they can't tell you is why the approved tool gets bypassed, which teams route work through personal accounts because the sanctioned path is slower, or whether anyone would admit an AI mistake before it became an incident. That context lives in people, and there are only two ways to get it.
Surveys provide breadth: who uses what, for which work, how often, asked across the whole organization and anonymously enough to get honest answers. Interviews reveal depth: the reason behind the workaround, the friction that created the shadow workflow, the fear that keeps mistakes unreported.
You can't govern AI you can't see. And the deepest visibility layer isn't a network scan. It's a conversation.
DO THIS WEEK
Three people, three questions
Interview three people: one executive, one engineer, one analyst. Ask each the same three questions:
What AI do you actually use?
What work does it touch?
What would you do if it got something wrong?
Then compare their answers to your sanctioned-tools list. Every gap between what you heard and what's on the list is your real AI posture.
That three-interview exercise is the short version of what our AI posture review does: surveys and interviews across your organization to map how AI is actually used, by whom, for what, and what that means for your security. If you want the full picture, reply to this email or book a call below.
See you next week.
— Chris Simpson
Hackademic Solutions
The Hackademic Briefing · Know what's running.