Skip to Content

The day the FAA halted departures nationwide

This article was first posted on LinkedIn. 

Nash Borges

On Friday afternoon, November 13, 2026, radar and communications at air traffic control centers went unstable, and shortly before 4 p.m. Eastern the FAA ordered a nationwide ground stop. Aircraft already in the air were allowed to land, and the stop lasted more than four hours, with some exceptions, before it was lifted at 8:15 p.m. By the end of the day, thousands of flights had been canceled in what Reuters described as only the second nationwide departure stop since September 11, 2001. Transportation Secretary Sean Duffy explained it that night with the plainness of someone reading off a whiteboard: "Our switches, our routers, actually got overloaded with data, and they went down. And I shouldn't say they went down. They started to give us inconsistency in the data from those routers. And because of that, we shut the system down."

The ground stop capped a week that was already heavy with change. Microsoft's November release lists fixes for 1,094 of its own CVEs compared to 63 in November 2025. On Thursday, Exchange Online began failing. Microsoft's preliminary account now says traffic shifts during network maintenance and upgrades exposed a latent bug, causing connection requests to queue until servers exhausted their processing capacity. Microsoft restored service gradually, validating stability at each stage. Both investigations remain preliminary, and both have said they see no evidence of an attack or AI involvement.

The week raises a familiar question: can defenders absorb a growing volume of fixes without increasing change risk? Extra approval layers can lengthen the queue without improving stability, while larger bundles can make faults harder to isolate and contain. A decade of software delivery research points to a different answer: ship smaller changes more often, roll each one out in rings sized to how varied the machines are, and give exploited flaws and other immediately exploitable, high-impact exposures priority in the fast lane.

Bigger haystacks, a smaller exploited fraction

Of the roughly 1,100 Microsoft CVEs in the November release, exactly one was flagged as exploited in the wild at release, and it wasn't one of the 63 rated critical. That one, CVE-2026-124194, is a moderate-rated Outlook flaw that can leak the user's NTLMv2 authentication hash when the user opens a specially crafted message file, potentially enabling impersonation or password cracking.

Satnam Narang of Tenable summed up the month: "For most of the year, AI has been building bigger haystacks across Patch Tuesday releases, and this appears to be the first time it has found a needle." The volume is real, but the observed exploited fraction is tiny. Google's threat intelligence group observed active exploitation of only 0.23% of vulnerabilities disclosed in 2026 through August. Prioritizing an exploited vulnerability often means accelerating a package of fixes. A cumulative Windows update bundles many fixes together, so administrators stage the package containing the urgent fix across representative groups of machines. Microsoft's validation before release and an organization's testing against its own hardware and applications provide complementary checks as deployment expands.

The small exploited fraction helps explain where to concentrate effort. How quickly attackers move helps determine how urgently those fixes need to reach exposed systems. Sophos's AI Security 2026 report described CVE-2026-10520 in Ivanti Sentry exploited within 24 hours of a public proof of concept. For CVE-2026-42208 in LiteLLM, Sysdig observed exploitation attempts about 36 hours after the advisory entered GitHub's global database. CISA's June 2026 Binding Operational Directive 26-04 gave federal civilian agencies 180 days to implement risk-based remediation deadlines as short as three calendar days, citing AI as a factor that may narrow the response window. A small share of the haystack can demand a fast response, which is why the highest-risk fixes need an accelerated, monitored rollout.

Speed and stability move together

The instinct when the queue grows is to slow down and add approvals, but software delivery research shows that speed need not come at the expense of stability. Nicole Forsgren, Jez Humble, and Gene Kim's Accelerate, built on four years of State of DevOps surveys, found "no tradeoff between improving performance and achieving higher levels of stability and quality," and it found approval by a change advisory board had "no correlation with change fail rate." In the 2017 data, high performers deployed 46 times more often than low performers, and their changes were one-fifth as likely to fail. The book uses deployment frequency as a stand-in for smaller batches. Google's DORA program, which grew out of that research, said it again in its 2025 report, calling the speed-versus-stability trade-off "a myth" and finding that the teams strong at both made up nearly 40% of its sample.

AI enters this picture as volume. DORA's 2024 survey associated a 25% increase in AI adoption with a 7.2% drop in delivery stability, and the 2025 report associated higher AI adoption with higher delivery throughput and continued instability, with "no evidence" that AI softens the damage instability does. The surveys are observational, so they show association rather than cause. DORA's proposed explanation is that AI can enlarge changes and accelerate code production without a matching improvement in review, testing, and deployment. Forsgren and Abi Noda describe the same shift in Frictionless: when developers can generate code in minutes but deployments take hours, "the constraint shifts from creation to validation and delivery."

Outages live where changes activate

Google Cloud's June 2025 outage shows three steps that the label "bad update" hides. A release adds a defect, a later configuration change switches it on, and automation spreads it everywhere. On May 29, 2025, Google Cloud rolled out new quota-checking code region by region, exactly as careful operators do, but the failing code path was not exercised during that rollout. On June 12, a routine policy update with some fields left blank copied itself to every region within seconds and triggered that code path. The code lacked a feature flag for staged activation and appropriate handling for missing values, so the service that approves Google Cloud API requests crashed and restarted in a loop everywhere at once. A change that switches on everywhere at once is the largest batch there is, however small the diff.

Cloudflare encountered a similar problem in December 2025 while expanding request-body inspection to protect customers against a newly disclosed vulnerability. That change was rolling out gradually. When an internal testing tool proved incompatible with the larger buffer, engineers disabled it through a separate configuration system that propagated globally within seconds. Turning the tool off triggered errors in part of Cloudflare’s proxy fleet. The main rollout was staged, but the supporting configuration change was not.

The controls around a release need the same discipline as the release itself. Turning a feature off, changing a flag, or retiring an old code path can change production behavior. Those operations need testing and containment too. Temporary rollout flags should be deliberately retired once their job is done, while retained recovery switches need owners and tested behavior.

Rings matter most where every machine is different

Small, frequent changes are easier when the operator controls the rollout and rollback path, though even a cloud fleet has varied hardware and configurations. Microsoft's Azure team, which has called change the primary cause of its reliability problems other than routine hardware failures, normally rolls changes through canary and pilot stages, waiting at each stage while it watches health signals. Distributed fleets are harder, because endpoints, branch sites, and air traffic control centers run different hardware, drivers, and configurations, and a bad change can knock out the very connection that would deliver the fix. There the early rings carry the load, because they're the first place a change meets machines nobody could test on.

The FAA's outage shows how far a local change can travel. By Sunday, the FAA had identified two potential sources of the traffic flood: a radio site in El Dorado, Arkansas, where L3Harris was deploying a new router, and a radar site in Amarillo, Texas. The agency reported 1,000 connection attempts every four to five seconds and instability at 17 of its 22 en route centers. Blocking both sites stabilized the network, and the broadcasting function was disabled on the new routers. The initiating fault remains under investigation, and the public account does not establish what testing or isolation preceded the work.

Security content can combine a fast cadence with operating-system-level consequences. At CrowdStrike on July 19, 2024, a routine detection update expected 21 pieces of data from the sensor, the sensor supplied 20, the check that should have caught the mismatch passed it anyway, and the part of the sensor running inside the core of Windows read past the end of what it had and crashed the machine. The file was released without staged rollout and was live for 78 minutes before CrowdStrike pulled it. Microsoft estimated 8.5 million Windows devices were affected. The bad content was withdrawn quickly, but restoring affected machines was a separate job, involving manual remediation guidance, scripts, and automated recovery efforts.

Microsoft's Defender for Endpoint splits its updates by what they can break. Monthly code and driver updates, which can reach the kernel, spend weeks in stabilization and then move through rings that start with Microsoft's "hundreds of thousands of employees and millions of internal devices," watched for reliability, performance, and compatibility "across hardware and software configurations." Detection updates ship several times a day, too often for that process, so Microsoft keeps kernel changes out of them entirely, still stages them from internal devices to early-access customers before everyone else, and says it can mitigate problems "through the cloud in minutes." Xbox uses five opt-in preview rings, ranging from invite-only testers to anyone who signs up, with one testing a separate branch for future releases.

This part does have a trade-off between speed, safety, and coverage. Those of us who ship detection content are being promised a lot more of it. AI can help a threat research team write detections faster, and Microsoft's split shows a good way to handle it. Content that only touches user-mode components and can be pulled back from the cloud can ship several times a day, while content parsed by kernel-mode code, like Crowdstrike’s Channel File 291, can fail with operating-system-level consequences. That content still needs staged rollout, so later rings wait for protection they could otherwise have received immediately. Where exploitation is moving quickly, I don't see a way around that trade-off, only a way to make it deliberately and with a clock attached.

At Sophos, the release principles we publish describe content moving through purpose-built test environments, then out internally, then to customers in stages while we watch telemetry and adjust releases, and we described the same process in July 2024, four days after Channel File 291. Joe Levy, our CEO, put the limit plainly at the time: "whatever precautionary steps we take, we are never 100% immune." What staging buys is the chance to catch a bad push on a small slice of the fleet before it reaches everyone.

Change lanes and detection cover

In practice, that means sorting changes into lanes by risk and keeping each one small. The Office update containing the exploited Outlook fix gets priority for an accelerated, monitored rollout, targeting rings measured in hours. Other packages follow risk-based staging, with exposure, exploitability, and consequence able to justify acceleration even before exploitation is observed. Rings should be sized by how different the machines are, so more uniform fleets can use shorter staging windows when their health signals justify it, while endpoints and remote sites need early rings that represent the important hardware and configuration families. Security content, configuration, and data that copies itself across regions need staged activation, limits on propagation, and tested halt or recovery mechanisms, just as code does, and the gate sits at the moment a change switches on. The tools for recovering sit outside the reach of the change.

The strongest objection is easy to state. If some newly disclosed flaws are being exploited within a day or two of a public proof of concept, isn't every hour of staging another hour of exposure? It is, and that is why staging needs defenses other than the patch. Zero-days are exploited before a patch exists. Detection, containment, and compensating controls can reduce exposure while a patch is unavailable or being staged, but their coverage has to be verified for the flaw and environment. For known flaws that already have a patch, the goal is rings measured in hours on the few changes that warrant them, concentrating the fast lane on the highest-risk packages rather than forcing every rollout onto that tighter clock.

Whatever the FAA eventually finds in its routers, its response made the right moves at national scale: departures stopped, flights in the air landed, and traffic resumed after the FAA confirmed stability. Rings and isolation boundaries give a rollout a chance to catch and contain a failure on a small slice of the fleet, before a broader shutdown becomes necessary for safety.

Research and blogging assistance provided by Hermes, an AI Assistant. It expected 21 edits on this draft, received 20, and decided to read past the end anyway.