Last Week, Everyone Agreed Something Must Be Done About AI. That's a Tell.
Last week the EU, the White House, OpenAI, and Anthropic all moved on AI governance. Every proposal protects the incumbents. Here's why that convergence is the tell.
By all appearances, last week was a banner week for AI governance and AI safety proposals. Everyone, collectively seemed to be waking up—The EU announced an ambitious AI/data sovereignty strategy for decoupling itself from US Big Tech, the White House issued its first real Executive Order setting any semblance of guardrails around frontier AI development, OpenAI released a blueprint calling for federal AI governance policy, Congress floated a bill purporting to do just that, and Anthropic suggested maybe it was time for a global AI pause. By all appearances, everyone was looking very decisive on the AI question.
Unfortunately, by my read, last week was mostly just a demonstration of my larger thesis on how big tech companies are increasingly becoming ungovernable.
But perhaps, I’m being a little too cynical here. I’ve got some thoughts (which I’ll cover in a future blog post) at the end. Also, I get that this sounds a bit contradictory, and don’t worry, I’ll explain my cynicism in a minute. But first, I need to break down the busy week we had.
The EU makes a play for sovereignty
On June 3, 2026, the EU announced its Tech Sovereignty Package, which consists of a few different components. The most ambitious of which being the Cloud and AI Development Act (CADA). It would prioritize homegrown data center and AI model development, including by tripling the number of data centers (and therefore compute) located on EU soil, as well as bolstering open source development. To get there, the Commission proposed a number of measures that would reduce red tape, and streamline procurement processes. For example:
Data center acceleration zones: Member States must designate zones where permitting for data centers is expedited to under 18 months.
Prioritizing open source first: Article 41 of CADA would encourage the Union and member states to take ‘the necessary measures’ to encourage the use of open standards, libraries, and open source software by public bodies and Union entities.
Common procurement processes: Extends the Commission’s ability to act as a central purchasing body for Member States to leverage collective bargaining power.
EuroCloud federation: A platform for public-sector bodies to share idle cloud and data center capacity. The proposal suggests that this idle-data sharing arrangement could itself be a revenue-generator through usage fees.
Sustainability framework: Adds a new EU rating scheme designed to incentivize energy-efficient data center development.
Financial commitments: Importantly, the package would also allocate EUR 34.55 million (2028–2034) in operational appropriations from the voted budget, plus EUR 54.33 million (expected to be covered by user fees for the EuroCloud federation), and supplemental funds from the European Competitiveness Fund (ECF), Horizon Europe, and the digital Europe program, amongst others.
A separate proposal, the Chips Act 2.0, would repeal and reinforce the 2023 Chips Act to address so-called “lab-to-fab” gaps (for example, including certain chip capacities that weren’t considered in the original Act), reduce regulatory hurdles, and address structural vulnerabilities in the semiconductor value chain. There’s also much discussion of frameworks for identifying strategic projects of “common Union interest” like advanced node manufacturing and memory fabrication.
Unfortunately, I didn’t see much in the way of economic commitments, beyond the first Chips Act, which allocated EUR 52 billion in public and private investments.
Two different strategic packages were also attached to the Tech Sovereignty package: the Open Source Strategy and a Strategic Roadmap for Digitalisation and AI in Energy. The former is designed to encourage wider use of open source software by the public sector (and is liberally sprinkled throughout the CADA and Chips proposals), while the latter sets out exactly how the EU should expand data centers in a “sustainable and transparent manner” and accelerate the deployment of digital and AI solutions for the energy sector. Truthfully, it reads like a plan to plan, but I did find the proposals on expanding smart grid and metering tech, and developing foundational AI models (with a preference for open-source AI tools & models) to be interesting.
I suspect that some or all of these proposals will be implemented in some form, because the EU has strong incentives to act.
See, the EU is very, very dependent on foreign technologies across almost every sector. Currently, the EU spends €264B/year on foreign technologies for critical systems, with 80% of the EU’s tech stack being foreign-owned. Most data is housed in data centers owned by a small cadre of mostly US firms (Microsoft, Google, and Amazon), which creates precarity given the fact that the United States is currently run by an erratic toddler with a vengeance streak, and the US CLOUD Act is still good law. Politico has a good overview on the political dynamics here.
The Commission’s proposals will now go to national governments and the European Parliament for negotiations on the final version.
Meanwhile, in the US …
The Trump Administration finally signed an AI Executive Order that purports to set some basic ground rules on the development of frontier AI models. The order, ‘PROMOTING ADVANCED ARTIFICIAL INTELLIGENCE INNOVATION AND SECURITY’, represents a major coup by the administration’s few remaining grown-ups—Chief of Staff Susie Wiles, Treasury Secretary Scott Bessent, and National Cyber advisor Sean Cairncross—over the rival faction of AI YOLO-libertarians (David Sacks, Mark Zuckerberg, Elon Musk, et al), who had only a week before, scared Trump into cancelling a substantially similar, but slightly more meaty EO hours before signing.
The core features of the EO are as follows:
Establishes coordination & hardening obligations for various government agencies—e.g., ‘prioritizing cyber defenses’, issuing directives, and mandating that Treasury, NSA, and CISA build a classified benchmarking process to identify ‘covered frontier models’ (i.e., AI powerful enough to pose security risks).
Provides a voluntary framework whereby model developers can (i) self-select whether model(s) under development meet the designation of a covered frontier model; (ii) voluntarily provide the Federal Government with access to said covered frontier model for a period of up to 30 days for pre-testing purposes prior to general release; (iii) provide ‘select trusted partners’ defined by the administration with early access to covered frontier models to strengthen the cybersecurity of critical infrastructure.
The EO explicitly states that this order, in no way, shape, or form, creates “a mandatory governmental licensing, preclearance, or permitting requirement for the development, publication, release, or distribution of new AI models, including frontier models.”
Basically, the EO is the administration begrudgingly admitting that companies could at least meet the barest of minimums when it comes to AI safety and cybersecurity. It’s less a governance edict, more like a hardening and resilience suggestion.
Anthropic & OpenAI’s parallel plays
Simultaneously, both OpenAI and Anthropic also pushed for more governance and oversight, with Anthropic even suggesting that maybe it’s time to talk about a global AI pause.
On June 2, OpenAI released a nine-page blueprint, “Democratic Governance of Frontier AI”, which consists of a three-part framework that places the US Federal Government as the decisive, and arguably, exclusive leader on this issue, applying a “reverse federalism” approach. Unlike Anthropic, there’s no mention of a pause. Instead, OpenAI proposes the following governance measures:
building a national framework through “reverse federalism” which leverages the emerging consensus reflected in state frontier safety laws. OpenAI name-dropped New York’s RAISE Act, California’s SB 53, and Illinois’ SB 315 as examples;
strengthening CAISI [the Center for AI Standards and Innovation] as the US government’s primary institution for frontier AI safety evaluation, but with no meaningful oversight; and,
mobilizing a broader resilience plan across governments to address the national security and public safety challenges posed by frontier AI.
As with the Trump EO, most of the state laws and proposals cited by OpenAI as model legislation (bar perhaps, Illinois’ SB 315, which isn’t law yet), are light-touch, and emphasize generalized transparency and alerting obligations for security incidents, modest fines and whistleblower protections, and in the case of Illinois’ bill, actual third-party audit obligations, but again, very manageable stuff for the incumbents.
None of the state laws OpenAI included, for example, prohibit specific harmful AI uses (cf: the EU AI Act’s Article 5), or provide a private cause of action for AI-derived harms. None of the laws set limits on AI development at all. Big Tech loves these kinds of laws because they’re easy to game, make the companies look responsible and serious in the press, and have the bonus of slowing scale-ups and competitors.
See, what OpenAI really wants is to enshrine the barest of minimums as a ceiling to further regulation—by urging the US government towards federal preemption. That’s the whole ‘reverse federalism’ angle. This is convenient, and it isn’t new. OpenAI sought essentially the same preemption in 2025, and the Senate killed a version of it 99–1. “Reverse federalism” is a rebrand of a deeply unpopular idea, dressed up in the language of laboratories-of-democracy.
Two days after the OpenAI Blueprint, on June 4, Representatives Jay Obernolte (R-CA) and Lori Trahan (D-MA) released a 269-page discussion draft bill in the House. The draft, known as the “Great American AI Act” is essentially the Blueprint, with way more words. It includes all the transparency, CAISI language, and importantly, a three-year preemption over state legislation on AI development.
I’m here to tell you that in practice, these laws are AI safety theatre.
Next up, we’ve got Anthropic and their very ambitious June 4 blog post, “When AI Builds Itself” (authored by Marina Favaro and Jack Clark through the Anthropic Institute). In the piece, the company lays out just how powerful Claude Mythos really is, and how much Anthropic relies on its AI models for everything from coding to research. According to Anthropic, these models are scary good, and the paper includes receipts. For example, as of April/May 2026, Claude now writes 80% of Anthropic’s code, steers 64% of the research, and identifies more mistakes in existing code than human engineers.
The paper also identifies a major threat: recursive self-improvement (RSI). RSI is what happens when AI models become so advanced that they can improve themselves with little or no human involvement. OpenAI’s blueprint also mentions RSI as the governance challenge “existing institutions are not equipped to address.” In response to this threat, Anthropic recommended a global AI slowdown.
But there’s one small problem with an AI pause, the authors note. It’s meaningless unless everybody agrees on the rules and follows them to the letter.
If it were possible to effectively slow the development of this technology to give ourselves more time to deal with its immense implications, we think that would likely be a good thing. But if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe. Without a global coordination mechanism, companies and governments will have to make difficult decisions about safety while under competitive and geopolitical pressures.
It’s a classic arms-race problem. And as was the case with nuclear and other disarmament treaties of the past, a big problem is around verifiability. Anyone can promise that they’ll agree to a pause, and then just lie and go on developing anyway. There needs to be a way to verify that frontier labs will honor their promises and hold them to it. But AI is different from an atomic bomb: first, building a new, more advanced model is a lot easier to hide than a nuclear stockpile. Second, unlike with nuclear treaties, humanity probably doesn’t have decades to sort this out.
Enter Anthropic with an answer: they’re going to organize a series of conversations with policymakers, researchers, civil society, and other AI companies to hash out how to do a successful slowdown that everybody will agree to:
These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.
Why now? (the honest version, not the comforting one)
Now, I’ll be honest: a month ago I’d have struggled to define RSI. It barely registered outside AI-safety circles. So, it’s a little interesting that it’s now a supporting thesis for two different corporate proposals released within 48 hours of each other.
That’s not two companies independently noticing the same fact about the world (who are both, coincidentally, also planning to go public in the near future). That’s two companies reaching for the same rhetorical device—a danger grave enough to justify governance, conveniently the kind of governance only the people already at the frontier can be trusted to design.
But maybe the timing isn’t coordinated, and instead, four separate, powerful institutions (the US and EU governments, OpenAI and Anthropic), all independently woke up on Monday and decided to push some policy.
Or maybe still, this is a salvo to address the other myriad anxieties and frustrations around AI that are percolating in the collective consciousness. Even the most insular billionaire can read the room, especially when the mood is this palpable. The speed of change is alarming. I’m alarmed, even though I see a lot of promise in the tools being developed. Something needs to be done.
Regardless of the exact motivations, the facts are still the same: Last week, we saw a convergence on incumbent-friendly governance. This is exactly what I expect to see if my larger theory of tech extensity is real. Namely, when a handful of firms define more and more of the substrate the rest of us depend on, the governance response bends sharply toward whatever protects that position, regardless of who’s drafting it or why.
Pay attention to the solutions being presented
The trick then, if you’re a powerful force who wants to stay in power, is to get in (preferably before the mobs come) and define what that something is. Defining the something gives the politicians and the tech execs an air of legitimacy. “Hey look, we hear you, and we’re committed to action!”, they can say. “See? Please stop attacking our CEOs, protesting outside our homes, lobbying your representatives, or blocking our data centers.”
For Anthropic, the same document that says “be afraid, the curve is bending” also says “and we, specifically, should build the machinery that decides what happens next.” Anthropic isn’t just calling for a pause— it’s volunteering to construct the verification regime a pause would require, the system that would adjudicate whether everyone else has actually stopped. The pause itself stays conditional on infrastructure and agreements that don’t exist, which means the commitment costs precisely nothing today while installing the most safety-branded incumbent as the architect of the rulebook tomorrow. That’s not a guardrail. That’s a land grab with a conscience.
Similarly, OpenAI isn’t advocating for regulation for regulation’s sake, or even because it’s worried about RSI. They’re shaping the rules on what global AI regulation will be, and have teamed up with a completely captured administration (and Congress), in order to get there. Trump is, at his core, transactional and easily distracted. He is influenced by the very entities who need to be regulated the most (or by those who fund them). I mean look at this room…
Molly White put this amazing chart together as part of her new Tech Influence Watch project. Take a look at the entities in black and light green. These are the folks actually calling the shots:
OpenAI and others are also betting that despite the rhetoric around sovereignty, the EU isn’t really being serious, and I’m inclined to agree.
There’s decent funding being allocated for operational expenditures, and Europe is experiencing a boom in growth, but there’s also a heavy reliance on private sector funds, with supplements from other sources. But I see two possible problems that should be prioritized for maximum effectiveness:
Data center development is frequently bottlenecked by power and grid availability. According to a recent report by the European Data Centre Association, power is “now the single most important inhibitor to growth, especially in major metropolitan hubs. Grid congestion, long lead times for new connections, complex permitting procedures, geopolitical uncertainties, and the escalating requirements of AI workloads all require coordinated action between policymakers, grid operators, regional authorities and the data centre industry.” The Chips 2.0 Act accounts for some of this, but I’m not certain if it goes far enough.
Even though data center development is dominated by dedicated hyperscale and colocation providers like Digital Realty, Equinix, and Vantage Data Centers,1 the EU needs to consider the anchor tenants, not just the buildings and energy. If the EU triples data-center capacity but the main clients leasing that capacity are Microsoft, Google, AWS, and Meta, then the sovereignty package subsidizes the incumbents’ physical footprint on EU soil. Will EU preferencing extend to the customers located in the data centers? If not, this doesn’t solve the dependency and extensity problems.
But separate from the capacity & compute questions, the EU AI Act and other laws make foundation model development and innovation tricky, expensive, and compliance-heavy. Finally, while open source and EU-first models will be preferenced, there’s nothing explicitly forcing the EU public sector to divest from US AI models & big tech.2
Nothing proposed last week meaningfully addresses the problem: that it’s not just the technology, but the companies themselves that need to be regulated. These are procedural not structural changes.
The EO and state & federal legislation are naked demonstrations of ungovernability, loosely draped in procedural regulation. A federal government that can barely extract voluntary 30-day access to the most powerful models (and who failed at a 90-day window a week before) with no mechanism to block or constrain release, shows exactly how little leverage it has. Federal legislation that leads with preemption and parrots OpenAI’s own talking points isn’t serious. State laws that have no meaningful enforcement or consequences are at best, procedural theatre.
The incumbents love the procedural stuff because it’s easy to game with a well-staffed compliance team, but what’s needed is actual, structural changes. Think forced interoperability, divestiture, private rights of action, hard development limits, and proactive, well-funded enforceability. These are the mechanisms that change the incumbent’s position, which is why they fight the hardest against it.
Even if they fund the development, the EU is merely shifting who it’s dependent on, not governing the unaccountable. The EU’s approach here is slightly different. By taking an innovation approach, e.g., tripling data centers, EuroCloud, open source, chip fabs, common procurement, etc., the EU is signaling that its response to ungovernable US tech companies isn’t to limit the incumbents’ power, but to create their own homegrown entrants and hope they catch up before things get much worse. And this is assuming that the US tech firms don’t take advantage of all those new shiny data centers.
My whole argument about tech extensity is that we’re becoming ever more dependent across different markets on a handful of tech companies who are selling tools that define more and more of our lives. I don’t think the solution to this problem is to add a few European replacements or build more data centers who lease to the same anchor tenants.
We also need to meaningfully constrain the incumbents.The Anthropic pause still preserves their power. Finally, even Anthropic’s suggestion of an AI pause favors incumbency by freezing the frontier roughly in place. As I noted above, Anthropic isn’t just asking for a pause—it wants to be the one defining the players, setting the rules, and shaping the rulebook. And in doing so, will anything meaningful come from this, or is it just going to be a ‘plan to plan’?
IMHO, if Anthropic is serious about this, they’d pull the CEOs, along with Ursula von der Leyen, Donald Trump, Xi Jinping, Emmanuel Macron, Vladimir Putin, and Bibi Netanyahu, and anyone else with actual power into a room, sit them down, show them the data, and get negotiations started. They wouldn’t be faffing about with blog posts and promising to organize. That’s my job.
We’re not at tech extensity yet, and I acknowledge I could be missing something. But to me, these specific moves, this specific week, are incumbent-shaped, procedural posturing. Not one of them is structured to bind the companies we’re growing dependent on.
The US embraces extensity openly through toothless, industry-favorable voluntarism. The EU fights US extensity by trying to manufacture its own. Anthropic proposes a freeze that happens to lock in its own lead on the eve of an IPO. OpenAI proposes a ceiling that binds states and exempts itself whilst planning to go public. These proposals do nothing to fix the problem.
The convergence isn’t a sinister plot, but I don’t think it’s entirely a coincidence either. But the one thing that would actually change the dynamic—applied friction on the firms that already define the frontier—is the one thing nobody’s reaching for.
This isn’t a foregone conclusion. I’m still optimistic that things can be done, and there are some hopeful signs, particularly in the EU and at the local level in the US. Regulators are indeed regulating against Google and Meta (multiple times over), particularly through the DMA and Competition Authorities. Citizen and government action is being mobilized against US tech dominance and data center developments, with New York even floating a one-year moratorium on new data center build-outs. This is the kind of structural reform I think can meaningfully address some of the problems. I’ll be pointing out other solutions which do appear to be working in a subsequent blog post.
Which, as Claude pointed out, are also US (or at least non-EU) entities. But we’ll ignore that for now.
First, the track record. Mandatory open-source procurement policies have the highest failure rate of any category of government OSS initiative. “Required to give preference” is softer than “required to use,” and preference requirements are exactly the kind of provision that gets diluted in trilogue, waived by exception, or quietly ignored at the procurement desk where “nobody gets fired for buying Microsoft.” A mandate is only as strong as its narrowest enforced clause.
Second, even a clean release mandate solves the wrong half of the problem. As one assessment of the broader “Public Money, Public Code” push puts it, mandating open-source release addresses ownership but not capacity. Code you legally own but can’t maintain, evaluate, or extend still routes you back to a vendor for support and updates. Breaking lock-in requires internal technical capacity. Without it, “open source first” becomes “open source on paper, proprietary support contract in practice.”