Distributed Alignment and the Rise of the Age of Intelligence

TLDR; With recent questions around the alignment of recent models like GPT-4.1, we introduce the concept of distributed alignment, where organizations beyond frontier labs assume responsibility for alignment of AI systems through robust monitoring and control systems.  

In the summer of 1859, Edwin Drake had a dream. He saw an opportunity that he thought others had missed: using techniques from salt well drilling to extract a different substance from the earth - oil. He had invested thousands of dollars into a machine to drill and was now testing it out near Titusville, Pennsylvania. It was not going well. 

For months, he has driven iron pipes through layers of stubborn gravel, fighting collapsed walls and bedrock, progressing at an excruciating three feet per day. Local farm hands had taken to calling his operation "Drake's Folly" and his financial backers had already abandoned him, leaving him to resort to taking out loans to stay afloat. Drake was on the brink of failure. 

But on arrival to work the morning of August 28, one of Drake's drillers peered down and witnessed something miraculous: crude oil, rising up the shaft. As Drake rushed to the site, they began pumping the mysterious black liquid into an old bathtub – a humble vessel for what would become the lifeblood of the industrial age. 

Besides being emblematic of many future startups’ struggles against impossible odds in the face of ridicule and doubt, this moment would unleash forces that, for better or worse, would reshape power dynamics in society and ultimately fuel the next century of technological progress. Before Drake's innovation, petroleum was a niche product, and whale oil lit the lamps of America. Within decades, petroleum would power the second industrial revolution, enabling unprecedented mobility, manufacturing, and economic growth. A person born at the time of Drake’s discovery would have started their life in a world of horse-drawn carriages and candlelit rooms and ended it in an era of space travel and nuclear power. 

Today we stand at a similar inflection point. The culmination of decades of innovation in model architectures, compute infrastructures, frontier data processes, and human feedback combined with the accessible, easy-to-use form factor of a humble chatbot has opened the door to a renaissance in Artificial Intelligence. Just as Drake's drill struck oil, we have struck a new kind of intelligence well.

2022 was the clear demarcating line for this wellspring with the release, and subsequent rapid adoption, of ChatGPT. In the time since, thousands of new startups have popped up to capture the rivulets of this intelligence well. It is difficult to throw a rock in San Francisco today without hitting an AI founder.

The previous technological paradigm,the Big Data Era, saw the rise of a common cliche - data is the new oil.  We are witnessing a new regime change, a renaissance in real-time, which is propelling the creation of a much more potent commodity - intelligence. We are entering the Intelligence Age where intelligence is the new oil.

The Bottleneck

Yet, despite all this progress, the economy remains ill-equipped for the mass adjustment to AI technology. According to Gartner, 85% of AI projects ultimately fail to deliver on their promises, with McKinsey reporting that only 10% of organizations achieve significant financial benefits from AI investments. In a world of abundant intelligence why are there these gaps? 

When John D. Rockefeller founded Standard Oil in 1870 (over a decade after Drake’s discovery), the automobile was still decades away from mass production. Most of oil's eventual applications were undiscovered, existing only as unrealized potential. Yet Rockefeller still understood something fundamental: the future belonged to those who could not just extract oil but refine and distribute it effectively. Standard Oil's dominance came not just from controlling oil wells, but from mastering both the pipeline of distribution to end users and the complex process of turning crude oil into useful products.

We are missing similar ingredients now. While hundreds of LLM wrapper startups have emerged as preliminary distribution vessels for this raw intelligence, we are still only beginning to develop a more robust infrastructure that also incorporates effective guardrails.  We have the raw intelligence, but we are still only developing the infrastructure and distribution for refining this intelligence into economic and social utility with minimal harm. The missing key is a concept that is not new in AI parlance but is often misunderstood and underestimated. To make this new form of intelligence truly useful and effective, we must solve the fundamental alignment problem. 

Infinite, Infinitely Intelligent Interns 

In its simplest terms, alignment ensures AI systems reliably pursue and achieve intended objectives while respecting human values and safety constraints. While often portrayed as an abstract concept relevant only to future scenarios of potential conflict between humanity and Artificial General Intelligence(AGI), alignment is fundamentally about something more immediate and practical. It is about ensuring AI systems, at any level of capability, are trustworthy, reliable, and genuinely useful for their intended purposes. Just as a well can produce unrefined crude oil that's too contaminated for practical use, an unaligned AI system – regardless of its raw intelligence – can be unreliable, unpredictable, or even counterproductive.

To understand alignment's practical importance, consider AI's promise: providing every organization, business, and individual access to what amounts to a (functionally) infinite number of (functionally) infinitely intelligent interns. Even considering just one such intern reveals critical challenges. Would any organization hire a highly intelligence intern who also:

• Gets your coffee correct 99% of the time but occasionally serves you battery acid by mistake
• Confidently presents incorrect information 15% of the time, not from malice but from sincere misunderstanding
• Asked to schedule a meeting with a busy executive, orchestrate a company-wide fire drill to ensure the executive is free at the desired time
• May secretly be plotting your demise

No organization would hire such a person despite any level of intelligence. Raw intelligence alone is insufficient—it must be refined through alignment to become trustworthy and useful.

In more formal terms, alignment is typically considered across a spectrum of dimensions:

1. Technical Alignment: Ensuring accuracy and reliability while minimizing hallucinations and errors. 
2. Value Alignment: Guaranteeing that AI systems understand and pursue intended goals while respecting human values and preferences. 
3. Robustness: Maintaining consistent and reliable behavior across diverse contexts and previously unseen scenarios.
4. Safety: Preventing harmful outcomes through comprehensive safeguards against misuse, manipulation, and unintended consequences.

A couple of years ago, I wrote about the POC to production gap in AI. I highlighted many of the same concerns that (now) legacy AI systems had in bridging the gap to meet production-level standards. The evolution of AI capabilities in the subsequent years has only broadened those concerns. Two years ago, technical alignment and robustness dominated the conversation. Today, as AI systems approach or exceed human performance in many domains, value alignment and safety have become equally critical. We're discovering that no matter their performance capabilities, humans have specific preferences for AI behavior and (surprise, surprise) would also prefer not to be harmed in the process of using AI. And as AI systems get better and better, alignment could be the most important problem we must solve.

The Imperatives of Alignment

Alignment is often seen in a context where it is motivated by resolving potential existential risk from AGI. Some of the field's leading researchers have published sobering estimates of what they call “P(Doom)” - the probability that advanced AI systems could lead to human extinction. Anthropic's Paul Christiano suggests this probability could be as high as 35%, while Deepmind's Stuart Armstrong places it at 40%. 

Moving down the probability of catastrophic doom for humanity is perhaps the highest moral imperative you can imagine. However, this framing may also do us a disservice. Focusing exclusively on the long-tailed risk misalignment poses to humanity risks overlooking the immediate, concrete challenges that demand our attention today. It may also, paradoxically, hinder our progress towards lowering P(Doom). 

An oil refiner in Drake’s time couldn't have conceptualized the complex dynamics of climate change or foreseen its global implications. Yet they certainly understood the immediate dangers of toxic fumes, industrial accidents, and local environmental damage. Had the early petroleum industry developed more rigorous safety protocols and pollution controls to address these immediate concerns, these practices might have naturally evolved into stronger environmental safeguards. It is not inconceivable that the compounding effects of early safety cultures and responsible industrial practices could have naturally altered the trajectory of climate change. 

Similarly, by solving today's tangible alignment challenges, we build the foundation for addressing more profound future risks. We should thus consider the immediate social imperatives for alignment. As AI systems become increasingly embedded in the fabric of society – making decisions in healthcare, criminal justice, education, and social services – misaligned systems could amplify existing issues and create new forms of social harm. These include areas such as:

• Information Ecosystem Pollution: As models become more sophisticated, misaligned systems could flood our information and media ecosystem with deepfakes and plausible-sounding but subtly misleading content, making it increasingly difficult for humans to distinguish truth from machine-generated fiction.
• Erosion of Social Trust: With the above pollution, it might be increasingly difficult for humans to trust any information or interaction
• Homogenization: Unaligned AI systems, in their drive for efficiency and optimization, might inadvertently homogenize human expressions, language usage, and artistic creation. Soon everything will sound like it’s from a machine. 

These and other social considerations are hugely important risks and challenges to get right in our transition to an AI-powered society. However, while we often think of the potential downsides of AI, we should equally consider the upsides of getting AI right. There is also a strong economic imperative for alignment which is often overlooked. 

Quantifying alignment's economic impact presents a challenge similar to measuring energy's contribution to GDP – it's so fundamental that it becomes difficult to isolate. Just as the difference between refined and unrefined oil marked the difference between transformative energy and mere combustible liquid, alignment determines whether AI becomes a transformative technology or remains a fascinating but limited tool. 

Alignment dictates how we use AI. The application of a technology inherently governs its eventual economic impact. For instance, instead of incorporating oil into economic processes, we could have just extracted it and used it to create a massive bonfire. How much GDP would that have produced? 

In his essay Machines of Loving Grace, Dario Amodei argues for many of the long-term benefits of AI. He foresees a world where AI has the potential to be the dominant force in economic life and, with that, lift billions in the world out of poverty. Even Bernie Sanders, far from what you would call a “tech bro, " recognizes AI's potential in improving economic reality. In an interview with the New York Times, he states:  “In terms of artificial intelligence, in terms of robotics, etc, we truthfully have the capability now of creating a society where all of our people have a decent standard of living.”

We don’t have to look just at the pie-in-the-sky to understand the economic impact of alignment. IBM's survey of business leaders reveals that 78% cite lack of trust in AI systems as a major barrier to adoption. Deloitte's State of AI report, 63% harbour concerns about AI reliability. McKinsey's analysis suggests successful AI alignment could unlock $13 trillion in annual economic value by 2030 through improved decision-making, increased productivity, and new product innovations. 

With all this at stake, how do we get alignment right for this new Intelligence Age?

The Next Era of Alignment

The imperatives of alignment are not lost on the current leaders of the AI wave. Sam Altman, CEO of OpenAI, has said: “Even if we stopped all AI progress today, the existing technology has sufficient capabilities to fundamentally transform our economy and society – we just haven't figured out how to properly implement and control it yet.” Demis Hassabis of Google DeepMind has noted that “the gap between AI's theoretical capabilities and its practical implementation remains substantial…In a world of increasingly abundant intelligence, what remains scarce is our ability to reliably harness and direct it." This is not just lip service. AI Labs are investing significant portions of both their budget and talent pool into alignment research.

This, however, also presents a challenge. The current paradigm of AI alignment research concentrates the responsibility for alignment in the hands of a few labs, creating a precarious situation akin to having oil refinement controlled by a single entity. While companies like OpenAI, Anthropic, and DeepMind have made commendable commitments to safety, they face inherent tensions in their position.

The AI Laboratory Dilemma

AI labs face an impossible balancing act. They must:

• Race to maintain competitive advantage in an increasingly crowded field
• Satisfy investor demands for rapid progress and market share
• Ensure rigorous safety standards and alignment protocols
• Navigate complex geopolitical pressures and regulatory environments

Game theory tells us this situation is unstable. In a competitive landscape where being first can mean market dominance, the pressure to prioritize capabilities over safety becomes nearly irresistible. Even if 99% of labs maintain strict alignment protocols, it takes only one actor choosing to prioritize speed over safety to force others to follow suit or become irrelevant. 

The labs themselves cannot possibly anticipate all use cases and deployment contexts for their models. An AI system that appears well-aligned in laboratory conditions may exhibit unexpected behaviors when deployed in specific industry contexts or novel situations. We must consider alternative solutions and demand a more distributed approach to alignment. 

The Case for Distributed Alignment

Just as the oil industry evolved from vertically integrated monopolies to a diverse ecosystem of specialized players, AI alignment must become a distributed responsibility. This requires developing robust tools and frameworks that enable organizations deploying AI to implement and maintain alignment independently of the original model providers, ultimately placing stewardship with the users, not the creators of the models. 

In the case of distributed alignment, where you may not have access to the underlying model and its internal parameters, you can still follow fundamental principles to mitigate potential alignment issues. 

• We should know what AI knows: Distributors of AI systems should thoroughly understand and audit additional data, prompting, and context they feed into the system. This corresponds to Input Monitoring and Control. We can call this Data Alignment.
• We should know what AI does: Every decision and action made by an AI system should be monitored. This corresponds to Output Monitoring and Control. We can call this Eval Alignment.
• Every AI system should have a human boss: Every AI system should have a human control system where human judgement can correct, override, and terminate AI action. This corresponds to Human Monitoring and Control. We can call this Agentic Alignment.

These principles should be followed throughout the entire AI distribution chain.

Consider this in a practical scenario: a healthcare organization deploying a third-party AI diagnostic tool. Without access to the underlying model, the organization could implement input controls by curating patient data to ensure relevance and quality, output controls by comparing AI-generated diagnoses against established benchmarks and ground truth, and human oversight by requiring a doctor to validate and sense check AI recommendations and sending back errors for correction. This would allow the healthcare organization to have some form of alignment over the system even without accessing the behind-the-scenes model.

In an ideal world we would have AI models that were 100% accurate, 100% honest, 100% explainable, and 100% controllable. Unfortunately, we do not have such guarantees. Without control of the underlying model, we must think beyond models and instead to AI systems. The development of a framework of distributed alignment can still contribute to the robustness, interpretability, and control of these AI systems that a model is powering. We must develop tools to improve our observability and to continually course-correct AI with strong protocols and human oversight. In future articles we will cover these controls in more detail and how they can be operationalized with concrete strategies, actions, and tools. 

Regardless of implementation details, the thrust of distributed alignment is that the responsibility for aligned AI systems should not just sit with the AI labs, it should also sit with AI application companies, end users, government agencies, and intermediate infrastructure providers. Only with a distributed approach will we have a chance of practically aligning AI.

Making Distributed Alignment Work

We would be remiss if we did not also mention the role of regulation in this framework. While some sort of AI regulation is almost inevitable(and certainly needed), we cannot expect that regulation will be a panacea to this issue. AI technology is moving faster than any other technology in history and regulation moves at the speed of, well, government. The complexity of the technology and the inherently fragmented nature of regulation will also make it extremely difficult to enforce consistently and uniformly. We must consider these frameworks with future regulations in mind, but we cannot wait for them in place before addressing the problem.

The ultimate success of distributed alignment depends on creating an ecosystem where:

1. Organizations have access to user-friendly alignment tools that automate tasks while keeping humans in the loop for auditing.
2. Best practices and metrics for alignment are standardized and understood across industries
3. Regular auditing and certification processes exist
4. Regulatory frameworks support and enforce alignment standards

Just as the oil industry developed standard safety protocols and quality metrics, the AI industry needs to develop standard alignment protocols that can be implemented and verified across different contexts and applications. 

The goal isn't to remove responsibility from AI labs but to create a robust system of checks and balances where alignment is maintained through multiple layers of oversight and control. This distributed approach to alignment will ensure that AI systems remain safe and beneficial as they become more deeply integrated into every aspect of society.

The Opportunity Ahead 

The cat is now out of the bag with AI development and we can’t put it back. It is now incumbent on us to also recognize the urgency of the moment. This recognition demands that we understand alignment not as a distant concern for some Terminator-like future, but as an immediate and pressing challenge that will define our era.

The current tenor of the conversation is around compute infrastructure and successively larger models from foundation model providers. But we may be missing the forest for the trees. We are on the verge of unleashing an infinite army of infinitely intelligent interns into a world without the institutional infrastructural or guardrails to absorb them safely or effectively. Unlike human employees who are governed by constructs such as management principles, legal frameworks, and self-inforced social norms developed over thousands of years, we have to figure this all out in the space of a few years. In this effort, we have the opportunity to really interrogate ourselves and our society deeply in the process. 

A common dynamic we have observed in various facets of society is what I call “Money-ball-ification”. This refers to the Michael Lewis book where he described how baseball was transformed by introducing quantitative methods and statistical analysis - to great success. We have seen this application of statistics and math to increasingly more areas such as finance, elections, and even content creation. In a world of AI systems governed by computer scientists and engineers, we will now have to do this on a global social scale.  In an effort to align AI systems, we must grapple with deep questions of ethics, values, and human nature that philosophers have pondered for millennia. Deep moral questions will come into the purview of technology, and we have the chance to not only positively affect the direction of technology but also to improve our understanding of ourselves. 

For AI, the stakes could not be higher. Like in the industrial age accelerated by Drake and others, in the Intelligence Age to come, we must navigate downside risks, social harms, and perhaps dizzying upside potential. The world will become unrecognizable from what it looks like today. It is on us to ensure these systems are aligned towards the greatest good. We must take the responsibility of alignment as a collective action. 

Our task is to ensure that when future generations look back on this moment, they see not just the rise of the Intelligence Age but the moment when humanity chose to take conscious control of its technological destiny. The time for that choice is now.