English

Based on "How Intercom 2X'd engineering velocity with Claude Code | Brian Scanlan" from How I AI Watch the original video

Unleashing the Code: How Intercom Doubled Engineering Velocity with AI and Reimagined the Software Factory

In the rapidly evolving landscape of artificial intelligence, many companies are grappling with how to integrate these powerful tools effectively. While some express skepticism about AI's true impact on productivity, Intercom, the customer messaging platform, stands as a compelling counter-narrative. Under the guidance of senior principal engineer Brian Scanlan, Intercom's R&D department has achieved a remarkable feat: doubling their engineering throughput in just nine months, largely thanks to a full-throttle adoption of AI, particularly Claude Code.

This isn't just about incremental gains; it's a fundamental shift in how software is built, challenging long-held beliefs about developer productivity, code quality, and the very nature of engineering work.

The Moment of Inflection: Imagination, Not Tools, Is the Barrier

For years, the promise of AI in software development felt just out of reach. Code completion tools offered minor conveniences, but nothing truly transformative. Brian Scanlan recounts a pivotal moment around November-December last year, coinciding with the release of more advanced models like Anthropic's Opus 46 and GPT-4U. "Suddenly you started realizing that you have to think bigger about things," Scanlan explains, "or that your imagination is now the barrier, not the tool."

This breakthrough wasn't just internal; it was a collective awakening. Scanlan describes the atmosphere during the Christmas break, with "everybody go wild on Twitter X," witnessing developers achieve unprecedented levels of output. The message was clear: "Everything's changed."

Intercom's journey was fueled by an internal AI-first mandate that permeated their product strategy. Having already transformed their customer-facing products with AI, the expectation for similar innovation within engineering was high. This created an environment where impatience for AI adoption was a driving force, pushing the team to go "all-in" rather than dabble.

Measuring the Unmeasurable: 2x Velocity and the Path to 10x

To quantify their progress, Intercom's CTO, Darra, set an ambitious goal: double the throughput of R&D. They chose a simple, albeit sometimes debated, metric: merged Pull Requests (PRs) per R&D head. This metric, encompassing product managers, designers, and TPMs alongside engineers, revealed a dramatic "number goes up" chart. Within nine months, the team saw a 2x increase in PR throughput.

This isn't merely about engineers typing faster; it's about unlocking "the physical limits of my ability to type code," as Claire Vote, the host of "How I AI," eloquently puts it. The shift is so profound that Scanlan now asks, "Why can't it be 10x?" This ambitious outlook stems from a belief that if an organization fully commits, prepares its team, and refines its codebase, it will see significant improvements in product quality and developer experience.

Addressing the common skepticism that such metrics might be "gamed" by shipping smaller, less impactful changes, Scanlan emphasizes Intercom's high-trust environment. The goal isn't just speed; it's about enabling engineers to do more meaningful work, faster and with greater enjoyment. Indeed, Scanlan shares, "I've been having the most amount of fun in my career over the last 3 months."

Beyond Speed: The Surprising Uplift in Code Quality

A frequent concern among engineers and leaders alike is whether increased velocity comes at the cost of quality. Will AI-generated code be "slop" or "garbage"? Intercom's experience suggests the opposite.

They've observed a consistent decrease in the time it takes from the first line of code written to a feature being announced on their news channel. More significantly, Intercom collaborated with a research group at Stanford, providing them with their internal data. The Stanford team's independent measures of code quality indicated that code quality was actually improving.

This counter-intuitive finding highlights a crucial benefit of AI-driven development: the capacity to tackle technical debt and improve the codebase. As Scanlan and Vote discuss, businesses often have limited capacity for "internal projects" like improving code quality, as these don't directly generate revenue. However, when the "cost of doing that compresses" due to AI's efficiency, organizations can afford to invest in developer experience, security, compliance, maintainability, flaky tests, and CI/CD improvements.

This leads to a powerful piece of advice for CTOs and VPs of engineering: "Everything you hate about the codebase, go spend a month fixing and see how fast we can speedrun that. That's going to feel really good." This strategic investment in core engineering health, made feasible by AI, ultimately unlocks even greater velocity and higher quality.

The Software Factory: Engineering the Golden Path

Intercom's success isn't just about adopting AI; it's about intentionally engineering their entire software delivery process around it, moving towards what Scanlan calls a "software factory." This involves treating the engineering organization itself like a product, complete with rigorous design, measurement, and continuous improvement.

One striking example is how Intercom addressed the issue of pull request (PR) descriptions. Initially, AI-generated PR descriptions were "terrible," merely regurgitating code changes rather than explaining intent – the truly valuable information for human reviewers. Intercom's solution involved:

Defining Quality: They established what a "good" PR description should look like.
LLM Judge: An AI model was trained to evaluate PR descriptions, revealing a negative trend in quality with early AI adoption.
Custom Skill: They developed a "create PR" skill for Claude Code that leverages session context to generate high-quality, intent-driven descriptions.
Enforcement: This skill was integrated as a mandatory hook. If an engineer or agent tries to open a PR without using the skill, it's blocked.

This approach mirrors the determinism of modern CI/CD pipelines, but applied upstream to the code-writing process itself. "We're on this movement towards a software factory," Scanlan explains, where predictability, reliability, and consistent quality are paramount. This ensures that while engineers move faster, they still adhere to the high standards of a mature, 15-year-old SaaS company.

The Invisible Infrastructure: Telemetry and Personalized Feedback

Underpinning Intercom's "software factory" is a sophisticated system for monitoring and improving AI usage. They don't "fly blind."

Skill Telemetry with Honeycomb: Every internal AI skill is instrumented with event-level telemetry, sending data to Honeycomb. This allows individual skill developers to see how often their skills are invoked, by whom, and when, fostering a data-driven approach to skill development.
Session Data Analysis with S3: All raw Claude Code session data (the chat logs) is anonymized, uploaded to S3, and then analyzed. This allows Intercom to identify broader organizational trends, common pitfalls, and areas where training or new skills are needed.
Personalized Feedback: Based on this session data, Intercom developed a simple internal tool that provides personalized feedback to individual engineers on their Claude Code usage. This helps new hires or those struggling to understand how to optimize their interactions, supporting a culture of continuous learning and self-improvement.

This robust telemetry system ensures that Intercom can identify bottlenecks, improve their internal tools, and provide targeted support, preventing the common problem of "throwing an API key and saying best of luck."

The Elephant in the Room: AI Costs

The exponential growth in AI usage inevitably raises questions about cost. Scanlan readily admits that their AI bill "looks exactly like this" (referencing the steep velocity growth chart). "It's like hiring whole new offices of people," he notes.

Intercom's current strategy is to prioritize speed over cost optimization. Their attitude is, "everyone just turn on Opus for everything... going as fast as possible and caring about the bill later." This reflects a strategic investment mindset, believing that the significant benefits gained from rapid innovation and increased throughput outweigh the immediate costs. However, Scanlan acknowledges this might not be feasible for every business and that future optimization will be necessary if costs continue at the current rate.

The Future: Agent-First and Higher-Level Concerns

Looking ahead, Intercom envisions a future where "all technical work will become agent first." This means AI agents will handle the foundational, repetitive tasks, freeing human engineers to operate at a higher level, focusing on complex problem-solving, strategic design, and truly innovative features.

The shift is profound: instead of engineers spending time on boilerplate code or debugging minor issues, agents will take on this "basic work," allowing humans to "move up to higher level to be able to like work on higher level concerns or just getting more stuff built more stuff out there or higher quality."

Intercom's journey with Claude Code is more than just a case study in productivity; it's a blueprint for reimagining the entire software development lifecycle. By embracing AI comprehensively, treating the organization as a product, meticulously measuring outcomes, and proactively engineering for quality, Intercom has not only doubled its engineering velocity but has also cultivated a more engaging, productive, and ultimately more fun environment for its engineers. For any organization looking to leverage AI in engineering, Intercom's experience offers invaluable lessons and a compelling vision for the future.

Based on "Inside the Five Days That Remade the Supreme Court" from New York Times Podcasts Watch the original video

The Secret Birth of the Supreme Court's Shadow Power

For the better part of a decade, the U.S. Supreme Court has operated with a parallel, often perplexing system for issuing major rulings – one that bypasses the traditional, deliberate processes that define American justice. This increasingly powerful mechanism, dubbed the "shadow docket," has been responsible for consequential decisions on everything from immigration policy to presidential authority, often with little to no public explanation. Now, a groundbreaking New York Times investigation has pierced this veil of secrecy, revealing the precise, dramatic five days in 2016 when this expedited system was born, effectively remaking the nation's highest court.

At the heart of this revelation are 16 pages of confidential correspondence among the justices themselves, offering an unprecedented look into their private deliberations. These documents, unearthed by Times journalists Jodie Caner and Adam Liptac, allow us to "eavesdrop on the justices at the exact moment that they are abandoning time-tested norms of judicial procedure and backing themselves into a new way of doing business."

Understanding the Shadow Docket

To truly grasp the significance of this shift, one must understand the stark contrast between the Supreme Court's traditional "merits docket" and its shadowy counterpart. The merits docket is the familiar image of the court: justices meticulously select cases, receive multiple rounds of detailed briefs, hear extensive oral arguments, engage in in-person deliberations, and then exchange numerous drafts of lengthy, reasoned opinions, concurrences, and dissents. This painstaking process, often stretching over a year, culminates in decisions that can be hundreds of pages long, providing binding law and clear guidance to lower courts and the nation.

The shadow docket, however, short-circuits all of this. It operates with extreme speed, relying on thin briefs, no oral arguments, and no in-person deliberations. Its rulings are typically brief, often consisting of scant or even no reasoning at all. Yet, over the last ten years, it has become a major part of the court's business, particularly during the Trump administration, where it was used to grant the president enormous leeway on issues like immigration, government spending, and agency power. Critics have consistently warned that this exponential increase in use, combined with the lack of transparent reasoning, raises critical questions: Is the court rushing to rule based on gut instinct, personal pique, or partisan impulse – precisely the elements a slow, deliberate, and judicious system is designed to avoid?

Defenders of the shadow docket argue that emergency orders have a long history, typically reserved for truly time-sensitive matters like death penalty cases or election disputes. They also contend that these orders are merely temporary, designed to maintain the status quo while cases proceed through lower courts. However, as Caner and Liptac point out, this "temporary" label often masks a very different reality. If the court allows the deportation of thousands of people, or the withholding of aid money, or the firing of employees, those actions, though nominally temporary, are often irreversible and conclusive in their practical effect. They are, in essence, final decisions made without the benefit of the court's usual rigorous process.

The Unprecedented Request: Obama's Clean Power Plan

The crucible for this seismic shift was a case in 2016 concerning President Barack Obama's Clean Power Plan. Facing legislative gridlock on climate change, Obama had instructed the Environmental Protection Agency (EPA) to issue regulations that would fundamentally move the American power system away from coal. This initiative, predictably, enraged industry groups and "red states," who challenged it in the DC Circuit Court. They asked the court not only to declare the plan unlawful but also to immediately halt its implementation. The DC Circuit agreed to fast-track arguments on the plan's legality but refused to pause its rollout.

In an "unprecedented request," the challengers then turned directly to the Supreme Court. They asked the justices to freeze the Clean Power Plan before any lower court had even ruled on its lawfulness – a move that everyone involved recognized as extraordinary. At this moment, the Supreme Court was evenly split, a 5-4 court leaning conservative, but with Justice Anthony Kennedy, a Republican appointee, serving as the crucial swing vote. Kennedy, famously "persuadable," had recently authored the majority opinion legalizing gay marriage, making the court's direction unpredictable.

Five Days That Shook the Court

The emergency request landed in the chambers of Chief Justice John Roberts, who oversees the DC Circuit. The ordinary expectation was a swift denial, or at most, a collective denial by his colleagues. What transpired instead was a five-day sprint of internal memos that would forever alter the court's operational landscape.

Day One: Roberts's Salvo Chief Justice Roberts initiated the debate with a three-page, single-spaced memo. He argued forcefully that Obama's plan must be halted due to the "enormous burdens" it would impose on states and the coal industry. He claimed there was "no time to waste" and questioned the EPA's authority under the Clean Air Act, invoking the "major questions doctrine" – a principle suggesting that unless Congress clearly grants an agency vast power, that power doesn't exist.

But beyond legal arguments, Roberts's memo betrayed a deep-seated grievance. He felt the EPA had "tricked" the court just months earlier in a mercury emissions case. In that instance, the court had ruled against the EPA after a three-year litigation, but by then, the regulation had effectively gone into effect, rendering the ruling somewhat moot. Roberts was "peeved," "irked," and determined "not to let it happen again." He viewed the traditional slow legal process as enabling the Obama administration to be "sneaky" in implementing its regulations.

Day One: Breyer's Counterpoint Justice Stephen Breyer, a Democratic appointee, immediately responded, laying out a contrasting chronology. He pointed out that the Clean Power Plan didn't require industry action for six years, with total compliance not due until 2030. There was, in his view, "plenty of time to do this in the usual course." Breyer found the court's intervention "quite unusual" and saw no urgent need to bypass the DC Circuit, advocating for the lower court to be allowed to finish its work.

Day Two: Roberts's Insistence Roberts's reply the next day revealed a growing irritation and an unyielding insistence on blocking the plan. He dismissed Breyer's procedural concerns: "I recognize that the posture of this stay request is not typical, but review is sought of what has been described as the most expensive regulation ever imposed on the power sector." The Chief Justice's impatience was palpable; he was "ready to rule now." He believed the court would ultimately strike down the Clean Power Plan, stating it was "highly unlikely to survive" a full review. He saw no reason to let the process play out when he already knew the answer.

This memo made it clear that Roberts was engaged in a power struggle with the Obama administration. He explicitly stated, "I am of the mind that a rule designed to transform a substantial swath of the nation's economy should be tested by this court before it is presented as a fait accompli." As Liptac observes, this was not the "even-handed, magisterial tone" typically seen in public; here, Roberts was "acting as a bulldozer."

Day Three: Kagan Sounds the Alarm Justice Elena Kagan, another Democratic appointee, filed an even more direct memo, opposing the Chief Justice's stance. She called the relief sought "unique" and used the word "unprecedented" to describe what Roberts wanted to do. Kagan stressed the complexity of the case, arguing that it involved "a complex statutory and regulatory regime" that demanded more time and consideration. "This is weird. Are we sure we want to do this?" she seemed to ask, highlighting the gravity of the situation.

Day Four: Alito's Existential Threat Justice Samuel Alito, a Republican appointee, then weighed in, echoing Roberts's sense of insult and grievance. He declared that a "failure to stay this rule threatens to render our ability to provide meaningful judicial review and by extension our institutional legitimacy a nullity." Alito framed the issue as an "almost existential threat" to the court itself, portraying the Obama administration as attempting to sideline the justices.

What's extraordinary about these private exchanges, the journalists emphasize, is how frankly the justices reveal their "actual agendas." Unlike their public decisions, which often present a "mask" of confining themselves to facts and legal materials, these memos expose the human element: grievances with an administration, fears about the court's institutional place, and concerns about legitimacy – factors not typically associated with purely legal reasoning.

Day Five: Kennedy's Decisive Whisper The entire debate, as predicted, came down to Justice Anthony Kennedy. On February 9th, the fifth day, Kennedy sent a terse, three-sentence note: he was voting with the Chief. The debate was over. Within hours, the Supreme Court issued its order, a single paragraph of "legal boilerplate" with no explanation, blocking President Obama's signature environmental initiative.

The Enduring Legacy of the Shadow Docket

The insights gleaned from these confidential memos profoundly vindicate the long-standing criticisms of the shadow docket. This was "not the court doing A+ work," Caner and Liptac conclude. Instead, it was a court "throwing ideas around, seeming to be motivated by grievances against a president...getting snippy with each other and just in general not doing the kind of work we associate with the nation's highest court." The justices were disregarding time-tested procedures without seemingly considering the long-term implications. As Kagan noted, it was "unprecedented," but no one seemed to ask: "where is it going to lead?"

Where it led was to an explosion of emergency applications under the Trump administration, with the court often quickly ruling in favor of presidential initiatives on major questions, a stark contrast to the Obama era. Political scientists have observed more partisan voting on the shadow docket than on the merits docket, suggesting that when acting fast, justices may rely more heavily on partisan impulses. For instance, in the Biden years, the court initially voted against Biden on three emergency applications, only to rule for him when those same cases returned to the merits docket for full consideration. This pattern suggests that deliberation truly "dampens partisan impulses."

Beyond partisan concerns, the rise of the shadow docket poses a fundamental risk to the court's legitimacy. Unlike elected officials, Supreme Court justices serve for life and derive their legitimacy not from votes, but from public trust. The act of writing a reasoned opinion is a judge's way of saying, "Here's why you should trust me; here's my work." When the court increasingly abandons this practice, issuing barely explained rulings on major national issues, it erodes that trust. As the Supreme Court's public approval ratings plumb historic lows, its increasing reliance on the shadow docket to make pivotal decisions only exacerbates this critical problem, threatening the very foundation of its authority.