May 26, 2021

What the End of “Building and Shipping” Means for Developer Tools

Jean Yang
Taylor Swift looking sad
Share This Article

It’s easier to build web apps than ever before. So why do developers feel so overwhelmed?

It’s well-accepted that software development as we know it has forever changed. Software development has gone from intricately crafting bespoke components, squeezing every ounce of performance possible, to combining off-the-rack functionality under an abundance of compute. T-shirts worn at takeout counters across Silicon Valley testify to the SaaS revolution.

SaaS + APIs make programming hard
SaaS and APIs have made software development more complex.

But here’s what people aren’t talking about: what this means for software engineering tools. As software engineering has shifted from building standalone systems to coordinating interconnected components, software development has been quietly shifting to a development/ops hybrid, supported primarily by ops methodologies like “testing” in production. With development-time tooling becoming more and more obsolete, developers are left to fill the gaps with their blood, sweat, and tears.

In this post, I’ll talk about why software engineering as we know it is dead and how we need to fill the gaps in developer experience. This post is an elaboration of a Taylor Swift themed talk I recently gave at DevX Conf.

And if you’re interested in helping us build a better developer experience for our new kind of API-centric observability tool, join our private beta.

It’s the End of the SDLC as We Know It

People have talked a lot about the shift from “waterfall” to “agile” software development, but they’ve talked less about what is arguably a much bigger change: the death of the software development life cycle as we know it.

The initial destruction of the SDLC happened with the move to deploying one’s software as always-available APIs. Providing software as a service transformed “building and shipping” to “continuously shipping,” with continuous delivery becoming the operative mode. Because shipping now means constant engagement with users, rather than lobbing code into the ether, monitoring production logs and traffic and logs became a bigger part of the process. This shift blended development, testing, and production into a continuous loop.

The illustrated death of the SDLC
The illustrated death of the SDLC.

The nail in the coffin of the SDLC was in the shift to consuming SaaS and APIs at scale. When software teams moved to adopting other people’s APIs in large quantities, they became vulnerable to other teams and companies changing things out from under them. In this interconnected world, it’s harder and harder to construct high-fidelity test environments, making “testing” in production look more and more attractive.

With software teams consuming and producing APIs left and right, the burden on developers is growing. Looking at source diffs doesn’t tell you if your change might cause a cascading failure because of services that depend on your API. Looking at single-service tests doesn’t tell you if you’re introducing an inadvertent performance bottleneck.  While it’s easier than ever for developers to add new functionality, they now need to wade through logs, metrics, and traces to understand the behavior of their own code, both intended and actual.

Next, I’ll talk about what it could look like to load off developers in this API world.

We Need Tools for Coding Against Production

Especially in the last decade, developers have moved from “programming from specification” to “programming by observation.” For instance, instead of writing data access policies based on what they’re supposed to be, developers are moving towards observing a system’s production behavior to identify what the effective access policies are. But the tools are still set up assuming developers know how a system is supposed to behave. And whenever tools assume, developers end up filling the gap with manual work.

In order to support this new mode of programming interconnected ecosystems, we need to look beyond tools for testing in test or “testing” in production. What we need is tools that support coding against production: developing software in the context of how the software will run. Here’s what that means:

  • Before production, during development and testing/analysis, developers should get as much context about how their code will affect production behavior as possible.
  • During production, tools for understanding software behavior should connect back to source code changes as much as possible.

When development is so intricately tied to production, we want to unify the experience of analyzing code or running unit tests with the experience of sifting through metrics, logs, and traces. What would it look like if analyzing a source code change showed how production behavior would look different? Or if querying logs could lead you to a pull request of interest?

Taylor Swift coding against production
What it could look like to code against production.

There are two ways to properly support coding against production: the familiar way and the right way. 😉 When I mention “coding against production” to people, the first thing their mind goes to is a framework where you can deploy code directly to production, like a traditional REPL (real-evaluate-print loop), or like in Dark Lang. Building the “one true framework” is the more familiar way. But the “one true framework” approach requires homogeneity—and one of the major reasons we’re in this mess is because software is becoming more and more heterogeneous. In the next section, I’ll talk about embracing software heterogeneity in our tooling.

Coding Against Production Means Explaining Complexity

To explain the kind of tooling that we need to build, allow me to reference a distinction I recently made in a Twitter thread, between tools that hide complexity and tools that explain complexity. Spoiler alert: we’re going to need tools that explain complexity.

Tools that hide complexity, or abstraction tools, automate something complicated. A compiler is the classic simplifying tool. An API like Stripe or Twilio also falls into this category. For simplifying tools with UIs, the dream is a one-click “set it and forget it” interaction.

Tools that explain complexity, or exploration tools, support the user in understanding something complex. A debugger is the classic complexity-embracing tool. While monitoring tools are largely simplifying tools, observability tools like Lightstep or Honeycomb are complexity-embracing tools. Sure, it’s important for complexity-embracing tools to simplify as much as possible, but it’s also important to recognize 1) some tools can’t boil down to one click and 2) some tools are better not boiling down to full automation.

The gap in developer tooling is this: the proliferation of simplifying tools (APIs; SaaS) has made it possible for systems to become large enough to have complex emergent behaviors. To identify and debug inadvertent performance bottlenecks and accidental cascading failures, there’s no getting away from the need for complexity-embracing tools. Today, programming with services and APIs is like what programming in C was like before gdb.

Here’s what’s going to be hard about building—and adopting—the tools we need to code against production in the services world:

  • We need to resist the urge to look for simplifying tools. Of course, we’re going to need some simplifying elements for monitoring and alerting, but we have a major tools gap when it comes to helping developers tie their source code changes to behavioral changes and we’re not going to be able to automate this away. We need tools that embrace the complexity of modern systems.
  • We need to resist the urge to look for “one true framework.” It’s tempting to look for yet another language, programming framework, or API to solve this problem. But the universal truth that different problems need different tools is how we got into this mess. “One true framework” will introduce yet another framework we’ll have to handle with our one true complexity-embracing tool. 😉

Supporting developers in the API era will mean building tools that help explore the API ecosystem that is emerging—and all of the implications on software behavior, reliability, and performance.

What’s Next?

The software development life cycle as we know it is no longer, but this doesn’t mean we’re done innovating on software development tools. If anything, we need new tools that support what programmers are effectively doing today: coding against production. And while there’s always room for more automation, there’s a dire need for complexity-explaining tools that help developers understand the dynamics across interconnected webs of APIs.

If you want to hear more about Akita’s approach of API-centric observability through passively watching traffic, you may be interested in watching my full talk at DevX:

And just as the SDLC of ye olde wasn’t supported by one kind of developer tools, we’ll need all kinds of different approaches to make developers’ lives better in the API world. But if what Akita is doing sounds interesting to you, we welcome you to join our private beta. 😉

Share This Article

Join Our Private Beta now!

Thank you!

Your submission has been sent.
Oops! Something went wrong while submitting the form.