Service Oriented What Again?
Technical organizations are bad now.
This is your host Can. Today, we talk software.
People commonly ask me what I did at Uber. I wrote code, is one approximation. In many people’s mind, many “companies” exist as some app on their phones. Most people realize there’s a “backend” or “datacenter” or “servers” somewhere, but the lines are blurry. The most high resolution view of Uber, or any tech company, is the demarcation between the “app” and the “server”. Engineers are engineers, as I've written before:
[Uber] felt like a tech company in that people used an app to get where there were going, but a lot of the work initially was really about keeping the lights on, while we either wired together off-the-shelf tech, or should have.
That’s a good starting point, but it belies some of the real challenges of running a sizable technology stack like Uber. The fluid operation hides much of the complexity, but there are many parts in play. There are obvious ones, like “matching riders with drivers” and “maps” but that’s barely scratching the surface. Think of the payments, automated support, authentication, communications, promotions, and such.
There are many different ways to draw an org chart here. You can have functional teams, or cross functional teams, or a matrix organizations. More thoughtful companies realize companies in different stages in their evolutions need different organizations, less thoughtful ones generally go with what is on the cover of HBR or top of High Scalability that week.
How to Hire Thousands of Engineers
Regardless of your overall organizational design, on the technical side, you need these different “features” be developed at the same time, by different teams, ideally without those teams stepping on each other teams’ toes too much.
Like many other technical organizations of its size, Uber settled on a service oriented architecture, or SOA in short. This largely means that different services are managed by independent teams, and they operate on a protocol they agreed on.
For example, a single “service” can be responsible for calculating “ETA between point A and point B” while another could be responsible for “calculating the estimated fare between point A and point B”. I am stylizing things here a bit, but you see the point. One of them is really a routing problem, whereas the other has to take into account many things like promotions you are part of, the distance between A and B” etc, etc. Then, there’s obviously a different service that talks to the “app” that collects all that information and sends it over the internet.
There are hundreds of nuances here, of course. Where the boundaries of these services are a common source of tension in many tech orgs, as well as how these things are independently monitored (i.e. ignored), deprecated (i.e. really ignored) and decommissioned (i.e. ignored forever). The benefits, generally, overweigh the costs though.
That is a lot of services.
There are significant technical benefits to a service oriented architecture. One big win is that since services are relatively isolated from each other, you can have more graceful failures rather than “Uber is down” type of catastrophes. Some of that is more aspirational than realistic, but it still a good rule of thumb. If you have a service that just manages “images of trips”, and it’s down, that’s probably fine for a bit. Your receipts might look funky for a bit, but Uber overall would work.
Services All the Way Down
A less obvious benefit is the ability to combine and compose these services make building new features much easier. Instead of simultaneously wading through thousands of lines of code and tip-toeing around other people's work, you can just see if there are "interfaces" that you can put together like bunch of LEGO pieces. Better yet, you can be inspired to build entirely new products by simply being inspired by all the features so clearly laid out in front of you.
The productivity benefits of these abstractions are hard to overstate. I am obviously biased here. But besides the "forward-leaning" work culture, Uber's high level and number of abstractions was one of its competitive advantage for a long time,. You could imagine a feature, and have it working in production (as in customers' hands) in less than a couple weeks, even including the App Store review process.
But of course, the flip side is also true. Sometimes things break in surprising or even amusing ways, and things can be hard to debug. There’s probably not a single person who can describe you in detail how Uber with all its services, it’s a living organism of its own. You can probably put enough people in a room, and they can draw you a systems diagram but probably, by the time they are finished, the technology would have changed significantly enough to make your diagram obsolete.
My co-host Ranjan likes to point at growing inaccuracies of Uber's ETA as a sign of impending doom. Surely, there's a point here about the ETA accuracy is an underrated proxy metric for people’s satisfaction with Uber in general, but fact of the matter is it's just a single service, owned by a single team.
Of course it didn't start out that way. Uber, like many other orgs, started out (way before my time) as a single, monolithic service, hilariously called api. For my first few months there, I worked on rationalizing that madness in small part. My favorite part of this process was seeing how this distributed architecture made it clear how we were "shipping our org chart", as they say. Most teams happily played along, but there were of course holdouts who believed their privileged status deserved some preferential treatment. Engineering, in the end, is as political as any other function.
SOA is not DOA
SOA is not a panacea, but it can return major business benefits. Jeff Bezos now is a household name thanks to his wealth —and a few other things—, but he was first and foremost an engineer who saw the potential upside of systematically tearing apart a technical organization and reshaping it around functions.
Here is a semi-famous rant by Steve Yegge, a former Amazon engineer who then went to Google from 2011:
Over the next couple of years, Amazon transformed internally into a service-oriented architecture. They learned a tremendous amount while effecting this transformation. There was lots of existing documentation and lore about SOAs, but at Amazon's vast scale it was about as useful as telling Indiana Jones to look both ways before crossing the street. Amazon's dev staff made a lot of discoveries along the way. A teeny tiny sampling of these discoveries included:
It's eerie to think Yegge wrote this in 2011, which is then a few years after Bezos sent his warring orders. Many consider AWS as something that Amazon has been working on for many years, but the reality is probably closer to (but not exactly!) Amazon slowly externalizing services that they have built for their internal services.
It’s easy to sound facetious now, as I’ve spent several years working on such services. Of course, "simply externalizing" a service is a major effort requires coordination of many different teams in a firm. And many of the benefits of SOA really become apparent at a certain size. Besides the obvious organizational challenges, the technical overheads in even the best implementations are real.
Yet as the software itself becomes more and more abstracted out, and more and more companies become “tech” companies in some vague sense, how your technical organization is built might end up returning huge yields. And as we software eats more of the world, and starts guiding, defining, ruling our lives, it could be useful to anyone working in technology, even in a business function, to understand how software really works.
What I’m Reading
Why High-Tech Commodization is Accelerating: HBS Professor Willy Shih writes about how abstraction of technology presents challenges to building competitive advantages in high-tech industries.
Sophisticated design and simulation tools are de rigueur for modern product design. Tool suites that allow companies to analyze structures, noise and vibration, acoustics, thermal behavior, fluid flow, motion, and dynamics have democratized design. They have lowered the entry barriers in engineering-intensive sectors, automated the process of cumulative innovation, and allowed new market entrants to stand on top of a pyramid of earlier innovations. In short, they have unleashed a powerful force that’s driving commoditization in globalized markets.
The Twilight of Combustion Comes for Germany's Empire of Engines: Electrifying cars changes entire supply chains, which employ thousands of people across many industries. What does that mean if your industry might is building that complex supply chain, that’s not just a simple engine?
BMW hasn’t been willing to forecast how many jobs might be lost, but the company’s human resources department acknowledges that making an electric motor takes roughly 30 percent less time than a gasoline-powered engine. “The numbers of hours worked to make an electric motor are smaller than for a combustion engine,” said Carreiro-Andree, the BMW board member. “That’s a fact.”