Software Eats Itself
Musings on career changes abound.
Hey. Can here! After a long break abroad, I am back in the land of the free, home of the brave. But more importantly, The Margins is back on its usual schedule.
One question I get often is why I decided to get an MBA, and one abroad. I had a degree from a fancy CS school, and brand name companies on my resume. And besides, aren’t all MBAs basically the same; a bunch of young kids who spend a year or two partying around the world and then getting the same jobs; be it consulting or product managers?
There’s both a personal and a professional answer to that question. The personal side is simple. I wanted to be momentarily away from the Silicon Valley bubble I spent a good 8 years in. While I throughly enjoyed most of that almost-decade, it felt like I was developing a blind spot to the rest of the world. Most of my conversations revolved around tech and most of my friends were in tech. Spending a year abroad, spread between Singapore and France, surrounded by people not in tech, seemed like a good idea. And it was!
The professional side is a bit more controversial. But here’s my contrarian bet on technology: While there’s seemingly an unassailable dearth of technology workers, especially in the Bay Area, the tech industry will also be automating itself, faster than most people are ready to appreciate. The lower end will is already experiencing this pain, but the bar will slowly move up.
The Cloud is not just a Server
Let me give you an example. When I started my career in Bay Area at Digg, we owned hundreds of physical servers, colocated in a facility. There were people hired, especially to maintain those computers, making sure the blinkenlights kept on blinking in unison. While Digg was a had a sizable user base, especially for its time (and even by contemporary standards), it’s hard to imagine a company do the same today. You simply click a button on the AWS console, and more machines than you can ever need will be ready at your disposal.
And this is really the most obvious example, but the bar keeps going up and up. What used to take 10 engineers 10 days to program is now a simple library on GitHub that you can import into your application with a few hours of tinkering. The always-eloquent Benedict Evans likens former white collar workers to cells in Excel. But you can take that analogy even further. Pivot tables replaced the lowest levels of analysts, and Tableaue reports went up even further.
Looking at just the offering of AWS, it’s hard not to feel the same fate will hold for technologists too. Most of those products on that chart used to be someone’s full time job at a sizable tech company. Why do you need to have a full time database administrator when AWS has a turn-key solution for you? I am simply taking Amazon as an example here as they are The Cloud for most tech companies, but it’s hardly alone. It’s Seattle neighbor, Microsoft has basically the same products, but also a few more especially focused on machine learning infused services. Simply drive alongside 280 in Peninsula and you’ll be inundated by billboard after billboard of companies offering similar services.
Of course, you could argue that I am simply contradicting myself here; clearly these are tech products built by tech workers. Yes, you don’t need to build your payment infrastructure if you just use Stripe, but someone needs to build Stripe. And same for all those features on Amazon, and Microsoft, and so on. These companies need to hire people, but there are qualifiers
Doing more with Less
And that’s the other side of my argument. I believe as more and more of the bottom rung of the technology work is either automated or abstracted away, there’ll be more and more demand for the upper end. In other words, more specialization which will require deeper, more intensive training.
Given most tech jobs are in states with unenforceable non-competes and at-will employment laws, these job markets are relatively liquid. This would ideally mean the compensation be relatively close to what would be suggested by the equilibrium of supply and demand.
Yet, there are other factors like housing shortage in California and New York and a frothy venture capital boom artificially increasing salaries. Of all the companies I worked at over 8 years, almost none of them turned any profit, and even the one that eventually did go public is still not profitable. There’s a horizon issue here, as profitability is hardly the goal of any growth stage firm. Yet, during all this time, I was gainfully employed and made a decent living primarily thanks to the largesse of the various venture capitalists and a zero interest rate environment. Is this forever sustainable?
And there’s the curious market of coding bootcamps and the like. It’s overall a good thing for society if more people can get high paying jobs without having to spend 4 years in college. And it’s a good thing for tech companies to come to their senses on giving up their obsession with fancy degrees, when a significant chunk of the work at even the most “high tech” firm is tying together different services, which hardly requires a Carnegie Mellon degree.
On the other hand, what seemed like an amazing market, charging a fraction of what would cost people tens and sometimes hundreds of thousands in student debt, is now ridden with dead company after dead company. Most of the smaller players are already out, and the some of the big ones are having to offer more and more specialized degrees. There’s also the Lambda School model, with clever financial incentive alignment and a heavy focus on initial student quality, but its success (if you take its outspoken CEO at his word) at this point is more the exception than the norm. Just because there’s a big demand for more tech workers, it doesn’t mean the demand is limitless and or we can or should convert our entire workforce to a bunch of coders.
Cash does not Rule Everything Around Me
Compensation is hardly the reason, though, I wanted to switch my function. If anything, it’s more likely that salary prospects of a software engineer is brighter than most other functions in tech. The reason for my switch is that I firmly believe in increasing abstraction of technology. While discussing this piece with my co-host Ranjan, we initially disagreed on what I mean by that word; abstraction. Let me explain.
In software engineering, abstraction means something specific. Namely, the idea is to hide the complexity behind an abstraction and only expose the important bits. A good analogy is the automatic transmission, which is an abstraction over (of?) the manual transmission. This means that you can either simply forget about changing gears entirely, which does come at a cost, or more recently, simply focus on changing gears instead of using a clutch pedal.
This notion of abstraction means that you can do more with less. In the case of driving, your cognitive drain is reduced, and driving becomes easier. In case of the larger software engineering industry, this means that you can slowly focus less on what software is, and focus on what it does. When you don’t have to worry about the intricacies of your software and infrastructure, you can focus on the larger vision, and increase your leverage, and impact.
In a world eaten by software, being a software person is going to be not just an advantage, but a requirement. But in a world of increasing software abstraction, the more in-demand skills will not be just ability to think in software. And not only that, software people will also have to think in terms of things they touch, which is everything. This is why, in the end, is why I decided to pursue a degree in business.
And hey, if it all fails, there’s always philosophy.
What I’m Reading
Wimbledon 2010 live blog: 23 June: The Wimbledon final between Federer and Djokovic was epic, but it’s still nothing compared to the 2010 Wimbledon game between Isner and Mahut. At 11 hours, it’s the longest professional tennis match in history. And this liveblog from The Guardian was an epic hit when it came out. I am both amused and disturbed that I remember it so dearly:
I'm wondering if maybe an angel will come and set them free. Is this too much to ask? Just one slender angel, with white wings and a wise smile, to tell them that's it's all right, they have suffered enough and that they are now being recalled. The angel could hug them and kiss their brows and invite them to lay their rackets gently on the grass. And then they could all ascend to heaven together. John Isner, Nicolas Mahut and the kind angel that saved them.
Principles for a More Informed Exceptional Access Debate: I talked here about end-to-end encryption before. As more happens in bits over atoms, our base assumptions on which our civilizations are built are tested. Who gets to see what? How do law enforcement and intelligence organizations work and how do we balance their demands on data with individuals rights on privacy? This is a good primer on some of the basics on end-to-end encryption. I don’t agree with everything here, but it’s sober and introduces some good points.
Law enforcement and intelligence agencies have been “going spotty” for some time, in that changes in technology continually changes what is available to law enforcement. There’s no panacea that can solve all the problems law enforcement has with access to information. This article outlines how to enable the majority of the necessary lawful access without undermining the values we all hold dear. For the purposes of this article, we’ll use the term “exceptional access” to mean a targeted government authorization to access, with the assistance of the service provider, the data belonging to a user when needed, for example as part of a criminal investigation or to stop terrorists. It’s exceptional because almost all users aren’t affected by it and it’s not very common, on the scale of the total number of devices and the total number of communications enabled by the platforms.