The Bloat Problem: Why Large Tech Companies Overhire Engineers

Why do large companies keep a ton of software engineers when they aren't building a bunch of new code?

I think this question was either inspired by, or at least has been given new life by, the recent news about Twitter firing (and/or losing) 1,200 software engineers.

And people are rightfully wondering why a company which has as its main product a microblogging site, with precious little other functionality, would need more than 1,200 developers, counting the ones who weren't fired and didn't quit.

And, well, they obviously don't need that many.

Not that Musk was right to fire people. I'm not going to say he's done anything right in his train wreck takeover of Twitter. It is a disaster in progress, absolutely, especially his goal of re-enabling hate speech and misinformation. But I digress.

The simple answer is that once a company grows to a certain size and has an income that's more than it needs to survive, it will frequently grow like a cancer. From what I can tell, that's almost the default state of large companies, and only a firm hand from the top and the correct culture throughout can prevent it.

Now, looking at the other answers that talk about maintenance and new features and so forth…No. Not 1,200 developers worth at Twitter.

Before I get unlimited hate mail for making such a claim, I have evidence that what I'm saying is true.

Exhibit 1: Craigslist .

Their income is reportedly $500M per year, and from what I understand that's only from classified ads in markets where the customers wanted there to be a fee . They are a top 100 web site that serves hundreds (thousands?) of communities. They have to deal with hate speech and spam and much more complex postings than Twitter's tweets. Posts that can be edited, updated, renewed, and reported. They also support both Android and iOS apps.

According to the Wikipedia article on Craigslist, they get 20 billion pages views per month. Not nearly as many as Twitter, but past a point you don't need more software engineers simply because you're getting more page hits. Coding so your site runs from 200 servers or 2000 doesn't mean a difference of 10x the number of programmers .

How many hundreds of software engineers do you think they need to support their code and infrastructure? 200? 500?

Craigslist has 50 employees.

Not 50 software engineers. Employees . I don't know if they hire freelancer engineers at all, but there's a limit as to how many they could manage with that few employees, so the total number of engineers is probably still less than 50.

I personally have never seen Craigslist go down, either. It generally just works. The design isn't inspirational, but a better design doesn't take hundreds of developers to implement.

Exhibit 2: Reddit .

A more sophisticated blogging platform than Twitter, with threaded comments, user-run subreddits with user moderation, a real time streaming platform with real time chat, Android and iOS apps, and a healthy 52 million daily active users, ranking it as the 9th most popular web site in the world and the 6th in the US.

Wikipedia pegs Reddit's total employee count at 700. That's 1/10 the number Twitter had before the purge, and Reddit is still smaller than Twitter after the purge . Again, not all of those employees are software engineers; given the ratio of community managers to software engineers in their job postings, it's likely only half that number who are software engineers.

And Reddit is a much more complex product in every way than Twitter.

No, really, Twitter didn't need that many software engineers.

Now to be fair, the reason that larger companies need larger teams than startups is more than just maintenance and new feature development. A startup can create a product like Reddit with a handful of developers, after all, if they choose the right team. Given tools and services available today, it could even scale to Reddit's size if the team and design is good enough.

But that's a team of irreplaceable world class developers who know all of the code inside and out because they wrote it.

And eventually those developers will decide to leave the company or take a long vacation—or get hit by a bus.

And then what? The company is screwed.

So to some degree, you do need more programmers than you would expect to manage a product. It's important to protect institutional knowledge. It's important to have enough redundancy for your developers to take vacations. And it's important because the brilliant developers who can create something like a scalable Reddit will get bored sitting around maintaining a product that is primarily in maintenance mode.

So you break the product down into smaller pieces that can be maintained and extended by teams of less-brilliant (or at least less ambitious, or with a different temperament) developers. You hire for redundancy. Your goal is reliability rather than innovation.

But if there's a lot of money floating around and little oversight, humans will do as humans do and will look for ways to build their little empires. Flimsy technical excuses will be made to build your teams—break this functionality into a microservice, then that, and then add layers of managers who want to add to their teams and eventually you have something like 3,000 microservices (reportedly at Twitter) and you need thousands of developers to maintain them all. Developers who often don't all reach a very high standard of ability due to the pressure to hire someone.

No, Twitter could have been well managed at 200-300 software engineers. But the mess they have right now probably requires many of those 1,200 software engineers who have left or were fired.

Maybe in six months of careful work they could have migrated to a system that only needed 300 developers. What actually went down is nothing less than a disaster for Twitter.

And honestly? Musk was being an idiot to let that many software engineers go at once. Spend time at first optimizing Twitter to need fewer developers and then figure out ways for Twitter to be more than a microblogging platform! Use the newly freed developers to build out the new services! Win win for everyone involved.

I mean, isn't Musk supposed to be a super-entrepreneur? Surely he could have come up with—or paid someone to come up with—a brilliant new direction for Twitter.

I do wonder if Musk is suffering from a similar affliction to his car company's namesake. Nicola Tesla was brilliant when younger, but exhibited increasingly irrational behavior as he got older, to the point where the inventor of the technology that enabled the modern electrical grid was surviving on handouts near the end of his life, unable to even hold a job.

And, well, Musk's behavior was always a bit eccentric, but he's really lost touch with reality. It's gotten to the point where I'd be more comfortable buying a Tesla if he stepped down and let someone else run the company. Which is probably 98% of the case now, given his Twitter hobby, but still.

But I've wandered away the domain of the question, so I should focus. The answer is above, but in summary: Companies that have poor or ineffective technical leadership often end up hiring more engineers than are strictly needed, sometimes without regard for talent and ability, resulting in workplaces that function more like Dilbert strips than they should, and insufficient leadership direction leads to poor designs and the inadvertent creation of problems that could have been avoided, creating the need for still more developers, which keeps repeating until the company can't afford to hire more. And I think that's where Musk found Twitter, though it's mostly a guess.