Ballmer, a 30-year Microsoft veteran who went to Harvard University with Bill Gates, has survived even as Microsoft faltered in the mobile and consumer markets and watched rival Apple take over Microsoft's former position as the world's most valuable technology company.
Even Microsoft's Board of Directors, in a recent Securities and Exchange Commission filing, referred to "the unsuccessful launch of the Kin phone" and "loss of market share in the company's mobile phone business" when it decided not to award Ballmer the full bonus he was eligible for.
Although you could argue Microsoft's Windows, Exchange and Office businesses are humming along (despite the Vista debacle), Microsoft's mobile struggles in the face of Apple's iPhone and Google's Android platform are among the issues that constantly fuel speculation about Ballmer's future.
"My personal opinion is he is long past due," says Craig Montgomery, who led a grassroots shareholder activism movement last year in an attempt to force changes at Microsoft. "Mr. Ballmer is not the correct leader for Microsoft."
But the biggest question may not be "Should Ballmer be fired?" A more important question may be "who can replace him?"
Despite the so-called "consumerization of IT," in which the worlds of business and consumer technology are slowly merging together, it is still very difficult for any one company to be the dominant technology provider in both the home and corporate markets.
"Ballmer may be facing an impossible task and it may be he's not doing a particularly good job, but it may be that no one could. And maybe even Bill Gates couldn't," says industry analyst Roger Kay, president of Endpoint Technologies Associates.
That being said, Kay says he has come to believe that Microsoft could do better with a different CEO. "The [Microsoft] board should have set a timer on Ballmer's tenure and said 'if we don't start seeing certain metrics then we're going to find a replacement for you,'" Kay says.
Microsoft dominates its strongest categories as much as Apple dominates its strongest areas, but "where Microsoft is getting killed is they're not competitive where Apple is," says analyst Rob Enderle of the Enderle Group. Ballmer has opened himself up to criticism even though Microsoft's Windows business has succeeded against Linux, and Exchange has "knocked Lotus Notes right off the map," Enderle says.
Kay suggests that Microsoft's ambitions are outstripping its actual capabilities. Microsoft built gigantic businesses with Windows and Office, and servers and tools, but creating monopolies in more than one or two businesses may be beyond the reach of even Microsoft, Kay says.
Enderle argues that "there aren't many people on the planet" who could run a company like Microsoft, saying that even Oracle's Larry Ellison and leaders of companies such as IBM don't have to manage the diversity of categories that Ballmer is tasked with. "They are a software and product company that spans consumer to large-scale enterprise products. That's more breadth than IBM has in terms of customers, and exceeds the existing skills of, probably, the leaders of the other major software companies," Enderle says.
Kay is a little more optimistic that someone in the industry could be an improvement over Ballmer. Someone like Larry Ellison would be more of an activist manager than Ballmer, killing off divisions that underperform and boosting investment in those that succeed, he says.
Ballmer is "not badly suited" to the Microsoft job, "but you have to remember he was a Proctor & Gamble product manager who happened to know the right person, and ended up sitting on a volcano that took him up to the stratosphere," Kay says. "It's not as if Steve Ballmer won out against many contenders to be the person who ran Microsoft. He basically just happened to be the right friend of Bill Gates, who had more business acumen than anybody Bill Gates knew."
As for who might replace Ballmer, Kay says "there's probably quite a lot of talent around, but the question is whether that talent is sporting the right type of resume to get that job," Kay says.
Even if Ballmer stays, Enderle suggests that a reorganization that revamps the consumer side of Microsoft while retaining Ballmer's strength on the enterprise side is in order. This could perhaps mean splitting the company up into smaller parts. "If you wanted to keep Microsoft together, finding someone to take it on the way it is now would be nearly impossible," Enderle says.
He's not the only one suggesting big changes at Microsoft.
Goldman Sachs removed Microsoft from its "buy list," while releasing a report on Oct. 3 that questioned Microsoft's strategy. Microsoft stock has performed worse than the S&P 500 over the past two years.
"Concerns over the longer-term sustainability of the Windows/Office franchise have clearly weighed on the stock," Goldman Sachs analyst Sarah Friar writes. "However, this is not just a 2010 phenomenon: since 2002 Microsoft has had a -8% return on a dividends-reinvested basis, the second-worst of the largest dividend paying tech stocks. For the shares to unlock value, we believe a call to action is in order."
A better long-term Microsoft strategy might involve splitting the company up into smaller parts, Goldman Sachs indicated, writing that Microsoft needs "a coherent consumer strategy that could involve paring back investments and/or divesting more peripheral assets such as gaming."
"The company could either aim to turn the consumer divisions into contributors to profitability and shareholder value by carving them out, or by meaningfully stepping up cost discipline. A break-up of the consumer businesses could potentially unlock hidden value, or more discipline on cost could turn the businesses into contributors to profitability and shareholder value," Goldman Sachs continues.
Microsoft has also not used its enterprise data center technology effectively enough in the emerging cloud computing market, and needs a stronger foothold in the smartphone and tablet space.
Microsoft has shaken up its mobile phone and consumer devices division with the departure of Robbie Bach, the company's highest paid executive.
But Goldman Sachs does not see a Microsoft mobile resurgence occurring until at least 2011 because "Apple's iPad and iPhone plus Google's Android operating system are well established."
On Oct. 11 Microsoft will unveil the first devices to run its new mobile operating system, and this launch of Windows Phone 7 could go a long way toward determining the ultimate success or failure of Microsoft's mobile business.
Based on public statements, it's not clear whether Microsoft's board is seriously considering replacing Ballmer. Although numerous media reports claimed that Ballmer's annual bonus was "cut in half," in reality his total compensation was slightly higher in 2010 than it was in 2009.
Ballmer's base salary was $670,000, and he was eligible for bonuses worth up to 200% of his salary. His awarded bonus ended up being 100% of his base salary, or $670,000, an improvement from his previous year's bonus of $600,000. His total compensation in fiscal 2010 was $1.34 million, short of his "maximum compensation opportunity" of $2.01 million.
In awarding Ballmer a $670,000 bonus, Microsoft's board praised him for numerous achievements, including "the strong financial year in which Microsoft reported a record $62.5 billion in revenue and $24.1 billion in operating income; the operating income performance of the Company relative to 25 large technology companies; his leadership in continuing the prior year's disciplined expense management; a series of successful product launches including Windows 7, Office 2010, Bing, Windows Server, and SQL Server; progress pursing new innovations that will position the Company to lead the transformation to the cloud (Azure and Office Web Apps) in active gaming (Kinect), and in other product areas in which work is well underway."
Some Microsoft employees are clearly dissatisfied with the CEO, however. On Glassdoor.com, a forum that lets workers rate companies anonymously, Ballmer received a 50% approval rating from employees, compared to 97% for Apple's Steve Jobs and 96% for Google's Eric Schmidt.
While Microsoft's board expressed dissatisfaction with Ballmer's performance in mobile, the company seems publicly optimistic that it can turn things around with Windows Phone 7. Last month, a few hundred Microsoft employees held a mock funeral for Apple's iPhone at Microsoft's headquarters in Redmond, Wash.
But whether the future performance of Windows Phone 7 is enough to save Microsoft's mobile business, and Ballmer's job, remains to be seen.
Follow Jon Brodkin on Twitter: www.twitter.com/jbrodkin
Read more about software in Network World's Software section.
Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.