066. Killing a Killer Feature In Outlook, Again

Enterprise software customers learn about roadmaps and plans long before the development team has robust execution plans. That’s part of the business. No matter how much these discussions with customers and partners are caveated, a failure to deliver is a big deal. Customers, partners, and salespeople all have a vested interest in those slides (promises!) coming to life when we said they would. When a project is spread across two separate and large development teams, failing to deliver or “cutting a feature” as we called it as if to minimize accountability, the cross-team dynamic is brutal. When BillG gets involved, the temperature rises even more.

Thank you again subscribers. For many readers, you will be receiving a renewal notice about a week before your subscription is automatically renewed. I am incredibly appreciative of your support this past year and look forward to another year of amazing stories: Office user interface, Windows 7, Windows 8, internet services, Surface, and more. So much to dive into. Again, thank you for your ongoing support. It means the world.

Back to 065. SharePoint: Office Builds Our Own Server

JeffR, my new manager and executive vice president, send me a note “LIS: Wow, the story just gets worse and worse. . .Let’s kill it.” He pushed to resolve this festering issue and suggested a meeting with BillG. This would be the second time we cut this feature, having shipped Office 2000 without it.

It was December 2000. In a hastily arranged lunchtime meeting (a BillG must was to always have hot lunch at noon), we sat in the Board Room where we normally met with him. It was a deeply technical meeting, the kind he liked. There were senior people from Exchange, Outlook, and the company experts on data storage that have been meeting as part of Bill’s project to improve storage. The meeting was to decide whether or not to abandon the work on LIS, or the Local Information Store, feature of Outlook and Exchange. The meeting was very tense. The broad view was that this feature was absolutely key to competing with IBM/Lotus Notes. The Exchange Platinum release was late with an uncertain ship date. Office10 was less than three months from being complete, and the code was essentially frozen.

After a tense hour that ran over, BillG said he would think about it, leaving everyone in a state of limbo. It was unusual for him to not reach a conclusion immediately. That meant he was either going to take an unpopular stance, perhaps try to keep the feature alive, or he knew it was a pretty grim situation and was going to avoid confrontation in the room. Counter to what many might have believed, Bill did not like to get involved in binary decisions that leave little room for optionality or win-win outcomes.

An hour later he sent mail saying it was a difficult decision but “I am for killing it because less than 5% of customers would end up using it.” That conclusion was based on the state of the feature as presented.

What followed was the rollout of a brutal cut, which included drafting mails for another month, communication with the field sales group, and removing dependencies on the code.

This recounting made the process seem straight forward, but in fact it was one of the most challenging last-minute product changes I experienced. Going through this surfaced many of the most difficult product strategy and cross-company execution challenges Microsoft exhibited.

During the meeting with Bill, I was not worried or concerned. I was frustrated (and it almost certainly showed). There was no need to have this meeting. The feature basically cut itself. There were no options other than delaying products for perhaps a year to complete this feature. This was the “physics of shipping software”.

Even though the classic by Frederick P. Brooks, Jr., Mythical Man-Month, was over 25 years old and found on every Microsoft engineer’s bookshelf, when decisions made it to the executive ranks, especially when they involved cross-group and strategic initiatives, the lessons from the book were forgotten.

BillG would ask if we could allocate more resources from either team to speed things up. We all knew that there was neither the expertise nor the available resources to do that. Plus, we all knew what the mythical man-month said about that, “The bearing of a child takes nine months, no matter how many women are assigned.”

BillG would offer suggestions on how we could scale back on some of the features or scenarios in order to require less work. We had been doing that for months. The feature had been scaled back so far that even if we shipped what we discussed at the meeting, it would not have moved the needle on competing with Notes. Quite the contrary, it would have disappointed.

BillG would say it would be acceptable to simply add some more time to the schedule, perhaps three months. Physics of shipping this amount of software meant that three months was hardly any time at all. Given all the work to test, stabilize, and for servers getting feedback from customers, adding three months was about the same as adding a few weeks of engineering at best. Besides, even without this feature Exchange Platinum would likely not ship until the end of 2000. Office10 was all but complete and “re-opening the patient” as we said was not an option. It was just physics.

There was something about how the company was working that we permitted ourselves to go through this sort of exercise when there really weren’t any options. Generously it was a process to come to grips with a difficult failure to deliver. Alternatively, it was a way of spending energy exploring options that didn’t really exist anyway. That’s why I was frustrated. It was decision theater. The bottom line was we didn’t deliver and there was no rescue mission.

There are times when these meetings can yield a new outcome. Projects have constraints and if BillG (or anyone in a position to do so) could relax some constraints—the broadly-defined trinity of ship date, features, or quality—then there is a new path to take. Usually, teams that forgo examining and changing these assumptions on their own tend to have other problems as well. Shipping software is managing the trinity and adjusting along the way, not blindly following constraints that aren’t working.

What was this feature and why was it so important? LIS was a new way to store all the email on a PC. This of course seems crazy today because no one wants email on their PC where it could be lost or stolen or worse. LIS was a new model of email where the PC would maintain a copy of the email that was on the server and keep the copies in sync. This made it possible to work without an internet connection while also enabling a rich level of capabilities to build apps on top of email that ran on the PC. This replicated storage was a key feature of Notes.

Underlying the feature was a new data storage architecture. Here’s the challenge. This model of software requires the code on the PC with the exact same capabilities as the code on the server in order to realize the benefits. Notes accomplished this with a smart architecture designed from the start. Exchange and Outlook evolved without such an architecture, and we were trying to retrofit a much more elegant architecture on top of Exchange. To do so would have required building a PC-based system with the same capabilities as the one running on big servers, but able to run with PC-level hardware not the much faster and more capable server hardware. The result: even by December 2000, LIS in Office10 was very best case 20% or more slower than Outlook 2000 and required a high-end PC. No one was accusing Outlook 2000 of being speedy, so this was a significant negative.

There were many other features that were slated to be delivered. Some of them were highly requested by customers. One was the ability to store email using the new UNICODE characters so one email storage file could easily have mail from any language. One feature that might be strange (or poorly architected) was that LIS was going to make it possible to connect from Outlook to the mail server using the web protocol HTTP and not the Windows networking protocol, which was really important for scalability and security. LIS aimed to provide a badly needed search capability that Outlook completely lacked. Another was the ability to store more than 2GB of email on a PC. This might sound crazy, but the cost of storing email on servers was so expensive that most Exchange customers were limiting email to 25-100MB (megabytes!) The rest of email could be stored in a separate file that existed only on a PC. The implication of such an architecture was that Outlook needed to be unbelievably rock solid and never ever damage that mail storage file. Any bugs or fragility in the code might mean a customer would lose all their email, permanently. The idea of inserting an entire new data storage format into the product at this late hour bordered on crazy.

Developing this feature was never going to be easy. It was another case of two major products with different processes, approaches to work, and schedules trying to align. Kurt DelBene (KurtD) was always calm and a great partner with the Exchange team leader Gord Mangione (GordM), but the tension over this work was palpable the entire release. The two products, while built by separate teams, were inescapably linked. Microsoft’s email strategy relied completely on both teams delivering an integrated product, while also serving another larger strategy (Exchange working with Active Directory, Outlook as part of Office).

The bet was even bigger for Office beyond Outlook. We had made a major bet on delivering “Office and Exchange for Corporate Groupware” as a significant pillar of the vision. While our vision process was still new (this was the second time we used it), the idea of losing a whole top-level focus area really hurt. In addition to Outlook, Office created a new tool called Designer, staffing an entire team of experienced engineers specifically for end-users to create applications like those in Notes. It was to be the cornerstone of Notes compete. Without LIS, there was no Designer—the work of that team would not ship at all.

The marketing team briefed important customers about the whole set of LIS deliverables including the features, Notes compete, and the new Designer product. There was a lot of excitement. It was easy to generate excitement with slides and mockups, especially when it made closing a big Exchange deal easier. Unwinding that excitement was brutally painful. Each customer meeting is incredibly difficult for the account and sets the relationship and business back. Immediately discussions turn to potentially turning to IBM for a solution to collaboration. Customers were tuned to escalate these failings straight to SteveB who in turn would again ask if there was anything to do or what he could say or offer. There was nothing. It was physics. The field hated physics. Customers did not understand physics. The press equated a missing feature with vaporware—software that never really existed except to muddy the competitive waters. How could Microsoft, the largest company in the world with the best software engineers anywhere not deliver? What did it mean for the future?

There were no answers, easy or otherwise. One way to say this was we failed to deliver. The lesson is not as simple as a failure to execute. Israeli military postmortems remind soldiers that there are no failures in battle only failures in intelligence. In software, failing to deliver was not a failure in writing code, but a failure in planning what code to write and how to write it. We were still planning products like the primary audience was retail customers or hobbyists who were more than happy to work through messy details, wait a little longer, or have some bugs as long as there was new stuff. Enterprise customers with their huge spend, multi-year planning horizons, and 5-10 year usage plans, were in no position to absorb this sort of attitude. We failed to plan, so our plans failed.

It took most of holiday season 2000 to make rounds with customers and all the teams to let them know we had cut the feature. The reactions across the company were varied depending on the team. The different cultures make quite an appearance at times like this.

The Office team already knew the feature would be cut by the time we told them it was official. More than anything they wondered why it took so long for Kurt and me to admit, finally, what they knew to be the case. Even our marketing team was somewhat relieved as this simplified our collaboration message to SharePoint and our email message to Exchange, without any redundancy. The Designer team, a whole new team, took it in stride as they knew it wasn’t coming together. I do not want to downplay the stress and strain on the individuals who committed a product cycle to the work. It was not their fault—they were at the receiving end of this failure.

The Exchange team was in a different state of mind. They tended to see things through the lens of Office decommitting at best or at worst Office was never committed. When a consumer of a dependency cuts a feature, it was often perceived as though they never really believed or somehow did not try, any evidence to the contrary. The presence of what was seen as a backup plan (SharePoint for collaboration) only made it seem as though there was a plan all along to “fail.” Exchange had a right to be anxious because they owned the Notes compete story and this was a big blow. It would take another couple of years, but Exchange would handily win in the market. The idea of building applications moved to the browser as quickly as customers decided they simply needed great email and scheduling more than applications, and Exchange plus Outlook was superior there. Competing with Notes, by 2000, was about skating to where the puck was as hockey legend Wayne Gretzky might say.

There was no escaping that the wounds were deeper on the Exchange team. Cutting LIS surfaced years later in a story highlighting Microsoft’s perceived morale difficulties. In the story in Forbes Magazine, “Microsoft’s Midlife Crisis”, a former Exchange engineer was quoted "They sent me a 200-page document that said our technology had to be 100% better than the current stuff. Then it failed, of course, so they did it themselves." The Outlook team would say (and did) that it just needed to be the same, not 4-5 times slower and bigger. Nothing was easy and when it doesn’t work and when failure is poorly managed, people can remember the worst parts in the worst way as they seek accountability.

There were hundreds of technical account managers who were anxious to begin to build Notes-like applications who reacted horribly to the cut. To them this was another case of Redmond not understanding what customers needed and worse failing to deliver what we said we would deliver. In the field, where salespeople have quota that they make or not, where general managers either make their numbers or find a new company to work for, failing to deliver what was promised was a first order failure. I attributed much of the enthusiasm and support of SharePoint for collaboration and Notes compete to the lack of an Exchange story. The enterprise team in marketing spent the better part of the next six months smoothing over each country and market, one at a time.

BillG was disappointed of course. Something he always did extremely well was bounce back from these setbacks and not put the team or leaders in any sort of penalty box. I previously shared the story of AFX, my first project, and how we wasted a year and got nothing done while Steve Jobs’s NeXT was on the rise. Bill was as anxious then as he was now and in a similar way—was there anything we could do sooner, more people, or a little more time? I bounced back. We bounced back. In this case, it is fair to say Bill re-doubled his efforts on the major project of the early 2000’s, data storage. In his eyes the failure of LIS only made it more critical to solve the company’s “storage problems.”

We held a big meeting in the atrium and announced the cut to the whole Outlook team. Everyone knew. This was a formality. As we were easing the team into the final decision, I thought of Andy Grove’s Only the Paranoid Survive. In the book he recounts the long process it took to come to grips with the need for Intel to exit the DRAM business only to come to understand the middle managers had long ago realized what was inevitable and made resource allocation choices in that direction. It was as if Grove was the last to really know. The team knew. We all knew. It just took a while to get there. It was the physics of cross-group collaboration, not just the physics of software.

A final reminder to the team was that we don’t market the features that didn’t make it into the release. No one was going to know there was something we did not do. In fact, the list of things we did not do was infinite. We did what we did, and it was going to be great.

I learned a lesson (again) about pre-committing to customers when basic engineering prudence said otherwise. With LIS out of the way, and Watson streamlining development, we were on a path to ship. We were feeling good. Like the end of every project, we were in the period where most people came to work and did nothing but make sure no one else did something rash. For a change, this holiday was going to be enjoyable and free from any project time crunch.

Cutting is shipping. We proved that once again.

Countdown to RTM blast-off and 3/2/01. At least I thought so.

Subscribers, share your story of the most difficult “cut”. In hindsight, was all the pain to make the cut worth it?

On to 067. MYR-CDG: Product Meets Sales

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit hardcoresoftware.learningbyshipping.com