Categories
All MacBU

Bugs stink! Yeah Yeah!

(my apologies in advance to the J. Geils Band)
Pierre Igot writes what I must politely describe as an impassioned discourse about my description of pseudo-localization, multi-lingual bundles in OS X, and MacBU testing. Pierre makes some good points, totally misses the idea of pseudo-loc in another, and generally castigates the MacBU for failing to fix a particular bug that is very important in French typography. He then invited me to comment.
Let me begin by saying what I am not. I am not the lead developer for Word, and I am not intimately familier with every bug that is entered against our products. I am also not a native French speaker, nor am I familiar with the rules of French typography. (In fact, my peers in high school often told me “Tu parles français comme une vache espagnole,” which, if my hazy memories of 1991 are correct, means “You speak French like a Spanish cow.”)
I do, however, know quite a lot about bundles in Carbon and Cocoa, how multi-lingual bundled apps should behave, and the benefits to users of software that takes advantage of them. Apple did indeed make them available to developers using Carbon in 2001 or so. Pierre is correct in saying that currently shipping Mac Office apps do not use them. (Well, Messenger does use them, and may have been the first major Microsoft app to do so.) I became a lead for Mac Office in June of 2002, well into the Office 2004 project and was not a part of the initial schedule planning that, for whatever reason, did not include moving Office to a bundled architecture. As the lead for localization, I understand and sympathize with users for whom this creates frustration. The reality is that for every user calling for a multi-lingual version of Office, there are at least two users who don’t care about that at all and want us to be working on something else. We can’t please everybody, unfortunately. As I said in the comments at the end of my localization post, I cannot comment on our plans for this aspect of the next version of Office, but please rest assured that I do understand and hear your request, Pierre.
Because I am not well-versed in French typography (or any real typography for that matter) I can only trust that Pierre’s assessment of the importance of the non-breaking space in French documents is indeed high. Assuming that is the case, then yes, this must be a very aggravating bug for anyone who uses Word 2004 to lay out text in French. I’m quite sorry, Pierre, that you have been so frustrated by this issue. I wish we had caught the bug and fixed it before shipping. Sadly, we did not.
Pierre then takes our non-use of bundles and conflates that with the existence of the non-breaking-space bug:

The fact that the language of the user interface in Office applications does not change depending on the user’s preferred language in Mac OS X is annoying enough. But that’s not the worst of it. The worst of it is that Microsoft’s process for fixing bugs is completely screwed. And there is no better example of this than the bug with non-breaking spaces and PostScript fonts in Word 2004.

He also implies that our use of pseudo-localization should have caught this ‘elementary’ bug:

The fact that Microsoft did not catch that bug in Word 2004 before they released the product is a clear illustration of how flawed their testing processes are. Whatever benefits this “œpseudo-localization” technique described by Erik Schwiebert provides, it is clearly not good enough to catch even such elementary bugs.

Here I must correct Pierre’s faulty assumption and say that the bug is not in any way connected to either of the other two items. Multi-lingual bundles are all about providing resources to the app, such as strings, dialogs, menus, windows, etc. Pseudo-localization is a method we use to test the localization of those resources and the code that uses them. It is intended to help find visual glitches, missed translations, improperly displayed dialogs, and similar items, and it does a pretty good job at optimizing our efforts for those issues. It is not designed nor intended to help with finding or fixing bugs related to the actual behavior of the application. Pseudo-localization in no way replaces good old-fashioned usage of the applications to uncover code defects.
In fact, the bug that Pierre is so frustrated with occurs in all languages that Word ships in — the faulty code is in no way connected to any localization. I can repro it in the English build. That doesn’t mean that our entire testing process is flawed, however. Every large software endeavor results in a product that has bugs. Microsoft is no better or worse than anyone else in this regard. Seen any OS X software updates lately? 🙂
Now, the fact that the bug is unrelated to our localized UI testing doesn’t address the issue that the bug is there, and is bad. Pierre blogged about the bug way back in September 2004. Unfortunately, the MacBU (and the rest of Microsoft, to my knowledge) has never had a good method for customers to report bugs to us. We’ve had the (now defunct) MSWish email address, and we do have the new direct feedback link on Mactopia, but we don’t have anything like Apple’s BugReporter tool. Interestingly enough Nathan Herring, another MacBU employee, just blogged about this specific issue and has some links you may want to read.
But let me continue with Pierre’s post. He finds it unjustifiable that we have not fixed this bug in the almost two years that have gone by:

Now what excuses does Erik Schwiebert have for this sorry state of affairs? I might find it acceptable (barely) that Microsoft is not able to catch such bugs in the hectic schedule that leads to the release of the initial versions of the product. But how can they justify not fixing the bugs in the next two years, even though the bugs are so obvious?

says it can be done on schedule:

it is perfectly possible to release multilingual software on schedule that automatically supports all of Mac OS X’s supported language without requiring the user to buy separate versions for each language and download separate updates for each language.

and casts shame on us:

And for all this, once more, all I can say is: Shame on Microsoft, and shame on the MacBU. They have absolutely no excuses here.

Ouch. That hurts. Me personally, as well as all of our hard-working employees. It’s a good thing I have a relatively thick skin.
I don’t think there is anything I can say that will appease Pierre. And I don’t mean that there’s something I know that I’m not allowed to post; I just don’t think that any insight I provide into MacBU’s bug process will soothe his wrath. However, I will try to shed some light on our processes because I think folks are interested in them anyway. I’m sure many people will individually disagree with many of of the decisions we make and the factors we weigh, but that’s part of the openness of blogging. Feel free to respond with your thoughts and comments — you may provide some insight that we’ve missed in all these years.
First of all, we do work darn hard to fix bugs in our products. Really darn hard. But, we’re not perfect. No developer team is. Not MacBU, not Microsoft, not Adobe, not Apple. Security defects? We’ve all got them. Software updates? Yep, them too. For OS X, we have 10.0.4, 10.1.5, 10.2.8, 10.3.9, 10.4.6 and counting. For Office we have 11.1.1, 11.2.4, etc. Updates are good, it means that we’re all working to make our products better, to fix the bugs and reduce the crashes.
However, not all bugs are fixed in dot-releases of products. Some bugs that we find are really obscure and don’t cause any real harm. For example, perhaps the File/Open menu item is grayed out when you hold down Cmd-Shift-Ctrl-Option and triple-click on the menu-bar. Not too many people will see that, and if they do, no real harm done. We can spend our engineering time and resources better elsewhere.
Some bugs are easier to hit, but fixing them may have a high risk of regression and may cause another bug that is even worse. One example might be a performance bug in Excel’s recalc engine. We could fix it and make things faster, but currently the code calculates correctly (albeit slowly) and any fix might totally break all function dependency analysis. It is sad but safer to leave things alone.
Other bugs are deemed worth fixing for a future release, but not worth back-porting to the current release. Every code change we make has to be tested around to make sure no regressions are caused, and that takes away from other work we could be doing. It may be more effective for us to take a bug that was reported against Office X and just fix it in Office 2004. And occasionally, some bugs are misclassified or accidentally forgotten. After all, we’re only human.
Here’s an example of an engineering decision. If you boot Mac RDC on a PowerBook and connect to a Windows server, try turning on Num Lock (the keyboard light should be on). Type something on the number pad in Excel. You get numbers, right? Good. Now, Cmd-tab back to the Finder and turn off num-lock. Cmd-Tab back to RDC. Type something on the number pad again. What, you still get numbers? Yes indeed you do. Geez, the MacBU really should have fixed that bug, right? We’d love to have fixed it, except that the bug is actually in the OS. The Carbon Event manager does not give you accurate information about the Num Lock state when you query for modifier key state after app activation and deactivation. I filed that bug with Apple in May 2002 against OS X 10.1.4, and they still haven’t fixed it. I wish they had, but I’m sure there is some reason (hardware? software complexity? who knows…) that the change is more expensive for them to make than to allow the bug to continue to exist. That decision is completely up to them, and I respect that.
In our move to Xcode, we’ve reported over 100 bugs to Apple. They’ve fixed the majority of them, and have postponed several to some later release. Some of the bugs they’ve postponed have been real bummers, but they’ve made the engineering justification to delay the fix and again, I respect that. After all, we have to do the same thing with regard to Office bugs too.
Second, the ability to meet a schedule or not has nothing to do with the ability to ship a bundled multi-lingual product. That is essentially a feature, and does not come for free with Carbon apps. (Yes, that is one aspect where Cocoa wins hands-down — it is impossible to create a monolithic Cocoa app. At the very least your app must be bundled, whether or not it has more than one locale for resources.) Perhaps Pierre would have preferred we have chosen a different schedule, moved Office to Mach and multi-lingual bundles, and shipped a product called Office 2005. We hit our internal schedule for Office 2004 with the features we planned for it.
Now, I’m not saying that any of this is an excuse. We’ve made lots of conscious decisions over the years about many many issues. Some of them, everybody likes. Some of them, everybody hates. Most of them are liked and hated simultaneously, and the overlap between the sets of people experiencing the two emotions is rarely the same from topic to topic.
The sad thing about this particular bug is that it falls into the category of ‘oops.’ The code was wrong in a very small way (an exception was made for the breaking-space character and accidentally forgotten for the non-breaking space character.) We actually found this bug in-house just a little while after Office 2004 shipped, and fixed it internally for the next major release of Office in August 2004. We even noted that it was important in the French market. Somehow, though, it was never flagged as a bug that should be migrated back to Office 2004. The tester for that area of Word (who found and opened the bug) left MacBU to go back to graduate school during that summer, and I think we probably just missed this one in the transition. A simple human mistake. Shameful? I personally don’t think so. Unfortunate and frustrating, I’ll certainly agree to that, but not shameful.
I’ve asked our Word team to take another look at this bug. I make no promises, but maybe we can get it fixed sooner than in the next major release of Mac Office. After all, the Windows team did that for a blogger recently.
So Pierre, we are listening. I think pseudo-localization has a greater value than you place on it, but for a different purpose. If you are ever in the Seattle area, please look us up. I’d be happy to take you out for coffee and a chat, and see if we can address some of your dissatisfaction with us. In the meantime, I’ll stand up proud and accept no shame, for I believe we work hard and produce a good product.

13 replies on “Bugs stink! Yeah Yeah!”

Thanks for an excellent post. I should send it to everyone who asks me why we haven’t fixed bug X yet.
For what it’s worth, I think you can actually build a monolithic Cocoa app. It wouldn’t be fun and I can’t imagine why anyone would want to do it, but I think it’s possible. If it was a weekend instead of Tuesday night, maybe I’d try it….

Great post,
There might be an advantage to have multi-lingual bundled apps. This might be a way to have the Mac Community create additional languages, even not officially. The OOo community did it, and you know find OOo localization ressources in many many languages… Eventually this might increase the adoption of Office:Mac abroad.
Another thing. Pierre is very vocal for this bug. That’s OK… but. As you know there are crashing bugs related to localization issues. Many have been correcting in PPT:2004. Word grammar checker crashing bugs are here since… Office 2001 (I think)… I hope this one will be on the top list thing to fix for Office:Next 😉

Thanks for the response, Erik. It’s greatly appreciated. My frustration (and the “impassioned” nature of my post) stems mostly from the fact that this is a situation that has been going on for years. I have been using Office for Mac since version 3.0, and I cannot remember a single version that did not have some glaring issues such as this one.
And that, in my opinion, is not just accidental. It suggests that there are serious systemic issues with the development and testing processes that Microsoft uses. And that Microsoft doesn’t take the issues seriously enough to do something about them. (The “pseudo-localization” reference was indeed just a starting point. I didn’t mean to imply that pseudo-localization should have caught that bug.)
Mac users are a pretty demanding bunch. They want great performance, attention to detail, and responsiveness. Most major Mac developers, including Microsoft, Adobe, and indeed Apple itself, appear to have serious difficulties meeting such expectations. But I am afraid I have to say that, of the bunch, Microsoft is most definitely the worst.
This bug with Postscript fonts and the non-breaking space is not an isolated incident. I could give you a very long list of very irritating bugs that have been there for years and haven’t been fixed. (You could also explore the “Microsoft” category on my blog to get an idea.)
Just to illustrate the on-going nature of the problem: PowerPower Point v. X had this horrible bug where using a “dead key” such as the circumflex accent on the French or French Canadian keyboard would cause the cursor to jump all over the place and make it impossible to type. Even if your French is not very good, you must know that the circumflex accent is an integral part of French spelling and is used in all kinds of very common words, including être, âme, maître, etc.
The bug was eventually fixed in PowerPoint 2004, but for several YEARS I as a French typist had to live with it in PowerPoint v. X (the English version; again, I don’t know whether the bug was ever fixed in the French version of the product; I cannot afford to buy two separate copies of every Office release).
This was the exact same situation: a glaring bug that mostly affected non-U.S. users and remained unfixed for YEARS. Was there a tester who went to graduate school at the most inappropriate time for that one as well? A simple human mistake? I think not.

Erik:
Thanks for the post and for your efforts. I agree with Pierre; there are a number of small issues that seem to linger with the Mac Office apps, and for those of us who work professionally with these apps these small issues cause continuous distraction. I hope that you and your colleagues can work out a reliable way to let the professional users communicate problems to you. I’ll bet that feedback channels usually have a low signal to noise ratio, so they may not be really very useful to you. As one suggestion, why not identify a small set of professional users (like Pierre) who really give your apps a continuous workout, and set up private channels of communication with them. These people will provide reliable problem reports for you that will be worth the time to try to address.

Hi,
I don’t really know anything about this bug so it would not be fair to comment, but I must thank you for being so honest in your blog. I wish more companies would encourage their employees to write blogs and this is something I must give Microsoft credit for.
Just a quick question – I was wondering if Microsoft had any plans to make a free Powerpoint viewer for Macs, as they do for Windows?

I must say it is sad to read so much “in your face” criticism in this blog over some bugs in products a company makes…
I mean, It suck to have bugs in a software app, or for that matter to have razor blades that gets oxidized because of the water (¿?), but c’mon, I don think a bloger should be accountable in his web page by any guy who has a keyboard, I think is it horrendous to go off to a guy just because he happens to blog about were he Works.
Every time I read this blog I think, that all (I mean ALL) the problems could be fixed with more people, because you are always taking a person away from something he is doing to finish some other stuff. A MacBU with a 100 persons or more, could EASILY, port Office to Cocoa, Get Messenger on par with Windows Messenger, revive explorer, fix all bugs you could ever imagine in all the previous apps, and do a WMP just for fun.
But he can’t, this guy must work with the card he has bean dealt and I don think he should be held accountable for that. (Maybe your employer, but that’s another topic)

Nadyne:
Thanks–that’s good to know. Still, I agree with Pierre–some of these problems are pretty obvious, and yet they persist for almost two years after the release of Office 2004. From the end user’s perspective, it is hard to understand what is going on when there is no information about whether the problem has been acknowledged and when or if it will be ever be fixed. Further, the updaters are accompanied with very little information about what the end users can expect when the updated versions of the apps. For instance, the last updater for Office 2004 was described as affording “improved stability” for Word 2004–that’s not enough information. It would be better to have a full technical release note, such as the ones that Adobe provides for the CS and CS2 suite, which lists all of the problems that were fixed in a given release. This would make it possible for an end user to issue another bug or problem report to remind you guys about the issues that remain.
Here’s just one issue that is on my personal radar screen: the unexplained lagging in the updating of the screen windows that accompany cutting and pasting or deleting blocks of text. I’ve got a pretty fast system, and still sometimes I wonder if I really did press the keystroke that I thought I did, because nothing happened, and then…. it happens. This behavior occurs even with a light load on the system, and it is highly disconcerting when one is concentrating on writing.
Thanks for listening, and thanks for keeping up the good work on the Office apps. We, the end users, are depending on you to do the best you can, and some better transparency about the process will make us feel better, I think.

Pierre (et. all),
You seem to miss the fact that a team of 50 odd engineers must work together across multiple campuses [with their supporting infrastructure as well], with a 20 year old, 20+ million lines of code [guess] project. Add in standard project scheduling, the humanity factor, actual knowledge of software development processes, some actual compassion and it’s all more understandable.
Perhaps you’ve forgetten you are but 1 in a sea of millions of users for Microsoft, Apple, and Adobe and that you’re individual issues and desires do not in any way shape or form map to the desires of the other millions other than simple statistical grouping?
I don’t work for the MacBU but I respect the fact that they maintain the largest Mac development team outside Apple and consistently deliver a product that addresses their customers needs.
“Mac users demand high quality” is a strawman. All consumers expect, nay, demand, high quality across any product. Flaggellating the MacBU to get your issues addressed using gross generalizations is sad.

I’m one of the developers of the OpenVanilla Project (http://openvanilla.org), which started as an input method framework on OS X. We have also hit the numlock bug quite badly, to the extent that Carbon Event Manager could mess up the whole thing and send totally wrong key masks to the input method component even after the num lock is turned off again.
In the end we had to ignore the numlock mask altogether and rely on virtual key codes to determine if the user is sending a numpad key to the input method component.
I’m glad to have found this article to support our findings. I’m attending this year’s WWDC (my first) and am hoping to present/reproduce this bug (along with a few more) directly to some Apple people…

Leave a Reply to Brian G. Cancel reply

Your email address will not be published. Required fields are marked *