Categories
All MacBU Personal

Vegas Mojo

Well, I’m a little late to the announcement party. Can I claim I’m just being fashionable? On Tuesday, the MacBU started up its new team blog called Mac Mojo! I’ll be one of the contributers there, along with a bunch of other current MacBU bloggers and several new folks. I see that Brad, Sheridan, and Joe have already dipped their toes into the blog ocean. So, come read us over there!
As for myself, I’ve been pretty quiet here. I’ve received a lot of great feedback on VB and made sure it got into the proper inboxes at work. Thank you all for your comments and feedback. Alas, there’s no way I can ever reply to every comment, but I have read them all. I plan to post some snippets of VB-to-AppleScript conversions so that people can get a rough idea of what you can do with AS.
That will have to wait a few days, however, because I’ve got a ton of stuff to do at work. I just got back this evening from a week’s vacation with my family in Las Vegas. Nope, no gambling, but we did spend a lot of time at the pool (my son now loves waterslides, provided I hold him as we slide down) and wandering through several of the mega-hotels on the Strip. My son had a very hard time believing that this was *all* Las Vegas. I think he thought the hotel itself was named LV, and that everywhere outside it was some other magical new place (Bellagio fountains, MGM lions, Mandalay Bay shark reef, Mirage volcano and dolphins, Caesars Palace Forum, it just went on and on…!)
So, after having been gone for a week, I came back to 772 new messages in my work inbox (110 per day!) that I need to cull through. Wheee!

Categories
All Blog

Sorry for the down time

My hosting service had a partition get corrupted during a normal server reboot early Monday morning, and it took over 26 hours for the fsck to verify the drive. Grrrrr.
It’s all back up now, with no lost content. Any mail sent to me in the last day or so will be trickling in today and I’ll try to get to it tonight.

Categories
All MacBU

So tell us what you want, what you really really want!

Thank you all for your comments about VB. There’s certainly a lot of passion, frustration, and outright anger being expressed here. For those of you with kind words, I greatly appreciate them. We’re reading every comment, no matter what the sentiment. I’ve personally seen MacBU leads reading your comments on my blog.
I had intended to post yesterday and answer some of the specific questions raised, but I’ve once again caught a nasty fever and cold from my preschool-aged son and didn’t get to it. Sigh… In lieu of that, I’ve got a couple of more general notes to make on yesterday’s post.
In my note on resources, I used a slightly extreme example of our headcount. I did have an opening for quite a long time, but our overall delta between having someone leave our group and hiring a replacement is probably something like 2 months or so. I have no idea how that compares to any sort of Microsoft- or industry-wide average. The complicating matter is that departures and candidates tend to happen in clumps, so we often end up with some abnormally long or short gaps in staffing instead of a more normal distribution. We’ve actually had a lot of folks very interested in working at MacBU (i.e, us having open headcount really is not due to a lack of people interested in working here). We hire world-class developers and they don’t simply grow on trees. For those readers offering a smackdown on the intelligence or capabilities of our developers, nothing could be further from the truth. We’ve even had some developers leave MacBU to go to other renowned Macintosh companies and then come back to MacBU because the grass really wasn’t greener elsewhere. We’re all proud to be here!
As for keeping VBA in PPC to run under Rosetta, that could be done but just vastly overcomplicates an already very complicated piece of source code. Rosetta only works on a per-process basis, so we’d be moving code and data out of the single app process into two processes. There are many many points in which apps call into VB and VB calls back into the apps (not to mention VB’s dependence on other lower-level shared library code that is now Mach-based) and marshalling all those calls and data cross-process is a heck of a lot of work. It gets even more complicated when you have more than one Office app running at a time, because then the new VB process would have to keep its internal state separate for each client app!
People have also offered the solution of porting the current Win Office 200x VBA over to the Mac. That’s pretty much a no-go too, because 1) it still has the assembly ABI problems, 2) makes calls to the Win32 APIs that don’t exist on the Mac (well, without WINE or any of the variants) and 3) relies on features in the Windows OS that don’t exist at all on the Mac. We’d have to port over large swaths of the Win OS and that might take even longer than writing VBA over from scratch.
VBA just wasn’t designed to be portable across architectures. The Mac and Windows implementations are very different, even though the actual language is the same. If I could go back in time about 15 years and tell the original designers to do it differently I would, but I can’t. Thanks to the heroic efforts of a few developers (Shuyi and Ying) we managed to Carbonize VB for OS X, but the move to Intel is gargantuan. We considered many, many alternatives. Each one turned out to be no simpler and as much or more work as the plain port would be.
None of this is work that we can’t do, it’s just work that would take away from everything else that we do want to do and would delay shipping Office 12. As I said on Tuesday, delaying Office 12 has other ramifications for cross-platform compatibility such as dealing with the new Office XML file formats and related features (like the larger cell grid in Excel, for example.)
Back to the idea of AppleScript… Folks who want to develop scripting modules on the Mac should check out AppleScript Studio. You can create GUI applications with it, and it is absolutely free from Apple. Also, a few bloggers have been wondering about scripting Office with Python and other languages. Yep, you can do that with the AppleEvent bridges available for those languages. We actually test Messenger in-house with a large suite of Python scripts that drive it through OS X’s Accessibility AppleEvent suite. We also drive Xcode via its AppleScript dictionary with Python scripts that send AppleEvents.
So, what now? Well, it is pretty obvious to me that the MacBU needs to provide some guidance for folks who currently use VB. The best way for us to provide the information that will help you the most is for you to give us some first. I encourage you to provide feedback to us — lots and lots of feedback! There are several ways you can do this:

  • Select Help/Send Feedback on ‘OfficeApp’ from within your favorite Office 2004 application
  • Click on this link and fill out the web form
  • Post a comment on any of the MacBU blogs
  • Send me email from this link and I’ll forward it into MacBU

Please give us specifics in your feedback. Just saying “I want VB!” or ranting about how we’re evil doesn’t provide us anything we can take action on. We’d like to know what you do with VB, what the goal was, did you share the macro with anyone, was it specifically cross-platform or just something on the Mac, what app was the macro in, and things like that. So yes, do tell us what you really want. Please keep it civil, it’s no fun reading through pages of insults. Your feedback directly helps us figure out the best ways we can help you.
[Aug 11, 9:26am — minor edit to correct the description of how we drive Messenger with Python]

Categories
All MacBU

Saying goodbye to Visual Basic

Well, its been less than two days since the MacBU announced that Visual Basic is being removed from the next version of Mac Office. The news has created quite a firestorm on many Mac forums (I’ve been scanning MacNN, Ars Technica, and a few others) and I received some very strongly expressed opinions about it in comments on yesterday’s post. I’d like to take some time to express my own views and experiences on the removal of Mac VB.
I should clear up one misconception about how the VB removal affects existing macros that has been making the blog and comment rounds. The removal of VB means that existing macros in Office documents will be round-tripped across file open and save, but you will not be able to edit them and you will not be able to run them on the Mac. Even previously compiled macros will not execute, because they have been compiled to PowerPC code that conforms to an older binary interface.
I want to say right up front that the MacBU is very aware of the pain this decision will cause for users, consultants, and enterprise organizations. I’ve personally seen the phrases “apoplectic with rage” and “absolutely livid” in two emails that crossed my inbox. Some people made comments on my post yesterday that were expressly clear about how this decision would drive them to one of the free Open Office variants instead of buying Mac Office 12, and other posts in other forums made similar statements. I’m sure some people will indeed decide that lack of VB is an absolute deal-breaker and they will plan to use other software. I’m truly sorry if that is the case for you.
The MacBU did not make this decision lightly. I personally spent several weeks digging into the VB source code to identify and plan what work would have to be done to move it to Xcode and make it universal, and I had several long discussions with our product planning folks to help our group leadership weigh the costs of doing the VB port vs. the costs of not doing it. I’ll try to lead you through some of the analysis here.
From my perspective, Mac Office has two primary driving requirements:

  1. it must be as Mac-like as possible, use Mac features, and take advantage of the Mac operating system, and
  2. it must be as compatible with Win Office as possible, and share as many features and commonalities as it can.

(We’ve got other requirements and product visions, but as I see it, they really act to refine these two basic needs.) As you may imagine, these two goals are many times not perfectly aligned. In the worst cases, they may actually be diametrically opposed, and we have to wrestle with making the best decision we can, knowing full well that whichever way we go it will help some users and hurt others. This VB decision is one where we’re truly caught between the Mac rock and the Win Office hard place.
VB on the Mac exists for cross-platform compatibility. There is no other software on the Mac that also uses VB, so it doesn’t help Mac Office integrate with other workflows based purely on Apple solutions. Thus, any work we do on VB only serves to satisfy one of the two major requirements. Doing that work then means we have less developer time to continue to improve Mac Office’s use of Apple-specific technologies (or tools, such as Xcode.)
Let me describe for you some of the technical challenges that would be involved were we to try to port VB to Xcode and to the Intel platform. For those of you reading who are not developers, bear with me for a little bit. Hopefully you’ll at least get a sense of the scope of work even if you don’t quite follow the nitty-gritty details.
VB on the Mac is really three parts: VBE (the editor), VBA (the execution engine) and Forms (the buildable windows and controls you edit in VBE and see when running a macro.)
VBE is pretty standard C++ code. However, the code is generally very old — it was originally designed and written several years before I came to Microsoft in 1996. VBE contains the top-level parser that converts the text of a macro into a series of mostly machine-independent opcodes (kind of like Java bytecodes, but not exactly the same). Thus you can’t just hook an external text editor up to VBA, because of the upper-level dependency. The VBE code actually isn’t too hard to port to Intel, but it is tricky to port to Xcode/GCC because of the age of the code. As I mentioned in an earlier post, GCC is very picky about code meeting the current standards and the VBE code most certainly does not. That’s not to say the code is ‘bad,’ it was just designed and written long before current modern C++ standards.
VBA, on the other hand, is incredibly difficult to port to Intel. The execution engine basically runs through the previously mentioned opcodes and, well, executes them. The hard part is that ‘executing’ them doesn’t mean interpreting them, it means converting one or more at a time into a block of assembly generated at runtime that looks and behaves like a regular function that can be called directly by other normally compiled code. This is in essense ‘self-creating’ code, and VBA is constantly flushing the CPU’s code cache in order to mark these chunks of data as executable. VBA’s generated code must adhere to the Application Binary Interface of the host platform (historically PowerPC and the Code Fragment Manager). This means register allocation, stack alignment, parameter passing locations, etc. VBA is basically a compiler that emits code at runtime. It does so by running a large state machine that tracks PPC register usage, stack location, mapping between PPC registers and VB variables, etc and then concatenates large blocks of pre-generated assembly together. VBA subsequently tweaks the assembly bit-field by bit-field to do things like assign registers to each opcode, set branch addresses, and create transition vectors for all function calls. The templates are very PPC- and CFM-specific and the state machine is designed for architectures that allocate static stack frames and pass parameters by register, unlike Intel which has dynamic stack frames (you can push and pop data to/from the stack any time you want) and parameters are passed on the stack. So, for us to port this to Intel we’d have to rewrite the entire state machine and create brand-new templates of IA-32 code. That’s basically writing a rudimentary compiler almost from scratch (we’d at least have the initial parsing and machine-independent opcodes already done.) Again, this is all a design that long predates me or most of my peers in Mac Office, and is code that we inherited when we created the MacBU (i.e, none of us wrote it in the first place.) There’s nothing inherently bad about the code, it was just designed for the constraints of the day and that design simply doesn’t lend itself to being architecture-independent.
Some folks might ask why not just port the Win Office VBA over to the Mac? Well, VBA circa Win Office 97 (which is the closest Windows VBA to what we have on the Mac) doesn’t implement their execution engine this way at all. Instead, they have tens of thousands of lines of IA-32 assembly that directly implements all of the opcodes. That assembly does so according to the Windows Intel ABI, which is different from the Mac ABI in several important ways (the specifics of which are described here.) Also, the assembly is in MASM format which is close to but not the same as NASM as supported by GCC. So, we’d have to edit the source to be compilable by GCC, and scrub it line-by-line to find and adjust the parts that aren’t compliant with the Apple Intel ABI. We’d also end up with two completely different implementations of VBA (PPC state machine and Intel straight assembly) that we’d have to maintain and keep in sync. That would be horribly bug-prone.
Lastly, we have Forms. Forms is also C++, but is backed by several thousand lines of gnarly custom assembly. This assembly ‘allows’ the C++ code to swap object virtual function tables and individual member function pointers between objects on the fly, to essentially do very fast object morphing. To do so, the assembly has to have specific knowledge of aspects of the C++ compiler (vtable layout, implementation of ptrs-to-member-functions, etc) and has to work in lockstep with the compiler. I spent almost two weeks massaging this code to try to make it compatible with just the PPC Mach ABI, which is only slightly different from the PPC CFM ABI. Even after all that work, I still didn’t get it completely right and internal builds had some really bad stability problems. We also don’t even have the Win Office 97 Forms source code, so I was not able to compare our code to how it was implemented for Windows.
I just noted that the assembly has to work hand-in-hand with the normal C/C++ compiler. That wasn’t too much of a problem when we were using CodeWarrior, as the C++ compiler only changed in small ways every few years or so. With Xcode and GCC, my understanding is that Apple has to merge in all the changes that external developers commit to GCC, and we run the risk of GCC changing much more frequently. That might not be a problem in reality, but the risk is non-zero and we have to take that into account.
One final problem is that all of this custom assembly is currently PPC 32-bit, and even the corresponding Windows assembly is Intel 32-bit. If we ever want to make a 64-bit native version of Office, any work we might do to solve all of the above problems would have to be done all over again.
So, in short: VB has lots of code and assembly that specifically assumes it is running on a PPC with the Code Fragment Manager, and to re-do it for Intel would involve writing a rudimentary compiler and relying on private compiler implementations that are subject to change at any time.
Whew, that’s a lot of technical stuff. I hope it provides some idea of the scope of work we were facing. We estimated that it would take roughly two years to of development time to move it all over to Xcode and to Intel. That would mean two more years before the next version of Mac Office made its way to consumers. In the meantime, Leopard will ship and Mac Office 2004 would still be running in Rosetta. Win Office 2007 and the new XML file formats will be ever more common. All Mac Office users would still be stuck with the old formats, unable to share in or use the great expansion of capabilities these new file formats bring. During that time, we’d also not be adding any other items our users have asked for.
Beyond that, if we were to port VB over to Intel in those two years, what you’d end up with is VB for Mac just as it is today. It still wouldn’t be feature-comparable to VB in Win Office, and the object model in Mac Office would still not be the same as the one in Win Office. That means that your macros would still be restricted to the same set of compatible items as you have today. Over the last 10 years, the Win Office programming model has become very different from that of Mac Office. We’ve tried to keep the object models in sync for the features that we have ported from Win Office, but we haven’t ported everything.
So, given that the developer cost was huge, that the consumer cost due to the delay while we did the work was quite large, and that the end result would be no better than what we have today, we made the very difficult decision to invest our time and resources in the other pillar of Mac Office, namely taking advantage of Apple tools and technologies to be more ‘Mac-like’. We’ve continued to improve the AppleScriptability of our apps (many many bug fixes post-Office-2004) and as announced are looking into adding some Automator actions to the suite. We’ve completed the rest of our transition to Xcode and to Intel and are forging ahead with the rest of the product.
I think a common question might be ‘if the cost is so huge, why doesn’t Microsoft just devote more resources to the problem? They’ve got a ton of cash, right?’ Well, the real question is ‘what resources do you throw at the problem?’ We’ve been working very hard to hire a bunch of developers, but it has turned out to be quite difficult to fill our existing open headcount positions. As an example, I’ve had an open position on my own team for 9 of the last 12 months (it took 8 months to fill the slot when one developer moved from my team to another one in MacBU, and only last week did we hire someone to fill the slot vacated recently when another developer moved to a different team at Microsoft.) The question of how Microsoft allocates developer headcount and funding to MacBU is a separate topic of its own which hopefully I or some other MacBU blogger will tackle later. In any case, there’s no point in adding new headcount to the MacBU when we haven’t yet filled the positions we already have open.
I know that explaining all this doesn’t make the fact of VB’s death any easier for those users who currently depend on it. As I said at the beginning, we in the MacBU really are aware of the difficulties you face. Our product planners, program managers, developers, and testers are working to alleviate some of that pain. Many people have only a few simple macros they use, and I do want to point out that those macros will translate very easily into AppleScript. Even large macros can be rewritten in AppleScript, although that takes some time and definitely some knowledge scripting on the Mac. The AppleScript object model and the old VB object model for our apps are roughly equivalent, so apart from the syntactical differences, if you could do it in VB you can do it in AppleScript. While I can’t comment on any more specific feature work for Office 12, I’m sure we will be working closely with enterprise customers to help them address their concerns. We’ll be saying more about our scripting plans as we get closer to the product release for Office 12.
For those of you contemplating a switch to Open Office, I don’t know if Open Office has any support for VB macros or other OLE Automation technologies so I don’t know if you’ll be any better off from a cross-platform perspective. You probably can’t be worse-off except that Open Office certainly doesn’t support any of the Mac OS scripting technologies that Mac Office does support and in which we will continue to invest, nor will it (at least for a while yet) support the new XML-based file formats. If you do switch, we’ll miss you.
Many people have viewed this announcement by MacBU as a sign that we are out to screw the Mac community, or that we’re just looking for an exit strategy. We’re not. Most empatically, we’re not. This decision was agonizing. My manager even said he felt ‘sick about the impact on those who really rely on xplat [cross-platform] VB support, particularly in Excel where we see it the most.’ In my post yesterday, I said that I wasn’t so sad to see VB go. I said that from the perspective of a developer who’s worked to maintain the code for many years. However, there’s nothing good about removing a feature that many people rely on, except that it frees up resources for us to invest more heavily in other important areas. Due to the age of the code, VB has been a very large drain on our resources for a long time with relatively little return. A couple of months ago I wrote that I hoped my blog would help people trust the MacBU a little more. I can see that many of you are very mad about this decision; I do hope that my post today helps you see some of the issues behind the press release. We had to make a hard decision one way or the other, and this is how it turned out.
(Please read my next post and consider giving us specific actionable feedback. Thanks!)

Categories
All MacBU

News of the Day

You’ve no doubt seen the news from WWDC today. Yes, it is true — the MacBU is halting development on Virtual PC and is discontinuing support for Visual Basic in the next version of Mac Office.
I’m sad to see VPC go. It was an amazing product to bring into the MacBU back in 2003 (I’d used it myself since version 2 when it was made by Connectix) and it still has features that Parallels doesn’t have. The biggest thing Parallels lacks is the ability to script the main app and the virtual machines. VPC let you query VM state, launch programs inside it, define shared drive mappings, and all sorts of other great stuff via AppleScript. You can’t do that yet in Parallels. Hopefully they’ll add scriptability someday soon. Them or VMWare!
I’m not so sad to see VB go. That code has been a major headache to maintain over the years. Lots of people are bemoaning the sudden lack of cross-platform scriptability, but to be bluntly honest, VB for Mac Office hasn’t been remotely compatible with VB for Win Office for years. Even back in Office 98 the VB IDE for the Mac had several major features cut compared to Win Office (watchpoints, etc) and the object models for the two platforms have diverged wildly in the 10 years that have gone by. I’d like to blog more about VB, but not tonight. It’s a long and sordid tale.
One positive thing that seems to have been lost in the rabid chatter about VPC and VB is that Mac Office has finally made the complete switch to Xcode! The last major checkin happened on Friday afternoon, and all of MacBU is finally switched over. Yay! That only took a month and a half longer than I guessed, back in early June… We’re still not ready to ship yet; we’ve been working on lots more than just the recompilation and those changes are not yet complete. I myself am hip-deep in the new file-format-converter compatibility work that we’ve announced.

Categories
All Personal

Long time no post

Sorry for the long delay in posting… When I went back to work a few weeks ago, I was still getting up at least twice in the middle of the night to feed my daughter. After a full day of work and playing with my son before his bedtime, I was absolutely exhausted by 9pm and pretty much crashed then. Now my daughter is down to eating only once during the night (!) but my son brought home his first cold from preschool and promptly gave it to my wife and me.
That was back on July 4th. I’ve had a cold since then, and a few days ago it got much worse. I had no energy, a bad cough, and my head hurt all the time — especially when I coughed or leaned over to pick up my son or daughter. I even took some time off of work because I felt so horrible. My wife finally told me to go see the doctor, and it turns out my cold turned into a sinus infection. Ugh. So now I’m on some massive antibiotics, and I’m finally starting to feel better. I have a post I’ve been mulling over for 3 weeks now, and hope to get it written up soon. I’ve got to have the energy to stay up past 10pm to get it done, though!

Categories
All Blog Personal

I'm not dead yet!

Just really busy and tired. I’ll hopefully get some post up this weekend. Sorry for the long wait!

Categories
All Blog

New contact page

I’ve added a new ‘Contact Me’ page to the sidebar on the right. You can use it to send me a private email if you wish to make a comment that will not show up on the blog.
Hopefully I won’t get too much spam.

Categories
All Blog

Small clarification

I realize in reading some of Pierre Igot’s latest posts that the intent of the title of my last post on bugs was not very clear.  In the absense of some specific cultural knowledge of American popular music of the early 1980s, my title “Bugs stink! Yeah Yeah!” could very easily be interpreted as condescending or as a brush-off of user concerns.
That is not what I intended, however.  See, there’s a song called “Love Stinks” (from an album of the same name) that was performed by the J. Geils Band in 1979 or 1980 that was very popular on the radio when I was growing up.  The chorus of the song is roughly this:

(Love stinks)
Love stinks yeah yeah
(Love stinks)
Love stinks yeah yeah
(Love stinks)
Love stinks yeah yeah
(Love stinks)
Love stinks yeah yeah

where each of the “yeah yeah” bits is an affirmation of the sentiment that “love stinks.”
That’s the spirit in which I used the lyric as a title for my post — bugs do stink, and there is much agreement about that. If you look at my post titles, you’ll see that many of them are silly puns or simple plays on words. As I was writing my last post, “Love Stinks” was playing on the radio and I thought it might be a silly little pun to use.
So, if my post title sounded like a brush-off or a put-down, I’m sorry. I definately didn’t intend it to be one.

Categories
All MacBU

Bugs stink! Yeah Yeah!

(my apologies in advance to the J. Geils Band)
Pierre Igot writes what I must politely describe as an impassioned discourse about my description of pseudo-localization, multi-lingual bundles in OS X, and MacBU testing. Pierre makes some good points, totally misses the idea of pseudo-loc in another, and generally castigates the MacBU for failing to fix a particular bug that is very important in French typography. He then invited me to comment.
Let me begin by saying what I am not. I am not the lead developer for Word, and I am not intimately familier with every bug that is entered against our products. I am also not a native French speaker, nor am I familiar with the rules of French typography. (In fact, my peers in high school often told me “Tu parles français comme une vache espagnole,” which, if my hazy memories of 1991 are correct, means “You speak French like a Spanish cow.”)
I do, however, know quite a lot about bundles in Carbon and Cocoa, how multi-lingual bundled apps should behave, and the benefits to users of software that takes advantage of them. Apple did indeed make them available to developers using Carbon in 2001 or so. Pierre is correct in saying that currently shipping Mac Office apps do not use them. (Well, Messenger does use them, and may have been the first major Microsoft app to do so.) I became a lead for Mac Office in June of 2002, well into the Office 2004 project and was not a part of the initial schedule planning that, for whatever reason, did not include moving Office to a bundled architecture. As the lead for localization, I understand and sympathize with users for whom this creates frustration. The reality is that for every user calling for a multi-lingual version of Office, there are at least two users who don’t care about that at all and want us to be working on something else. We can’t please everybody, unfortunately. As I said in the comments at the end of my localization post, I cannot comment on our plans for this aspect of the next version of Office, but please rest assured that I do understand and hear your request, Pierre.
Because I am not well-versed in French typography (or any real typography for that matter) I can only trust that Pierre’s assessment of the importance of the non-breaking space in French documents is indeed high. Assuming that is the case, then yes, this must be a very aggravating bug for anyone who uses Word 2004 to lay out text in French. I’m quite sorry, Pierre, that you have been so frustrated by this issue. I wish we had caught the bug and fixed it before shipping. Sadly, we did not.
Pierre then takes our non-use of bundles and conflates that with the existence of the non-breaking-space bug:

The fact that the language of the user interface in Office applications does not change depending on the user’s preferred language in Mac OS X is annoying enough. But that’s not the worst of it. The worst of it is that Microsoft’s process for fixing bugs is completely screwed. And there is no better example of this than the bug with non-breaking spaces and PostScript fonts in Word 2004.

He also implies that our use of pseudo-localization should have caught this ‘elementary’ bug:

The fact that Microsoft did not catch that bug in Word 2004 before they released the product is a clear illustration of how flawed their testing processes are. Whatever benefits this “œpseudo-localization” technique described by Erik Schwiebert provides, it is clearly not good enough to catch even such elementary bugs.

Here I must correct Pierre’s faulty assumption and say that the bug is not in any way connected to either of the other two items. Multi-lingual bundles are all about providing resources to the app, such as strings, dialogs, menus, windows, etc. Pseudo-localization is a method we use to test the localization of those resources and the code that uses them. It is intended to help find visual glitches, missed translations, improperly displayed dialogs, and similar items, and it does a pretty good job at optimizing our efforts for those issues. It is not designed nor intended to help with finding or fixing bugs related to the actual behavior of the application. Pseudo-localization in no way replaces good old-fashioned usage of the applications to uncover code defects.
In fact, the bug that Pierre is so frustrated with occurs in all languages that Word ships in — the faulty code is in no way connected to any localization. I can repro it in the English build. That doesn’t mean that our entire testing process is flawed, however. Every large software endeavor results in a product that has bugs. Microsoft is no better or worse than anyone else in this regard. Seen any OS X software updates lately? 🙂
Now, the fact that the bug is unrelated to our localized UI testing doesn’t address the issue that the bug is there, and is bad. Pierre blogged about the bug way back in September 2004. Unfortunately, the MacBU (and the rest of Microsoft, to my knowledge) has never had a good method for customers to report bugs to us. We’ve had the (now defunct) MSWish email address, and we do have the new direct feedback link on Mactopia, but we don’t have anything like Apple’s BugReporter tool. Interestingly enough Nathan Herring, another MacBU employee, just blogged about this specific issue and has some links you may want to read.
But let me continue with Pierre’s post. He finds it unjustifiable that we have not fixed this bug in the almost two years that have gone by:

Now what excuses does Erik Schwiebert have for this sorry state of affairs? I might find it acceptable (barely) that Microsoft is not able to catch such bugs in the hectic schedule that leads to the release of the initial versions of the product. But how can they justify not fixing the bugs in the next two years, even though the bugs are so obvious?

says it can be done on schedule:

it is perfectly possible to release multilingual software on schedule that automatically supports all of Mac OS X’s supported language without requiring the user to buy separate versions for each language and download separate updates for each language.

and casts shame on us:

And for all this, once more, all I can say is: Shame on Microsoft, and shame on the MacBU. They have absolutely no excuses here.

Ouch. That hurts. Me personally, as well as all of our hard-working employees. It’s a good thing I have a relatively thick skin.
I don’t think there is anything I can say that will appease Pierre. And I don’t mean that there’s something I know that I’m not allowed to post; I just don’t think that any insight I provide into MacBU’s bug process will soothe his wrath. However, I will try to shed some light on our processes because I think folks are interested in them anyway. I’m sure many people will individually disagree with many of of the decisions we make and the factors we weigh, but that’s part of the openness of blogging. Feel free to respond with your thoughts and comments — you may provide some insight that we’ve missed in all these years.
First of all, we do work darn hard to fix bugs in our products. Really darn hard. But, we’re not perfect. No developer team is. Not MacBU, not Microsoft, not Adobe, not Apple. Security defects? We’ve all got them. Software updates? Yep, them too. For OS X, we have 10.0.4, 10.1.5, 10.2.8, 10.3.9, 10.4.6 and counting. For Office we have 11.1.1, 11.2.4, etc. Updates are good, it means that we’re all working to make our products better, to fix the bugs and reduce the crashes.
However, not all bugs are fixed in dot-releases of products. Some bugs that we find are really obscure and don’t cause any real harm. For example, perhaps the File/Open menu item is grayed out when you hold down Cmd-Shift-Ctrl-Option and triple-click on the menu-bar. Not too many people will see that, and if they do, no real harm done. We can spend our engineering time and resources better elsewhere.
Some bugs are easier to hit, but fixing them may have a high risk of regression and may cause another bug that is even worse. One example might be a performance bug in Excel’s recalc engine. We could fix it and make things faster, but currently the code calculates correctly (albeit slowly) and any fix might totally break all function dependency analysis. It is sad but safer to leave things alone.
Other bugs are deemed worth fixing for a future release, but not worth back-porting to the current release. Every code change we make has to be tested around to make sure no regressions are caused, and that takes away from other work we could be doing. It may be more effective for us to take a bug that was reported against Office X and just fix it in Office 2004. And occasionally, some bugs are misclassified or accidentally forgotten. After all, we’re only human.
Here’s an example of an engineering decision. If you boot Mac RDC on a PowerBook and connect to a Windows server, try turning on Num Lock (the keyboard light should be on). Type something on the number pad in Excel. You get numbers, right? Good. Now, Cmd-tab back to the Finder and turn off num-lock. Cmd-Tab back to RDC. Type something on the number pad again. What, you still get numbers? Yes indeed you do. Geez, the MacBU really should have fixed that bug, right? We’d love to have fixed it, except that the bug is actually in the OS. The Carbon Event manager does not give you accurate information about the Num Lock state when you query for modifier key state after app activation and deactivation. I filed that bug with Apple in May 2002 against OS X 10.1.4, and they still haven’t fixed it. I wish they had, but I’m sure there is some reason (hardware? software complexity? who knows…) that the change is more expensive for them to make than to allow the bug to continue to exist. That decision is completely up to them, and I respect that.
In our move to Xcode, we’ve reported over 100 bugs to Apple. They’ve fixed the majority of them, and have postponed several to some later release. Some of the bugs they’ve postponed have been real bummers, but they’ve made the engineering justification to delay the fix and again, I respect that. After all, we have to do the same thing with regard to Office bugs too.
Second, the ability to meet a schedule or not has nothing to do with the ability to ship a bundled multi-lingual product. That is essentially a feature, and does not come for free with Carbon apps. (Yes, that is one aspect where Cocoa wins hands-down — it is impossible to create a monolithic Cocoa app. At the very least your app must be bundled, whether or not it has more than one locale for resources.) Perhaps Pierre would have preferred we have chosen a different schedule, moved Office to Mach and multi-lingual bundles, and shipped a product called Office 2005. We hit our internal schedule for Office 2004 with the features we planned for it.
Now, I’m not saying that any of this is an excuse. We’ve made lots of conscious decisions over the years about many many issues. Some of them, everybody likes. Some of them, everybody hates. Most of them are liked and hated simultaneously, and the overlap between the sets of people experiencing the two emotions is rarely the same from topic to topic.
The sad thing about this particular bug is that it falls into the category of ‘oops.’ The code was wrong in a very small way (an exception was made for the breaking-space character and accidentally forgotten for the non-breaking space character.) We actually found this bug in-house just a little while after Office 2004 shipped, and fixed it internally for the next major release of Office in August 2004. We even noted that it was important in the French market. Somehow, though, it was never flagged as a bug that should be migrated back to Office 2004. The tester for that area of Word (who found and opened the bug) left MacBU to go back to graduate school during that summer, and I think we probably just missed this one in the transition. A simple human mistake. Shameful? I personally don’t think so. Unfortunate and frustrating, I’ll certainly agree to that, but not shameful.
I’ve asked our Word team to take another look at this bug. I make no promises, but maybe we can get it fixed sooner than in the next major release of Mac Office. After all, the Windows team did that for a blogger recently.
So Pierre, we are listening. I think pseudo-localization has a greater value than you place on it, but for a different purpose. If you are ever in the Seattle area, please look us up. I’d be happy to take you out for coffee and a chat, and see if we can address some of your dissatisfaction with us. In the meantime, I’ll stand up proud and accept no shame, for I believe we work hard and produce a good product.