I’m very happy to note that with the release of Snow Leopard (aka Mac OS X 10.6) today, Apple has made significant improvements to the Spaces architecture and Office 2008 SP2 now participates quite nicely with the Spaces feature! We worked with a number of people at Apple to identify the set of issues involved and had some interesting back-and-forth discussions on what changes needed to be made where.

So, if you’ve been avoiding using either Office 2008 or Spaces because the two don’t mix well, you can now have the best of both worlds by installing Snow Leopard.

Some bugs are easier to hit, but fixing them may have a high risk of regression and may cause another bug that is even worse. One example might be a performance bug in Excel’s recalc engine. We could fix it and make things faster, but currently the code calculates correctly (albeit slowly) and any fix might totally break all function dependency analysis. It is sad but safer to leave things alone.

Let’s look at performance as an example. Over the ten or so months that Office 2008 has been available, we’ve received a lot of feedback about the suite. One of the most common areas mentioned is performance, such as the amount of time it takes Office apps to boot, the amount of time it takes a chart to draw, or the amount of time it takes Excel to recalculate a worksheet in particular. Each of these three areas are places where we’ve actually put in some significant research and investment in improving Office 2008 performance. Each of the Office 2008 updates (12.0.1, 12.1, 12.1.1, 12.1.2, and 12.1.3) have contained a number of performance improvements, among many other fixes. That said, we’re continuing to make additional improvements. Some of the code changes needed are invasive and we’ve had to evaluate the risk-vs.-reward tradeoffs and make decisions to delay these changes until a more appropriate time.

Let’s look at one of these scenarios. Pretend you are working in Excel and want to create a simple line chart with a large data set that contains as many as 30,000 data points. I’ll presume that you have a pretty good monitor, perhaps running at a resolution of 1600×1200 pixels. Now, your chart probably doesn’t actually fill the entire screen from left-to-right — after all, you need space to show the columns of data that you are working with, right? So let’s further assume that your chart is roughly 1000 pixels wide. That means you are displaying 30,000 data points in 1000 pixels, or about 30 data points per pixel. Even if not all of those data points fall directly on top of each other in the y-axis, the only possible line to draw to connect all 30 of those points in one vertical pixel slice is a single line up and down and up and down again, on top of itself many times. Your chart looks kind of smushed together. Or, let’s pretend for the sake of simplicity that all those 30 data points are approximately the same on the y-axis, and you just get one single pixel, drawn over and over and over again. And, as you may have noticed, the current charting engine that we share with Windows Office is rather sluggish as it faithfully draws that line or pixel over and over and over again.

Now, the fact that 30 data points end up in one pixel slice on the x- axis isn’t the charting engine’s fault, as it didn’t choose the data set or the size of the chart — you did, or you at least accepted the charting defaults. Now, to improve things, you could stretch the chart out to be a full 1600 pixels wide, right? But that doesn’t help much; you’re still at almost 19 data points per pixel. Or, if you had $17,207+tax on hand, you could set up a Mac Pro with four ATI Radeon HD 2600 XT video cards and eight 30″ Apple Cinema Displays (2560 pixels of horizontal space per monitor) for a total horizontal space of 20,480 pixels, which is still not enough space to achieve one data point per pixel! Ok, so perhaps it’s not reasonable to ever expect that such a data set would ever have more than one pixel available for each data point. What would you do? How might you redesign the charting engine to handle this?

One option might be to only plot at most one data point per pixel. But, how do you decide which of the 30 data points is the best one to plot? The first? The middle one? The average value? Do you add math to best-fit the overall line, which adds processing time, which is what you are trying to cut down on? I’ve only mentioned one single possible change, and there are many. The point here isn’t to actually craft a redesign in this blog post (or in the comments either…), but to demonstrate that a redesign can be a complex thing — I’ve posed some design questions and probable decision points already, and I haven’t even considered how the current code is actually designed. (Neither have you, since you don’t have access to it!)

This example is just a very long way of saying that whatever redesign is planned, it has to fit into the current architecture of the code or be willing to accept the risk of changing that architecture and possibly breaking something else. That breakage may in fact be worse than the current bug. If we make a major change to the code, we run the risk of introducing a variety of different possible bugs — incorrect visual representation of the data, or a crash, etc. Testing architecture changes is particularly challenging because it’s impossible to craft a series of tests that exercises every possible type of data that could be charted, and if we miss one edge case we could let a critical bug slip through that is worse than the current problem. We’d rather make the change when we have sufficient testing resources lined up for a longer period of time than we usually have for a dot-release. That way we have all the test coverage needed to validate the change, and is why a bug fix in a feature as important as charting may be delayed.

Bear with me while I shift gears now, and talk about another issue that the MacBU has heard a lot about: Office 2008 and the OS X feature called Spaces. If you read through the links in that previous sentence, a couple of themes pop up:

1. Mac Office 2008 doesn’t work properly with Spaces
2. It happens most often in Word or when the Formatting Palette is open
3. People rarely see the bug in non-Microsoft applications
4. People assume the Mac Office 2008 code base is the cause of the problem

Let me give you some of the background of the Formatting Palette, to help explain why the problem shows up so much more readily in Office 2008 than in Office 2004 or in other applications.

When people talk about the “Formatting Palette” in Office 2008, they usually mean the Toolbox window. The Toolbox is actually two separate windows, bound together by Carbon Window Groups. The first window has the title bar and the row of buttons across the top (the buttons that toggle between the Formatting Palette, the Scrapbook, the Reference Tools, the Object Palette, and whatever else is there that I can’t remember off the top of my head.) That first window is a true floating window created by OS APIs. The second window is everything below that row of buttons, and is the instantiation of one of those toolbox items. These windows are slightly customized, in that we tell the OS to create them with no border or shadow, again through OS APIs. When the Formatting Palette is showing, you’ve actually got the root toolbox window showing first and then the FP window bound tightly to it, on top in the z-order. If you click on the Scrapbook button, the FP window is destroyed and a new window is created to hold the scrapbook, and that new window is bound against the root window. I think that Spaces and Exposé don’t take the window bindings into account (my understanding is that they manipulate windows at the Core Graphics level, which is a lower-level private system interface upon which both Cocoa and Carbon windows sit), and that is why Spaces and Exposé seem to get confused by the root floating window and the upper child window.

The reason MacBU uses this window separation is that most of these child windows are hosted in different modules of code, most of which have their origins in different architectures. The Carbon Window Group APIs allow for very rich and precise control over how windows are presented to the user, and gave us the ability to combine UI from a variety of sources in our codebase with minimal rearchitecting of each of the individual components. The Scrapbook window, for example, is a PowerPlant window because it actually lives deep in the Entourage code (due to the fact that Entourage is currently PowerPlant-based, and that was the easiest way to get access to the Entourage database). PowerPlant is very picky about owning its entire window, which is why we use a separate window here — it misbehaves rather badly if you try to put PowerPlant objects in a sub-frame of a window that is not fully under PowerPlant’s own control. The Formatting Palette is actually a special instantiation of the toolbar code, which has its own assumptions about the sort of window it lives in, and the Compatibilty Report is actually an instantiation of what was originally a modeless dialog.

We have long-term plans to overhaul the entire architecture of the Toolbox and all its clients to use Cocoa, but that didn’t happen in 2008. The Cocoa AppKit window APIs do not yet contain functionality that supports the full richness of window management features that the Carbon APIs do. The Toolbox and its use of Carbon Window Groups were introduced in Office 2004 and predate both Spaces and Exposé. The Office 2004 Toolbox has the same issues with Spaces and Exposé, but you only notice it if you show the Toolbox. In Office 2004, the Formatting Palette was separate from the Toolbox, so the Toolbox was not shown by default. In order to reduce the amount of screen space the Toolbox and Formatting Palette obscured at the same time, we merged the two together early in the 2008 cycle, long before Leopard and Spaces were demoed or available for us to test with in beta.

After we received a beta of Leopard with Spaces, we tested our apps and identified a number of issues that our apps have with the feature. We had an engineer spend several days digging into these issues. He did some serious spelunking into our windowing code and determined that we were not moving the windows incorrectly in our code so we reported them to Apple to investigate. Apple has fixed a number of problems with Spaces in OS X 10.5.3, but some still remain.

So, let’s circle this discussion back to the point I opened with, and the real point of this post: some changes are just too risky for dot-releases, and every company that writes software has to deal with that. Microsoft does and certainly Apple does too. I don’t know the Spaces code — as I don’t work for Apple, I’ve never seen it. Microsoft and Apple work together to troubleshoot customer issues as situations warrant, and as part of that joint effort I’ve spent some time talking to some of the Apple Carbon and Spaces developers over the last few weeks about Spaces. We wanted to see if there’s any sort of change we could make in our code to avoid the issues. If there was anything we could reasonably change in our code at this time I would love to do so. However, changing our windowing system to not use Carbon Window Groups entails a complete rewrite, and is not something we can feasibly do in a bug-fix release. Given the direction Apple is taking with OS X, any significant rewrite of our windowing system should be done in Cocoa, and that is a tremendous task to do in a dot-release. It would almost certainly cause other serious bugs as I alluded to in the charting example above. The Apple developers I’ve spoken with have been unable to come up with any simple code changes to that we could make to work around the issues, and have indicated that our code is generally acting correctly.

For now, you’ll have to wait for Apple to release an update to or new version of OS X where Spaces works with apps that use Carbon Window Groups. Our User Experience team has put up a brief help topic about the issue. We’re keenly aware of our customers’ frustration with Office 2008 and Spaces, and we will continue to work with Apple to help find a solution or workaround. Please continue to share your product experiences with us on Mactopia, or privately with me if you wish. As long-time software developers, it’s frustrating to me and to my peers when we’re unable to fix every problem instantly. Improving your experience with Mac Office on a continuous basis is part of our job. Sometimes we can do it with a quick fix, sometimes not. Your input helps us get it done.

(Edited to clean up the extraneous formatting tags in the last paragraph. Also fixed spelling of Mactopia.)

Rick Schaut, one of our long-time developers in the MacBU, has a few blog posts about what it’s like tracking down a bug from a developer’s point of view (see Repro, Man and Anatomy of a Software Bug on his blog).

On a related note, I haven’t forgotten about my promise in the comments on my last post about the Office 2008 updaters. We’re still looking at some things internally. I’ve received a number of emails listing details about user setups and what they think the problem is, but when I’ve tried to reproduce the same setup and problems at work, the updaters function just fine. So, I don’t have anything concrete to offer yet. If you are running into updater problems, please consider reinstalling from your original DVD, and keep careful notes about what you do as you run the updaters (see, there was a point to the initial paragraph and title!) so that you can tell me/us as much detail as possible.

Shortly after the update went live, I started seeing recurring reports of people being unable to apply the update. Typically, the updater starts to run, and then mysteriously (from the user’s perspective) says “You cannot install Office 2008 12.1.2 Update on this volume. A version of the software required to install this update was not found on this volume.”, even though the user has Office 2008 installed and has been using it.

We’ve seen these reports regularly, and they almost always happen because of one of four reasons:

1. The user has renamed the Office 2008 folder or some application name inside it
2. The user has deleted some part of the Office 2008 folder they didn’t think they needed
3. The user has run a tool such as Monolingual to remove extra languages from Office
4. The user has run a tool such as Xslimmer or Monolingual to remove extra code architectures from Office

In an ideal world, our installers could handle these cases and complete the install, but installers are complex beasts, and in the interest of reducing complexity, doing any of the above actions is unsupported. Let’s get into each of the issues:

Supporting the renaming of the folder or an app adds complexity because then we have to derive heuristics to guess what bits on your hard drive might be Office. A heuristic is by definition a guess, and for every time we guess right, we might guess wrong. As a very simplistic example, if we have a heuristic that says “look in the /Applications folder for any folder that contains the word Office, and assume that is the Microsoft Office 2008 folder”, then what happens if some other app uses a folder with that name? We’d have to add another rule that says “ok, then look for an app inside that folder named Microsoft Word.app”. That rule can fail too, if users rename the app to be just Word.app. So, we could add another rule, and another, and suddenly the updater complexity just to find Office becomes unmanageable. I don’t think Apple allows their apps to be renamed or moved, either (although I’ve been told by an Apple engineer that they do. I’ll have to check on that.)

Supporting the deletion of parts of Office adds complexity because updates are not necessarily full file or bundle replacements. Some updates are just patches of the executable, or of sub-resources in the app bundle. If you have deleted a particular component that a given update wants to modify, you can end up with only a partial version of that component updated and the rest missing. Office is designed to act together as a suite, and as such, dependencies between the various apps and libraries can change at any time under the hood. If you’ve deleted a library that version 12.1.1 of Word doesn’t need, but version 12.1.2 does, your install of Office would be completely broken if we updated Word after you deleted the now-needed library. So, we err on the side of caution and refuse to update the suite if it has been modified by deleting components.

Supporting the removal of ‘unnecessary’ languages with an app like Monolingual is basically the same scenario as above, although a bit less risky. Interestingly enough, since very few parts of Office actually ship with multiple languages (I think that only MERP, the Help Viewer, and one or two other items have more than one language), removing the ‘extra’ languages only saves you a few KB of disk space, which is pretty small compared to the overall size of Office. Again, however, if you’ve modified Office by removing parts of it, we can’t tell up front whether the bits you currently have on disk will work with the updated bits the updater wants to install, so we err on the side of caution and refuse to update the suite if it has been modified by deleting languages.

Supporting the removal of ‘unnecessary’ architectures with apps like Xslimmer or Monolingual adds complexity because we don’t know exactly what has been done to the software. Did the tool adjust the loader commands in the Mach-O binary? What if we’re patching the binary instead of replacing it? It’s always safer to do nothing to your current install than it is to blindly make changes based on an unknown current state, so we err on the side of caution and refuse to update the suite if it has been modified by deleting architectures.

Some web forums have posted a workaround that involves copying the updater to a write-able location and then modifying one of the scripts inside the installer, so that the installer always blindly goes ahead and applies the update.  That may appear to work, but leaves your Office install in a possibly precarious and probably unsupported state.  [Update: I did a little digging into this script in the updater, and it is checking a number of specific files to make sure they match their expected state.  It is pretty obviously not validating every file in the Office suite (for performance reasons, maybe).  If we're not validating every file, then it is possible to update some modified file, which undermines my own discussion here.  Hmph.  I'll have to talk to the folks at work to get the background on this script.  My guess is that we check a representative sample of files to see if they've been modified, since the usual suspect processes modify almost every file in the suite.  If one of this set of files has been modified, it's likely that the rest of the suite has been changed too, and it's safer to abort the update.  Again, that's just my guess.]

Now, having said all that, I think the current user experience is not ideal and could be improved. One change might be to tell you that we did in fact find what looked like Office, but that we detected some change that is unsupported (such as language or architecture deletion, etc), but even doing that adds some heuristics, and if we’re looking by name and don’t find the right name, then the current error message is technically correct. So, we’ve got some thinking to do about this.

Hopefully this post explains some of the reasons users are seeing the updater error message. If you are seeing it for reasons I haven’t listed above, please add a comment.

During Office 2008, we converted most of our drawing code over to use Core Graphics instead of the historical imaging APIs known as QuickDraw. CG is much richer than QD, has full alpha-channel support and hardware acceleration, and uses floating-point coordinates for full flexibility, so it is to our obvious advantage to use it. Even though we did most of that conversion during 2008, I’m amazed at the amount of QuickDraw cruft I keep finding.

I went through a bunch of our toolbar code today with a peer of mine, and we found dozens of references to GetPort/SetPort, even though the code right next to it was fully CGContext-based. Sigh. Some of them have been trivial to remove, but I’ve also found a lot of mouse-tracking code that uses QuickDraw APIs for comparing points to rectangles. Even the so-called modern OS tracking APIs take CGrafPtrs (I’m looking at you, TrackMouseLocationWithOptions()…) A little more inspection of headers this evening revealed a new replacement API in Leopard called HIViewTrackMouseLocation, but at the moment the Office source code still runs on Tiger, which means I can’t use it easily. Ugh.

Anyway, Apple has deprecated lots of stuff over the lifespan of Mac OS X, and we’ve got a lot of cleanup to do. Reading the Leopard Carbon Release Notes for the HIToolbox gives a sense of the direction to take for the Carbon code that needs to survive. It does feel nice to rip out old crufty code (as my peer said today, “Kill it! Kill it!”) and modernize some of our subsystems. It will feel even better as we transition some of those subsystems to Cocoa. None of this cleanup directly helps our users, but as we modernize our code we make it more maintainable and sustainable for the future, and that gives us a better platform on which to build things that do help you.

When Office 2008 was released in January, all of the keys used in our private beta program expired, but it wasn’t until the SP1 update that we added code to enforce the expiration. While we communicated to all of the users in our beta program that their license keys were going to expire when the product became publicly available, one of the beta versions did escape onto the Internet in November and we didn’t know who was using it. During our work yesterday we found that some of our users installed their retail copy of Office 2008 over that beta version and ended up running the final release of Office with the beta key.

Our testing and user assistance team scrambled today and have put a new help topic about the problem up on Mactopia, along with another link to how to remove the old product key information and re-enter your valid key. This is about the fastest we’ve put up new help content — about 26 hours from first report of the problem to active help content!

Also, the English and Japanese versions of the SP1 update are now live in the Microsoft AutoUpdate tool, so rather than finding them on Mactopia for manual download, you can simply go to the Help menu in your favorite Office 2008 app and select “Check for Updates”. The autoupdaters for the European languages should be available soon.

Lastly, we actually put up a very minor revision to the SP1 installer today. The new package has a fix so that the update can actually be deployed via Apple Remote Desktop or via the command-line. If you want to deploy SP1 via ARD and downloaded the update yesterday, please redownload it. If your install is already done and working, there’s no need to redownload and reinstall the update — the fix was in the installer only, not in any of the actual application bits.

I’m particularly excited to see this update go live, because it’s been the project consuming most of my time for the last several months. Shortly after we shipped released Office 2008 to manufacturing in December 2007, my manager asked me if I’d be willing to act as the development lead for the SP1 project. I agreed, and have spent many hours working on the project since we kicked it off in early January. Since our fine UA department has already written up the list of highlighted changes, I thought I’d spend a little time today describing how we approached SP1 and how we did the work.

First off, while I was the development lead for SP1, I most certainly cannot take any majority of the credit. SP1 was a MacBU-wide effort, with significant contributions from every single developer, tester, program manager, content writer, lab engineer, builder, and all the other disciplines I’m forgetting at the moment. I am incredibly thankful for all the hard work that everyone around me did.

So, why SP1, and why now? Well, it’s no great secret that we released Office 2008 later than our original plans intended. Even after that delay, there was a decent list of issues in the product that were postponed during the original release cycle that we knew we wanted to fix, so we knew that we had to plan for an update sometime in 2008. Mac Office 2004 was first released in May 2004, and its first service pack came out in October 2004, about 5 months later, and that’s pretty much the same schedule we came up with for Office 2008. In addition to fixing the issues we know about internally, we like to have a few months of real user data in hand (crash reports, newsgroup/forum postings, etc) that help us to identify problems that we didn’t know about. I’ve been spending a bunch of time in Ars Technica‘s Macintoshian Achaia forum over the last year or so, and have received a number of good bug reports from particpants there. (In fact, we have a ‘SWAT’ team of folks in the MacBU who participate in a whole variety of forums across the net.)

Anyway, the entire MacBU began development and testing work on 2008 SP1 in January. Initially, teams pretty much just took their list of issues postponed from the regular release and started fixing those bugs. Our test team once again dove into the product and found more issues that needed fixing, and we started to collect MERP logs (Microsoft Error Reporting Program — aka the dialog you see when an Office app crashes) from the wild. Devs took all those bug reports and began working on classifying them by severity and importance, and then started fixing them.

Like any release, however, there needs to be a modicum of control applied to the whole process. If everybody just keeps on checking in fixes, the product can actually end up being less stable than before. The Office codebase is large, and code changes in one part can have unintended consequences somewhere else, so we don’t let code churn run on forever. Many teams at Microsoft (and, I’d imagine, at other large software companies) have instituted ‘triage’ for bugs.

When we triage bugs, it basically means that we take a look at all the aspects of a bug before deciding whether to accept a fix it:

1. How bad is the bug? (Is it a crash? corrupted file? data loss? simple redraw glitch?)
2. How hard is it to encounter the bug? (every time on boot? Only when saving? On Feb 29? Only when dragging a shape across a page boundary when the page is in PubLayoutView and the shape has a semitransparent shadow and the document has never been saved?)
3. Can the user work around the bug? (use the mouse instead of the keyboard?)
4. How complicated is the fix? (simple typo (= instead of ==) or re-architecture of entire function?)
5. Where is the fix? (in the middle of Excel’s recalc engine? Or for populating a single popup menu in one dialog?)
6. How close are we to shipping? (Months, weeks, days?)

All of these factors come into play, and some combination of the magnitude of each of these leads to a decision of whether to fix a bug or not. It can get pretty complicated, and much of it comes down to a gut decision (although we try to collect hard data when we can, like ‘exactly how many MERP reports do we have of this one bug?’) In January, when we were 4+ months away from shipping SP1, pretty much any bug was allowed to be fixed. In April, when we were trying to lock down the code and minimize code churn in order to keep the product stable, we punted a known crashing bug because although the impact of the bug was large, the fix was a little complicated and the testing team didn’t think they could verify that the code change had no unintended side effects with the amount of time left before shipping.

So, what was my role in all this as development lead for SP1? I worked closely with one of our Program Managers and one of our Test Leads as the three-disciplined Sustained Engineering Lead Committee (I have no idea if we had some formal name, so that’s what I will call it here.) Scott, Pat, and I set the product schedule (kickoff, first day of triage, code-complete, zero bugs, release candidate target, etc) and acted as the final arbitrators in daily triage meetings, putting our roughly 33 (ok, now I feel old) combined years of software development experience to use. Mostly that meant that as teams brought their bugs to the triage meetings, we asked them about the above list of questions and tried to get a sense of how each bug fit in importance. We were responsible for using that information to ‘raise the bar’ each day/week — making it harder for bugs to qualify for fixing. That sounds somewhat anti-quality or antithetical to the purpose of a service pack, but actually it is a very important part of ensuring that we ship a high-quality, stable product for our users. It can be very hard to say no to a bug, particularly when people really are passionate about the specific problem or user scenario, but someone has to do it. The three of us all made a final call on each bug, and I think I tended to be the most hard-core about locking things down. Toward the end, a few people jokingly (at least, I hope they were joking) called me “Dr. No.”

As I said at the top, there are over 1000 fixes in SP1, including the re-addition of some features that were glaringly absent when compared to Office 2004. That doesn’t mean Office 2008 is now perfect — I know we haven’t fixed every bug in the product, and some folks are bound to note that we haven’t fixed some of the bugs they are most frustrated by. However, I hope you’ll agree that this update shows that I and the rest of the MacBU are committed to this product and to its future.

Update: I added a new post the other day about some installation problems people have been running into with SP1. If you are having difficulties with SP1 apps, please take a look and see if it helps you.

Almost two years ago, back at WWDC in August 2006, the MacBU announced that Office 2008 would not have support for Visual Basic. I blogged about it at the time, and that one post has proven to be my 15 minutes of Internet fame. It continues to be the most popular post on my site — 21 months later, it still accounts for almost half of all the hits I get each week. While most of our customers don’t require the cross-platform scripting enabled by VBA, a section of the Mac community spoke out very vocally against our decision, and I still hear echos of it to this day. At the time, I wrote about the challenges we faced in bringing it forward with the rest of Mac Office 2008 and why we ended up deciding to remove the feature, but while some people understood or at least accepted the details, some in the community did not. I’ve been told that we must have cut VB to intentionally drive users to use virtualization and Windows Office 2007 on Macs, or that we were ordered by upper Microsoft management to slowly kill the Mac, or any one of a zillion other “Microsoft is evil” conspiracy theories. None of these theories are true, but it’s rather hard to prove that, except by deeds.

This isn’t a done deed yet, but I’ve got a new commitment for you. Quoting from a press release that went out from the MacBU at 12:01am PST today:

VBA Returns to Future Versions of Office for Mac

The Mac BU also announced it is bringing VBA-language support back to the next version of Office for Mac. Sharing information with customers as early as possible continues to be a priority for the Mac BU to allow customers to plan for their software needs. Although the Mac BU increased support in Office 2008 with alternate scripting tools such as Automator and AppleScript — and also worked with MacTech Magazine to create a reference guide, available at http://www.mactech.com/vba-transition-guide — the team recognizes that VBA-language support is important to a select group of customers who rely on sharing macros across platforms. The Mac BU is always working to meet customers’ needs and already is hard at work on the next version of Office for Mac.

Yep, you read that right. VB is (well, will be) back, baby! When we came to the realization in 2006 that there was no way for us to keep VB in the product and still ship Office 2008 on any semblance of the schedule we wanted, we announced its removal, but kept looking at how to bring it back into the suite even before we shipped. Many of the technical challenges I wrote about then still remain, but for a while now I and several others have been working with a group of people who know a heck of a lot about the internals of VB, and once we determined that we could achieve the revival VB in the new schedule for the next version of Mac Office, we locked it into place on the feature list.

Personally, I think it’s really cool that we’re announcing this now. For all the wringing-of-hands and gnashing-of-teeth in the Mac community over the lack of VB, Mac Office 2008 has been selling really well (Craig Eisler, our General Manager and all around cool-boss-guy, said “The response has been amazing — since we launched in January, the velocity of sales for Office 2008 is nearly three times what we saw after the launch of Office 2004″ in that same press release) which seems to indicate that most of our users don’t find the lack of VB to be a major issue. I think our management is confident enough in our ongoing sales of Office 2008 to tell you about something very significant in the next version, even if that defers some sales to that next version. Based my own experiences talking with people in various Internet forums, I don’t think too much of that will happen, though. And if you were wondering, the delta between Office 2004 and 2008 was longer than we normally expect between versions, so my understanding is that this next version will be available somewhat sooner than 2012 (I can’t give any specifics at this time, however.)

So, if you have a dire need for Visual Basic, you can continue to run Mac Office 2004 (it will even run side-by-side with Office 2008) and we’re publicly committing to VB as good (maybe even better, if things go well) in the next version. My team is responsible for that reintegration, and I’ve been meeting frequently with a number of people as we’ve planned exactly what we’re doing and how we’re bringing VB back. This seems to me to be a strong example for the MacBU naysayers that we’re really listening to what all of our users want, and that we’re most definitely not slow-marching to some bagpiper’s funereal drone!

I’m excited to be able to blog about this now, after almost two years of keeping my lips zipped, and I can’t wait until we reveal everything else about the next version of Mac Office. In the meantime, let me ask you something. What parts of the Visual Basic experience are most important to you? The IDE? Macro UI, such as dialogs? Object model parity between Mac Office and Windows Office? (and if so, which features in the Windows object model do you most want brought to the Mac?) Or something else altogether? I can’t promise to achieve anything in particular, but I’d love to hear how we might be able to improve upon the 2004 VB experience for you.

(Made a few edits this morning, and added a link to the press release.)

That’s a great idea, but what about work? I’m still quite mentally fit (aside from the foggy effects of taking Percocet to dull the pain) and didn’t relish the idea of spending 3 weeks staring at the ceiling. Luckily, Microsoft has a great HR and benefits team, and I have an understanding manager (who happens to be a long-time friend and is actually the very first person I met at Microsoft on my first day as an intern in 1995!). Together with my general doctor, we crafted a formal ‘accomodation’ plan that acknowledges my limitations and still allows me to work from home part-time (and account for the rest of my time with sick leave). This way I get to be productive, keep my mind off of my back, and yet not stress my system with long commutes or hours sitting at my desk. I’ve got a laptop at home and can use Apple Remote Desktop to drive my Mac Pro at work, and thus can do pretty much anything from my bed that I could do at work.

The one tricky thing is that as a development lead, I have several people who report to me, and I need to be able to meet with them on a regular basis. Handily enough, between the new A/V capabilities of Microsoft Messenger 7 and the nice corded mic on my cell phone, I can attend meetings and talk to all my direct reports face to face. I did a number of one-on-one meetings that way today, and it seemed to work rather well.

So, I’m spending my days lying on my nice comfy bed with a couple of pillows and a freaking hot MacBook Pro. Not exactly where I’d like to be, but given the circumstances, I’ll take that over a stay in the hospital or a blah view of my ceiling. And, *knock on wood*, with any luck I’ll avoid a third surgery and be physically back in the office soon.

On a more personal note, I’ve not posted here in a very long time. As you might imagine, I’ve been very busy at work over the last several months, with more to do before Office 2008 becomes available in January. Instead of blogging, I’ve been spending my free time at home playing with my kids and doing small fixit projects (I replaced two faucets over the weekend so that my son could reach the handles and turn on the water by himself.) I do hope to write more regularly soon.

I’ve talked about the relationship that the MacBU has with Apple developers before, and it’s nice to hear Steve himself say that “it’s one of our best developer relationships.”

I obtained a new AEBS from our lab and set it up in my office. I rebooted my Intel iMac into Tiger, applied all the recent security updates (that Mac has been running Leopard seeds for a while, so the Tiger partition was a little out of date) and installed the new AEBS software. I then booted up Office 2008 (since that’s easier to debug right now) and tried to reproduce the problem. No luck — Excel and Word both had no problems saving to the wirelessly-mounted disk drive. So, I tried Office 2004. Again, it just worked! So, I sent the bug back to my colleague saying I couldn’t reproduce the problem.

Today, he brought his own AEBS in from home and we tested it. Sure enough, we could reproduce the problem on his AEBS. So, we pondered the problem for a little while and realized that when I set up our lab’s AEBS the other day, the first thing I did was update the AEBS firmware as prompted when I first ran the Airport Utility tool. Comparing the two AEBSs showed that the one I set up was using firmware v7.1, whereas my colleague’s was using v7.0. A quick update of his AEBS and a new test, and the problem went away!

I then dug around on Apple’s web site and found the release notes for the new firmware update. Lo and behold, the new firmware says that it has “improved support for third party applications saving files to a USB disk.”

So, if you are having problems saving files from Office 2004 applications to your new whiz-bangy Aiport Express Base Station, go to your Airport Utility and check to see if you have the latest AEBS firmware. Your Office installation will thank you (or at least, save your files…)!

MacTech Magazine announced on Wednesday that their upcoming April 2007 issue will contain the entire 150-page book! I saw a slightly-pre-release copy of the book at MacWorld and I think it will be an excellent reference/HowTo for anyone who has created custom solutions in VB for the Mac.

I know this doesn’t address the cross-platform issues surrounding VB but it should give you some ideas of the richness of AppleScript and our support for it in Mac Office 2008.

Anyway, as far as work goes, MacWorld was pretty good this year. I met a number of interesting folks in the Microsoft Blogger Lounge and answered a ton of questions over at the main Microsoft booth. Apparently I’m a “docile developer” — I’m glad that Brandon liked the personal tech support. On Tuesday evening I got to chat with Jacqui Cheng of Ars Technica at the Microsoft Press Party. Apparently Jade had wandered off to someother event so I didn’t get to experience the full snark effect, but Jacqui and her friend Herschell (sp?) were quite pleasant to talk to. I’ve been following a few threads on the Ars Macintosh forum, where a few commenters insist on dragging the MacBU through the mud as often as they can (and you can see a few of their comments on some of my other posts here). Somehow Jacqui and I got to talking about the perceptions you get as you scan people’s comments. So many of them are negative that it is easy to get kinda down about blogging. I mean, who really wants to share some personal insights only to get cursed out all the time? It’s a little odd, but most of the positive comments I get are sent to me in private email, whereas the people who have some issue or complaint about me or the MacBU usually post public comments (dare I call them diatribes, at times?). Jacqui said she’s noted the same thing with her columns on Ars. I wonder why that is? Are there any human behaviorists reading this who care to hazard a guess?

On Wednesday at the Blogger Lounge I spent a long time talking with Eric Albert, a former Microsoft employee (although not with the MacBU) who’s now been involved with development on both the Intel Macs and the new iPhone. We chatted about the development of Xcode, including his perceptions of how easily 3rd party (ie, non-Apple) developers would be able to switch their codebases to it (hint — not as easily as Steve Jobs said!) and the various experiences we’ve both had with various chip and system architectures. I asked him about working on the iPhone, and aside from the normal “I can’t tell you that” sorts of stuff, it sounds pretty cool. He had some funny anecdotes to share about the secrecy involved, including having to stand guard around the demo model with his coat flared out for privacy at 5 or 6am the morning of the keynote as someone else ran through the demo to ensure it worked! (Here you can see Eric chatting with Nadyne and myself. I’m in the center wearing black with the yellow Office logo all over my shirt, Nadyne the red-haired woman to my left, and Eric is facing us wearing a grey shirt.)

Oh, and I’d like to welcome all you Macintouch and MacFixIt readers — it seems that my post on VB is making the rounds again. I guess that will be my 15 minutes of Internet fame, doled out in little bits a month or so at a time. Some of the comments I’ve seen recently about Office 2008 wonder why we didn’t demo much in the booth at MacWorld. I’m not in any position to answer that, but I do know there’s a lot more to Office 2008 than you’ve seen. There are a bunch of screenshots that show visuals for features we haven’t demoed publicly yet. By the way, all those rows of buttons in the Elements Gallery collapse — they are only expanded while you are working in the EG, so it doesn’t take up so much screen real estate on a permanent basis. I’ve asked some folks in MacBU to get a screenshot of the default state so you can see the difference. In the meantime, AppleInsider has a few rough camera shots from our booth demo that show the EG in a collapsed state.

So beyond all that, what’s up? Lots more coding at work. My manager and I got a large new chunk of code to link on Friday that we’ve been working on for several weeks, and that removes the block on the Excel converter work I’m scheduled to do next. I’ve got some work to do on integrating Cocoa nibs into our localization process, so that we can apply localization transforms to them automatically, rather than doing them by hand each time the English nib changes, which in turn lets us autogenerate a complete localized build every day (something that we haven’t had working for the internal builds yet this cycle, yeesh!)

Ok, I’d better go to sleep so I can get something done tomorrow. If you hear me whistling on the bus tomorrow morning, say hi!

Anyway, the big news for MacBU today was our announcement of Mac Office 2008. (Finally, I can stop calling it “the next version of Office” all the time!)

There’s a couple of screenshots over on Ars Technica (along with some good old Charles Jade commentary — I see he had fun tlaking to Geoff) and MacFixit has a short writeup as well.

Please let us know what you think. There’s still more to come; we’ve got converters coming out in beta in a few months, as well as more information on new Office 2008 features in the months leading up to our release in the 2nd half of this year.

I spent a little while at the main Microsoft booth today. One person asked me about VB and another about file converters, but most of the questions were about things that users didn’t know they could do in Office 2004. It’s always nice to be able to show someone how to solve their problem on the spot!

Not this time!

Since the meeting is happening during the 2nd week of January, I’m combining the travel for that meeting with a jaunt over to the Moscone Center. I’ll be at the 2007 San Francisco MacWorld Expo on Tuesday and Wednesday January 9-10. I haven’t been to a MacWorld in a long time; I think the last one I went to was MacWorld Tokyo in March 2002! The main reason I’m going this year (other than to avoid a brutal day of travel!) is to participate in some of the blogger events that Microsoft is sponsoring. In particular, I’ll be hanging out at the Microsoft Blogger Lounge (booth #702 in the South Hall) on Tuesday and Wednesday afternoons from 3:00 to 5:30pm or so, and a little bit at our main booth (#1112). Depending on what time my plane lands, I might get there a bit earlier on Tuesday.

So, if any of you reading this are planning to be at MacWorld, feel free to drop by with any questions or comments you may have, or your resumé if you’d like. If you want to be sure I’m around, you can send me an email from this page. I believe some of the other MacBU bloggers will be at MacWorld as well, so look for several of us there over the course of the week.

The short version is: there is no urgent need to uninstall the patch because we’ll release a new patch by the end of this week to fix everything up.

Much of the sentiment expressed seems to stem from a number of theories, such as

1. We have been sitting around doing nothing
2. We have not talked to the Win Office team
3. File format converters are trivial

None of these three beliefs are true, however. Let’s look at each one in turn.

1. The MacBU has been doing nothing
The reality is we have been working very hard on underlying technologies we need in order to make these converters. The new file formats are XML-based. On the Windows side, there is a very complete and modern XML parser built into the operating system that the WinOffice team uses. That parser does not exist on the Mac (there are certainly a variety of XML parsers out there, including libxml, but the only one that ships on Mac OS by default is libxml and it doesn’t support everything that the new file formats need.) So, we have spent significant development resources on porting MSXML over to the Mac and testing it to ensure it works. You may recall from earlier posts of mine that porting code from Windows to the Mac is not a trivial task, even when the code doesn’t use any OS-specific APIs. The compilers used by each team are very different, even the behavior of some of the standard runtimes is different, and of course we have to test in great detail any new code and especially any changes we make to it. Beyond that, there are changes that needed to be made to Office 2004 applications to support the upcoming converters, and we’ve been busy doing that. I spent much of the month of October going through Excel 2004′s load and save code path to make it aware of the new converter (I even went so far as to create a fake converter that pretends to handle the conversion commands so that I could actually test Excel 2004′s new behavior as much as possible in advance.)

2. The MacBU never talks to the WinOffice team
Various people in MacBU talk to their counterparts on the WinOffice team pretty frequently. For example, I coordinated the work I did on Excel 2004 with a developer on the WinOffice side who did similar work for WinXL 2003. Our Program Managers have had weekly and monthly meetings with WinOffice folks for the past few years. We’ve known the general plan for the scope of work we’ve needed to do for some time, and have been doing that work. Knowing what to do, however, doesn’t make the work happen instantaneously.

3. File format converters are trivial
Let’s step back and look at the problem again. Office 2007 for Windows and Office 12 for the Mac both need to support the old and the new formats. They need to support the formats natively — it would be pretty silly to invoke an external converter for the new native file format, right? Ergo, the code for the new file format should be written directly into these newest versions of the suites. Ok, so now you’ve got that code in place, and you need to create a converter for the older versions of the suites. What do you do? Should you rewrite or duplicate that file code in the converter project, and risk the converter and the actual product getting out of sync? Or do you take that existing code and repackage it as a converter itself? Well, the WinOffice team decided to do the latter, so their converters for Office 2003 are based off of a significant portion of the Office 2007 code. They shipped their final converter versions after shipping the actual suite that natively implements the formats.

That’s exactly what we’ve chosen to do as well. Rather than duplicating code and having to test each set, we are writing it once and repackaging the appropriate parts as the converters. The problem for us is that we’re following along behind WinOffice in a temporal sense. Because we’re shipping after WinOffice, there is a very real time delta where WinOffice users will be creating files with the new formats and the Mac converters won’t be ready. We’ve been porting the WinOffice code over to MacOffice in stages for many months now, but since they just shipped their final bits in November, we’ve only had those final pieces of source code available for less than a month. We’re porting it over as fast as we can. However, as I mentioned earlier, porting code across is not a trivial task. Our apps have diverged from WinOffice over the last 10 years, so we have to deal with internal implementation conflicts as well as compiler pecularities, OS differences, etc. Some parts even need to be rewritten — anything that deals with graphics on WinOffice probably uses the GDI+ library on Windows, which doesn’t exist on the Mac. So, we need to take time to rewrite it to use CoreGraphics, etc. Those changes need to be tested thoroughly! The converters even need that new graphics code, so that they can convert Office 12-style pictures with all the fancy new effects into some format recognizible by Office 2004.

So, you can see that our Office 2004 converters are very dependent on the progress we make on Office 12 itself. We can’t just drop work on Office 12, as some have suggested, to get the converters ready faster, because the converters use a core subset of the actual Office 12 code. The overall Office 12 project, however, is not yet ready for beta use. The WinOffice file format code changed in some pretty significant ways between their Beta 2 and their final shipping bits, and we had to wait for them to ship so we could get those bits. Our code (not necessarily the file format code, but the whole of Office 12) has some rough edges still; they need to get smoothed out before we can ship.

None of these issues are intractable, insolvable, or something that has surprised us. These issues are just part of the facts of developing and porting code cross-platform (especially code that was not written with cross-platform specifically in mind.) We’re making great progress, and can even see the light at the end of the tunnel (and it’s not a train!) As Sheridan said in a followup comment to her post, back in November we weren’t ready to talk about dates because we didn’t have the latest WinOffice code and couldn’t predict our progress based on missing data. Now that we’ve had the code for a few weeks and have made significant forward progress, I can actually tell you about where we are and when you can expect to see some of the fruits of our labors.

We plan to release a beta of the converters in late March or early April (roughly 3 1/2 months from now), and final versions of them after Office 12 ships. It is imperative that we do proper testing on even the beta converters, because a buggy converter that destroys your files would probably be even worse than not shipping a converter at all. I’m sorry that the wait is so frustrating to you. Many people in the MacBU are working very hard to make the converters available as soon as possible. I’m even helping out by putting on my old Excel developer hat (the one I put on the shelf back in 2001!) and doing some of the Excel port myself. The converters are coming, just not immediately.

The good news is that I can now publicly say that we are adding A/V support to a future release of Messenger for the Mac. I’ve personally seen the code and heard some of the status reports. I can’t speak about any sort of version or date of availability, but it is coming!

The converters are coming along but we’ve got a long way yet to go. Stay tuned here or to MacTopia for announcements on their availability.

As for myself, I’ve been pretty quiet here. I’ve received a lot of great feedback on VB and made sure it got into the proper inboxes at work. Thank you all for your comments and feedback. Alas, there’s no way I can ever reply to every comment, but I have read them all. I plan to post some snippets of VB-to-AppleScript conversions so that people can get a rough idea of what you can do with AS.

That will have to wait a few days, however, because I’ve got a ton of stuff to do at work. I just got back this evening from a week’s vacation with my family in Las Vegas. Nope, no gambling, but we did spend a lot of time at the pool (my son now loves waterslides, provided I hold him as we slide down) and wandering through several of the mega-hotels on the Strip. My son had a very hard time believing that this was *all* Las Vegas. I think he thought the hotel itself was named LV, and that everywhere outside it was some other magical new place (Bellagio fountains, MGM lions, Mandalay Bay shark reef, Mirage volcano and dolphins, Caesars Palace Forum, it just went on and on…!)

So, after having been gone for a week, I came back to 772 new messages in my work inbox (110 per day!) that I need to cull through. Wheee!

I had intended to post yesterday and answer some of the specific questions raised, but I’ve once again caught a nasty fever and cold from my preschool-aged son and didn’t get to it. Sigh… In lieu of that, I’ve got a couple of more general notes to make on yesterday’s post.

In my note on resources, I used a slightly extreme example of our headcount. I did have an opening for quite a long time, but our overall delta between having someone leave our group and hiring a replacement is probably something like 2 months or so. I have no idea how that compares to any sort of Microsoft- or industry-wide average. The complicating matter is that departures and candidates tend to happen in clumps, so we often end up with some abnormally long or short gaps in staffing instead of a more normal distribution. We’ve actually had a lot of folks very interested in working at MacBU (i.e, us having open headcount really is not due to a lack of people interested in working here). We hire world-class developers and they don’t simply grow on trees. For those readers offering a smackdown on the intelligence or capabilities of our developers, nothing could be further from the truth. We’ve even had some developers leave MacBU to go to other renowned Macintosh companies and then come back to MacBU because the grass really wasn’t greener elsewhere. We’re all proud to be here!

As for keeping VBA in PPC to run under Rosetta, that could be done but just vastly overcomplicates an already very complicated piece of source code. Rosetta only works on a per-process basis, so we’d be moving code and data out of the single app process into two processes. There are many many points in which apps call into VB and VB calls back into the apps (not to mention VB’s dependence on other lower-level shared library code that is now Mach-based) and marshalling all those calls and data cross-process is a heck of a lot of work. It gets even more complicated when you have more than one Office app running at a time, because then the new VB process would have to keep its internal state separate for each client app!

People have also offered the solution of porting the current Win Office 200x VBA over to the Mac. That’s pretty much a no-go too, because 1) it still has the assembly ABI problems, 2) makes calls to the Win32 APIs that don’t exist on the Mac (well, without WINE or any of the variants) and 3) relies on features in the Windows OS that don’t exist at all on the Mac. We’d have to port over large swaths of the Win OS and that might take even longer than writing VBA over from scratch.

VBA just wasn’t designed to be portable across architectures. The Mac and Windows implementations are very different, even though the actual language is the same. If I could go back in time about 15 years and tell the original designers to do it differently I would, but I can’t. Thanks to the heroic efforts of a few developers (Shuyi and Ying) we managed to Carbonize VB for OS X, but the move to Intel is gargantuan. We considered many, many alternatives. Each one turned out to be no simpler and as much or more work as the plain port would be.

None of this is work that we can’t do, it’s just work that would take away from everything else that we do want to do and would delay shipping Office 12. As I said on Tuesday, delaying Office 12 has other ramifications for cross-platform compatibility such as dealing with the new Office XML file formats and related features (like the larger cell grid in Excel, for example.)

Back to the idea of AppleScript… Folks who want to develop scripting modules on the Mac should check out AppleScript Studio. You can create GUI applications with it, and it is absolutely free from Apple. Also, a few bloggers have been wondering about scripting Office with Python and other languages. Yep, you can do that with the AppleEvent bridges available for those languages. We actually test Messenger in-house with a large suite of Python scripts that drive it through OS X’s Accessibility AppleEvent suite. We also drive Xcode via its AppleScript dictionary with Python scripts that send AppleEvents.

So, what now? Well, it is pretty obvious to me that the MacBU needs to provide some guidance for folks who currently use VB. The best way for us to provide the information that will help you the most is for you to give us some first. I encourage you to provide feedback to us — lots and lots of feedback! There are several ways you can do this:

• Select Help/Send Feedback on ‘OfficeApp’ from within your favorite Office 2004 application
• Click on this link and fill out the web form
• Post a comment on any of the MacBU blogs
• Send me email from this link and I’ll forward it into MacBU

Please give us specifics in your feedback. Just saying “I want VB!” or ranting about how we’re evil doesn’t provide us anything we can take action on. We’d like to know what you do with VB, what the goal was, did you share the macro with anyone, was it specifically cross-platform or just something on the Mac, what app was the macro in, and things like that. So yes, do tell us what you really want. Please keep it civil, it’s no fun reading through pages of insults. Your feedback directly helps us figure out the best ways we can help you.

[Aug 11, 9:26am -- minor edit to correct the description of how we drive Messenger with Python]

I should clear up one misconception about how the VB removal affects existing macros that has been making the blog and comment rounds. The removal of VB means that existing macros in Office documents will be round-tripped across file open and save, but you will not be able to edit them and you will not be able to run them on the Mac. Even previously compiled macros will not execute, because they have been compiled to PowerPC code that conforms to an older binary interface.

I want to say right up front that the MacBU is very aware of the pain this decision will cause for users, consultants, and enterprise organizations. I’ve personally seen the phrases “apoplectic with rage” and “absolutely livid” in two emails that crossed my inbox. Some people made comments on my post yesterday that were expressly clear about how this decision would drive them to one of the free Open Office variants instead of buying Mac Office 12, and other posts in other forums made similar statements. I’m sure some people will indeed decide that lack of VB is an absolute deal-breaker and they will plan to use other software. I’m truly sorry if that is the case for you.

The MacBU did not make this decision lightly. I personally spent several weeks digging into the VB source code to identify and plan what work would have to be done to move it to Xcode and make it universal, and I had several long discussions with our product planning folks to help our group leadership weigh the costs of doing the VB port vs. the costs of not doing it. I’ll try to lead you through some of the analysis here.

From my perspective, Mac Office has two primary driving requirements:

1. it must be as Mac-like as possible, use Mac features, and take advantage of the Mac operating system, and
2. it must be as compatible with Win Office as possible, and share as many features and commonalities as it can.

(We’ve got other requirements and product visions, but as I see it, they really act to refine these two basic needs.) As you may imagine, these two goals are many times not perfectly aligned. In the worst cases, they may actually be diametrically opposed, and we have to wrestle with making the best decision we can, knowing full well that whichever way we go it will help some users and hurt others. This VB decision is one where we’re truly caught between the Mac rock and the Win Office hard place.

VB on the Mac exists for cross-platform compatibility. There is no other software on the Mac that also uses VB, so it doesn’t help Mac Office integrate with other workflows based purely on Apple solutions. Thus, any work we do on VB only serves to satisfy one of the two major requirements. Doing that work then means we have less developer time to continue to improve Mac Office’s use of Apple-specific technologies (or tools, such as Xcode.)

Let me describe for you some of the technical challenges that would be involved were we to try to port VB to Xcode and to the Intel platform. For those of you reading who are not developers, bear with me for a little bit. Hopefully you’ll at least get a sense of the scope of work even if you don’t quite follow the nitty-gritty details.

VB on the Mac is really three parts: VBE (the editor), VBA (the execution engine) and Forms (the buildable windows and controls you edit in VBE and see when running a macro.)

VBE is pretty standard C++ code. However, the code is generally very old — it was originally designed and written several years before I came to Microsoft in 1996. VBE contains the top-level parser that converts the text of a macro into a series of mostly machine-independent opcodes (kind of like Java bytecodes, but not exactly the same). Thus you can’t just hook an external text editor up to VBA, because of the upper-level dependency. The VBE code actually isn’t too hard to port to Intel, but it is tricky to port to Xcode/GCC because of the age of the code. As I mentioned in an earlier post, GCC is very picky about code meeting the current standards and the VBE code most certainly does not. That’s not to say the code is ‘bad,’ it was just designed and written long before current modern C++ standards.

VBA, on the other hand, is incredibly difficult to port to Intel. The execution engine basically runs through the previously mentioned opcodes and, well, executes them. The hard part is that ‘executing’ them doesn’t mean interpreting them, it means converting one or more at a time into a block of assembly generated at runtime that looks and behaves like a regular function that can be called directly by other normally compiled code. This is in essense ‘self-creating’ code, and VBA is constantly flushing the CPU’s code cache in order to mark these chunks of data as executable. VBA’s generated code must adhere to the Application Binary Interface of the host platform (historically PowerPC and the Code Fragment Manager). This means register allocation, stack alignment, parameter passing locations, etc. VBA is basically a compiler that emits code at runtime. It does so by running a large state machine that tracks PPC register usage, stack location, mapping between PPC registers and VB variables, etc and then concatenates large blocks of pre-generated assembly together. VBA subsequently tweaks the assembly bit-field by bit-field to do things like assign registers to each opcode, set branch addresses, and create transition vectors for all function calls. The templates are very PPC- and CFM-specific and the state machine is designed for architectures that allocate static stack frames and pass parameters by register, unlike Intel which has dynamic stack frames (you can push and pop data to/from the stack any time you want) and parameters are passed on the stack. So, for us to port this to Intel we’d have to rewrite the entire state machine and create brand-new templates of IA-32 code. That’s basically writing a rudimentary compiler almost from scratch (we’d at least have the initial parsing and machine-independent opcodes already done.) Again, this is all a design that long predates me or most of my peers in Mac Office, and is code that we inherited when we created the MacBU (i.e, none of us wrote it in the first place.) There’s nothing inherently bad about the code, it was just designed for the constraints of the day and that design simply doesn’t lend itself to being architecture-independent.

Some folks might ask why not just port the Win Office VBA over to the Mac? Well, VBA circa Win Office 97 (which is the closest Windows VBA to what we have on the Mac) doesn’t implement their execution engine this way at all. Instead, they have tens of thousands of lines of IA-32 assembly that directly implements all of the opcodes. That assembly does so according to the Windows Intel ABI, which is different from the Mac ABI in several important ways (the specifics of which are described here.) Also, the assembly is in MASM format which is close to but not the same as NASM as supported by GCC. So, we’d have to edit the source to be compilable by GCC, and scrub it line-by-line to find and adjust the parts that aren’t compliant with the Apple Intel ABI. We’d also end up with two completely different implementations of VBA (PPC state machine and Intel straight assembly) that we’d have to maintain and keep in sync. That would be horribly bug-prone.

Lastly, we have Forms. Forms is also C++, but is backed by several thousand lines of gnarly custom assembly. This assembly ‘allows’ the C++ code to swap object virtual function tables and individual member function pointers between objects on the fly, to essentially do very fast object morphing. To do so, the assembly has to have specific knowledge of aspects of the C++ compiler (vtable layout, implementation of ptrs-to-member-functions, etc) and has to work in lockstep with the compiler. I spent almost two weeks massaging this code to try to make it compatible with just the PPC Mach ABI, which is only slightly different from the PPC CFM ABI. Even after all that work, I still didn’t get it completely right and internal builds had some really bad stability problems. We also don’t even have the Win Office 97 Forms source code, so I was not able to compare our code to how it was implemented for Windows.

I just noted that the assembly has to work hand-in-hand with the normal C/C++ compiler. That wasn’t too much of a problem when we were using CodeWarrior, as the C++ compiler only changed in small ways every few years or so. With Xcode and GCC, my understanding is that Apple has to merge in all the changes that external developers commit to GCC, and we run the risk of GCC changing much more frequently. That might not be a problem in reality, but the risk is non-zero and we have to take that into account.

One final problem is that all of this custom assembly is currently PPC 32-bit, and even the corresponding Windows assembly is Intel 32-bit. If we ever want to make a 64-bit native version of Office, any work we might do to solve all of the above problems would have to be done all over again.

So, in short: VB has lots of code and assembly that specifically assumes it is running on a PPC with the Code Fragment Manager, and to re-do it for Intel would involve writing a rudimentary compiler and relying on private compiler implementations that are subject to change at any time.

Whew, that’s a lot of technical stuff. I hope it provides some idea of the scope of work we were facing. We estimated that it would take roughly two years to of development time to move it all over to Xcode and to Intel. That would mean two more years before the next version of Mac Office made its way to consumers. In the meantime, Leopard will ship and Mac Office 2004 would still be running in Rosetta. Win Office 2007 and the new XML file formats will be ever more common. All Mac Office users would still be stuck with the old formats, unable to share in or use the great expansion of capabilities these new file formats bring. During that time, we’d also not be adding any other items our users have asked for.

Beyond that, if we were to port VB over to Intel in those two years, what you’d end up with is VB for Mac just as it is today. It still wouldn’t be feature-comparable to VB in Win Office, and the object model in Mac Office would still not be the same as the one in Win Office. That means that your macros would still be restricted to the same set of compatible items as you have today. Over the last 10 years, the Win Office programming model has become very different from that of Mac Office. We’ve tried to keep the object models in sync for the features that we have ported from Win Office, but we haven’t ported everything.

So, given that the developer cost was huge, that the consumer cost due to the delay while we did the work was quite large, and that the end result would be no better than what we have today, we made the very difficult decision to invest our time and resources in the other pillar of Mac Office, namely taking advantage of Apple tools and technologies to be more ‘Mac-like’. We’ve continued to improve the AppleScriptability of our apps (many many bug fixes post-Office-2004) and as announced are looking into adding some Automator actions to the suite. We’ve completed the rest of our transition to Xcode and to Intel and are forging ahead with the rest of the product.

I think a common question might be ‘if the cost is so huge, why doesn’t Microsoft just devote more resources to the problem? They’ve got a ton of cash, right?’ Well, the real question is ‘what resources do you throw at the problem?’ We’ve been working very hard to hire a bunch of developers, but it has turned out to be quite difficult to fill our existing open headcount positions. As an example, I’ve had an open position on my own team for 9 of the last 12 months (it took 8 months to fill the slot when one developer moved from my team to another one in MacBU, and only last week did we hire someone to fill the slot vacated recently when another developer moved to a different team at Microsoft.) The question of how Microsoft allocates developer headcount and funding to MacBU is a separate topic of its own which hopefully I or some other MacBU blogger will tackle later. In any case, there’s no point in adding new headcount to the MacBU when we haven’t yet filled the positions we already have open.

I know that explaining all this doesn’t make the fact of VB’s death any easier for those users who currently depend on it. As I said at the beginning, we in the MacBU really are aware of the difficulties you face. Our product planners, program managers, developers, and testers are working to alleviate some of that pain. Many people have only a few simple macros they use, and I do want to point out that those macros will translate very easily into AppleScript. Even large macros can be rewritten in AppleScript, although that takes some time and definitely some knowledge scripting on the Mac. The AppleScript object model and the old VB object model for our apps are roughly equivalent, so apart from the syntactical differences, if you could do it in VB you can do it in AppleScript. While I can’t comment on any more specific feature work for Office 12, I’m sure we will be working closely with enterprise customers to help them address their concerns. We’ll be saying more about our scripting plans as we get closer to the product release for Office 12.

For those of you contemplating a switch to Open Office, I don’t know if Open Office has any support for VB macros or other OLE Automation technologies so I don’t know if you’ll be any better off from a cross-platform perspective. You probably can’t be worse-off except that Open Office certainly doesn’t support any of the Mac OS scripting technologies that Mac Office does support and in which we will continue to invest, nor will it (at least for a while yet) support the new XML-based file formats. If you do switch, we’ll miss you.

Many people have viewed this announcement by MacBU as a sign that we are out to screw the Mac community, or that we’re just looking for an exit strategy. We’re not. Most empatically, we’re not. This decision was agonizing. My manager even said he felt ‘sick about the impact on those who really rely on xplat [cross-platform] VB support, particularly in Excel where we see it the most.’ In my post yesterday, I said that I wasn’t so sad to see VB go. I said that from the perspective of a developer who’s worked to maintain the code for many years. However, there’s nothing good about removing a feature that many people rely on, except that it frees up resources for us to invest more heavily in other important areas. Due to the age of the code, VB has been a very large drain on our resources for a long time with relatively little return. A couple of months ago I wrote that I hoped my blog would help people trust the MacBU a little more. I can see that many of you are very mad about this decision; I do hope that my post today helps you see some of the issues behind the press release. We had to make a hard decision one way or the other, and this is how it turned out.

(Please read my next post and consider giving us specific actionable feedback. Thanks!)

I’m sad to see VPC go. It was an amazing product to bring into the MacBU back in 2003 (I’d used it myself since version 2 when it was made by Connectix) and it still has features that Parallels doesn’t have. The biggest thing Parallels lacks is the ability to script the main app and the virtual machines. VPC let you query VM state, launch programs inside it, define shared drive mappings, and all sorts of other great stuff via AppleScript. You can’t do that yet in Parallels. Hopefully they’ll add scriptability someday soon. Them or VMWare!

I’m not so sad to see VB go. That code has been a major headache to maintain over the years. Lots of people are bemoaning the sudden lack of cross-platform scriptability, but to be bluntly honest, VB for Mac Office hasn’t been remotely compatible with VB for Win Office for years. Even back in Office 98 the VB IDE for the Mac had several major features cut compared to Win Office (watchpoints, etc) and the object models for the two platforms have diverged wildly in the 10 years that have gone by. I’d like to blog more about VB, but not tonight. It’s a long and sordid tale.

One positive thing that seems to have been lost in the rabid chatter about VPC and VB is that Mac Office has finally made the complete switch to Xcode! The last major checkin happened on Friday afternoon, and all of MacBU is finally switched over. Yay! That only took a month and a half longer than I guessed, back in early June… We’re still not ready to ship yet; we’ve been working on lots more than just the recompilation and those changes are not yet complete. I myself am hip-deep in the new file-format-converter compatibility work that we’ve announced.

Pierre Igot writes what I must politely describe as an impassioned discourse about my description of pseudo-localization, multi-lingual bundles in OS X, and MacBU testing. Pierre makes some good points, totally misses the idea of pseudo-loc in another, and generally castigates the MacBU for failing to fix a particular bug that is very important in French typography. He then invited me to comment.

Let me begin by saying what I am not. I am not the lead developer for Word, and I am not intimately familier with every bug that is entered against our products. I am also not a native French speaker, nor am I familiar with the rules of French typography. (In fact, my peers in high school often told me “Tu parles français comme une vache espagnole,” which, if my hazy memories of 1991 are correct, means “You speak French like a Spanish cow.”)

I do, however, know quite a lot about bundles in Carbon and Cocoa, how multi-lingual bundled apps should behave, and the benefits to users of software that takes advantage of them. Apple did indeed make them available to developers using Carbon in 2001 or so. Pierre is correct in saying that currently shipping Mac Office apps do not use them. (Well, Messenger does use them, and may have been the first major Microsoft app to do so.) I became a lead for Mac Office in June of 2002, well into the Office 2004 project and was not a part of the initial schedule planning that, for whatever reason, did not include moving Office to a bundled architecture. As the lead for localization, I understand and sympathize with users for whom this creates frustration. The reality is that for every user calling for a multi-lingual version of Office, there are at least two users who don’t care about that at all and want us to be working on something else. We can’t please everybody, unfortunately. As I said in the comments at the end of my localization post, I cannot comment on our plans for this aspect of the next version of Office, but please rest assured that I do understand and hear your request, Pierre.

Because I am not well-versed in French typography (or any real typography for that matter) I can only trust that Pierre’s assessment of the importance of the non-breaking space in French documents is indeed high. Assuming that is the case, then yes, this must be a very aggravating bug for anyone who uses Word 2004 to lay out text in French. I’m quite sorry, Pierre, that you have been so frustrated by this issue. I wish we had caught the bug and fixed it before shipping. Sadly, we did not.

Pierre then takes our non-use of bundles and conflates that with the existence of the non-breaking-space bug:

The fact that the language of the user interface in Office applications does not change depending on the user’s preferred language in Mac OS X is annoying enough. But that’s not the worst of it. The worst of it is that Microsoft’s process for fixing bugs is completely screwed. And there is no better example of this than the bug with non-breaking spaces and PostScript fonts in Word 2004.

He also implies that our use of pseudo-localization should have caught this ‘elementary’ bug:

The fact that Microsoft did not catch that bug in Word 2004 before they released the product is a clear illustration of how flawed their testing processes are. Whatever benefits this “œpseudo-localization” technique described by Erik Schwiebert provides, it is clearly not good enough to catch even such elementary bugs.

Here I must correct Pierre’s faulty assumption and say that the bug is not in any way connected to either of the other two items. Multi-lingual bundles are all about providing resources to the app, such as strings, dialogs, menus, windows, etc. Pseudo-localization is a method we use to test the localization of those resources and the code that uses them. It is intended to help find visual glitches, missed translations, improperly displayed dialogs, and similar items, and it does a pretty good job at optimizing our efforts for those issues. It is not designed nor intended to help with finding or fixing bugs related to the actual behavior of the application. Pseudo-localization in no way replaces good old-fashioned usage of the applications to uncover code defects.

In fact, the bug that Pierre is so frustrated with occurs in all languages that Word ships in — the faulty code is in no way connected to any localization. I can repro it in the English build. That doesn’t mean that our entire testing process is flawed, however. Every large software endeavor results in a product that has bugs. Microsoft is no better or worse than anyone else in this regard. Seen any OS X software updates lately?

Now, the fact that the bug is unrelated to our localized UI testing doesn’t address the issue that the bug is there, and is bad. Pierre blogged about the bug way back in September 2004. Unfortunately, the MacBU (and the rest of Microsoft, to my knowledge) has never had a good method for customers to report bugs to us. We’ve had the (now defunct) MSWish email address, and we do have the new direct feedback link on Mactopia, but we don’t have anything like Apple’s BugReporter tool. Interestingly enough Nathan Herring, another MacBU employee, just blogged about this specific issue and has some links you may want to read.

But let me continue with Pierre’s post. He finds it unjustifiable that we have not fixed this bug in the almost two years that have gone by:

Now what excuses does Erik Schwiebert have for this sorry state of affairs? I might find it acceptable (barely) that Microsoft is not able to catch such bugs in the hectic schedule that leads to the release of the initial versions of the product. But how can they justify not fixing the bugs in the next two years, even though the bugs are so obvious?

says it can be done on schedule:

it is perfectly possible to release multilingual software on schedule that automatically supports all of Mac OS X’s supported language without requiring the user to buy separate versions for each language and download separate updates for each language.

and casts shame on us:

And for all this, once more, all I can say is: Shame on Microsoft, and shame on the MacBU. They have absolutely no excuses here.

Ouch. That hurts. Me personally, as well as all of our hard-working employees. It’s a good thing I have a relatively thick skin.

I don’t think there is anything I can say that will appease Pierre. And I don’t mean that there’s something I know that I’m not allowed to post; I just don’t think that any insight I provide into MacBU’s bug process will soothe his wrath. However, I will try to shed some light on our processes because I think folks are interested in them anyway. I’m sure many people will individually disagree with many of of the decisions we make and the factors we weigh, but that’s part of the openness of blogging. Feel free to respond with your thoughts and comments — you may provide some insight that we’ve missed in all these years.

First of all, we do work darn hard to fix bugs in our products. Really darn hard. But, we’re not perfect. No developer team is. Not MacBU, not Microsoft, not Adobe, not Apple. Security defects? We’ve all got them. Software updates? Yep, them too. For OS X, we have 10.0.4, 10.1.5, 10.2.8, 10.3.9, 10.4.6 and counting. For Office we have 11.1.1, 11.2.4, etc. Updates are good, it means that we’re all working to make our products better, to fix the bugs and reduce the crashes.

However, not all bugs are fixed in dot-releases of products. Some bugs that we find are really obscure and don’t cause any real harm. For example, perhaps the File/Open menu item is grayed out when you hold down Cmd-Shift-Ctrl-Option and triple-click on the menu-bar. Not too many people will see that, and if they do, no real harm done. We can spend our engineering time and resources better elsewhere.

Some bugs are easier to hit, but fixing them may have a high risk of regression and may cause another bug that is even worse. One example might be a performance bug in Excel’s recalc engine. We could fix it and make things faster, but currently the code calculates correctly (albeit slowly) and any fix might totally break all function dependency analysis. It is sad but safer to leave things alone.

Other bugs are deemed worth fixing for a future release, but not worth back-porting to the current release. Every code change we make has to be tested around to make sure no regressions are caused, and that takes away from other work we could be doing. It may be more effective for us to take a bug that was reported against Office X and just fix it in Office 2004. And occasionally, some bugs are misclassified or accidentally forgotten. After all, we’re only human.

Here’s an example of an engineering decision. If you boot Mac RDC on a PowerBook and connect to a Windows server, try turning on Num Lock (the keyboard light should be on). Type something on the number pad in Excel. You get numbers, right? Good. Now, Cmd-tab back to the Finder and turn off num-lock. Cmd-Tab back to RDC. Type something on the number pad again. What, you still get numbers? Yes indeed you do. Geez, the MacBU really should have fixed that bug, right? We’d love to have fixed it, except that the bug is actually in the OS. The Carbon Event manager does not give you accurate information about the Num Lock state when you query for modifier key state after app activation and deactivation. I filed that bug with Apple in May 2002 against OS X 10.1.4, and they still haven’t fixed it. I wish they had, but I’m sure there is some reason (hardware? software complexity? who knows…) that the change is more expensive for them to make than to allow the bug to continue to exist. That decision is completely up to them, and I respect that.

In our move to Xcode, we’ve reported over 100 bugs to Apple. They’ve fixed the majority of them, and have postponed several to some later release. Some of the bugs they’ve postponed have been real bummers, but they’ve made the engineering justification to delay the fix and again, I respect that. After all, we have to do the same thing with regard to Office bugs too.

Second, the ability to meet a schedule or not has nothing to do with the ability to ship a bundled multi-lingual product. That is essentially a feature, and does not come for free with Carbon apps. (Yes, that is one aspect where Cocoa wins hands-down — it is impossible to create a monolithic Cocoa app. At the very least your app must be bundled, whether or not it has more than one locale for resources.) Perhaps Pierre would have preferred we have chosen a different schedule, moved Office to Mach and multi-lingual bundles, and shipped a product called Office 2005. We hit our internal schedule for Office 2004 with the features we planned for it.

Now, I’m not saying that any of this is an excuse. We’ve made lots of conscious decisions over the years about many many issues. Some of them, everybody likes. Some of them, everybody hates. Most of them are liked and hated simultaneously, and the overlap between the sets of people experiencing the two emotions is rarely the same from topic to topic.

The sad thing about this particular bug is that it falls into the category of ‘oops.’ The code was wrong in a very small way (an exception was made for the breaking-space character and accidentally forgotten for the non-breaking space character.) We actually found this bug in-house just a little while after Office 2004 shipped, and fixed it internally for the next major release of Office in August 2004. We even noted that it was important in the French market. Somehow, though, it was never flagged as a bug that should be migrated back to Office 2004. The tester for that area of Word (who found and opened the bug) left MacBU to go back to graduate school during that summer, and I think we probably just missed this one in the transition. A simple human mistake. Shameful? I personally don’t think so. Unfortunate and frustrating, I’ll certainly agree to that, but not shameful.

I’ve asked our Word team to take another look at this bug. I make no promises, but maybe we can get it fixed sooner than in the next major release of Mac Office. After all, the Windows team did that for a blogger recently.

So Pierre, we are listening. I think pseudo-localization has a greater value than you place on it, but for a different purpose. If you are ever in the Seattle area, please look us up. I’d be happy to take you out for coffee and a chat, and see if we can address some of your dissatisfaction with us. In the meantime, I’ll stand up proud and accept no shame, for I believe we work hard and produce a good product.

There’s been lots of commentary today on what this change means for Bill and for Microsoft. My question for you is what does it mean for the MacBU?

My answer? Probably not much at all. Bill and Steve have both been very aware of our products and plans, and both have been very supportive of our group. I don’t anticipate that changing. As for the effect of the actual shuffling of which top exec does what, in my 10 years at Microsoft I’ve experienced exactly one re-org that had any real effect on my day-to-day work, and that was when the MacBU moved from the Desktop Applications Division to whatever the Home and Entertainment Division was called back in 200x. (And you can see what a large effect it had, given that I can’t even remember the year it occurred.)

The MacBU, like many smaller groups at Microsoft, runs pretty autonomously. That’s one of the things I love about our group — we’re really like a small company, just one that has a lot of support from a much larger organization.

Perhaps that’s a topic to blog on. Another day, however; I need sleep!

I’m curious to know what the termites are in that analogy.

I saw a very interesting essay on Daring Fireball today. I particularly like this quote from John Gruber:

…in some cases, some people seem unwilling to concede that any criteria other than the ones they themselves deem important actually matter, or even exist.

Now, I’m not pointing any fingers at any specific people. I do believe that the choice to use Cocoa or Carbon is, loosely adapting from John, not a binary choice. It isn’t either-or, and neither option is always better than the other. I firmly believe that there are very valid reasons to use each ‘package’. Saying that ‘your house is going to collapse; you can’t avoid it, so go rebuild it now’ does two things.

1. It denigrates all the work Apple has put into Carbon and continues to put into it.
2. It ignores all the reasons I listed in my post for why we have heretofore continued to use Carbon.

Now, I don’t want to get too dogmatic in defense of Carbon, myself. After all, as I said in my first post and in the preceding paragraph, I believe there are good reasons to choose either platform. That’s why I’m curious to know why some folks are so dead-set on the idea that Cocoa is a magic bullet that can fix anything wrong with an app written in Carbon.

Let me know; we’ll chat about it.

Anyway, I was very surprised to see how little commentary my Cocoa post generated. I’ve been playing with some simple stats plugins, and it looks like lots of people have subscribed to my RSS feed (114 feed hits in the last 90 minutes) but I received almost three times as much feedback on my Messenger post. Interesting. (Oh, and jhawk28, I spent lots of time in Middlebury!)

Rick Schaut showed me his entry on bad code today before he posted it. I saw the ambiguity in about 5 seconds, but that’s because it was very carefully contrived. The best part of his particular example is that neither CodeWarrior nor Xcode give any sort of warning that you’ve written Dumb Code. While this example may have been easy to spot, it gets very tricky for devs to hold a mental map of more than a few dozen lines of code in their head. Spotting similar errors when one function is perhaps inlined in a header and another function that interacts with it is at the bottom of a 3000 line source file somewhere else is nigh impossible without actually running the code under the debugger.

Now that I’m back at work, I expect to be blogging less frequently. Please do keep the comments coming; I like to hear from you.

The house itself was built in the 1870s and was originally just a summer home, so the kitchen that had been added on in the 1960s was always a little small and never had enough cupboard space for all of my mom’s pots and pans. Last year, my parents announced that they were going to do a massive kitchen remodel and replace the kitchen with a real ‘farmhouse’ kitchen with lots of space for cooking, playing with grandchildren, hanging up wet snow-clothes after sledding, etc. Their plans are pretty impressive — practically doubling the size of the kitchen and adding a mudroom, putting in a fireplace and space for large comfy chairs, a full basement underneath, new fancy refrigerator, and other cool stuff. I just hope it is as warm and friendly a place as the old kitchen where we used to sit around the woodstove with a cup of hot cocoa after an afternoon of sledding down the hill onto the pond across the road.

Ahh, hot cocoa. What, you didn’t guess where I was going with this? Yep, Cocoa. That Cocoa, the one that everybody keeps saying that if only the MacBU would rewrite Office in Cocoa, that it would be magically bugfree, run at warp speed, and somehow still maintain compatibility with legacy fileformats and code ported from Win Office. Well, let’s see, shall we? I’m going to touch that third rail of Mac Development. Let’s see if I live…

If you will pardon me for a little bit, I’m going to draw an analogy to my parent’s house here. I grant you up front that this will not be a perfect analogy, and I fully expect to see comments explaining how badly I miss someone’s point, but you know, no analogy is ever perfect and I’m not a very experienced Cocoa developer (whereas I’ve been writing CarbonLib and InterfaceLib code since 1992).

I see the idea of ditching the Office source code and starting over in Cocoa as similar to the idea of taking an existing house and tearing it down with the intent of rebuilding it precisely as it was before, just with modern tools, components, and technologies. You want to replicate all the character of the house — the creaky boards you love, the rattly shutters, the custom-sized windowpanes, etc. You can do it, but it will cost you a lot of time and money, and you’ll probably exasperate both yourself and your contractor trying to duplicate something that was hand-built long ago.

Or, you can keep your house generally the way it is, and you can take that time and money to add on a neat new addition (perhaps that indoor Olympic swimming pool you always wanted) or rebuild just one part of it in a new style with more capabilities than it had before. You live with the existing creaky floors (you never wanted to change them anyway), maybe you rip out the old furnace and put in a new one with new efficient radiators, but you concentrate your efforts on the new part of the house. You make it extend the old house, yet, because you had a great architect, make it fit in flawlessly so that you don’t feel a sense of shock or dissonance when you walk into that new space.

Moving Office to Cocoa would be kind of like that first scenerio. I’m not saying it can’t be done, or that we’ll never do it; we just haven’t done it yet (as of Office 2004) because the time and effort required are huge, and we can get a lot more out of extending the current source code. If we spent 2+ years rewriting Office in Cocoa, and came out of it with exactly the same feature set as before, how many people would buy it? Be honest now, would you really spend $239 to upgrade Office Standard to a version that was no different than before, from a feature and useability perspective? Oh, you say we could add features at the same time? Yep, we could, but that would take us even longer, and the computer industry moves faster than that!

Rewriting in Cocoa is also no guarantee of achieving bug-free software. There is no such thing. Your contractor will always make a mistake, or your architect will, or the electrician. (When my parents added on a room to the house in 1991, the architect miscalculated the height of the roof by 3 inches, and the builders only figured it out when they went to start putting up the trusses! They had to take the wall frames down and redo them.) Your really smart developer will also always have bugs. We all try really hard to ship clean code, but any system that grows beyond a few thousand lines of code will have flaws. Even Apple has been constantly updating Cocoa in every system release.

Another thing for the MacBU to consider is code compatibility with its own older software and with Win Office. We can’t literally ditch all of our code because we have to be able to read and write legacy files. Moving to Cocoa doesn’t help with that. And, Windows doesn’t have Objective-C, so any code we write in Obj-C cannot be ported to them, and all the code they write in C/C++ is harder to port to us. Plus, our apps still share a lot of common code, so any gratuitous differences in the code make it harder to port features from Win Office to Mac Office (like the new XML file formats.) It’s already hard enough when they write code in C#!

Beyond that, there is always some code you just don’t want to touch (the creaky floorboards and rattly shutters). That might be things like Excel’s recalc engine or Word’s text layout engine. Both are incredibly complex and it is easy to introduce subtle flaws that make the apps almost worthless. Consider the ramifications if Excel miscalculates some spreadsheet for a large Swiss bank. Think they won’t blame Microsoft? Or if Word suddenly doesn’t lay text out the same on OS X 10.5.2 as it does on 10.4.6 because of a change to the Cocoa text engine?

Now, having said all that, don’t get me wrong — Cocoa is a great thing. Our internal infrastructure team has written tons of tools in Cocoa. Just ask David Weiss. Cocoa is really good for setting up a UI and hooking controls and behavior together, and then letting you write the backend. It’s also cool because you can easily subclass all sorts of OS objects and behavior. That’s a pain to do in Carbon where everything is a C interface and you have to write all the glue yourself.

We have lots of plans for Cocoa for upcoming work (no, I won’t tell you when, where, or how — yet) and lots of areas where it makes total sense for us to use Cocoa. In fact, some areas it would be downright stupid of us to do some new features in Carbon. The point of this post is to hopefully help you see why moving to Cocoa is not a slam dunk simple decision. We’re well aware of both the great advantages and great disadvantages of Cocoa and of Carbon, and are making educated engineering decisions on which to use where.