Beta Perversion article (by Rick Strahl) (from: Alex Feldstein)
http://www.bloglines.com/blog/AlexFeldstein?id=247
There is an interesting discussion written a little while ago by Rick Strahl [MVP] titled “Beta Perversion“.Rick complained that Microsoft is diluting the value of a Beta tothe point that it is not a Beta anymore. There is confusion on Betaswhich is a limited exposure of a new, near-complete piece of softwareto a select group of expert customers or advisors to try to find morebugs and otherwise make constructive criticisms on this software.Instead, it has now become a marketing tool with a wide exposure as aCustomer Technical Preview which does nothing more than show off anot-ready-for-prime-time product with no effective way to manage thefeedback. In many cases, the next CTP has had substantial featurechanges as to make some of the previous tests by these users irrelevant.
Josh Ledgard from Microsoft posted an extensive rebuttal explaining the point of view from his side of the aisle. Rick answers in “Beta Perversion Redux” defending his position and clarifying the points made earlier.
Botharticles compare the excellent way the Microsoft Visual FoxPro Teamhandles its Betas (most recently Europa-VFP 9.0) and the mess thatVisual Studio 2005 (Whidbey) Beta has become. Acknowledging the pointJosh makes on the difference in scope between the two programs, I agreemore with Rick’s comments that on a vast product like VS2005 one canbreak it into subgroups (sub-betas) for each piece of the program, withsmaller teams of a knowledgeable and dedicated group of beta testersworking on each.“There’s really two issues that the discussion boils down to:
Previewing the software is quitedifferent than the actual beta process which is designed to help ironout problems and inconsistencies. While there’s some similarity thedifference is usually in the amount of involvement by the beta programprovider - ie. Microsoft.
- Previewing the software
- And beta testing
My contention is that Microsoft is doing a lousy job interactingwith people here. While Previews are clearly not production quality andare meant for previewing and experimenting, with a product the size ofVS.NET I find it almost too much to sit down figure out how somethingworks without documentation, without support and somebody to ask that’sinvolved. Then find out that the code and concepts we spendhours/days/weeks figuring out have been pulled or changed so it’scompletely different in the next build.”
I’ve been in smaller-audience betas (from Visual FoxPro to FlightSimulator) along with bigger ones like VS2005. Having had thatexperience I have to agree with Rick’s comments and concerns. Microsoftshould do a better job overall by separating the beta testing and themarketing processes, labelling each one as it should be. They shouldpay more attention to their MVPs, a group of very knowledgeable peoplethat will tell it like it is. After all the goal is to worktogether to make a better product, is it not?