Ask the Author of the Latest MS-Funded Windows vs. Linux Study 449
Last week on Slashdot you saw a (Microsoft-funded) research
study on Windows vs. (Novell) Linux reliability by Dr.Herbert
Thompson. Novell disagreed
with the study's conclusions. So did most Slashdot readers.
Thompson's work been mentioned on Slashdot before, especially his
famous five-line
script that could change electronic voting machine results
and his novel, The
Mezonic Agenda: Hacking the Presidency. He's a real,
genuine-article computer security expert (and regular Slashdot reader)
who is happy to put on his flame-resistant
suit and discuss his Microsoft vs. Linux study with you. So
ask whatever you like, one question per post. We'll send him 10 of the highest-moderated questions and publish his
answers next Monday. He'll jump into the discussion then, which ought
to make it rather lively.
My Question (Score:5, Insightful)
Re:Why risk your creditibilty? (Score:5, Insightful)
Aren't these studies worthless? (Score:5, Insightful)
My personal bias is that Windows systems are good for being domain controllers and file servers for Windows clients, and the UNIX/Linux is better for your typical "headless" dull day to day server stuff like web servers, email, database servers, HPC machines, etc.
So my questions are: Are these studies worth anything more than pseudo-science advertisements, and if so why? And why is the end of life so rarely discussed?
Re:My Question (Score:5, Insightful)
This isn't something that I think can be answered as no matter what he says most of the readership here won't believe him (myself included).
Regardless of any study *I* have interpreted data for, I'm always looking to slant it in *my* favor. There's no way that *any* one person is able to present a set of data, paid for or not, in a neutral manner.
Even if they can, we won't believe them unless it's for our side
Better question (Score:3, Insightful)
Re:What about negative results? (Score:5, Insightful)
How would Dr. Thompson ever know that? Has he been in charge for a lot of MS-funded studies lately?
Re:Why risk your creditibilty? (Score:3, Insightful)
Contrary to popular belief, except in circles like slashdot few people have trouble accepting a report that favors a commercial product from the market leader over a distro based on a free product from a minority player. It is after all the way most reports go (which tend to make little fuzz because preserving status quo isn't very exciting), and there's nothing uncommon about being commercially funded by one side. You make it sound like this is something rare and dangerous. It's common and everyday, even when the reports go against Linux. Is a Linux study funded by GNU/FSF/OSI/OSDL or whatever any more impartial? No. Do you have problems finding people doing it? [sarcasm]What? Wouldn't they be afraid to be considered communist hippies?[/sarcasm]. Most people go out there and try to make a honest living (or at least not further than sensationalizing headlines to draw page hits). Of course there's a small group delivering FUD on demand, but they are far from the majority.
Re:My Question (Score:4, Insightful)
Because if you don't, no-one will fund your research again. Anyone can find marketing people and spin doctors. Quality researchers are hard to find, and if there is evidence of biased or forged research, their career is at an end.
Re:Why risk your creditibilty? (Score:1, Insightful)
i.e. - Any idiot knows that a study becomes worthless when independent funding is lost (even if the results are truthful). Exactly how much did Microsoft pay you in order for you to submit to such a fiasco and lose your credibility for life?
To someone like me (who believes that Windows is much cheaper than Linux in many cases), this is something that I'd like answered.
Meta-credibility? (Score:5, Insightful)
To be sarcastic, I'd ask "who the heck actually takes these studies seriously?", but obviously *somebody* does. Who are these people, and why do these people take these inudstry analyst firms/journals/reports seriously? Are they right or wrong to do so? This isn't an attack (or endorsement :) of your research -- I'm talking about the credibility gap in industry research, and my observation that it's an industry-wide problem.
The meta-credibility question is this: Given the amount of shoddy pay-for-play research out there, does being published in an analyst journal tend to cost (a researcher, his consulting company, his financial backers) more credibility than it can gains him/her/them? If not, why not -- and more importantly, if so, is there any way to reverse the trend?
Re:My Question (Score:1, Insightful)
Re:My Question (Score:3, Insightful)
Re:Why risk your creditibilty? (Score:5, Insightful)
This really isnt about Linux its about making computers and their software be as standard as the internet.
Somewhat less germaine (Score:3, Insightful)
ATMs vs. Voting Machines (Score:5, Insightful)
Also, does the flame-resistant suit come with its own matching tinfoil hat? (don't answer that one)
Re:Results (Score:5, Insightful)
I mean, by whatever deitys protect sysadmins, _manually_ upgrade _glibc_??? I havent done that since before package systems were invented.
"If the conditions were different"
You mean, if the Windows admins spent most of their time manually copying files in dos shells from floppy disk because they for some inexplicable reason didnt want to use more modern methods for handling such problems?
If the conditions are to benchmark people doing things the wrong way then I rather doubt the value of the conclusions.
What are you talking about? (Score:3, Insightful)
Re:Why risk your creditibilty? (Score:2, Insightful)
Yes. As far as I'm concerned, they are whores, just like this guy.
[sarcasm]What? Wouldn't they be afraid to be considered communist hippies?[/sarcasm].
Oh! You assumed I was going to answer some other way. Huh. So much for your whole damned post having any point at all.
Integrity (Score:3, Insightful)
-everphilski-
Re:Why risk your creditibilty? (Score:4, Insightful)
I think many here would disagree. Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on. They represent what are largely hobbyists, almost to a maddening degree.
OTOH, in Microsoft we have a callow and selfish for-profit entity with a rather abusive track record right up through their financial, er, daliances with SCO.
Need I say more?
Given their dynamics and history, being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
If you actually read the report... (Score:2, Insightful)
Comment removed (Score:5, Insightful)
PHBs who listen (Score:3, Insightful)
First, let's recognize that anyone experienced enough with both operating systems will have their own experiences that will tell them which OS is better in various ways. These people are unlikely to be swayed by studies. Therefore, the first thing that is critical to understand is this: these studies are aimed at people who are NOT experienced with both OS's.
As such, it seems there are two potential groups who are targeted by such studies: 1) CIO or sysadmin types who are experienced with windows systems, and who were thinking of trying linux; and 2) PHBs. For the first type, the MS studies are meant to deter. For the second type, the MS studies are meant to indoctrinate.
For example, let's say MS saturates WSJ, Fortune, and similar newspapers/magazines likely to be read by PHBs. They read it enough times, and given they have no field knowledge of the various TCO variables, they believe what they read from seemingly "objective" sources. What MS then wants is this: when an intelligent CIO or sysadmin goes to the CEO and says "Let's try linux, it's great!" the CEO says no, and considers the CIO incompetent for even considering such a blatantly horrible idea.
So basically these studies are meant to influence decision makers who don't have hands-on knowledge. It's a very good idea, really. It will keep Linux adoption a lot lower than it would be otherwise.
Re:My Question (Score:3, Insightful)
Garter,
Enderle consulting,
Do any of these names ring a bell? all but the last on claims to be unbaised but their reports can be shown in minutes to us predetermined Data.
So what was that about being neutral again? Which group is making more money than you do every minute?
Re:Why risk your creditibilty? (Score:5, Insightful)
Hang on, you're saying you believe that you would trust a FSF or OSDL-funded study to be impartial? You're saying that if the FSF funded a study comparing GNU to Windows, and the study came back saying "Windows saves you money in the long term, and Microsoft's Shared Source is as good as Free Software for 99% of users", that the FSF would then be happy to publish that study?
I don't think so, and I suspect you won't either, if you pause to think about it.
Nonprofits are not driven by motives which could be considered the mirrored opposite of commercial corporations. There is not the tremendous pressure to turn a profit (or some analog to monetary gain), and in your examples they're run by mere handfuls of individuals receiving very little compensation with only their reputations to fall back on.
But that doesn't make them impartial! All it means is that the profit motive is replaced by other motives. And there are plenty.
Think about how much time the major contributors to free software projects put into those projects. Hours, days, months, years of personal time, freely given. Time that could have been spent earning money, or doing charitable work, or even just spending time with their families. Time that was wasted, if it turns out that the software they produced is not actually going to help many people do anything at all.
When you reach middle age, and the end starts to heave into sight on the horizon of your life, you start to get very, very uncomfortable about the idea that you might have devoted your precious time to an unworthy cause.
Being so dismissive of FOSS organizations as to just say 'well, eveone's biased anyway' really doesn't seem like an acceptable attitude.
What's dismissive about that? Microsoft really does think that everyone ought to use Microsoft software, and the FSF really does think that everyone ought to use free software. Everyone is biased. Pretty much everyone does have a pre-existing investment, either of time or money, in one of the options. And human nature does dictate that when you have an investment in something, you are biased towards accepting studies that support it and disregarding studies that don't.
What's wrong with telling the truth?
Re:Why risk your creditibilty? (Score:3, Insightful)
Re:My Question (Score:3, Insightful)
When you're not being paid, though, it is possible. Take a look at science, for example; if you replace "study" with "experiment", you'll see that the whole thing isn't much different really, and in science, most people manage to be at least reasonably objective. You may already have an idea about how things work (that is, you may already have a scientific theory); but if you do an experiment and the results don't fit into your theory, then you'll most likely modify the theory until it works again. (Or you might even throw it away altogether, but I think that's less common.)
Of course, some scientists don't manage to do that and instead get so attached to a particular theory (for whatever reason) or come to dislike a new theory so much that they simply reject it on emotional grounds, without a real, rational, scientific reason. This is unfortunate, but it's not the norm - most scientists manage to be objective.
A good example might be Einstein and the "spooky action at a distance". Did he dislike it? Certainly. Did he think it couldn't be something that actually occured in reality? You betcha. But he didn't reject quantum mechanics because of it; rather, he tried to find a way to modify QM to get rid of what he believed was an artifact.
Of course, we nowadays pretty much know (to the extent that you can truly positively "know" something in science) that he was incorrect and that SAAAD really exists. But the point is that Einstein, while he did not like the idea of SAAAD, reacted in a scientific fashion instead of resorting to dirtier tactics like spreading FUD, buying studies, and all that, which unfortunately seems to be all too common outside of the scientific community.
Regarding Mr. Thompson, I'm not sure if he's a man of questionable ethics who doesn't mind creating a study that comes to a predetermined result if the money is right, or if he is naive enough to truly believe that he can be objective in a study commissioned by and paid for by Microsoft.
And just to make it clear, I don't want to comment on the findings of the study as such at all. I do have my own opinion, of course (Linux/Unix rox, Windows sux, and yes, I've used both for more than a decade), but I don't know exactly what question the study was supposed to evaluate, so I can't comment on it, naturally. I just think that the methodology in this case is rather fishy - praise for company A paid for by company A always is, even if it should be well-deserved.
Re:Meta-credibility? (Score:3, Insightful)
So who do you go to when you have question then? Eventually you have to trust somebody when it comes to a topic that you've reached the limits of your knowledge of.
More importantly, when you make a decision, and something goes wrong, how do you explain your decisions? You could say "I used the recommendations from $RAG about $CORP." Or you could say "I didn't trust the people at $RAG, but I asked my colleagues Jimbo and Cleatus," and have a difficult time explaining problems to the non-techie management.
The meta-credibility question is this: Given the amount of shoddy pay-for-play research out there, does being published in an analyst journal tend to cost (a researcher, his consulting company, his financial backers) more credibility than it can gains him/her/them? If not, why not -- and more importantly, if so, is there any way to reverse the trend?
I think once you figure out who to ask, and that multiple people can agree that the source is reliable, then it's just a matter of getting people to use that.
Linux by other names (Score:2, Insightful)
Re:If you actually read the report... (Score:5, Insightful)
OK, I've found and read the report now, and this is just bollocks. From the report:
So the test involved installing on SuSE 8 two applications that (effectively) required SuSE 9. Rather than upgrade to SuSE 9, the test mechanism required the operators to hack their systems to make this work. Some of them did this by taking the ill-advised step of compiling their own glibc; doing this broke the vendor supplied version of 'rpm', leaving them unable to undo their changes. Others did it by partially upgrading their system to SuSE 9 by installing SuSE 9 rpms over their SuSE 8 equivalents.
The Windows equivalent test worked fine because the equivalent applications that the Windows operators were required to install were intended for use with the version of Windows they had installed.
Basically, the test wasn't fair. If SuSE-9 dependent applications were to be used, then SuSE 9 should have been used as the basis of the test. If SuSE 8 had to be tested, then equivalent applications that functioned on SuSE 8 should have been found (chances are, slightly older versions of the same 2 apps would have functioned fine).
So, no, glibc wasn't "mucked up because SUSE's YAST was broken". The operators broke YAST by trying to install a glibc upgrade in order to use an application that wasn't compatible with the system they were running. The test was unrealistic; they weren't given the option of upgrading the system properly. They were told, "make this application run on this system." It's not surprising that some of them failed.
Re:If you'd read the study... (Score:5, Insightful)
They broke RPM by hand compiling glibc, not the other way around. It says so quite explicitly. They hand compiled glibc because they were asked to install (without upgrading to SuSE 9) an application that wasn't compatible with the version in SuSE 8.
Debugging in Linux (Score:3, Insightful)
The two are largely equivalent.
I use emacs gdbsrc mode to debug my code, and I can set breakpoints, conditional breakpoints, step in, step over, print any expression, or call any function I want in the debugger. If I recall correctly, you cannot really manually call functions in the Visual Studio debugger, but correct me if I'm wrong.
There are also advantages to gdb frontends though:
Please explain what extra productivity or features you gain from the Windows debugger.
As for your selection of tools:
Missing the point (Score:3, Insightful)
Its that for certain kinds of purposes, the current situation in the real world, is that, for no good technical reason, software only exists for Windows.
Due to this unfortunate situation, Windows is superior at achieving certain real world tasks.
People who just accept this and go through the path of ethical lazyness get bitten in the ass by the lockin they are themselves creating.
Re:If you actually read the report... (Score:3, Insightful)
They should have had them on Debian Stable or Slackware. For fuck's sake, Gentoo would have been a better choice for this than SuSE. RPM-based distros always seem to be the hardest to change or upgrade piecemeal, without doing a full upgrade to the latest version of the whole OS. I've used Mandrake and Fedora extensively, and pre-Fedora Red Hat and SuSE quite a bit, too. They all have these sorts of problems. You learn to be very careful with upgrades to individual packages, and you learn to upgrade to the newest version of the OS at the first sign of trouble with a package upgrade, before you've dicked with it so much that the system gets broken. If you can't do that for whatever reason, then you use a different distro. Simple as that.
Eh, this is mostly just a "me too!" post... but damn, that's just so dumb that I had to say something!
Re:Why risk your creditibilty? (Score:3, Insightful)
Cut: Ctrl-X
Paste: Ctrl-V
That uses one buffer (i.e. KDE's or Gnome's buffer). Not only is it standard, but it is the exact same shortcuts as those used in Windows!
X's buffer is used by:
Select: Copy
Middle-mouse : Paste
You don't need to know about one to use the other. They work independently of one another. I've been using Linux since 1998 and I've never has a problem cutting and pasting between application under Gnome, KDE and Ion3.
Config files? System wide are in
I find it odd you would choose these things to complain about. Now, if you want to talk about wireless drivers, it's a whole different ballgame... =)
Statistical Significance. (Score:5, Insightful)
You note yourself, in your study that the sample is based upon 6 system administrators/systems. That number is, as you yourself note, too small to be considered definitive. That being the case I would argue that this makes the report viable not as a decisionmaking tool but a marketing tool. Were I a CIO I would feel unwilling to base my conclusions soley on a sample size of 6. What is your opinion on this? Do you expect further, more statistically-significant, work to take place? Or do you feel that this is not a problem?
Vendor Tools (Score:4, Insightful)
Upgrade Glibc? (Score:2, Insightful)
First of all, thank you for participating in this flamefest
I read the Executive Summary of your report and skimmed the rest, so pardon me if I failed to notice something vital.
It seems to me that the demand that your Linux Admins were asked to upgrade Glibc led them to fail the majority of tasks, creating an artificial bias against Linux.
Any Admin worth his weight in pizza knows that you Just Don't Do That.
If you absolutely, positively need some component, you get the version which works with your Glibc. All hell will break loose as soon as you upgrade Glibc and especially if you don't recompile the rest of the system. For an organization which needs commercial support from the OS vendor, this is unacceptable and your Admin should have refused to comply. If your web programmers need a specific component, they should get the component which works with your system.
I understand that this induces "pain" on your organization, but that pain should be much milder than the one your Admins experienced, and as a result, your organization.
My question is therefore: How can you defend the demand to upgrade Glibc when it is so obviously designed to force the Admin to fail?
Thank you very much for your answer, I look forward to reading your reply.
Three Questions about the Study... (Score:2, Insightful)
Re:Why risk your creditibilty? (Score:3, Insightful)
The FSF keeps its nose squeaky clean, because they know they have to to keep any respect from their members and from the world at large.
Here is my question (please read it). (Score:1, Insightful)
According to what I have read, you did not use the most recent version of Redhat Linux in your comparison. Also, the specific hardware build you selected was one that is publicly known to give very poor performance with Redhat Linux.
Why did you select this version of Linux, and why this particular hardware build? More to the point, what decision-making process do you use to ensure that the OS and hardware configurations you use for your studies are good matches for one another? I guess what I am really asking is, what measures do you take to prevent the "bad match" of which you have been accused here on slashdot? And please do be specific, we love the details.
Thank you.
Re:Why risk your creditibilty? (Score:3, Insightful)
When you reach middle age, and the end starts to heave into sight on the horizon of your life, you start to get very, very uncomfortable about the idea that you might have devoted your precious time to an unworthy cause.
Which is supportive evidence that your argument is too biased in itself to consider.
How is helping to produce freely given and very secure software for EVERYONE ELSE TO USE FREELY an unworthy cause? Indeed, one could (and I do) consider it "charitable work".
Oh, and I'm on the approach ramp to middle age; and my feelings about helping only get stronger year after year.
It's not the fact of bias that you're not considering, it's the *reasons* behind the bias. (Yeah, I've heard a lot of people say that altruism doesn't pay the bills. But as a blanket statement, that's bullshit).
SB
Comment removed (Score:3, Insightful)
Re:If you actually read the report... (Score:3, Insightful)
My experience is about 6 hours for the upgrade, plus another 3 or 4 to check everything still works afterwards. My experience of compiling my own glibc suggests that this will take about twice as much work.
Downtime can cost a lot of money, this would have been a pressure on the admins.
Any real company employing the kind of solutions described (which included so-called "best of breed" commercial applications) would certainly have a staging server to use, and could then swap the staging server for the live one in order to deploy. This would result in no more than a minute's downtime if done correctly. It's possible to do it with zero downtime.
And isn't that kinda the point of the study?
I think by insisting they stick with SuSE8 and use applications on it that blatantly aren't compatible with it they skewed the results. I know if I'd had all of the other requirements given, I'd have done a complete upgrade.