Ask Dan Kusnetzky About Linux Server Counts

How many people run Linux as a desktop OS? How many servers run Linux? Is the Linux server market share 8.6 percent or 24 percent or somewhere in between? Dan Kusnetzky is a heavily quoted analyst at IDC who wrestles with questions like these for a living. This is your chance to find out how analysts come up with all those numbers -- and why analysts seem to disagree with each other so often. One question per post, please. We'll send 10 of the highest-moderated ones to Dan, and post his answers as soon as we get them back.

What was your methodology.

How did you go about researching your figures? Who was involved? How many people companies or vendors did you ask? What statistics did you track?

In short how did you come up with your figures. We could ask about the other figures too but since you didn't publish those someone else may have to answer that.

Factoring in the Captive Audience

How do you handle large blocks of users that are forced (often against their will and better judgement) to use a particular operating system? For example, 360,000 Navy users are forced [eds.com] to use Windows 2000.

SuperID
Free Database Hosting For Developers [freesql.org]

Linux can't feed you

During Linux servers market share study (or similar studies involving Linux), are you concerned about the fact that there's no big Linux company who will ever buy you such work, meanwhile Microsoft or other giants sure will ask again for it in the future ?
Does it affects you or your team work ?

A question about your report

I've read your report, and I didn't find any description of the mechanism by which you came up with your estimate of the average number of times that a given purchased copy of Linux is installed. That's probably my own error, but since you're answering questions, would you be willing to enlighten me?

You use the number "15". Frankly, I'm surprised that it is so big. Upon what data did you base that estimate? Who did you interview to get it? I realize that any such estimate would have to account both for the very large number of installs at large colocation and/or service provider shops, as well as the number of untrackable network installs that take place. But it would also need to account for the number of times where a machine had Linux installed upon it, was used as an experimental development platform, and then was wiped, not to mention the number of cases where somebody bought a distribution, and then never installed it at all.

Re:I just have to ask...

I would like to see a breakdown of server O/S weighted by usage. For example, a server that receives 1000hits/hour would have 1000 times more weighting than a server which receives 1 hit/hour. Such a weighting would give a much more honest view of usage. In other words, which server O/S is getting the most work done. I suspect this would lead to significantly different results from what we have seen so far.

linux ports.

Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?

no, they count as targets for l33t haX0rs, especially since the people who just installed mandrake from that wal mart cd don't even know they're open or use them...

--saint
----

Hardware sales mean very little

When most desktop and server Linux users are building their own boxes, how can sales of server and pc systems be relevant? Most people I know that use Linux (myself excluded) first bought a PC with wDOS on it, removed that OS and installed something else. What kind of formula do you use to make up for this disparity between systems sold and systems built ?

Question

Hi Dan,

According to IDC figures, the Linux desktop market share was 5% and now it's less then 2%. Gartner numbers are of course differently (way lower if I'm not mistaken)

Yet, when I look at developments of projects like XFree, KDE, Gnome, Linux kernel - or when you get the daily list from freshmeat, or even talking to the the ISP who host mirrors of ISO images of Redhat, Mandrake etc - then you see that linux get FAR more then 2%. Hell - if it was 2% and you account the developments of Linux - then each developer works 25 hours per day on a porject!

So, as you can see - the numbers here are definately wrong here - and those numbers are actually hurting the Linux community. If an ABCD company wants to make a software for the Linux desktop and they see those IDC figures - then they will say something like "oh, 2%? no thanks - we'll make it for Mac - they are %5+", and we'll loose..

Comments?

Company size.

Is there a relationship between company size and linux use?

Do we have evidence of any companies with multi-million dollar revenues that rely on linux solutions for their servers or do most companies that use linux servers do it because they lack the money? (choice/only option)

-S

Scott Ruttencutter

Assuring accurate results?

Dan,
How does IDC determine if the answers they are getting are valid? What I mean by this is how does your organization insure that the people they are surveying being truthful in their responses? I can see scenarios where a well co-ordinated group of people could be swayed by a vendor to influence the results of a survey, especially in a field as specialized as the IT industry.
Also, if you suspect that the results of a survey are tainted, do you publish it anyway (with caveats included, of course) or redo it (at your time and expense)?

Re:Funding

In studies of market share(or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.

Even if the funder does not infulence the questions asked they may well control if the "results" see the light of day.

Re:Polling questions

I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method?

It's quite trivial for methodology to affect the results. That is why in any kind of scientific publication the method is considered at least as important as any conclusion.
Not only is no methodology available "conclusions" are being called "results".

How do you discount overcounting?

Hi Dan,

This is probably going to be a tough question for you (if you ever get it, that is - given the bias on Slashdot, this probably won't be modded up).

A big problem I see with your methodology is that you probably overcount Linux server shipments. From what I understand (I may be incorrect here) you count each sale of Linux as a server shipment. However, many of these copies may be tested but never used, a great many are used in home computers or development workstations. The problem is, unlike with Windows or OS/2, there is no ``client version'' vs. a ``server version''. There is no easy way to tell if a copy of Linux is used in a server environment or a client environment by sales figures alone. So my question is: given total Linux sales figures, how do you estimate the number used in server environments?

Secondly, is it possible that the descrepancy between your numbers and Gartners' is due to a problem in the way you estimate Linux server sales?

What difference does it make?

What is the real world impact of these numbers? Is it just an ego boost for the producer of product X? Does it influence buying? Does it change R&D strategy? Especially since the numbers differ so much from analasyst to analasyst, how does anyone make any sort of decisions bade on this information?

---

why not just ask?

The technical approach of talking to publicly accessible servers and attempting to fingerprint the OS is fraught with methodological problems.

The naive approach of asking hardware vendors how many units of hardware ship with what OS is obviously flawed.

So why not just do the dumb brute-force thing? Dial phone numbers at random and ask people what they use? Too expensive?

How much is accurate data worth, anyway?

Trends

Do you have any insight about trends that can be gleaned from these surveys - that is, regardless of what number one uses for the percentage of users that adopt Linux, is it static, taking off, steadily increasing, or what?

*what* consititutes a Linux server?

At what point does a computer become a server? Many Linux desktops have ftp, telnet & http ports open, so do they count as severs too?

Re:Question(ADDNUM)

I just remembered an Idea that a friend of mine suggested:

What if IDC could work with the Linux distributions (RedHat, Mandrake, SuSE, TurboLinux, Debian) to add a small program which will run after the first internet connection has been succsessfull..

When this program runs - it will ask the user to "register" his copy of the distribution. If it has been purchased from one of the distributors - then the user can add his serial number. Some other questions like will this distribution be used as a server, a workstation, or combination of the 2, or a development workstation. The survey SHOULD be annonymous (unless the user wants to give some details about himself)

By that way - the distributors can give the numbers back to IDC - and IDC can publish a report which will tell that the number of Linux installations - and that number is X. X is combined of Y free download version and Z purchased copies of Linux.

What do you think, Dan? what the slashdot readers think about it?

Data origins

Do you base your data mostly on marketing analysis or do you actually go a pay a consultor to scan machines on the net? If there are scans involved, how do you pick the IP blocks to be scanned and what's the uncertainty associated with such a method (and how is this uncertainty guessed)? If there are no scans involved, why not? If this is "maket analysis", can you defined that for me? Which factors are involved? And a different question: who's the target market for this kind of study? How much does such a thing cost?

Reproducibility

Has any thought been put into doing a "reproducibilty challenge" against Gartner? That is,

1. Ask Gartner their methodology,
2. Document it so that it can be reproduced;
3. Make sure and get Gartner to say "yep that's how we did it".
4. Dan&Co reproduces the methodology and compares the numbers

Might be overly scientific. Maybe consider it an "Open Source" version of market research because of the peer review and verification. Some may think this approach has no place in the market research area.

But, I (for one) think it would be interesting to call Gartner's Bluff (if indeed that's what this is). I personally place them (and ZD, etc) into the Shill [tuxedo.org] category.

Thoughts anyone?

I just have to ask...

Might he have the stats on the BSD's. People from the BSD community are curious. :)

TIA

Self-fulfilling analysis?

Do you think that there exists a possibility (or can even provide examples of) self-fullfilling analysis, such as "analysis says X is losing market share=>people get skeptical about X=>X loses market share although it may not have done so otherwise"?

If it is possible or already happened, do analysts in general (and you in particular) find it a worrisom possibility, and if so, are there any attempts/ideas to deal with the issue?

Thanks,
DVK

Funding

In studies of market share(or studies in general), we often hear quotes about who funded them. This seems to somehow imply that those who funded the study had some influence in how the data was gathered/interpreted.

My question is: Do those who fund a study influence how the study turns out (ie. Microsoft studies show higher MS market share). Or is it that these corperations only decide to fund groups who they know will most likely return results in thier favor.


---Lane

Who Keeps track of the Predictions

Mr. Kusnetzky,

I have a question on the area of predictions in general. For example, fellow IDC analyst Jill House has been severaly negative on Palm over the years, with regard to the Win CE operating system and devices. A sample quote from her in Feb. 2000 read, "If I was Palm, I would be beside myself with panic. [zdnet.com]"

The issue is, that over the last 3 - 4 years she's been predicting the demise of Palm and the rise of Win CE, a claim that has never materialized. Who verifies the reliability of these predictions and keeps the analysts accountable. With the frequent sound bytes and one-liners that they give to the press, these analysts have significant influence over public perception of the issues. But how is policing done when the analysts don't analyze very well?

Sincerely,
Daniel McCarty
Palm OS Developer

What about the so-called "third world"?

Do these analyses factor in the so-called third world? Most of these analyses are US-centric or some times do include the continent across the pond but what about Africa and Asia? Do these analyses *really* take inputs from these continents?

Here's a simple one...

Do you think Gartner Group caved in ethically when Microsoft paid them for this study? That is, do you believe it to be a purchased fabrication? Or might it be an honest mistake, a difference in data analysis, or something less sinister?

Thanks for your time,
- Brad Heintz
--

Polling questions

I have often wondered how biased polls are based on the questions asked, the demographics of the people polled, etc. When results about polls are made public, is it also possible to obtain information about how the poll was conducted in a simple, by request method? Now if the answer to that question is, no, how much can we rely on polls, since we have no way of verifying if the questions asked and the people interviewed were heavily biased to favor one outcome over another? (Such as in the recent large discrepancies of the 8% vs. 24% use of Linux as a server results that we've seen on Slashdot recently).

More breakdown needed

When People magazine does an issue devoted to "what's hot" in fashion, do they interview Jane Doe from Des Moines, Iowa? No.

So why are OS numbers reported with equal rating? Not all users are equally suited to *choose* an OS, therefore not all users *choices* are equally interesting. I'd really like to see a breakdown of OS by user-type (levels of education, field of degree if applicable, occupation, etc). Keep in mind this applies just as much to business. A technology company presumably put more informed thought into their choice of server than an art supply house or whatever.
--