Ask Ingo Molnar About TUX

Ingo Molnar is the guy behind the TUX Web server, which produced those astounding SpecWeb results reported here last week. He's agreed to a Slashdot community interview. So ask away at the man who created what appears by some measures to be the world's most powerful Web server at present. Please make Ingo's job easier by first reading the LinuxToday articles (here's the first LW story, and here's the second LW story) commenting on the SpecWeb numbers and the background of how they were achieved, as well as Ingo's informative post in the initial Slashdot story, and the SpecWeb results themselves. The moderators may have no mercy otherwise.

My name is Ingo Molnar

You killed my penguin.

Prepare to die.

I already asked this once

Given that part of the explanation for Tux's impressive performance is the use of a kernel-based httpd server, how much how stable and secure do you expect it is.

BTW, this isn't a flame. I'm sure it's better than IIS/NT on both fronts--but is it better than Apache/Linux, even after factoring in the speed?
--

Versions for other OSs?

Will there be versions of Tux for other Operating Systems, such as FreeBSD, OpenBSD, BeOS, MacOS, Windows and others?

Why

What is the reason for moving the http server into the kernel, I do not see any benefits to it, could you enlighten me about it.

TUX as a real web server

Would you ever advocate using TUX as a real life web server?

Think of a high availability environment, where you are building a highly dynamic application such as an e-commerce system. Would you even think of using TUX in such a situation, or would you go with the far more sensible Apache + mod_backhand + (pick one of mod_perl, php, or servlets)?

The problem is, its all too easy to generate web server software that can withstand a high "hit" rate. But the pressures on web servers, and particularly web developers, lie in completely different areas: Time to market, ease of development, and configuration capability.

Server Extensions

Are there any plans for or existing features in Tux that allow for adding custom modules such as WebDav? How about custom protocols?

Version 1

This is a version 1 of the web server, and it has proven itself to be pretty nifty when it comes to serving both static webpages (through a kernel level httpd) and dynamic webpages. Do you see TuX getting more lean and faster as time wears on, past versions 2, 3, ... or do you see it getting bogged down in mostly unnecessary cruft and bloat?

Will there be a way to port an existing Apache configuration across to the TuX configuration? How about IIS, Netscape, Zeus, etc? Will TuX have the option of a GUI setup screen for those who don't like the command line? Will TuX have a simple installer?

How portable is this?

This is based very much on the Linux kernel, so I presume that it couldn't be ported easily to different kernels. Hwever, are there any major optimisations dependent on the use of a x86 architecture?

What is TUX's goal?

What is the target purpose of TUX? From the benchmarks, it appears to be for very high traffic sites. I ask because I'm curious if that is its specific purpose, or does it serve well in other areas as well (i.e, Apache flexibility)?

This is important because it will also help indicate what Red Hat's stance will likely be in either replacing Apache with TUX or just including it in their Professional distributions.

It works with Apache...

It works with Apache but is TUX generic enough to be interfaced with another server?

Load balancing

Does/Will TUX provide any sort of load balancing for a cluster of heterogenous TUX servers?

Kernel modules decrease portability?

You mentioned in the second Linux Today article that you intend to integrate TUX with Apache. However, Apache has always been a cross-platform server and is heavily used on *BSD and Solaris. Do you feel that this integration will undermine the portability work of the Apache team, or will it simply provide an incentive for web servers to be running Linux? If you intend to encourage people to move to Linux, can a similar idea as TUX be applied to an SQL server to make up for the speed deficit between Linux SQL servers and Microsoft SQL?

could you have made it faster

You've said before on slashdot that TUX supports a number of usability features, although not the full complement. Could TUX have been made faster at the job of serving static pages if you had ripped out every single extraneous bit? What if TUX had been small to the point of being basically unusable in the real world, but served pages faster even than it does now?

The question is, would that have been a fair benchmark?

If your answer is No, then the followup question is, how is that materially different from what you *did* do?

-konstant
Yes! We are all individuals! I'm not!

Re: Ingo's informative post

Ingo's informative post [slashdot.org].
-russ

patent plans?

In the second LW article, ingo writes:

- So in our opinion TUX is a new and unique class
- of webserver, there is no prior art implementin
- such kind of 'HTTP stack' and 'abstract object
- cache' approach. It's i believe a completely
- new approach to webserving. Please read this
- comment too, which (i hope) further explains
- the in-kernel issue:

Maybe I'm paranoid, but "new and unqiue"
and "prior art" in the same sentence mean
patent filing to me.

Are there plans to see patent protection for
TUX? As I recall, the RTLinux folks got a
patent for RTLInux's prioritization stuff.

Is a patent in the works?

Regardless, TUX is an interesting idea and I hope
to try it out soon.

--chuck

Re:BAD LINK IN STORY

I think the link it should be is here [slashdot.org], though he answered a bunch of questions in that story. If you browse through his user-info [slashdot.org], you'll be able to see all of the informative posts he's made recently...

Threading and Linux

Unix programmers seems to dislike using threads in their applications. After all, they can just fork(); and run along instead of using the thread functions. But, that's not important right now.

What is your opinion on the current thread implementation in the Linux kernel compared to systems designed from the ground up to support threads (like BeOS, OS/2 and Windows NT)? In which way could the kernel developers make the threads work better?

Primary Market

What do you see is the primary market for TUX? With the ability to handle such high traffic it would be suitable for busy e-commerce sites, but I'm sure there will be more than a few people who are wary about a http server in their kernel. However, the embedded market might see this as a good thing and allow for web-based configuration, monitoring, etc of embedded devices. I would think there would be more people willing to just throw the webserver in kernel to save space if nothing else, but of course this still raises the security/stability concern.

So the main question is really just where and in what applications do you see TUX in the future?

Performance using dynamic content.

How would TUX perform using CGI/Servlets/PHP/etc. compared to Apache or IIS? The ability to serve static pages fast is not that useful in the real world, as all the sites that get really big hits-per-second are those with dynamic content (Yahoo, Slashdot, Amazon.com, etc.)


"Evil beware: I'm armed to the teeth and packing a hampster!"

Integration into RedHat?

How will the TUX Webserver integrate with RedHat's Linux distributions? Will RedHat create a special distribution with an identical setup to yours? Will RedHat start releasing more specialized distributions, preferably ones more suited to a secure server environment but focused on performance like your setup was?

As it is RedHat seems too insecure and bloated for a streamlined server environment. Ideal would be installation options where I can say "This server will do these 3 things (i.e. DNS, Mail, HTTP) so make it suited for that and nothing else." This kind of flexibility would be a HUGE boon to the server market, giving customers a high performance machine running TUX + Apache that was secure and did the functions they needed it to.


Yea that was a long question, you can chop off last paragraph if you like. Hehe, insecure and bloated, can we say WinNT/2K?

"I want to get more into theory, because everything works in theory." -John Cash

Caching

I have a few questions about TUX's caching system. Before I go any further, I want to say that I'm incredibly impressed by the results. I've been following specWeb99 for a while and have been wondering when someone would manage to build a great dynamic cache like this one. I hope it'll get the wide acceptance it seems to deserve.
First, it seems that basically the entire test file set was loaded into memory ahead of time for use by TUX. How adaptable is TUX to more dynamic, limited-memory environments in terms of setting cache size limitations, selectivity (e.g. "cache all .GIFs, but not .html files"), and expiration/reloading algorithms?
Second, can a tux module programmer modify the basic tux commands, or do they always do the same thing? For instance, if I were adapting TUX to work with a web proxy cache, I'd want TUX_ACTION_GET_OBJECT to actually go out over the network and do a GET request if it couldn't find a requested object in the cache. You can imagine lots of other circumstances where this would come up as well.
Third, is it possible to execute more than one user-space TUX module at one time?
Fourth, when can we play with the code?
Thanks a lot!
--JRZ

How will TUX leverage marketing?

I know it's a sleazy thought, but the reason why Red Hat is the most popular distro has more to do with marketing and deals than the code itself.

So my question is this - how will TUX market itself and what kind of deals are you looking at making so that it becomes more widely adopted?

I don't think we need specifics, just some of the general methods you plan to use for marketing and some probable categories of companies you are looking at making deals with.

[yeah, I know, free bheer - but it's a good question]

Good question!

I've heard one of the reasons for Windows' historical instability is that user applications are permitted (nay, even encouraged) to corrupt the kernel, whereas in a typical vmunix implementation this is not allowed. I.e. my bad calls to free() result in core dumps not BSoDs... But here is a Linux application program right there with hooks in the kernel, and not only that it is hooked up to the network! Is Ingo a) Godlike or b) Nuts?

WWJD -- What Would Jimi Do?

Relative Impact

TUX includes a variety of kernel and apache changes. Can you give a rough measure of how each of the changes improved the http performance? I'm interested in the amount of improvement as well as why it improved performance. Do those particular changes have negative impact on the performance of other applications?

Defamation of Character?

Are you afraid that if a particularly buggy version of TUX slips out the door and trashes people's systems and loses valuable data, that a certain angry penguin [tamu.edu] will open a whole can of whoopass on you for defamation of character? Are you investing in reserves of herring and icecubes in anticipation of this event? Perhaps an adapted Ursus anti-bear suit [slashdot.org]?

TUX performance

You deserve a beer. I'll send you some. What kind do you want?

(Assuming you're not like Raster and would rather have spirits....)

TUX Architecture

You appear to have take an "architectural" approach to designing TUX, so I have some architectural questions.

1. The choice of a kernel space implementation is probably going to be a controversial one. You suggest that HTTP is commonly used enough to go in the kernel just as TCP/IP did years ago. What performance or architectural advantages do you see to moving application protocols into the kernel that cannot be achieved in user space?
2. What is your approach to concurence? In particular, you refer to "event driven". What do you mean by that and what did you choose as the core to your event engine? Also, how do you handle threading to scale on SMP machines?
3. Are there any plans to generalize the infrastructure elements of TUX so that other protocol servers can take advantage of the TUX architecture and also go fast as hell?

Re:Versions for other OSs?

Actually there is a specific feature that would probably make TUX incompatible with the BSDs. TUX is licensed under the GPL and the BSD maintainers would probably be very reluctant to port it to their OSes. Especially since it is possible that this would require them to release the derivative work under the GPL.

Which leads to the obvious question for Ingo. You mention a specific disclaimer that would allow the Apache to be linked with TUX, do the BSDs get the same privilege?

Not that I particularly care, as I am not a BSD user, but the putting such a nifty program as TUX under the GPL is bound to cause weeping and gnashing of teeth in the BSD camp. Which brings up another question. How much pressure do you get from your BSD compatriots to release software like this under a more liberal BSD-friendly license?

Apache Modules

How will our favorite Apache modules, like mod_perl and php, be helped by TUX? Will my mod_perl code or php code run faster with TUX?

How are benchmarks used?

Have you tuned TUX for any particular benchmarks, or do you just write it as best you can and throw it in the ring? If it's tuned to some benchmarks, does that hurt its performance on other benchmarks?

Have any benchmark tests ever been particularly useful for revealing bugs/inefficiencies in your code? That is, are the benchmarks tools to you, or are they just the end product?

Not a question for Ingo...

But rather for the /. crew: when would you see deploying TUX as a server for /.?

This is the real question: when will people for whom serving web pages is their bread and butter adopt this? Apache already has this level of trustworthiness, how long until TUX has it?

The death of MultiPurposeEverything?

TUX appears to me to be rather specialized, eg. if you own a web server, eventually you'll use TUX; if you also serve web pages, you probably won't, just because you don't need to.

Do you see TUX as indicative of a growing realization that general purpose computing might not be perfect for everything? More specifically, do you see it as part of a movement towards more specialize hardware and software? For instance, why should a web server run the same kernel as a workstation, and why should the be built of the same parts?

Re:Why

I hope you will accept my apology, there was no reason for me to lash out at you in that manner.

Pardon me. I thought I was logged into Slashdot.

carlos

Comment on the SPEC Web Results.

Since your goal is to incorporate HTTP into the Linux kernel, I'll assume you are intimately familiar with how the OS deals with things like caching, memory, etc.

First of all, great job. For those of us whom speed is a primary concern, integrating HTTP into the kernel is a godsend. Obviously this will be a great improvement.

That said, don't you think the hardware differences in this last test are big enough to discredit the results? The W2K machine had an Ultra2 SCSI channel, 80MB per second data transfer vs. an Ultra 160, 160 MB per second data transfer rate of the Linux machine. The test operator claimed that since the machines more memory than the total size of the files they were serving that the SCSI bus speed did not matter. Is this true? Secondly, the Linux box had a dedicated 1000MB/s ethernet adapter while the W2K machine was using a 10/100/1000 NIC. The tester claimed that since they were plugged into the same network, that the NIC's were functionally equal.

In your opinion, do the hardware differences mean anything? I'm asking because if this were the other way around(and the Windows machine won) I think the Linux community would have been up in arms about it.

Color me stupid if need be; Just what is this?

I've downloaded the code and browsed it; I'm not entirely certain precisely what Tux is.

Is it:

• A "hack" designed to make IIS look as stupidly slow on carefully "hacked-at" benchmarks as the IIS benchmarks make Apache look?

  No particular flaming intended here; in either direction, this represents "benchmarketing" as opposed to anything realistic.

  It may be as unrealistic to "real world" situations to use a highly tuned combo of TUX and Apache and make IIS "look sick" as it was for Mindcraft to use a heavily tuned IIS to make a poorly-tuned Apache look bad.

• Something that would be embedded into a sophisticated web application framework to make certain cases of page accesses run ravingly fast ?

  In which case someone building the next Slashdot might care, as they need to write finely-tuned code, whereas I, when running a lightly loaded web server at home, will have a hard time detecting differences between Roxen, [fsf.org] Apache, [apache.org] Boa, [boa.org] and WN. [nwu.edu]

• Something that I could run in lieu of Boa [boa.org] as a tiny, fast web server?

This isn't quite a flame; it truly is important for a piece of software that you want people to use to be described in an economical manner that makes it easy for people to determine its relevance.

Re:could you have made it faster

• I think that the rule in the commercial world is "all's fare in war and benchmarks. Love is irrelevant".

You don't have to defend the TUX benchmarks as being exploitive of some weakness with the SpecWeb test. Nobody did anything "unfair" here.

I'm finding the pro-Microsoft moderation bias around here lately a little hard to stomach. If I had wanted to read FUD surrounding Linux Benchmarks, I'd just tune into ZDnet.

I guess that moderators think someone is brave for expressing pro-Microsoft opinions that will likely catch derision from all the close-minded Microsoft bashers here. The fact is, if you write anything even vaguely pro-Microsoft here these days, and keep a cool, even tone, you're likely to be moderated way up.

An MS employee posting pro-MS Comments brave? Hardly... That employee surely have nothing to fear from an eWatch [businessweek.com] investigation.

konstant intimates that making design tradeoffs (features for speed) somehow makes a benchmark invalid.

Those who develop benchmarks are supposed to take into account the "real world". If you feel that the benchmark allows someone to compare impractical, unusable software to more fully featured software, then you should criticize the benchmark and be specific about how the benchmark is not addressing these "real world" concerns so that we can be educated and the benchmark can be improved. Don't ask leading questions that suggest that features were thrown out to the point of making a product that's not usable in the "real world". Perhaps he didn't really suggest that TUX was unusable in the "real world". No, he did something more subtle. He suggested that if features were thrown out to benefit performance, then this test was no different than if features had been thrown out to the point that it was unusable (asking "how is this different...").

Both Spec and Ingo Molnar have been quite open about the conditions of the test and the capabilities of TUX. As Ingo Molnar says here [slashdot.org]:

• "...while it's not as feature-full as Apache, TUX is a 'full fledged' HTTP/1.1 webserver supporting HTTP/1.1 persistent (keepalive) connections, pipelining, CGI execution, logging, virtual hosting, various forms of modules, and many other webserver features."

The list of capabilities given above for TUX covers what is needed by the overwhelming majority of Web sites. Sure, there may have been some usability tradeoffs, but look at the HUGE performance benefits.

So, exactly what is konstant suggesting? That it's not a "fair" benchmark because it doesn't support all of the usability features that Apache has? Or is it only a fair benchmark if TUX can do everything that IIS does?

-Jordan Henderson