Scott Trappe's Answers About Code Quality 113
1) sample size and conclusions
by tim_maroney
How can any conclusions about the relative virtues of two development methodologies with a universe in the millions of components be drawn from a single sample, and one as small and atypical as a TCP/IP stack?
Scott:
You raise an excellent point, and answering it will take some exposition, so bear with me.
You're right, these results do not "prove" that Open Source is superior to proprietary development, nor were we trying to prove superiority of either development process. However, it does counter the numerous articles and studies written alleging that Open Source software is inherently inferior in quality because there is no formal development process, no comprehensive test plan or infrastructure and no dedicated resources to provide follow-on support and maintenance. Our study is a single counterexample which discredits those assertions. We are in the process of inspecting other Open Source projects to see if these results are anomalous or not, and will report our findings.
We chose to examine the TCP/IP stack because it's "atypical" in a way that makes it perfect for comparison: it has a very well-defined set of published requirements that have been stable for several years; adherence to published standards is an essential element of any implementation; there are hundreds of books and articles covering design alternatives, performance measurements, and sample implementations; and there are publicly available conformance tests.
Thus, unlike most software projects, all the implementations we looked at had a common set of requirements to meet and access to a wealth of information about how to design a high performance, well behaved implementation. All of the implementations have been in existence for several years. Thus in comparing them, we didn't have to consider the quality of the requirements document, the uniqueness of the design challenges, nor the resources available for validation.
In short, all the ingredients exist for all of these implementations to be of very high quality. The commercial implementations should enjoy the significant advantages that other studies have pointed out, i.e., resources available to them to insure comprehensive testing, response to bugs, etc. The five commercial implementations were developed at large corporations that have well-deserved reputations for developing high quality products, and invest significant resources to insure that quality. I don't believe it is a question of developer skill; the companies all have smart, capable, experienced engineers.
That an Open Source implementation would compare so favorably to commercial implementations was a surprising result and one that we thought others would be interested in. The question remains, what is different about the Open Source development process that, at least in this case, resulted in such a difference in measured quality? One of the most obvious differences between Open Source and proprietary development projects is the peer review process -- code inspection -- that is very intense for major Open Source projects, and underemployed in most commercial development projects because of resource and time constraints.
2) What needs to happen?
by argmanah
If open source has such a direct correlation to better quality, why do you feel companies are still keeping their source proprietary? Do you think that we should try and convince them to open source their code in every case, and if so, what do you think needs to happen before they can be convinced to change their minds?
Scott:
As I mentioned in the answer to the previous question, we haven't proven that Open Source is always superior, just that it can be. One of the clear advantages enjoyed by Open Source development projects is the luxury of time: the developers can take as long as they deem necessary to get it "right." Commercial projects rarely have this opportunity. Customer demands or competitive pressures often force aggressive schedules, and teams simply run out of time.
Even if Open Source was always superior in quality, quality and profitability is not the same thing. Most commercial software companies are leery of Open Source because they don't see how to make money at it. Red Hat, perhaps the most successful company with a business model based on Open Source reported only a tiny profit last year. Until executives are convinced that making their software Open Source will not negatively impact their profitability or valuation, it won't happen.
I think trying to lobby companies to make ALL their code Open Source will fail. However, I can see a hybrid approach where companies make some of their software Open Source and keep some of it closed. For example, some of the engineers and marketing staff inside Reasoning lobbied me to make our "Code Collection" tools Open Source. They convinced me that this provided a benefit for our customers at relatively little cost. Other companies may be persuaded to do the same on at least a limited basis.
3) Influence of project size
by arvindn
The parallelizability of bug-fixing is quite clearly very effective for high-visibility projects such as the linux kernel and apache. However, considering that most open-source projects have only between 1 and 5 developers, how popular do you think a project needs to be for it to significantly benefit from people looking at the source code?
Scott:
I think a project of ANY size benefits from peer review, and that it is even more important if there is only a single developer! That said, there is no question that as the number of active developers on a project grows, the opportunity for misunderstandings and miscommunication grows geometrically. A significant portion of the errors we find are clearly the result of two (or more) developers working on different parts of the code not understanding the interfaces between their components.
4) Quality Software vs Fewer Bugs?
by gosand
I work in software Quality Assurance, and have for going on 10 years now. My experience tells me that true software bugs are only part of the quality of software. So much can get lost in the software development lifecycle. An unclear requirement can travel through the lifecycle and come out the other end as a bug to the customer. Usability is another part of quality. It could be bug-free, but if it is really difficult to use or doesn't fit the needs of the customer, it may not matter.
It sounds like your company focuses on analyzing the code bugs, and not necessarily the perceived bugs. What are your opinions on this? I know that locating and eliminating the bugs *is* a critical part of software QA, but do you feel that bug-free ensures true quality? A bug-free Open Source project may still be too difficult to use or confusing for the non-technically inclined.
Scott:
You are absolutely right, and we don't claim that Illuma is a panacea. For the two issues you raised, there are good methods available today. Conventional black-box testing is a good way to confirm that an application meets its specified requirements, and usability testing with "real" users is important to determine if the customer's needs are truly being met. The problem that we see is that typically there are so many "code bugs" in the software that QA ends up spending most of their time running into those and not having enough time to adequately confirm conformance to the requirements. What Reasoning seeks to do is change that by providing developers with frequent, in-depth feedback on code bugs so that when QA does get the application, it is largely functional and robust.
5) General quality of programming
by pro-mpd
Do you find that the quality of the programming depends upon the geographic location of the programmers? So, for instance, an open source program will be troubleshot and combed over by people from potentially a dozen different countries. Closed source software is checked by people where it is written. Since, as a general rule, education varies in quality and areas of emphasis around the world, does it help having people attacking a program from many different angles (i.e., open source, cheked world wide) rather than simply drawing from a set of people who may share many of the same abilities, backgrounds, etc.?
Scott:
You have really posed two different questions: (1) how does software quality vary based on the location of the developers, and (2) do geographically distributed teams result in better quality because of the (implied) diversity of the developers?
To answer the first question, there are certainly significant differences in the quality of education around the world, and I believe cultural factors also play a role. However, numerous studies have shown that the programmer productivity and quality varies over a huge range - 10 to 1 or more, even when controlling for ethnicity and education. So I believe quality depends more on the individuals that make up the team than does their location.
For the second question, I disagree with the underlying assumption. Open Source projects could well be composed of people from similar backgrounds and education. In contrast, many commercial development teams, whether geographically concentrated or distributed, have a tremendous diversity among their developers. Again, I think the individual programmers are by far the most significant factor, and that location plays a much smaller role.
6) Issues behind test cases for proprietary vs. open
by Tekmage
One of the bigger challenges facing open source projects as compared to their proprietary equivalents is how to manage confidentiality of test cases. With companies such as Red Hat and Ximian involved, it's certainly less of an issue for their core products and projects they over-see, but there will always be cases where there is friction when the best/only person who can fix a particular problem is on the outside, unable to work with the test cases in question.
What are your thoughts on this trade-off between test case management and confidentiality as it relates to proprietary v.s. open source code development?
Scott:
This is a common problem regardless of the state of the source. For example, a semiconductor company may run into a serious defect with a device simulation program. They may view their "test case" -- the chip they are designing - as being so proprietary that they are not willing to share it with anyone outside, regardless of whether the application is Open Source or closed. Open Source may have an advantage if it is practical to bring in an outside expert on the application and do the bug-fixing work on the customer's premises. The framework that exists for open source projects (SourceForge.net, etc.) is better suited for this than most proprietary development environments. However, companies will always look for a competitive edge to deliver superior results relative to their competitors. If the company is based on Open Source, then they must identify other means to differentiate themselves. Red Hat, Suse, and others differentiate based on (a) components included in their distribution; (b) a "certification" that the integrated components will work together; and (c) quality of service and support. Thus, keeping test suites proprietary is a logical way for a company like Red Hat, for example, to differentiate its version of Linux from others, and I think this will be common in businesses based on Open Source.
7) How do you maintain your neutrality?
by arvindn
Given that on more than one occasion "independent institutions" which conducted similar studies (and concluded that closed source is superior) were revealed to have been sponsored by the other side [microsoft.com], how do you convince other people of your neutrality? Since you are selling a service [reasoning.com], not a product, I would guess that the confidence of your customers in your independence is pretty important from a business perspective. How do you win and keep that confidence? The article notes that you agree with ESR's pro open-source reasoning. Wouldn't the perception of your having a OSS bias be something you'd want to avoid?
Scott:
As I answered in question #1, we didn't set out to prove Open Source superior or inferior to proprietary software. Our mission is to give our customers a competitive edge by helping them deliver great software--by accelerating their development process, saving time and money. Code inspection has been demonstrated to be the single most effective way to eliminate defects, and we offer a way for organizations to perform thorough inspections quickly and cheaply. When we started our comparison, we didn't know how it would turn out, but the results confirm our belief that code inspection -- which happens naturally for major Open Source applications -- has a dramatic impact on software quality.
8) So if open source is so good...
by anthony_dipierro
Where can I get the source code to these automated inspection tools?
Scott:
How much are you willing to pay? :-)
Seriously, as I discussed in the answer to question #2, I haven't seen a way to provide Reasoning's shareholders with an equivalent (much less superior) return by making our source code Open. I think this is one of the most significant challenges that advocates of Open Source have yet to successfully address.
9) The future of automated code inspection
by phamlen
According to the article, it appears that you look for buffer overflows, freeing memory early, and other memory issues.
What errors are currently hard to detect automatically but which you would really like to be able to find? What is the next category of errors that you're trying to detect with automatic code inspection?
To give you some ideas, what about:
- "unrefactored" code - code which has a lot of duplication and should be cleaned up
- "untested" code - code (or branches in the code) that are currently untested by unit tests?
- "programmer intention" errors - code which doesn't do what the programmer intends
Scott:
Well, I've always wanted an editor with a "DWIM" command (Do What I Mean) so I wouldn't have to think so hard when I'm writing -- whether it's code or just a letter. :-)
More seriously, as was alluded to in the answer to question #4, automated inspection can detect "errors of effect" but not "errors of intent". By that, I mean that we cannot validate that the application does what the requirements document says it should. For example, if the requirement for a routine is that it calculate the geometric mean of a list of numbers, but instead it calculates the arithmetic mean, we cannot detect that. Today, we detect NULL pointer dereferences, references to uninitialized variables, array bounds violations (a superset of buffer overflows), memory leaks, and bad deallocations.
In general, a static analysis approach is limited to identifying defects resulting from (a) internal inconsistencies -- such as setting a pointer to NULL at one point in the program and dereferencing that pointer at a later point; and (b) violations of constraints imposed by the language semantics or standard library functions.
So, staying within those bounds, I'd like to find: (a) Concurrency-related defects: deadlocks, race conditions, etc. These are notoriously difficult to find and even harder to test. (b) Resource leaks: similar to memory leaks but applied to other types of resources such as file descriptors, handles, etc. (c) Protocol violations: misusing an API, such as trying to read or write a file after it has been closed. (d) Dead code: code that could never be executed. (e) Performance bottlenecks: either inefficient algorithms (such as using Bubble sort), or indications of "hot spots" in the code.
Your ideas are good too; so far we have focused on "critical defects" -- those likely to make the application abort, hang, or silently corrupt data. Your first example is more in the category of "poor programming practice". I can see integrating this type of analysis in the future, because there is a correlation between overly complicated routines and the likely presence of bugs.
10) Test first
by neurojab
What do you think about the new "test first" software development methodology? For those that haven't heard of it, it's a method wherein the test cases for a program are written, and no code is written that doesn't cause a failing test case to pass. All test cases are automated and run after every code change. Would you advocate this in an open-source project? This would mean every contributor would write test cases for each new feature, and add it to a project's common test case repository... What do you think?
Scott:
In my experience, programmers like to write code. Period. They don't like to write documentation, they don't like to write system tests, and they don't like to write unit tests. Programmers are also optimists--how else could they tackle building these enormously complex systems and think they had any chance of working? Programmers like instant gratification (who doesn't?). They enjoy coming up with a solution to a problem and seeing that solution implemented immediately.
Because programmers are optimists, that is reflected in their unit tests. Time and time again I've seen developer-written tests that demonstrate the feature works -- because the tests reflect the thinking of the developer about how the feature will be used. They rarely do a good job of testing corner cases, limits, or "unusual" situations (like running out of memory or other finite resources).
I think the "test first" methodology is too at odds with what motivates programmers to do what they do. Would Linux have ever been created if Linus' original postings to the net had been test cases for a UNIX-like operating system? And invited others to write more test cases? How many would have responded? How many would have become excited about the prospect of building an Open Source operating system if the first year was going to be spent writing unit tests?
Maybe I'm just a skeptic, but Test First reminds me of so many other software development methodologies proposed over the years that promise great benefits but rarely deliver them. That said, I am excited by other aspects of some of the newer methodologies, such as the "program in pairs" in Extreme Programming.
Independent code checking (Score:5, Interesting)
Re:Independent code checking (Score:2)
That is far from what happens here where I work, sounds nice not to be harassed about every little thing... If it isn't spaced properly I get the code back, if a variable isn't named right, I get the code back.
so..... where do you work? =P
Re:Independent code checking (Score:4, Interesting)
Here's another scenario: you write a class, it gets reviewed by someone and they suggest you use a Factory pattern. You rewrite it to use Factory, and then it gets reviewed by someone else and they say "why are you adding in that Factory pattern? that just makes it more complicated to use - you're over abstracting!". Both reviewers are right in a way... it all depends on how the class is being used.
There's a nice discussion of alternatives to code reviews here [develooper.com].
tom
Re:Independent code checking (Score:5, Insightful)
Re:Independent code checking (Score:2, Interesting)
I saw two major problems with our code reviews, other than the one noted above, which was also present.
First, timeframe issues. Inspections quickly become the enemy (rightly or wrongly) when you have irrational deadlines. Then the deadline causes managers to exert pressure to "streamline" inspections, which, regardless of the intent of the manager, means that inspections get turned into puppet theater, and goo
Re:Independent code checking (Score:2, Interesting)
One benefit of using an outsourced automated inspection service like Reasoning's Illuma is that there is no axe to grind, no preconceived notions, and no fear of retribution ("give me a bad review and I'll give you a bad review").
Re:Independent code checking (Score:2)
Why Didn't They Ask The Metrics Question? (Score:5, Interesting)
Re:Why Didn't They Ask The Metrics Question? (Score:3, Insightful)
That's what THEIR tool does (or at least is supposed to do). The original article states a few of the checks they make (using uninitialized pointers and memory, buffer overflows, etc), but is indeed lacking in telling us exactly what bugs were found in their actual comparison.
Are there checks for use of unsafe functions like gets
Re:Why Didn't They Ask The Metrics Question? (Score:3, Insightful)
Even when strncpy is used correctly, it's almost always better to use strncat instead. Your reviewers are jokers. I sympathize.
Isn't it correct? (Score:2)
Re:Isn't it correct? (Score:2)
15 A new-expression that creates an object of type T initializes that object as follows:
If the new-initializer is omitted:
If T is a (possibly cv-qualified) non-POD class type (or array thereof), the object is default-initialized (8.5) If T is a const-qualified type, the underlying class type shall have a user-declared default constructor.
Otherwise, the object created has indeterminate value.
This is one of the 'otherwises'.
Re:Isn't it correct? (Score:2)
It depends on the usage. IIRC (but without checking my copy of the standard), the code
does not initialise the integer to which p points, but
does.
Very telling example... (Score:3, Interesting)
Your example is particularly apt to a discussion about the quality of reviewers. You comment that those reviewing your code suggested (wrongly) that the second version was safer. My immediate suggestion, had I been doing the r
Length sensitive checks on strings (Score:3, Interesting)
Hey, strcpy is no buggier than anything else if you use it properly. I recently was forced to change my code as follows:
char *foo = new char[strlen(bar) + 1]; strcpy(foo,bar);
to
char *foo = new char[strlen(bar) + 1]; strncpy(foo,bar,strlen(bar));
Shame your reviewers knew enough to know about strncpy() but not enough to know about strnlen() and how to handle the out-of-range conditions from strnlen()... But then of course, you must have rewritten your code better than you indicate here.
So what were
Re:Length sensitive checks on strings (Score:3, Interesting)
Well if you're referring to the various buffer overrun bugs that allow "arbitrary code" to execute, this would not be an example since the memory is allocated on the heap and guaranteeing the heap would be in the right condition to create say an overwrite of a vtable would be very difficult without explicit knowledge of the code itself and this code is not open source. Most of those issue
Re:Why Didn't They Ask The Metrics Question? (Score:2)
Grab.
Re:Why Didn't They Ask The Metrics Question? (Score:2)
Now before anyone pipes up about how lame the code is (no checking for null
Checking for null after every new is very wasteful anyway. Much better to have a new handler defined. Esp. since you don't want to saddle EVERY developer by having them try to think up something creative to do when you run out of memory (which for most apps is something that is non-recoverable anyway since the most common reason for running out of memory is that you have a memory leak). FYI, this ma
Re:Why Didn't They Ask The Metrics Question? (Score:3, Funny)
Well, if we take a quick look at reasoning.com [reasoning.com] we get..
Illuma identifies defects that cause application crashes and data corruption. Examples of the C/C++ error classes covered include:
Re: Why Didn't They Ask The Metrics Question? (Score:2)
> Does anyone else have an opinion about reducing number of bugs?
Yeah... program in the much despised Ada and you'll get built-in protection against all the bullets in your list except the first one.
It's not like this technology isn't 20 years old. Though I'm glad that geeks are finally starting to see the importance of this really basic stuff.
Re:Why Didn't They Ask The Metrics Question? (Score:3, Informative)
by Kevin Smith
They are probably counting lint or other types of compiler warnings.
What about HIS best practices? (Score:2, Interesting)
Couple of points... (Score:5, Insightful)
Second, re:
However, it does counter the numerous articles and studies written alleging that Open Source software is inherently inferior in quality because there is no formal development process, no comprehensive test plan or infrastructure and no dedicated resources to provide follow-on support and maintenance. Our study is a single counterexample which discredits those assertions.
This is yet another case of ESR-driven confusion between open source licensing and community-based development. There is no requirement that open-licensed code include patches from any teenager who can compile hello.c, let alone that it be developed without a formal development process or test plan.
Not that his point is completely misplaced, since the Linux TCP stack _is_ a community effort, but it's wrong to define open source in terms of community. The overwhelming majority of open projects are not developed the way Linux and KDE are.
Re:Couple of points... (Score:2)
Yes, but the overwhelming majority of Free Software that actually gets used is developed in some sort of a community fashion. Sure, there are plenty of small projects that are basically one or two person shows, but these projects generally only have one or two users as well. Now take any piece of Free Software that sees a lot of use and you will undoubtedly see that there is truly a community built around it. Heck, r
Re:Couple of points... (Score:2)
Re:Couple of points... (Score:3, Insightful)
yehhh cool tcp/ip (Score:4, Funny)
Making a return (Score:3, Informative)
Really? No open source advocates [catb.org] have addressed this [catb.org] at all?
Re:Making a return (Score:2, Insightful)
Note the word "successfully" (Score:2)
Success is a value judgement, and without knowing his criteria for "success", you can't know whether ESR's works meet those criteria.
Re:Making a return (Score:3, Insightful)
You mean to say that you can't violate the GPL (Score:2)
That's just the way the GPL is, (by design) everyone knows it by now (or should). Get over it. Saying "OSS use may == legal trouble" is using such a broad brush that I'm sure that someone could easily accuse you of spreading FUD.
There's no reason not to use GPLed source in building your closed source app. Want to use gcc? Fine. Want to
Re:Making a return (Score:2)
Grab.
Re:Making a return (Score:2)
Zope Corporation [zope.com] (Zope)
Covalent Technologies [covalent.net] (Apache)
ActiveState [activestate.com]
I deliberately haven't included Apple for reasons cited above, but Apple is almost as much a software company as it is a hardware company.
Re:Microsoft Ad? (Score:1, Offtopic)
Re:Microsoft Ad? (Score:2)
They do this already. Look into it.
The whole coding process (Score:5, Insightful)
He said that TCP/IP was a good example because the requirements, and specifications were already there and were the same for everyone. That may have made the example easier to analyze, but in my opinion it gives an incomplete picture. How good the requirements, specification, and design are effects how good the implementation is. In order to have a complete comparison of open source and closed source you need to compare how well each of these steps (or their equivalents) occurs in each.
I personally suspect that the "think it through first" part could also be better in open source (if less formal) because there's no deadline pressure involved. Also, the people programming had to convince themselves that it was a worthwhile endeavor somehow, which requires them to convince themselves that at least the requirements are better.
BSD stack ? (Score:1)
Seems to be a rather solid implementation, perhaps even more so.
Re:BSD stack ? (Score:1)
Re:BSD stack ? (Score:1)
-MDL
Blah, everyone knows, we're just too lazy (Score:5, Interesting)
Lets face it, msot of us do printf debugging and assume that if it doesn't core dump, it works.
Re:Blah, everyone knows, we're just too lazy (Score:1)
Actually, I do System.err.println() debugging, so nyah
Re:Blah, everyone knows, we're just too lazy (Score:2, Interesting)
Seeing a JUnit report page like this [sourceforge.net] certainly gives me a bit more confidence in a project...
Tom
"Programmers like instant gratification" (Score:3, Funny)
Well then, we should opensource it! Gratification was meant to be free.
Testing (Score:5, Informative)
In fact, they shouldn't be the same people. That way there are two sets of people going over the requirements. Sometimes a developer will interpret a requirement differently than a tester. It allows ambiguous requirements to be found during the development cycle rather than having a person write the test, write the code, and have the customer say "I didn't want it to work like that."
Re:Testing (Score:3, Informative)
This is particularly true in companies where the hand-off is somewhat formal and may require paperwork. Of course, any bugs found have to be documented as well which takes more time.
So the bottom line is that the total cost involved for having only the testers test the code, is often higher tha
Re:Testing (Score:1)
Re:Testing (Score:2)
Re:Testing (Score:3, Interesting)
I agree that ultimately, one needs a seperate QA department for the end game assessments, because after a long time of staring at the same code, a set of people will no longer be able to see any further defects.
However, I think it's important for programmers to take an adversarial approach to their own code. I definitely don't feel that it "wastes p
You're right but it almost never happens. (Score:4, Interesting)
That was a small company ( 100 employees) who really, really cared about the quality of thier code - and it showed, they had a very stable and well-written product. It was the only time I've seen real SCM and QA in action and it was great.
Slowly over time I moved into larger and larger companies - never again did I see that level of attention to the SCM and QA process. Instead I see a lot of beurocracy that calls itself SCM, but is really just wastes a lot more of my and my team's time than maintaining my own CVS tree. There is more emphasis on knowing who to blame when something goes wrong, than to help produce working code leveraging as many people as possible to fix something that might be wrong.
So you are right, the QA writing tests is the idyllic case. But alomost no-one does it, at least at the moment. The funny thing is that using code workshops in India to do QA tests while keeping development in the US would probably be the optimal model, since it would actually take advantage of the time difference and not be too affected by seperation, but instead companies go too far (as usual) and just ship everything overseas (well, some companies anyway).
I Heart Unit Testing (Score:3, Interesting)
Both sorts of testing are helpful.
Functional tests make sure that the program meets the requirements. Good QA people are invaluable.
But even if I had the world's best team of testers, I'd still do test-first programming. Why?
Re:Testing (Score:1)
Re:Testing (Score:2)
Also, having a defined
Umm (Score:4, Interesting)
I think a project of ANY size benefits from peer review, and that it is even more important if there is only a single developer!
Maybe my question was not quite clear. I didn't mean how big should a project be for peer review to be benificial. I meant how big should a project be so that it gets any peer review at all in the first place. In particular, I was asking about the thousands of OSS projects that have between 1 and 5 developers. Do these get any significant amount of patches/bugfixes to justify their being open source?
Any opinions?
Re:Umm (Score:2)
In particular, I was asking about the thousands of OSS projects that have between 1 and 5 developers. Do these get any significant amount of patches/bugfixes to justify their being open source?
Any opinions?
My opinion is that they don't get input based on how many developers are on the project, they get input based on how many developer-esque people use the project's product. I've never been an offcial developer on any OSS project but I've submitted code fixes when I've had to fix something. The actu
test case management as competitive edge (Score:4, Interesting)
I like the point you make, about maintaining proprietary and confidential test suites being a key ingredient in developing a business (aka making money and being profitable) based on Open Source solutions. The business relationship is built on trust and survives based on a sustainable, mutual relationship. It isn't dependent on code remaining secret, and if the relationship breaks down (lost trust and confidence) the business/individual using the software isn't stranded.
Thanks for the insight!
Open source opportunities for peer review (Score:4, Insightful)
One of the benefits of the open source model is that even projects with only a single developer can get peer review. The reasons are simple, and obvious once they are stated. The only barrier to entry as a reviewer on an open source project is knowledge, of the project, the language, and the address to send bug reports to. It is often touted that open source turns every user into a potential co-developer. It should not be overlooked that projects with a large user base can receive not just bug reports, but also patches, from that user base.
I've found that when I investigate a problem to determine if there is a bug, the investigation often yields a solution because I have the source to inspect. Thus, when I submit a bug report, I can frequently include a candidate patch to fix it. Even projects with a single active developer can often have a number of occasional or even one-time contributors.
Why is open-source better? (Score:5, Insightful)
Re:Why is open-source better? (Score:2)
Amateur vs Professional (Score:4, Insightful)
It might be interesting to look at exceptions to that, though. I wonder how the GPLed code coming out of some place like Red Hat, happens to stack up against amateur-written code. (Do Red Hat programmers have the "luxury of time"? I suspect not, but don't really know enough about how the place works.)
fallacious argument (Score:5, Insightful)
We chose to examine the TCP/IP stack because it's "atypical" in a way that makes it perfect for comparison: it has a very well-defined set of published requirements that have been stable for several years; adherence to published standards is an essential element of any implementation; there are hundreds of books and articles covering design alternatives, performance measurements, and sample implementations; and there are publicly available conformance tests.
Ok, I love open source as much as, no more than, the next guy. But being knee deep in LSAT preparation materials, I immediately spotted the weakness in the argument here.
1. Open source is believed to be inferior because of the lack of formal development process.
2. TCP/IP is an atypical situation where an external standards body has created requirements, test cases, and the other trappings of a formal development process.
3. TCP/IP open source implementations are greatly superior to non-open source implementations.
Scott goes on to talk about the virtues of increased code inspection in open source projects. I think this is a fair and interesting point, namely, when the "formal development process variables" are held constant (i.e. all other things being equal), the increased code inspection leads to greatly increased code quality.
However, it seems clear to me that the original contention that open source lacks formal development processes and therefore produces inferior code has not been directly addressed. We are left to wonder whether the virtues of increased code inspection can overcome the lack of formal development processes.
-- p
Re:fallacious argument (Score:1)
Even so, I don't think Scott's reasoning is quite right, though I can't put my fin
Their goal may not have been that big (Score:4, Insightful)
1. All Open Source implementations are inferior because of the lack of formal development process
2. Comparing the TCP/IP handling, we found that the Open Source implementations are superior to proprietary implementations
Ergo: The common belief is not an absolute truth.
I don't think it was their intent to show that a lock of formal development is more or less likely to produce inferior code, only that it isn't an absolute result.
Of course, moving from pure logic to statistics it is clear that one test result isn't really enough to disprove the original assertion. However, it is enough to plant doubt and possibly lead to more comparisons along these lines.
revised 'Test first' (Score:2, Insightful)
If the test can be automated then test data is produced to verify that the bug has been corrected.
Also, I frequently write 'test scripts' when consulting change requirements. This helps resolve the requirements, makes testing easy and covers saves you neck if the client says that's not what they asked for(the tests are unambiguous).
Now, if only they p
Taking the time to get it right (Score:4, Insightful)
That luxury of time comes both before and after the release for open source. Without the pressure of a specific feature list and release schedule being directly tied to a revenue stream it is obvious that open source projects have more leeway before the release date. However, the Release Early, Release Often [catb.org] paradigm has two parts. It presents a model that makes a project available to people all the way along the adoption curve. There is less pressure for make-or-break releases. This presents an opportunity for a more continuous cycle of releases and improvements.
Detection of unrefactored code (Score:3, Interesting)
The other day I hacked up a similarity [membled.com] tool which gives an index between 0 and 1 of the cross-redundancy between files. (It gzips the concatenation of the two files and compares this with gzipping them independently.) Maybe a version control system could run this on all new file checkins, so a file is rejected if it is too similar to some file already under version control. This would cut down on cut-and-paste of existing code, which is generally a Bad Thing (refactor instead).
Why go open source --or-- why go extinct? (Score:2)
And advocates of Open Source never will, nor should they try.
Your products will: survive because they are unreproducable; survive because they go open source; or be replaced by an open source package
Re:Why go open source --or-- why go extinct? (Score:2)
Aren't you confusing open source with free software?
Re:Why go open source --or-- why go extinct? (Score:2)
"Open source" is a term coined by ESR (I'm pretty sure, he was at least the first person I heard pushing it) which refers to the general physical property of having access to source code in a "useful way".
"Free software" is a term coined by Richard Stallman to indicate a way of thinking about software and its impact on those with whom you collaborate.
While what I said would play into RMS' definition of free software just fine, the very act of giving people the source code to your software
Re:Why go open source --or-- why go extinct? (Score:2)
You make an interesting argument, but I disagree with your conclusion, probably because I'm not sure I agree with all your underlying assumptions. First, however...
OK, we're using somewhat different definitions of "open source" here. I use the term simply to mean software for which the source code is
Re:Why go open source --or-- why go extinct? (Score:2)
That's a fine idea, it's just not "open source". The Open Source Initiative are the keepers of the actual definition of the term [opensource.org] as it was originally coined by the founders of the OSI. They certify licenses on the basis of several criteria, of which access to source is only one. If you want to use another, looser def
It isn't really a cultural diversity issue. (Score:2)
pro-mpd seemed to be attempting to steer a cultural-related response out of Scott Trappe. However, I really think geographically-distributed culture has little to do with the actual code quality.
What is more significant is that distributed developers tend to communicate mostly through written e-mail or newsgroups. Their very act of communication is documenting their decisi
Ah, but you *can* have it both ways (Score:1)
There's a lot to be gained in getting a few people together in a room with a whiteboard to brainstorm -- you can review (and discard) a much greater volume of ideas, because the feedback is almost instantaneous... you can stop explaining as soon as the other people understand the idea, you get non-verbal hints that someone else sees a problem with the first
customer pressure is an itch waiting to be scratch (Score:3, Insightful)
If a program being developed has an urgent customer demand, it applies pressure on the developers. Commercial developers can end up releasing too soon, and selling the customers an inferior product. Release early and release often means that open source developers also give the "customers" incomplete programs. But they are not selling to the customers, though, so they can clearly label this stage as Beta. If there is a genuine urgent need for the program, though, the open source developers now get an important advantage. Some of those anxiously waiting customers may decide that their itch is too big, and they stop waiting and assist the development team. They scratch their itch by sending in bug reports and possibly fixes for the parts that don't yet fill their needs, or they do whatever else they see that will help fulfill their own needs faster. The open source project starts moving faster (as long as the communication load doesn't start to get in the way).
How to make $$ with Open Source (Score:5, Interesting)
Sell your binaries and give away the source. That's how easy it is.
Free Software/Open Source does not equal no-cost binaries. It never has, and it never will. If a company offers a binary, it's a gift. If the company feels obliged to give away a binary, they've misunderstood FS/OS. If a hacker feels that the company is obliged to provide binaries, he's less of a hacker and more of a slacker. All a hacker has a right to is unrestricted access to and use of the source code. Binaries are not a right; they are a privilege.
Linux distros lose money because they give away binaries to everything, pure and simple. Why they give them away, I don't know. I suspect it's part ignorance, part hope that a 'free' product will find an audience that will get hooked now and pay for support later. I suspect that RedHat has taken this a step further -- release a basic distro that's buggy enough to encourage users to upgrade to RHAS. SUSE has a better solution: binaries are available four weeks after the product goes on sale. Who can't wait four weeks for free binaries, if you can't afford the boxed set?
To be blunt, companies that are serious about turning a real profit from FS/OS products must put their source code on their FTP server and charge for about everything else except basic documentation. If a hacker can compile OpenWhazzit all by his lonesome, let him. He poses no threat to the profit margin. If he posts a HOWTO on the subject, there's still no problem. He and everyone else who reads it will probably document and/or fix the bugs they find in the same way. He may even give away or sell binaries of FreeOpenWhazzit, but should that really matter to the bottom line? Do they think Boeing or Pepsico or Mitsubishi would pass up 1,000 copies of BlueGlove Advanced Server or in favor of Joe's Blue Garage Linux? Odds are, even if Joe's distro represents a siginificant increase in quality, they would still buy the name brand. And so will the average home user who wants to use a real, inexpensive OS. That is, unless BlueGlove Advanced Server is so bad that Joe's distro finds a niche in the marketplace, in which case, shame on BlueGlove for selling a bad product.
That means, in the end, more people are using your product, so more people are buying your product, and more people are fixing your product at not cost to you. I'd say that's what you want your investors to hear.
And before the flames arrive, understand this: I have nothing against giving away binaries, as long as it's understood by everyone that they are gifts, not obligations. And if businesses want to survive, they're going to have to learn this lesson quickly. That will mean that hackers will either have to get used to compiling or switch to Debian.
Daniel.
Re:How to make $$ with Open Source (Score:1, Insightful)
*When your argument is based on the assumption that the best solution has been overlooked because everyone else is just too stupid to see it , it's likely you're the one being stupid.
Re:How to make $$ with Open Source (Score:2, Interesting)
But doesn't this just encourage developers to ship slightly broken source? Say, all the Makefiles deleted or something?
Test First misunderstood (Score:2)
I'm going to use the term "Test Driven Development", or TDD, since that's the current usage.
First, TDD doesn't start by coding a lot of tests. That would indeed be discouraging and counterproductive. It starts by coding _one_ test, and then immediately implementing just enough code to satisfy that one test. The point here is immediate feedback.
Second, TD
Re:Test First misunderstood (Score:1)
"Test First" in XP is about design, not about unit testing in the classical sense. You specify something that your program should do, write a test to verify that it does it, and then write the code to make it do it.
A lot of QA people criticize the resulting suite of test cases as being incomplete. The're absolutely right: they are incomplete from the formal testing viewpoint.
So where do issues like corner cases get handled? Where th
smatch (Score:4, Informative)
Check out kbugs.org [kbugs.org]. These are the smatch results from testing the 2.5.65 kernel. We found 1400 possible bugs in the 2.5.65 kernel but probably over half of those are false positives.
Smatch [sf.net] is an open source checker that finds similar sorts of problems to the Reasoning software. For example, both look for dereference [kbugs.org] bugs.
The bad news is that smatch is still in the pre-alpha stages and it only works on C for now. And also the kbugs.org site is crappy.
The cool thing about smatch is that you can write checks which are custom to your code. Mostly it is used for the kernel, but Michael Stefaniuc has used it for Wine specific bugs as well.
OSS alternatives to code checking tools (Score:1)
Basically, what does Reason offer over running gcc -Wall -pedantic -anything else, running lint on the code, and then running the code with something like valgrind to detect dynamic memory problems? Okay, its a single tool as opposed to many of these and probably produces nicer reports, but what I think I'm really asking is, for the poor student like me, is there anything similar/have I missed out a
Re:OSS alternatives to code checking tools (Score:2)
The cool thing about smatch as oposed to lint etc is that you can look for bugs specific to your codebase. For example, in the Linux kernel you are not allowed to call copy_to_user with a spin_lock held.
It's pretty simple to throw together a smatch script that looks for that kind of bug.
I'm not sure how useful writing specific checks like that is for smaller projects, but for a huge project like the linux kernel, you are always going to f
Test first works by changing mindset (Score:1)