Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Software

QA Under The Open Source Development Model 180

Posted by timothy from the sound-the-pdf-klaxon dept.
carrowood writes "A survey was conducted questioning open source developers from both large and small projects concerning their quality assurance practices. A research paper based on the survey result was just published in the Journal of Systems and Software. Some comparisions between QA practices of open vs closed source projects are made with some interesting observations. While on the whole it looks like open source QA can be as good as that in traditional software development, there were a few areas pointed out where the open source community does not do so well, such as regression testing and setting release dates. A thought provoking read."
This discussion has been archived. No new comments can be posted.

QA Under The Open Source Development Model More

QA Under The Open Source Development Model

Comments Filter:

  • ISO (Score:4, Interesting)

    by Gortbusters.org ( 637314 ) on Sunday July 20, 2003 @04:21PM (#6486527) Homepage Journal
    Do any open source projects get audited for ISO 9001 compliance? They have a quality certification that many [enterprise] customers desire.

    I know at work ISO audits are both fun and exciting! (must contain laughter)

    • Re:ISO (Score:5, Insightful)

      by zagy ( 565897 ) on Sunday July 20, 2003 @04:28PM (#6486566) Homepage
      I suspect clients requiring ISO 9001 just go for commercial software. One part of OSS often is to refuse warranty and alike - IMO this does not quite fit into ISO 9001.

      • Re:ISO (Score:2, Interesting)

        by korgull ( 267700 )
        here are probably a lot of companies who don't bother to spend money to be ISO9001 compliant and deliver just as good quality as their ISO9001 competitors.
        BTW how much is a warranty actually worth when the shit hits the fan ?
      • It does if the company is prepared to take the risk onto itself. If they have proper routines for validating the correctness of the OSS libraries they are using, this isn't a problem.

        What they'd basically say is "Apache provides no warranty for the parts of this software that they have developed. We, ACMEsoft Inc., however, guarantee that it will work."

        This is indemnification in EULAspeak.

      • Re:ISO (Score:4, Funny)

        by ratboy666 ( 104074 ) <fred_weigel@ho[ ]il.com ['tma' in gap]> on Sunday July 20, 2003 @11:16PM (#6488464) Journal
        ISO 9001 is about process. Not warranty or bugs, etc.

        To be certified, you must have a process, and must be accountable to the process.

        For example - I could say that I take all defect reports, shred them, and that is how I deal with quality. This is ok, as long as I tell my customers that this is what I do, and I am completely accountable. Of course, it wouldn't be worth anything to have this quality method certified.

        Ratboy.

    • Re:ISO (Score:3, Insightful)

      by Surak ( 18578 ) *
      Do any open source projects get audited for ISO 9001 compliance?

      Who would pay for it? ISO 9001 auditing is VERY expensive.

    • Re:ISO (Score:5, Informative)

      by stwrtpj ( 518864 ) on Sunday July 20, 2003 @05:20PM (#6486849) Journal
      Do any open source projects get audited for ISO 9001 compliance? They have a quality certification that many [enterprise] customers desire.

      Interesting thing is, having been through the ISO 9001 rigamarole, I can tell you that it does NOT necessarily enhance quality. ISO 9001 has very little to do with quality, regardless of what their claims are to the contrary. ISO is all about process and repeatibility. It simply validates that you have a well-defined process. You can have a wonderful process, with everything down to the number of times the developers go to the bathroom and for how long documented and validated, and still have a product that's a piece of crap.

      An organization I used to work for did the ISO thing. The software had no fewer problems after ISO than it did before. It's just a nice thing to put on my resume, really.

      Scott Adams of Dilbert fame put it best: If you never went through an ISO audit, you probably don't know what ISO is about. If you did go through an ISO audit, you definitely don't know what it's about.

      • Re:ISO (Score:3, Funny)

        by bentcd ( 690786 )
        You can have a wonderful process, ..., and still have a product that's a piece of crap.

        Certainly, but it is predictably, reliably and consistently a piece of crap. Your customers have some guarantee that it's not going to get significantly worse than that.

        And, of course, if your product is crappy in ways your customers don't really care about, then it's not even a problem.

        • Re:ISO (Score:3, Insightful)

          by ClosedSource ( 238333 )
          I would say that the process used is predicatable but the outcome for each project can still be better, worse, or the same as a previous project.

          Why? Because using the same process to develop different projects is like using the same algorithm for every mathematical problem, in general it's going to be inappropriate most of the time.

          Each project typically has its own goals, deadlines, customers, implementors, managers and economics. To be successful, the process has to reflect those realities.

          Trying to c

      • Re:ISO (Score:3, Informative)

        by GlassHeart ( 579618 )
        You can have a wonderful process, with everything down to the number of times the developers go to the bathroom and for how long documented and validated, and still have a product that's a piece of crap.

        True. However, if your development processes are unpredictable and unreliable, your product is most likely going to be a piece of crap. You may have superstar coders who will "save the release" by working through the night for weeks, but the result is not going to be a well thought out and maintainable p

      • Amen! (Score:4, Interesting)

        by wowbagger ( 69688 ) * on Sunday July 20, 2003 @06:46PM (#6487268) Homepage Journal
        I like to make this point about ISO-9000: if they are not now ISO-9000 certified, McDonalds could be in about five minutes - they have a proceedure for everything. Does this mean that McD's good is "good quality"?

        IF you have a proceedure, AND IF that proceedure included required analysis of failures, AND IF that proceedure requires improvements to the proceedures to correct the failures, THEN you MIGHT begin to approach quality over time.

        However, far too many company's ISO proceedures fail to require analysis and actively discourage improvement to the proceedures ("You want to change the proceedure! OK, here's a ton of paperwork to fill out, and you will have to get the signatures of fourteen people who would rather not sign anything other than their own paychecks. Have Fun!").

        ISO is really just a big peer-pressure MLM scam the way it is run now.

        And yes, the company I work for is ISO 9001 qualified.
      • ...well it's a lot, at least.

        ISO is all about process and repeatibility. It simply validates that you have a well-defined process. You can have a wonderful process, with everything down to the number of times the developers go to the bathroom and for how long documented and validated, and still have a product that's a piece of crap.

        Process is the heart of engineering. There is at least one thing that links civil engineers to electrical engineers to software engineers, it's process. In science to, ther

        • So what's the standard electrical engineer's process? I suspect it varies a lot more then you might think.

          It's interesting that despite the fact that software engineering is always compared unfavorably with Electrical and Civil engineering, there has been far more work done on formal processes in software engineering than in the other branches combined.

          I suspect the reason is that SW engineering is a much broader discipline in a practical sense than the others. In a sense, the other disciplines are subset

    • ISO 9001 et al. (Score:3, Interesting)

      by hughk ( 248126 )
      I have worked before in QA, my wife works for a QA Certification Company. The meaning of ISO 9001 and its successors (15442, I think), and it is just about having a documented process so the 'customer' doesn't have any false expectations about what the 'supplier' is producing (be it a car, a lump of plastic or a computer program).

      I don't know of ISOed OS software, but I am aware of organisations that have gone through a quality audit who use open source software. The main issue is having a test and intern

  • Release dates? (Score:5, Interesting)

    by mrjb ( 547783 ) on Sunday July 20, 2003 @04:24PM (#6486544)
    Considering the fact that open source software is often not commercial, it doesn't sound strange to me that release dates are not very strict. As there often is no budget, there is no maximum allowed time that may be spent on the feature set. Thus, software is released whenever the planned features have been built in, rather than cutting the desired feature set to fit the release date.
    • Adherence to release scheduals is also almost always contrary to quality.
    • Hmm ... whatever happened to ESR's RERO recommendation in all of his admired open source paraphernalia? Release Early, Release Often seems to have been thrown away to the auspices of elitism. Oh well, I guess time will tell which projects will be successful.
    • Good point. I hate it when people try to compare OpenSource and closed source in this way. Why the hell should it ship on friday!? You're not paying me, it was my idea in the first place, and I'm going camping.

      The idea that open source projects need to improve anything is stupid. It's free, for christ's sake, you get what you pay for.

    • I do as much formal QA as I'm required to by client sites. Sometimes that means unit and regression test frameworks, sometimes it just means making it "good enough" for daily use.

      Regardless of the reasons for the formal QA, you won't find too many people who actually enjoy doing it. It's a necessary part of doing a good job, like documentation. But as with anything that requires tedious, painstaking detail, it's hardly something I want to do with my free time.

      And there is the crux of it: my free tim

    • The importance of release dates varies according to individual perspectives:

      * A company developing a scheduling system for use in a new factory needs that system working correctly by a particular date or the factory will not run.

      * A company delivering a commercial software product may need to promise release dates for certain features in order to attract business. The importance of meeting these dates may vary from avoiding a minor credibility hit to avoiding the company being bankrupted, depending o

  • At last we actually have a peer reviewed published article on Slashdot main page! Actually, I could not believe it at first, but the link confirms it. Beeing somewhat wise, I downloaded the article before it got Slashdoted :-)

    Kudos for editors for publishing the story.

  • If you want a release date.... (Score:5, Insightful)

    by Boss, Pointy Haired ( 537010 ) on Sunday July 20, 2003 @04:26PM (#6486561)
    ...then you can go and buy software developed out of a motivation for financial gain.

    Surely open source projects exist for other motivations, and hence have less (if any) need to aim towards release dates.

    After all, in the business world the sheer fact that you _have_ to get something out by a particular date (because marketing / the bean counters said so) contributes significantly to the quality problems.
    • Well least there is no marketing dept so OSS projects don't get stupid names like scoop :)

      Rus
    • Open source projects do not necessarily "exist for other motivations", or at least not completely. Commercially-sponsored open source projects (e.g., RealNetworks' HelixCommunity.org) and commercial applications built using open source components (e.g., IBM Websphere) most likely have release date considerations. Even non-commercial projects have to release *sometime*...

      Commercially-sponsored open source projects can attempt to dictate release schedules for the open source projects to match their commercia

    • Re:If you want a release date.... (Score:4, Funny)

      by stwrtpj ( 518864 ) on Sunday July 20, 2003 @05:38PM (#6486941) Journal
      ..then you can go and buy software developed out of a motivation for financial gain.

      Yes, like, say, Duke Nukem Forever.

    • If you want a release date, go for nightly builds. They're always released on time.

      The focus on release dates, I think, is due to not allowing access to the code at other times. If you've implemented some functionality, and people want it, and they can't get it because you haven't released the next version, there's pressure to hurry the next version. If anyone can get the current code at any time, there's no reason to change the version number without implementing everything you want to get into this relea

  • QA is harrrrd (Score:5, Insightful)

    by The Bungi ( 221687 ) <thebungi@gmail.com> on Sunday July 20, 2003 @04:27PM (#6486563) Homepage
    It's hard when everyone is sitting on the same floor of the same building - it must certainly be a *lot* harder under the distributed development model.

    Ultimately, while development can, in certain cases, be done in a vacuum, QA cannot (and should not). It's by nature a collaborative and interactive process.

    I have nothing but respect for the few (good) QA engineers I've worked with.

    • Donald Knuth wrote, about TeX, that a programmer should also do QA, use the program, and write the manual.

      I don't know about other places, but where I work, QA, support, development, and documentation are different groups. Anecdotally, and intuitively, if the development team had to support the product after it was released, they'd fix a LOT of bugs that they don't consider important from their ivory tower.

      In the open source world, everything seems less hierarchial. Users are more likely to be develop

      • Re:QA is harrrrd (Score:2, Insightful)

        by bentcd ( 690786 )
        I don't believe in having a programmer do the formal QA of his own code. He's already fixed the bugs that pop up when it's being used in the way he uses it and I even suspect that we subcounsciously avoid using it in ways that we suspect may touch upon flakey bits of code.

        Having the programmer do the QA on something written by an altogether different programmer is something else entirely though. Programmers tend to be more expensive than testers though, so it might get a tad on the costly side.

      • Re:QA is harrrrd (Score:3, Insightful)

        by __past__ ( 542467 )
        Donald Knuth wrote, about TeX, that a programmer should also do QA, use the program, and write the manual.
        He also said that hard programs can only be written by a single programmer. This is just not an option in a world where you can't just take 10 years off to write TeX because the look of your books isn't satisfying. Most people just aren't Don Knuth - they are neither geniuses nor professors for the Art of Computer Programming.

  • Not setting releases dates ? (Score:4, Insightful)

    by roard ( 661272 ) on Sunday July 20, 2003 @04:27PM (#6486564) Homepage
    Well, the fact that no releases dates are setted are more a good point than a bad one ! Of course, in an ideal world where software would be released on dates (!), they won't have bugs either. But in the real world, must proprietary software aren't on schedules, and anyway, when they are, this is often at the detriment of the number of remaining bugs and/or dismiss of some features.
    In the free software world, the software is released when ready. So, of course they don't set release date (generally speaking -- some projects have regular releases). But I hardly see that as an obvious bad point. It could be on the contrary one of the strength of the free software.
    At least, programmers on free software releases when they are happy with the code.
    • What are you taking about? GCC has an estimated release date usually it slips by a week or two like right now 3.3.1 was planed for last friday but it slipped because there are some bugs that need to be fixed before the release (but most of these bugs are not regressions from 3.3 or 3.2.3 but from 2.95.3 or 2.91.66). The problem is that most bugs only are fixed in the last month before the release of the minor version.
    • Well, the fact that no releases dates are setted are more a good point than a bad one ! Of course, in an ideal world where software would be released on dates (!), they won't have bugs either.

      Yep, I heard Duke Nukem Forever will be the most perfect and bug-free software, ever.

  • Setting release dates? (Score:3, Funny)

    by BitwizeGHC ( 145393 ) on Sunday July 20, 2003 @04:29PM (#6486573) Homepage
    Lemme guess... "When it's done" isn't good enough?

  • Open Source Development HOWTO (Score:5, Funny)

    by Anonymous Coward on Sunday July 20, 2003 @04:29PM (#6486574)

    1. Introduction

    As everyone knows, Open Source software is the wave of the future. With the market share of GNU/Linux and *BSD increasing every day, interest in Open Source Software is at an all time high.

    Developing software within the Open Source model benefits everyone. People can take your code, improve it and then release it back to the community. This cycle continues and leads to the creation of far more stable software than the 'Closed Source' shops can ever hope to create.

    So you're itching to create that Doom 3 killer but don't know where to start? Read on!

    2. First Steps

    The most important thing that any Open Source project needs is a Sourceforge page. There are tens of thousands of successful Open Source projects on Sourceforge; the support you receive here will be invaluable.

    OK, so you've registered your Sourceforge project and set the status to '0: Pre-Thinking About It', what's next?

    3. Don't Waste Time!

    Now you need to set up your SourceForge homepage. Keep it plain and simple - don't use too many HTML tags, just knock something up in VI. Website editors like FrontPage and DreamWeaver just create bloated eye-candy - you need to get your message to the masses!

    4. Ask For Help

    Since you probably can't program at all you'll need to try and find some people who think they can. If your project is a game you'll probably need an artist too. Ask for help on your new Sourceforge pages. Here is an example to get you started:

    "Hi there! Welcom to my SorceForge page! I am planing to create a Fisrt Person Shooter game for Linux that is going to kick Doom 3's ass! I have loads of awesome ideas, like giant robotic spiders! I need some help thouh as I cant program or draw. If you can program or draw the tekstures please get in touch! K thx bye!"
    Thousands of talented programmers and artists hang out at Sourceforge ready to devote their time to projects so you should get a team together in no time!

    5. The A-Team

    So now you have your team together you are ready to change your projects status to '1: Pre-Bickering'. You will need to discuss your ideas with your team mates and see what value they can add to the project. You could use an Instant Messaging program like MSN for this, but since you run Linux you'll have to stick to e-mail.

    Don't forget that YOU are in charge! If your team doesn't like the idea of giant robotic spiders just delete them from the project and move on. Someone else can fill their place and this is the beauty of Open Source development. The code might end up a bit messy and the graphics inconsistant - but it's still 'Free as in Speech'!

    6. Getting Down To It

    Now that you've found a team of right thinking people you're ready to start development. Be prepared for some delays though. Programming is a craft and can take years to learn. Your programmer may be a bit rusty but will probably be writing "hello world" programs after school in no time.

    Closed Source games like Doom 3 use the graphics card to do all the hard stuff anyhow, so your programmer will just have to get the NVidia 'API' and it will be plain sailing! Giant robot spiders, here we come!

    7. The Outcome

    So it's been a few years, you still have no files released or in CVS. Your programmer can't get enough time on the PC because his mother won't let him use it after 8pm. Your artist has run off with a Thai She-Male. Your project is still at '1: Pre-Bickering'...

    Congratulations! You now have a successful Open Source project on Sourceforge! Pat yourself on the back, think up another idea and do it all again! See how simple it is?

  • Hrm... (Score:3, Funny)

    by Duncan3 ( 10537 ) on Sunday July 20, 2003 @04:29PM (#6486578) Homepage
    Open Source... Quality Assurance... I didn't know you could use those 4 words together in a sentence...

    "51% projects have one developer" - Now we have proof geeks really can't work well with others :)
    • Or that the itches they're scratching are quite personal ;-)
    • Open Source... Quality Assurance... I didn't know you could use those 4 words together in a sentence...

      Given what passes for QA in many companies, I don't think it's unreasonable for open-source developers to claim that many of their practices qualify as QA, even if an academic might disagree. :)
    • "51% projects have one developer" - Now we have proof geeks really can't work well with others :)

      I have a few projects on Sourceforge, so let me comment here. Most of my projects are small and usually written around a specific need.(see my fwreport tool for example).

      For Hermes (see my sig and journal), the project is gigantic (>10000 lines of code) and although there are many contributors to the code, I do nearly all of the coding. One of the real problems has been developing a real community of use
      • The way I handle QA usually is to write, test and release the code myself and then request feedback from the user community. QA is not just about bug testing, but it is also about making sure that the program actually enables people to get their work done. So usually, I release beta then rc then final but request feedback at every point regarding general usability.

        Also maintaining a roadmap is also essential, but it should be flexible based on user feedback.

        Ideally one of the strengths of OSS is that it
    • "51% projects have one developer"

      I wonder how many only have 1 user...

    • Now we have proof geeks really can't work well with others:)

      Hey, another advantage to OSS development. OSS develpores can choose who they work with a lot easier the developers working for some company.

      The fact that so many OSS projects have one developer is more in keeping with modular tool aproch to design that the OSS community has adopted from UNIX. Anyways, most propiatary projects are realy just collections of projects, many with single developers, placed under one name.

      As a developer, if my name i

  • hmmm..... (Score:5, Insightful)

    by Xtifr ( 1323 ) on Sunday July 20, 2003 @04:31PM (#6486588) Homepage
    In my many years of experience, I have to say that regression testing is not as common in proprietary software development as it should be (and frequently not as common as claimed). Furthermore, while I don't want to say that regression testing is less important for Libre Software, I will say that I think it's probably more important for proprietary software, where the programmers are writing for a paycheck, rather than for pride, and are frequently under intense deadline pressure (which in turn, frequently leads to testing/QA of all types being skimped).

    As for no release dates, anyone who doesn't recognise that as an advantage of Libre Software simply lacks any clue about the process. Sure, there are downsides, but nothing's perfect. Free, reliable, on-time, pick any two. :)
    • It's also my experience as a proprietary software developer that QA varies wildy, even internally in the company. This is not just a developer issue; we need managment on our side. If the managment sees QA as just "money bloat", we've got a problem.

      As for regression/unit testing : unless the software arcitecture supporting this, it will not be done. Quite simply because it's time consuming, and changing the arcithecture might very well be too costly. However, there are Open Source tools available, like f

    • Re:hmmm..... (Score:3, Insightful)

      by deranged unix nut ( 20524 )
      I only have three years of professional software testing experience, and the company that I work for seems to be good about running regression tests every week, sometimes every day and even though there are deadline pressures they do let testing drive the release schedule.

      I have chatted with testers at other companies who have disasterous process and for which testing isn't taken seriously, so it isn't universal.

      I would venture to say that software quality is only as good as the people in charge of it's d
    • Free, reliable, on-time, pick any two. :)

      Reliable, on-time, pick one IMHO :)

  • If your not trying to sell it... (Score:3, Insightful)

    by Faith_Healer ( 690508 ) on Sunday July 20, 2003 @04:31PM (#6486596) Homepage
    When I release code I realy dont care about getting any certifications and accridations linked to it. I write code to be usefull not to be noticed. You use open source at your own risk, and the beauty of it is if you dont like it you can just change it. Why would you need a quality assurance process in place for open source. If its open and its quality people will use it. If it doesnt work then people will just blow it off. Does any one else share in my oppinion?

  • Free Doctoral Thesis (Score:3, Interesting)

    by PaddyM ( 45763 ) on Sunday July 20, 2003 @04:45PM (#6486666) Homepage
    I skimmed the article. I'm not sure what they were trying to prove. Release dates didn't jump out at me. As a software engineer, I don't see what release dates by you. Regression testing is obviously missing.

    BUT HERE's the free idea. Maybe someone (aka doctoral candidate) could prove that because Free software is used by ALL possible users in whatever way they are going to use it within, say 4 months, that it doesn't matter if there's some obscure bug, in say, the Intellivision drivers for Linux, because no one uses those. I.e., software has an astronomical amount of states, any 1 of which could be broken, but after all (5 billion people) actually use the software, all HUMANLY possible states are VERIFIED.

    • Re:Free Doctoral Thesis (Score:4, Insightful)

      by Xtifr ( 1323 ) on Sunday July 20, 2003 @05:00PM (#6486761) Homepage
      Free software is not used by "ALL possible users", it's used by interested users. The size of the user-base varies greatly from project to project. And no, speaking from experience, I can assure you that not all bugs are found w/in 4 months. Subtle timing or edge-case bugs can lurk for years before leaping out to destroy someone's critical data. (And this is true with both proprietary and libre software.)

      One thing this study didn't seem to look at though, was the size of the user-base of the projects studied (of course, this is a hard thing to measure, but interesting). I think it would be useful to see what sorts of correlations (if any) there are between QA practices of a project, size of the user-base (popularity), and the overall quality of a project. Can a large user-base help make up for poor QA practices, or is a project with better QA more likely to attract users due to its higher reliability? Do users even care about quality and reliability, or do they just say they do? Interesting questions for which I don't have answers.
      • And no, speaking from experience, I can assure you that not all bugs are found w/in 4 months. Subtle timing or edge-case bugs can lurk for years before leaping out to destroy someone's critical data. (And this is true with both proprietary and libre software.)

        I'll second that! I have been responsible for, and paid to test a reasonably large administration tool for the last three years. This product was developed and tested for at least 4 years prior to my employment. Even after 4 years of someone else

  • Site is /.ed -- Abstract (Score:2, Informative)

    by Anonymous Coward
    Quality assurance under the open source development model

    Luyin Zhao, Sebastian Elbaum.

    The open source development model has defied traditional software development practices by generating widely accepted
    products (e.g., Linux, Apache, Perl) while following unconventional principles such as the distribution of free source code and
    massive user participation. Those achievements have initiated and supported many declarations about the potential ofthe open
    source model to accelerate the development of reliable s

  • No Deadlines does not mean No Pressure (Score:5, Informative)

    by Commykilla ( 107585 ) on Sunday July 20, 2003 @04:55PM (#6486723) Homepage
    All the comments so far have been arguing that "no set release date means more QA". From my experience with open source software, this is complete crap. In many cases, commercially used open source software is developed by organizations such as Red Hat and IBM who, we hope, have significant QA teams and resources.

    But, for the projects contributed by individuals, with a single or small group of developers -- who wants to do testing? Everybody wants to be a developer, and the more new features, the more bang, the more people will be drawn to the program, so more often than not new features and frequent releases rather than testing are the focus. Most of the time, you can guess from the sheer frequency of builds with new features that QA is not being taken into account. This system works great for projects that enthusiasts use -- individuals request new features, play with the product, and in return, report bugs. This does not work for companies of any size who expect software to work properly right out of the box, where the "feedback loop" of reporting bugs is a last resort.

    Open source programmers and commercial programmers alike are under pressure to release early and release often. And in both the open source world and the commercial world, there are groups that excel at QA and groups that don't take QA seriously enough.

    • "who wants to do testing? Everybody wants to be a developer, and the more new features, the more bang, the more people will be drawn to the program, so more often than not new features and frequent releases rather than testing are the focus."

      There is a solution to this problem.

      1) Stop using C. Use object oriented languages and languages that offer garbage collection. You will immediately reduce bugs by 80%

      2) Make code more literate. Use pre and post conditions, demand that all contributors use lots of a
    • This system works great for projects that enthusiasts use -- individuals request new features, play with the product, and in return, report bugs. This does not work for companies of any size who expect software to work properly right out of the box, where the "feedback loop" of reporting bugs is a last resort.

      Most mature OSS projects have a "stable" tree, and a "development" tree. The company that wants something that "works right out of the box" uses the "stable" tree, the guy who wants new feature X us
  • and I have my new release.

    When does any commercial project match that ?

    I hope cvs won't ask me for a credit card number in the future....
  • >>such as regression testing and setting release dates

    Scheduling (Release dates) have nothing to do with QA. A schedule is a separate leg of the stool upon which a project sits; quality, scope and cost being the others.
  • Release dates for open source projects are more honest. They say beta or alpha when they really mean beta or alpha, and they say release when it actually works.

    Most proprietary software fails for reasons that manufacturing failed in the 1980s. Top down, hieararchical processes simply do not work. You need to have cross functional teams, have a quality culture that allows developers to take risks and the initiative, and interact directly with requirements. Most of today's processes simply don't do that
    • Release dates for open source projects are more honest. They say beta or alpha when they really mean beta or alpha, and they say release when it actually works.

      For commercial projects, its interesting to see what happens when you really REALLY make the projects adhere to schedule or don't release. I am now in my second organization where the group I work in has cracked down hard on release schedules. The bottom line was: you release on time, and according to established process, or you don't release. In ot

  • OSS QA will always be two-faced. (Score:5, Insightful)

    by Dthoma ( 593797 ) on Sunday July 20, 2003 @05:11PM (#6486813) Journal
    The open source development model allows tremendous flexibility, allowing members of a development team to be dropped or added at a moment's notice. With the source readily available, one can become familiar with a project's code before applying to be given access to the CVS or equivalent repository. Gradual accretion may produce code in a style not unlike that of James A. A. Joyce's Ulysses manuscripts, but, like James A. A. Joyce, all of the core developers can easily jump from point to point in the code and comprehend the necessary sections and the C/PHP/Python/Java equivalent of their allusions.

    Unfortunately, as a result of this decentralised development system, commercial QA, support and RHQ are not as readily available. I'm a middle manager and my company has had a double-sided experience with the MySQL AB organization. They produce a fine product which is perfect for a medium sized corporation such as the one I work for (which shall remain nameless). MySQL worked very well for us, but unfortunately at one point we started receiving segmentation faults when there were more than 30 connections or if a query was greater than 2,048 characters in length. We have reported these bugs to MySQL AB but they have not yet fixed them in their latest gamma/production release. However, they have been very polite and are always willing to cooperate with us; even if small portions of the code are not yet fixable and have escaped the relatively poor QA of the EOD, their TOS have always been reasonable and our MD has always been able to CWT regarding the slight problems and BS our way around them.
    • Why not try out PostgreSQL? Perhaps you have some conditions not best met by MySQL.

      For our product (on Windows, alas) we use Mimer SQL [mimer.com] that is available on several platforms. Granted, they are closed source, but runs on several platforms, and a trial version (up to 10 concurrent connections) are freely downloadable.

  • The open source development model gives tremendous flexibility and leeway for various methodologies such as XP or the more traditional but cumbersome waterfall method, allowing members of a development team to be dropped or added at a moment's notice. With the source readily available, one can become familiar with a project's code before applying to be given access to the CVS or equivalent repository. Gradual accretion may produce code in a style not unlike that of James A. A. Joyce's Ulysses manuscripts, b

  • Say what? (Score:4, Informative)

    by curtlewis ( 662976 ) on Sunday July 20, 2003 @05:24PM (#6486871)
    Since when does QA set the release date in open OR closed software? Certainly not with any company I've worked for.

    The role of QA is, unfortunately for the state of software, rapidly diminishing. Open Source has only rarely done QA in a professional manner due to it's very nature. And in closed source in today's economy something has to give between time, money and quality. We all know it's not going to be money and time to market is viewed as the all-important element, leaving one thing to suffer.

    I have literally worked for a startup where it was essentially "It boots? SHIP IT NOW!" That's how sad it is with respect to professional development these days.

    These days the only people that care about Quality are the customer and QA.

    • Since when does QA set the release date in open OR closed software? Certainly not with any company I've worked for.

      I've worked for Microsoft, where QA literally decided the ship date. If it wasn't ready to ship, then it wasn't ready to ship.

      (which, of course, provokes flames from the Slashdot crowds as Microsoft never can stick to a release date. Consistency is not a feature of Slashdot...)
  • I'm tired of all this talk about open source development vs. closed source development. That's not the issue here.

    It doesn't matter so much whether the source is open or closed as it does who is in charge of the project. Any company could use standard software development methods to produce open source software. Similarly, any company could hire developers all over the world, and have them work together on a project without ever having met.

    It's not an open vs. closed thing, and it has little to do with
  • What I find interesting about this algorithm is that it is applied individually by each node; there seems to be no need for nodes to share data over some complicated protocol as in many distributed systems. Yet (I think we can believe Clarke) this change improves response time through the system as a whole. It's a validation of the basic Freenet model of systems acting alone but providing a service greater than the sum of its parts.

  • No surprise to QA folks (Score:5, Insightful)

    by Anonymous Coward on Sunday July 20, 2003 @06:21PM (#6487167)
    QA is a weak point of OSS. Over the years I've occasionally tested OSS to check out what seemed to be inflated claims of quality. Everything I've tested so far has more or less failed to pass muster. The biggest failures are in complexity metrics such as the Halstead or McCabe tests. I don't mean marginal failures or borderline cases. I'm talking huge blowouts. Nothing seems to be immune, from nasm to the OpenBSD ftpd. And fixing the problem is usually so simple--decompose a complex function or procedure into several simpler ones.

    Other areas of problems are attributable to slovenly or "don't give a damn" attitudes--unused variables, unreachable code, "magic number" constants, and so on. Ignoring values returned by a function are very common. Maybe it is acceptable with a library function, but why return a value if you aren't going to use it? It's better to make the function into a procedure by returning void. On a more theoretical level, the use of weak typing even when the language allows for tighter specification of variables. Strong typing is designed to prevent such oddities and inadvertently multiplying a color by date.

    However, in the end it all comes back to complexity. And that is where the biggest improvement in OSS quality can be obtained.

  • why is is that IBM open source proejcts have more regression testing than SUn's own?

    Inquiring minds want to know :)

  • Comercial software has relase dates for one reason: money. If they don't release it at the right time they don't make money. Release too soon and you get a reputation for bugs. (beta programs help relase earlier, but you still can't do them too soon). Release too late and someone else will beat you in.

    However it is more complex. I've worked with projects targeted at telecoms. Back then they had a lab, and nothing was put into production until they bought a bunch of them, and it was in the lab for 3

  • GNOME (Score:3, Informative)

    by asobala ( 563713 ) on Sunday July 20, 2003 @06:57PM (#6487320) Homepage

    http://mail.gnome.org/archives/gnome-hackers/200 2-June/msg00041.html is the historically significant mail for GNOME in the post-2.0 era.

    We're sticking to 6-month releases (in fact, 2.2 hit the streets after 5 months since the first bit of time was bugfixing 2.0). The advantages to having time-based releases are if the features aren't ready, they wait while the bugs from the finished stuff are removed and the new features released. This is instead of adding more features, making the software (presumably) buggier and buggier, which would make the release QA much harder and more painful.

    6 month releases means that a feature that misses a given release will not have long to wait before it can be in the main development branch again.

    So far, it's working.

    Andrew Sobala, GNOME QA dude, variable hacker, and release team member

  • " there were a few areas pointed out where the open source community does not do so well, such as regression testing and setting release dates."

    It really depends. Setting release dates is not mandatory for an opensource project. Those projects are not made by a company for a customer requiring a specific deadline. OSS projects are mostly made during spare time. So the most reasonnable release dates are "when the code will be completed".

    Regression tests are another story, but it really depends on projects
  • In my experience, most (not all, of course) Open Source projects aren't concerned with backward compatibility outside of the scope of the project itself. Regression testing in OSS is folded into the bug testing.

    That's one of the downsides of OSS. The biggest example: When you upgrade your libc, you have to recompile all of your dependent apps. One thing Windows and Solaris have going for them is that you can run a 5 year old binary on the newest version of the OS and it will almost always work.

    It's a big
  • So are we referring to the software testing side of QA? You see, depending on the size and maturity of your organization, and if you have some kind of maturity rating (ISO, CMM, etc), QA can be very different things.

    QA (for those who don't know) stands for Quality Assurance. For many small companies, that means testing. Is bigger companies, it means lots of different testing, such as installation, integration, subsystem, system, performance, load, stress, regression, automated, etc. When you get to th

Slashdot Top Deals

Trap full -- please empty.

Close