Testing Frameworks in Python 120
An anonymous reader writes "This article looks at Python's two standard modules for unit testing: unittest and doctest. These modules expand on the capability of the built-in assert statement, which is used for validation of pre-conditions and post-conditions within functions. The author discusses the best ways to incorporate testing into Python development, weighing the advantages of different styles for different types of projects."
Traditional testers might be interested... (Score:5, Informative)
It talks about eliminating some of the tediousness from testing web applications, mainly by using automated solutions like WTR [rubyforge.org].
He's also got a list of testing resources [io.com] that's got some good stuff in there...
Re:Traditional testers might be interested... (Score:2, Insightful)
bull (Score:5, Informative)
1. I've never had a problem with memory management in python. Can't say it doesn't exist, just never impacted my production applications.
2. Implementing great built-in test frameworks doesn't need to wait for memory management improvements. I'm seeing almost immediate pay-offs from this kind of built-in testing.
3. I'm implementing python as an alternative to java in large applications - with complete success. Easy to learn, easy to maintain, fast enough to handle millions of rows of data a day - what's not to like?
Re:bull (Score:1)
For large application maintainability, I do not like that I cannot expect to look at a Python function definition and see what are the expected types of the arguments.
Larry
Re: bull (Score:2)
Yes, with the type of the formal parameter an ancestor of the class of the actual parameter.
Testing is useful after chan
Re:Python's dirty little secret (Score:4, Interesting)
Circular references are what weak references [python.org] are meant to handle. But it'd be nice if the garbage collector would handle those cases automatically.
Re:Python's dirty little secret (Score:4, Informative)
Re:Python's dirty little secret (Score:1, Flamebait)
I would never use Python for server applications, but I love it for quick'n'dirty scripts (although I prefer Ruby more and more these days.)
Re:Python's dirty little secret (Score:2, Interesting)
Snide remarks about "dirty little secrets" notwithstanding, I know which platform I trust to ke
Re:Python's dirty little secret (Score:4, Informative)
It's Guido van Rossum.
As for your claims about the GC, I was under the impression that current versions of Python have a hybrid GC that cleans up circ-refs. I certainly could be mistaken, though, as I've been exiled to Java-land for the last several years.
Re:Python's dirty little secret (Score:3, Informative)
According to this [python.org] you are correct.
:)
BUT.. please note it also says "You can run gc.collect() to force a collection, but there are pathological cases where objects will never be collected."
It also mentions the use of weakrefs.
merit (Score:1)
Re:merit (Score:2)
Would you not expect ridicule if you said that Bill Gables was a fool because his OS, Windows 3.11, was full of security holes that hadn't been dealt with
Re:merit (Score:2)
BTW, a troll is criticized because it is designed merely to eli [catb.org]
demerits (Score:2)
Starting Score: 1 point
Moderation -1
100% Overrated
Karma-Bonus Modifier +1 (Edit)
Total Score: 1
Sharkbite!
Re:Python's dirty little secret (Score:3, Interesting)
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:5, Insightful)
It's Guido van Rossum, not Guido van Sustren. And Python's garbage collector works fine on cycles, as it has done since 2.0, iirc. The only way you'll get a memory leak with Python these days is by writing a C module that forgets some DECREFs or by writing cyclic objects with __del__ methods, and even in the latter case, you can easily take care of your leaks by breaking the cycle yourself.
You're uninformed and incorrect; take your troll to the next Perl story, please
Jeremy
Re:Python's dirty little secret (Score:1)
Re:Python's dirty little secret (Score:4, Insightful)
Neither of those references point out he weakref [python.org] module that allows you to work nicely with circular references.
Python's memory issues need to be fixed before the language can break out of its niche.
Yep, it's a niche [google.com] language [python.org]. Crawl back under your bridge.
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:5, Informative)
The second hint is that you cite a page from 1999 [nightmare.com]. A Python developer would know that that was before Python had cyclic garbage collection. Here is an article [ibm.com] from 2001 that describes how the issue was fixed.
The third hint is that you point to a page [zope.org] that describes how to avoid creating a memory leak by appending an infinite number of items to a list. Guess what: appending an infinite number of items to a list causes a memory leak in Java, C++, C, assembly, Scheme, sh, Perl and every other programming language in the world. If you ask the computer to store a continually growing list of items, it will do so...in any language.
If you think that Python can leak memory in circumstances where other languages would not, post an example program and we'll all test it out.
Re:Python's dirty little secret (Score:3, Insightful)
Properly implemented, Scheme should never leak memory. Even if you try to add an infinite number of items to a collection, you'll get an out-of-memory error before you'd get a leak.
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:2)
Re:Python's dirty little secret (Score:2)
Leak has a very specific meaning. A memory leak is memory that is no longer referenced by the program, but is not freed within an indefinite period of program runtime.
If this is your definition of "leak" then Python cannot leak either. In my mind, if you are running a server application and over days, months or years the process grows for no outwardly observable reason, that application is "leaking".
I think FOLDOC agrees [ic.ac.uk] with me: A leak in a program's dynamic store allocation logic that causes it to
Re:Python's dirty little secret (Score:2)
Your example about a server process isn't necessarily a leak though. If the app is keeping references to that data (ie: it's not finished using that data), then its not leaking. If it *isn't* keeping references to that data, but it still does not get reclaimed, then it is a leak.
Re:Python's dirty little secret (Score:2)
If Sc
Re:Python's dirty little secret (Score:2)
The only reason I brought up Scheme is because you mentioned it.
Re:Python's dirty little secret (Score:2)
The process grows without bound, just like in every other language...just as in the original URL [zope.org]. Sorry to keep on about it but it annoys me when I am stomping a troll and somebody pops up with a non sequiter that confuses the issue.
Gentoo (Score:3, Interesting)
Re:Python's dirty little secret (Score:2)
In 1.5.x you were definitely correct. Except that it wasn't a secret.
I don't know about the versions between 1.5 and 2.1. Somewhere in there they changed the garbage collector, but the change to 2.x had to do with a change in sponsorship/licensing, not with a change in technology...so I dont' know just where.
In 2.1 the new garbage collector was introduced that handles cycles. I don't know h
Re:Python's dirty little secret (Score:5, Informative)
b.foster's misguided comment gives a link to a 1999 article about problems in Python with cleaning up circular references. Well, yeah: in 1999, Python 1.5.2 could leak on circular references. It was actually less of a problem in practice than advocates of mark-and-sweep collectors tended to complain about; but it really was a limitation. That was a long time ago.
Nowadays, Python still uses references counting as its primary collection mechanism. It's a nice system: fast, deterministic, etc. (despite what you might think, refcount *really is* faster than Bohm generational; and is well-tested as such by very smart Python developers). However, the last half-decade of Python versions also perform occassional checks for orphaned objects, and cleans them up too. The "problem" pointed to is of historical interest, at most.
Re:Python's dirty little secret (Score:3, Funny)
Sorry bud, I think I've heard that line before.
Re:Python's dirty little secret (Score:1)
Circular references, in my experience at least, are a) rare and b) easily avoided if they cause problems.
Different types of project ? (Score:1, Offtopic)
Call me silly here, but that article talks about developer ONLY testing, and doesn't seem to discuss different types of projects at all. This was about basic code testing, and mainly unit-test.
No UAT, no System Test, no Integration Test... no how test cases should be defined.
Please go an get a decent, non-language specific book on testing before reading and listening to this article.
Re:Different types of project ? (Score:5, Interesting)
> article.
Sure usability, user acceptance, system, string, and integration testing are all valuable. But why can't developers without any knowledge of these start with built-in unit testing?
And keep in mind that few books on testing cover the fairly recent phenomenon of test-driven-development (or test-centered-development).
The use of built-in test harnesses and focus upon developing tests as documentation of requirements is probably the biggest single improvement in testing in twenty years.
In one fell swoop the debugger is rendered obsolete...
Re:Different types of project ? (Score:3, Insightful)
> in testing in twenty years
Right on. Being able to make some changes and run a suite of 500+ tests to make sure things still work is a thing of beauty.
And when a bug slips through, that's just an opportunity to write another test and make sure that bug never happens again. Good stuff.
Re:Different types of project ? (Score:3, Interesting)
The links he provides in the Resources section at the end of the article provide well-rounded background information on testing in general.
Re:Different types of project ? (Score:2)
I'm learning Python and I think the unittest section will help greatly while stumbling through the process of building applications.
I've found that writing tests before writing the code they test requires a deal of self-discipline (ie. I don't do it as often as I should. When I have made this effort however, rather than stumbling through, the unittest module will took me by the hand and led me through.
Running the test suite while writing really focuses you on a specific part of your code (ie. that par
Re:Different types of project ? (Score:3, Insightful)
Yes, this is exacly what this article is about. What is wrong with that?
No UAT, no System Test, no Integration Test... no how test cases should be defined.
No it is assumed that anyone programmer worth his salt knows some test methodology, and this usefull article explains tools that make that task easier.
Ple
Re:Different types of project ? (Score:5, Insightful)
I wrote a 2300 word article for a column on Python! I didn't write a book. Well, actually, I did write a book, but it isn't the above article (and it's not about testing frameworks). It's certainly not a good idea to think my short article is the alpha and omega of testing. But I am confident that my article -does- address a topic that some Python programmers can benefit from. And other installments of _Charming Python_ each address similarly small, but useful, topics.
Re: (Score:3, Informative)
Testing.. (Score:4, Informative)
I hate to break it to the hack and slashdot crowd, but Testing is actually a whole career in itself, and the application of different testing processes and methods to different projects is a critical part of ensuring projects succeed.
This article covers NOTHING about the different types of testing on a project, or indeed how test cases should even be constructed. Its basically about some UnitTesting elements that could be done by testing.
I know its unpopular here on Slashdot to claim that there are more developers working on big projects than people hacking in Python. Buts its articles like these that underline the difference between professional software development and hacking.
This is about hacking.
Unit tests are just one aspect of testing (Score:5, Insightful)
What you're saying puzzles me. You're absolutely right that this article is not about testing as a whole. The title should have been "Unit Testing Frameworks in Python."
But your statement that "this is about hacking" and not professional software development puzzles me.
I believe unit tests are a very legitimate piece of testing - a kind of first-line defence. They're intended to test individual software modules for their low-level behaviour. Typically, a developer would be expected to run them before submitting any change or bugfix, as a kind of "smoke test" to make sure things are okay. Certainly, some organizations might make the mistake of thinking that this kind of testing is all that's required - which is dreadfully wrong - but I don't think there's anything hackish about it.
In a large organization, the testing team might not consider it testing because unit tests are necessarily maintained and performed by the developers only.
But I would argue the exact opposite with regards to underlining the difference between professional software development and hacking. If you don't have unit tests, I would say that what you're doing is closer to hacking.
Re:Testing.. (Score:4, Interesting)
Agreed. I can impress someone with my knowledge and professionalism, but when they ask what languages I can program in and I state "I'm not a developer", they just don't get it. That I can whip a program up is one thing...that I know that I'm not the person to do that seems to lead to puzzlement.
These days, testing (and CM) is so far off the radar for most folks that it is no longer considered an issue. Yet, developers are constantly required to know all details of the specs and to deliver code that does not immediately fall apart. Some can, but very few can do it rapidly.
The extra burden of detailed testing -- work that the developers are usually not suited for or have time for anyway -- does not make for a productive environment. Limited testing is OK though full testing by development is a distraction -- both VV&T and development groups know it.
That said, initial unit tests of the core services can be the responsibility of the development group, though it should never fall to the same person writing the code being tested. The unit test framework will be incomplete and faulty since the developer does not have the proper distance from the code they are testing.
A skilled VV&T developer can do the job much better, though time for that person to write code to test the developer's work is usually not allocated in the schedule...even if such a person is hired at all.
(Case in point: I was hired last year to automate testing and test 2 applications...10 months later, and here I am having tested about 20 applications and have not written any automated test sripts. Yes, I warned them that it takes a specific type of environment to support automated testing, one that can allow substantial amounts of time and effort to creating and maintaing those scripts, and I didn't see that environment. Yes, they understood and said 'do it anyway'. Yep, that worked.)
Re:Testing.. (Score:1)
The posting and the article it links to are just what they claim to be.
Software Testing is a whole different discipline, in its entirety, from software development. This does not negate the fact that unit and integration testing should first be performed by the developer, since they know their expected results.
This article is "a good start" for information on Unit Testing Frameworks for Python.
As a
Re:Testing.. (Score:3, Interesting)
What to me is so strikingly insane is th
Re:Testing.. (Score:2)
Hard-earned experience shows that it's risky to depend on developers to test their own code. It's kind of like playing a chess game with yourself. That said, I disagree with MosesJones's claim that unit testing is just hacking. It's an excellent part of a balanced breakfast. Ironically, i
Languages form an ecosystem (Score:4, Insightful)
When I want to solve a program I choose the language I will use, taking into account the abstractions and facilities it offers. * I chose Java when I wanted to leverage the javadoc applets (doclets) to convert a Java-like syntax into UML with my UMLgraph tool
* I chose C++ to implement the CScout refactoring browser for C programs. In this case I wanted rich and efficient data structures, with minimal speed and space overhead. CScout datasets can require more than 1GB of RAM, and runtimes can span more than a day; any overhead of object boxing, garbage collection, or bytecode interpretation would in this case be unacceptable.
* I chose Perl to o Convert digital photographs and GPS track logs into annotated photo albums and trip maps
o Examine the availability of 4500 URLs cited in computer science research papers.
o To create the diagrams and the index for my book Code Reading: The Open Source Perspective.
In all the above cases, I needed a typeless language with a rich set of operators, functions, and libraries to minimize the time I would spend to convert my ideas into code. Ruby and Python would have served me equally well.
* Finally, I chose C to write
o the *BSD sed implementation.
o The socketpipe zero-overhead network pipe tool.
o The Outwit Windows-Unix shell integration tool suite.
o The fileprune backup file prune utility.
o A device driver for interfacing with my home's alarm system.
In these cases, I did not require any fancy data structures or framework APIs, but I did want tight integration with the underlying system, absolute efficiency, and minimum-fuss portability. For code that will be executed billions of times on tens of thousands of systems, spending some additional effort to provide the absolute efficiency and reasonable portability that are possible in C, is a proposition one should take into account.
top of the foodchain (Score:3, Insightful)
Jython for unit testing (Score:5, Interesting)
Why Python? (Score:1, Interesting)
Re:Why Python? (Score:4, Insightful)
For example, it's easier to see what's going on in:
than in the equivalent C or Java code. Also, terseness isn't the ultimate goal--I could make this more terse (and less readable or efficient) by saying:
The biggest benefit of Python over Java, as far as I'm concerned, is that code written in Python looks like what it does--that's the source of the "Python is executable pseudocode" meme you often hear from Python fanboys. The lack of a compile step and the dynamic typing help, but they're secondary to me.
Re:Why Python? (Score:2)
for line in file("/var/log/messages"):
Re:Why Python? (Score:2)
Anyway, I figured anyone who read it would either have enough Java and Python experience to parse it, or too clueless to catch it.
Re:Why Python? (Score:4, Informative)
Can someone help me understand why I would want to use Python as opposed to Java?
Python is not just about brevity, but I think you underestimate writing less lines of code. Take the fact that if you are writing twice as much code, you're probably writing twice as many bugs. Also a consise 50K LOC program is much easier to get around in than a 200K program.
But anyway, Python is an interpreted, interactive, object-oriented programming language. [python.org], where as Java is only the last of those three. Python also has dynamic documentation, meaning that the documentation string can be inspected at run-time, without the need to maintain a separate doc tree, ie Javadoc.
Classes are dynamic ie. you can change their behavior at runtime. All python objects can be marshalled or serialized, while in Java you have to define the interface, and account for anything that you use that can't be serialized.
There are also thing which I am indifferent about like tabbed blocks, and some things which I don't like, like not having enough compile time type checking. IMHO you should always define the type of a function parameter, it makes learning the API faster.
In general, Python is great for script-strength stuff, ie. in situations where you would use perl. I can see it getting unwieldly as it got bigger but it generally scales much better then perl. Java tends to be too complex and too restrictive. In short there are many situations where you would choose one over the other.
Re:Why Python? (Score:1)
In general, Python is great for script-strength stuff, ie. in situations where you would use perl. I can see it getting unwieldly as it got bigger but it generally scales much better then perl. Java tends to be too complex and too restrictive. In short there are many situations where you would choose one over the other.
Python actually scales very well. I feel most people tend to think that it doesn't thanks to its "interpreted language" label. While the label is correct, the argument that it doesn't s
Re:Why Python? (Score:2)
Also, my python programming isn't quite yet "pythonic" so my concerns are likely unjustified, and it does remain on my short list of implementation languages.
Re:Why Python? (Score:2)
Re:Why Python? (Score:2)
Less lines of code != more efficient
It boils down to what those lines of code are translated into. You can just as easily write a single line program which makes a library function call containing 50 machine instructions as you can write a 50 line assembly language program which gets converted to 50 machine instructions. Lines of code is not a good indicator of efficiency - not by a longs
Re:Why Python? (Score:3, Informative)
It's a beautiful language, I'm sorry I can't code in it more...
Re:Why Python? (Score:2)
We should also keep in mind the reality that we as programmers make mistakes and typing a few extra characters to improve code readability/maintainability may not be at that bad of an idea.
One of Python's great virtues is its readibility/maintainability. Python is not a terse language, if your really want to minimise line count (and decrease readibility) you'd do much better with Perl (which is not to say that you cannot write readible Perl). In fact part of Python culture is the eschew the one-liner.
Integrating the two approaches (Score:2, Informative)
Here is a presentation, with PDF article, about integrating the two approaches, straight from recent PyCon:
Literate Testing: Automated Testing with doctest [python.org]
Re:true programmers... (Score:5, Insightful)
Despite that fact that you are clearly trolling I think there's a valid point in what you are saying: there are programmers who think that they are too good to write unit or system tests for their code. And that's a real danger.
The adage "a line of untested code is a line of broken code" is so often true that I still find it scary when examining untested code. It's just amazing how much of a difference the discipline of writing automatic unit tests makes in improving code.
If you think you are too good to write tests, then perhaps you are too good for the software industry?
John.
Re:true programmers... (Score:3, Insightful)
Moreover, unit tests make t
Re:programming languages... (Score:2)
Re:Python and Perl... (Score:1)
Just thought you might want to know.
Re:Python and Perl... (Score:1)
There are httpd's written in pretty much every language -- although I don't think BrainF*ck [muppetlabs.com] has been used yet. And Malbolge [mines.edu] certainly hasn't.
Re:Python and Perl... (Score:1)
> that he or she can't do with awk, sed, and grep.
And there is nothing a that a real Unix professional can do with awk, sed and grep that cant be done with emacs.
But then, there is nothing that cant be done with emacs.
sufficiency (Score:2)
"Any technology, sufficiently advanced, is indistinguishable from a rigged demo." - Luke McCormick