How Would You Improve Today's Debugging Tools? 731
redelvis asks: "I recently came across an article by MIT's Media Lab on 'The Debugging Scandal and What to Do About It'. It's a few years old now, but it really got me thinking about how little the debugging process has improved over the last 5,10 or even 30 years. I have developed applications using modern IDE debuggers such as Borland's JBuilder, Microsoft's Visual C++, as well as standard tools like gdb and jdb. Despite the slick graphical interfaces, nice thread stack traces and local variable browsers, I still make sure I have on hand plenty of notepads, graph paper, pens and pencils so I can try to build up a picture of what state the program is in and to help me play detective in pinpointing what is going wrong with my (or other peoples) programs. Do other developers have similar problems? Do they find modern IDEs and debuggers have shortcomings in helping track down bugs? What would make a better debugger? Why do you think so much effort been invested in areas such as advanced modelling tools but so little in improving debugging tools?"
I'm sure loads of debuggers have got it... (Score:4, Interesting)
Other than that, I'd like generalized hot-swapping of code and 'step back' (which I don't know if it's always factible); or at least be able to view a variable's history.
Shame that my IDE of choice (Eclipse) doesn't (AFAIK) support those.
Forget the Debugger (Score:3, Interesting)
Still nothing to beat (Score:4, Interesting)
and the mighty grep
Improved Variable Watch (Score:5, Interesting)
Debugger improvements (Score:5, Interesting)
Problem Solving Skills (Score:2, Interesting)
Remember those high school days of comparing the green text on your TRS-80 monitor to the print in your Softside magazine. Now that was fun... har har.
Visual Basic debugger.. (Score:3, Interesting)
How about a "previous" command? (Score:2, Interesting)
Back at Silicon Compilers, we actually had a hardware simulator that could do this. It was a pretty neat trick: it would checkpoint the entire system state every few hundred cycles, then when you wanted to back up, it would restore to the most recent checkpoint and run forwards until it got to the right clock cycle, the one just before where you asked it to back up.
println debugging (Score:4, Interesting)
Recently I remember seeing some justification/defenses of this technique, I think it's sometimes called "logging based debugging".
When something goes wrong I figure out the related assumptions I'm making, and then test 'em by output to STDOUT or STDERR or a file. It avoids some of the "Heizenbug" problems debuggers can bring on (so long as dumping output can be reasonably expected to be reliable w/o bringing on crossproblems) and by thinking of your assumptions, you're tying into principles similar to "write tests first", part of XP practices.
One set of things this doesn't handle well is memory leaks and bottlenecks and looking out for runaway object creation and the like, but garden variety debuggers usually don't cover that too well either, you need special profiling tools anyway.
some ideas on why debuggers are in this state (Score:5, Interesting)
b) making a good debugger is definately not easy
c) many developers prefer puzzling out the code on pen and paper then walking it in a debugger. Quite a few I have talked to believe that they make better code if they can solve the problem without resorting to debuggers. This BTW is why Linus does not like kernel debuggers.
d) most of the debuggers are good enough and as we all know from open source, a good enough solution will live forever. The existence of a debugger that mostly works gives most people a reason to spend their time elsewhere.
In my experience I have truly benefited more from programs like Purify or Insure (and now the great valgrind) than I have from a debugger.
Fix the programmer, not the program (Score:3, Interesting)
The best way to learn how to write bug-free code is to NOT use a debugger at all. Every programmer should begin writing code in Notepad or some other basic, no-frills text editor. Just like everyone should learn how to type on a typewriter and not a computer. It forces you to NOT make mistakes, instead of correcting them (you don't have to "debug" a program if you don't put the bugs in in the first place!) It forces you to slow down and think about what you're doing, instead of blasting through a procedure and then run the debugger on it. If everyone programmed for at least a year without ever using a debugger, the quality of their code will improve far greater than any fancy IDE debugger will improve it.
Reversable debugger (Score:3, Interesting)
mahlen
Re:Visual Basic debugger.. (Score:3, Interesting)
The call stack is at your fingertips, I add/remove conditional watches with a couple keystrokes, stepping into, out of, and over functions, and the immediate pane makes a great scratchpad/interface to the internals of the program. You can break on all errors, unhandled errors, or arbitrarily.
It wouldnt be nearly as easily to quickly put together apps in VB without it's debugger. It truly turns a half-assed language into a really good tool for rapid prototyping.
It's a deep complex issue (Score:1, Interesting)
I've seen that most of the high level languages (java, Visual Basic, c#) promote RAD via lack of formal trainging. Thus the real problems start to show up when someone who has learned the syntax but knows nothing about how IPv6 packets handle little endian sockets tries to debug a complext multi threaded app and spend all their time basically guessing instead of analytically and systematically debugging the application.
Fred Brooks in the "Mythical Man Month" stated that unless a language is written in the same language that you increase the time to debug it by Olog(n) and that's true today when you look at java (not written in java) or perl or c# as opposed to languages like C and Fortran that are optimzed for debugging by being written in C and Fortran. Much of what Fred wrote about is not pertinat any more that this one still stands the test of time.
Basically it's a low level problem and not a "debugging tool" problem as I see it.
Warmest regards,
--Jack
My Dream Debugger (Score:5, Interesting)
1) The ability to set conditional breakpoints.
2) The ability to see not only a variable's current value, but a stack of all of its previous values.
3) The ability to select a variable's previous value and jump to the line of code that set it to that.
4) The ability to change the value of a variable at any point.
5) The ability to add/change code on the fly.
6) The ability to jump into the debugger at any point in the program, even when I hadn't planned to before running it.
7) Auto-logging of method calls and (optionally) variable values, to be started and stopped as I see fit either while stepping through code or running it.
That's all that comes to mind off the top of my head.
Perhaps it's an economic issue. (Score:4, Interesting)
A debugger is going to help you find and fix the bugs that got through:
Studies have shown repeatedly that the cost of fixing each bug increases at each stage. So if you are going to invest $1000 in new development tools, is it more economical to buy tools to detect bugs when they're cheap, or when they're expensive?
Great Development Tools (Score:2, Interesting)
Why on earth do so many people feel all development tools need to be built into a software debugger or they are inferior???
One great tool he did not mentiong is the napkin.
Getting it right the first time (Score:5, Interesting)
The more work goes in before you get to the debugging stage, the less buggy your product will be. A well-designed system does not need as much debugging later. I know I'm idealizing, but I've seen this bear out, especially when version N > 1 comes out and it's been hacked to smithereens because of poor design choices in the previous iteration.
Probably the most amazing bit of self-fulfilling prophecy you'll ever hear on a software project is a manager saying, "We'd better start coding now, because we're going to have a lot of debugging to do."
As for catching things ahead of time, I've always put breadcrumbs in my code to spew to stderr or a Java error console class or wherever. Very easy to #ifdef out later and trivial to turn back on later.
Memory Threshold Watch (Score:2, Interesting)
Debuggers are for chimps (Score:2, Interesting)
printf();
Seriously, an over-reliance on debugging tools is something I've *definitely* seen come into this industry.
Bah. Chimps.
Re:Perhaps.++ (Score:2, Interesting)
I write CPU simulators. In this kind of environment, there may be hundreds of threads each performing one little iota of work per (simulated CPU) cycle. When a bug is encountered, the code that encounters the bug is not in any way related to the code that created the bug. And the cod that created the bug may have made its mistake thousands of simulated CPU cycles (billions of actual CPU instructions) before the code that stumbles over the bug-incarnate.
Stack back traces, and other vonNeuman-like debuggers are totally useless in this kind of programming environment. I am simulating real parallel things (circuits) on a sequential turing machine. Its no wonder that a debugger is useless--it is based on the wrong mental model of what I am programming.
I have to build the degugger into the actual simulation code--which I proport--is the right way to debug programs anyway.
The best debugging tool I've ever used... (Score:2, Interesting)
It's interesting to note that a lot of those day-to-day bugs can be solved by simply straightening out your train of thought by posing a question in a manner that someone who doesn't know the system can understand.
Re:Perhaps.... (Score:5, Interesting)
This is also experienced based -- as I get older I use debuggers less and less but my code output has remained constant or improved (even though I don't put in all nighters any more :-)
I'm finding it really hard to get excited by the fact that there is no effort being put into debuggers where as 10 years ago it would have been a big deal.
Perhaps that's also part of the problem -- when I evaluate tools etc. I don't spend a lot of time looking at the debugger these days ...
Re:Debugger improvements (Score:3, Interesting)
A stack trace gives you the current calls but what were the previous 5 calls made?
An execution history would tell you what the last N calls were and in what order they occured.
It would be easy to implement as a series of stack snapshots.
Re:Reversable debugger (Score:2, Interesting)
Being the junior member of the programming team, I was naturally the primary suspect. And my boss was quite vocal about it.
Finally, he broke down and brought in a logic analyzer. It only had something like 32K memory, so by the trap triggered, the error was typically out of the buffer. And each time, my boss would again loudly complain about my poor coding skills, and how much money it was costing him because of some bug in my code.
After many attempts, he finally trapped the bug. It turned out to be a trivial error in his code, of course - we were working with zero based arrays, and he allocated n slots. So when message number n came along, it offset just beyond the jump table and grabbed some random address.
Of course, he never apologized for the weeks of insults - just changed the n to n+1, and went back to his office.
The moral: a good debugger is worth its weight in gold.
Write-time debugging (Score:3, Interesting)
For a lot of code, the compilier should be able to work out the pre and post conditions of a function, and then just check that when you call that function you don't violate them.
Also I'd like to be able to hover my mouse over a variable and see what range of values it could have. From here you can check that you never go outside the bounds of a variable, and so on.
Obviously you can never get it to work on all cases due to the halting problem, but it will work with a lot of code - and on confusing code you can manually add the special comments for the post and pre conditions - a good idea anyway for complex code.
Re:Perhaps.... (Score:4, Interesting)
Then you haven't written enough tests. Or you're trying to integrate too much at once.
If you do test-first development [c2.com] and continuous integration [c2.com] (backed up with good black-box integration testing), then the number of bugs you get should be low. In my experience, that's circa one shipped bug per man-month.
If I run all the tests frequently as I develop and check in often, I rarely need a debugger. If I break a test, I generally know exactly what the problem is.
If I forget to run all the tests and check in, then the automated build box will send me email promptly, so it's still pretty easy to find the problem.
It is essential, like an x-ray / CT-scan / ultrasound is to a surgeon.
Note that the only time you see a surgeon is when something is pretty seriously wrong. Isn't it much better to make sure that things don't go so far wrong that a surgeon is necessary?
I don't use debugging tools for much.. (Score:5, Interesting)
But I only use debuggers for two purposes.
Purpose 1. Segmentation Fault (core dumped). Uhm, now where did that happen? Whip out gdb, find the line that generated SIGSEGV, and it's usually obvious how it happened. If not, I have it print out a stack backtrace. If I really can't figure it out then, a 5 minute walk around the block and I'll have figured it out as soon as I sit back down.
When writing in high level languages, I'm finding that debuggers are wholly unnecessary. In fact, I can't remember the last time I spent more than 20 minutes trying to track down a bug. *shrug*
It'd be nice if debuggers solved my problems automatically, but I'm really finding that I don't need them. I might even go so far as saying use of debuggers encourages dependency on debuggers, which in turn discourages thinking about the program itself. Not saying that EVERYONE does this, just that some of the best work gets done remarkably well even without debuggers.
The Linux kernel, for example, was largely developed without the aid of a debugger, and the core developers seem to eschew them. Here's a good thread [insecure.org] on why the developers don't want to include a debugger.
Purpose 2. On the other hand, debuggers are remarkably good at helping you break program code. With having almost no experience using gdb, I was able to break the license key check on Intel's C Compiler in about an hour. I was amazed at how easy it was to attach a debugger to the compiler and skip the subroutine that performed the license key check. With no debugging symbols to work with. Disassemblers rule. It took another 10 minutes to turn this into a script that could be distributed as a wrapper for icc (called xicc), so all you had to do was set CC=xicc in the Makefile.
Sure I could have used LD_PRELOAD so that time() always returned a date within the trial period, but breaking program code with a debugger is just so gosh darn fun.
Re:As somebody who uses printf... (Score:2, Interesting)
Debuggers are useful to a point ... (Score:2, Interesting)
GDB the millstone (Score:3, Interesting)
Even front end tools are little help. DDD is much better than the cli but it still suffers the same performance faults as GDB and introduces some fun issues of its own. Even a 'commercial' tool such as Project Builder on Mac OS X groans to a halt with GDB running underneath and has some very odd concepts concerning when breakpoints are hit or not.
All this would be understandable if Win32 were the same but it isn't. Debugging on Win32 is a snap - just in time debugging, compile in time debugging, a debugger that works, integration with the editor and more. It is sad to say that if asked and irrespective of open / closed source issues, which OS were the better for development I would say Windows. In fact, if I'm faced with a bug in Mozilla, I'd rather fire up Windows and replicate the problem there than wade through the shit that GDB throws in my face.
Die GDB! Linux really needs a decent debugger and all the mod cons that Windows developers have benefited from in the past decade.
Debuggers do more than debug! ("Ask the code") (Score:3, Interesting)
As I've gotten more skilled at software development, I've become better able to read a bunch of code and analyze it
An MIT animal physiologist is said to have told his students, "The animal is always right. When in doubt, ask the animal." Ask the code; the code is always right. (It may be right in knowing what the bug is, but sometimes that's what you need to ask.)
P.S.: I've learned more Java since then. Running some programs through its debugger was one good way to help learn it.
what about non-iteractive debugging ? (Score:2, Interesting)
Most of this discussion seems to be about interactive debugging tools, but as an individual concerned with mission critical telecommunications software where an outage can take out customer private networks, I tend to find less need for interactive debugging, and greater need for post-mortem diagnostic analysis that results from the following:
- faults generated in house system / integration testing
- faults generated in house automated unit / component testing
- faults generated in house automated system / scalability / performance testing
I need to isolate sometimes highly obscure run-time failures and discover what went wrong, then resolve the issue, which I do with:
- high levels of run-time instrumentation using (a) method / object / runtime c++ trace style classes, (b) trace back mechanisms for exception handling and, (c) assertion handling and (d) run-time verification mechanisms; all of which generate output to debug logs (sometimes when returned to support, these logs are multi-gigabyte in size, fortunately our development workstations are very high spec)
- post-mortem analysis scripts to interpret the contents of debug logs and look for faults, inconsistencies and other issues
- application level mechanisms to allow testing, deployment and other experts to enable run-time debug instrumentation (if it were enabled all the time, it would constitute considerable performance loss) sometimes at the direction of developers or deployment / field support personnel, and sometimes these personnel don't have access to machines where the components are installed (because the components are distributed CORBA components, and these personnel have access to host / user interfaces, but not to components in a silo in another country)
The problem is that we've had to design all of this from scratch: (a) general purpose c++ based debugging / tracing modules with command line / run time configuration options, (b) tools to stimulate the debugging facilities to do things [set / change debug trace levels / activation tags / identifiers], (c) tools to capture / wrap up and return trace logs back to support, including capturing host information (processes, memory, installed modules, environment, configuration) and other debugging items (debug stack traces, core dumps) and version information (component version, etc), (d) tools to analyse / work with / process logs for developers, (e) work instructions / process / documentation to educate developers, deployment experts and others about the use of these facilities, (f) make sure that there's a balance between getting the right level of detail, yet avoiding gigabytes of detailed code / method level information
Typically, once I have a debug log returned to me (as I am developer), I need to look at what was going on, and try to reproduce the fault with a unit test case, then inspect the component for faults of a similar type, and correct those with appropriate unit tests as well, then verify regression test for the component, make changes to multiple development branches, and wait for the scheduling of scalability / system / automated testing to ensure no other significant side effects, then safely know that the next release will have corrections.
We shouldn't have had to develop all of this in house: there should be standards for this. Other companies that I have worked with have the same adhoc construction of debugging facilities - for these situations were you need to analysis a running system, or obtain post-mortem results.
Interative debugging dwindles in comparison: unless you are working with obscure hardware, or related sorts of embedded systems, I think that an excessive need to use interative debugging says more about the lack of design / defensive coding / other engineering approaches to reliability of the constructed software. Interative debugging consumes about 10% of my debugging efforts (not including time spend on unit / component / system test design).
Re:Fix the programmer, not the program (Score:1, Interesting)
Must be nice (Score:2, Interesting)
Out here on the wild frontier we're stuck working on code that was written many many years ago by people who weren't quite so clever as the average slashdot poster. These people left behind badly structured code with inelegant data structures and comments that don't accurately describe how the code works. This code consists of several tens of thousands of lines in the component I'm directly involved with and several hundreds of thousands of lines in the larger project.
So while it sure would be swell if all the functions were small, the data structures elegant, the code tidy, and the comments accurate, that's just not the way it is for many of us.
And it pains me to say, but much of the debugging we do is done using printf() because most debuggers are such unredeemable sacks of $#!].
I do much of my debugging using msdev from VC++ v6, and I find it to be even less useful than the CodeView debugger that I used in 1987. There's a few things that are better but there are a lot of things that are worse. How can I get msdev v6 to show some of my variables in hex and some in decimal? Is this such an esoteric request that the developers at MS couldn't be expected to anticipate it?
The number of things that the MS debugger can't do that I could do in 1992 using IBM's VisualAge C++ debugger is even longer. Is this progress?
Savestates. (Score:3, Interesting)
If you've ever followed the popular system computer simulators known as emulators, you've likely seen savestates. It's effectively saving all the variables of a system en bulk, to be leaded back again later so another approach might be taken. They have been very useful to emulator developers who often need to return to an exact point where a virtual machine breaks down, without having to re-run to a certain point.
It would be nice if debuggers could automate the process of saving the state of functions (and everything they immediately touch), so you could go "back in time" rather than need to restart a program in order to return to a non-bugged state. On small programs, it would also be very convenient to be able to save the state of the entire program.
Possible extentions of this idea would be a breakpoint-like auto-savestate at a certain line, and a "step back" feature to be able to rollback one line.
Of course, there would be many possible complications involved in using savestates - but I believe the potential dangers are far outweighed by the opportunities for quicker and more comprehensive testing and code-observation.
Ryan Fenton
Superdebuggers (Score:5, Interesting)
I used to work for a company called Saber, who later changed their name to Centerline. We produced a product called Saber-C (and later Saber-C++, and the products were renamed CodeCenter and ObjectCenter).
This product was a C/C++ superdebugger. It started out as a C interpreter developed at Harvard. Now, an interpretive environment allowed a lot of neat features: You could tell whether or not variables had been initialized at reference time, you had complete stack and context information, it was pretty easy to unload and reload code in source modules (dynamic reloading). You also got dynamic type checking, which caught errors in downcasting (for instance). And everyone's favorite, array bounds checking.
With this tool debugging C code often meant: Loading the code, running your test case, and seeing where the debugger complained. The first time you ran your code through it was a humbling experience ... almost all production code has serious (but normally benign) errors.
Unfortunately the tool didn't scale well because it took time to load the interpreted code into the debugger and the heap and runtime requirements of interpreters stressed machines of the time (this was when Sun3s were common, and 3/50s couldn't even have more than 4MB memory).
I think it was 1990 when we released version 3, whose big new feature was object code debugging (it's not really that easy to have object code debugging as well as interpretive code, although we could certainly have implemented it more easily than we did). We of course maintained the ability to dynamically reload code fragments, so you could do a "make", reload the affected modules, and be on your way in a few seconds. This sure beat the several minute wait you normally had with the linker and debugger restart. But what you lost was some of the runtime checks -- including most of what made the product interesting. Still, the ability to have superdebugger capabilities on some code and still scale to large programs was very valuable.
About a year, maybe two years later we got visited by the people who eventually started Purify. They had a neat approach where they'd get most of the benefit of our tool, but in object code, by disassembling the object code, adding extra code to perform checks, and putting it back together. This would give about 80% of the functionality of our debugger but with much lower runtime costs and work even if you didn't have source code.
We ended up re-implementing this functionality such that we had a Purify competitor (and getting sued, and losing, even though Sun later patented the same technique we used).[1] Integrating it into the debugger allowed you to fine-tune the debugging capability you wanted on a module-by-module basis. This was a superb tool.
Now, what are the raw capabilities that made this stuff really neat?
The tool also provided two additional interesting features. One was called "action points". Since we had a C interpreter in the product it was no big deal to allow you to type C code into it and run it at arbitrary times. We'd allow you to attach any code you wanted to any line in the program or to just type code at the debug prompt and it would run in the current scope. This was useful for things like adding print statements without recompiling, for instance, and for adding assertions, and many other things.
The other feature was a graphical heap inspector. All this did was display structures as tables of their internal values and allow traversal of references to other structures, which would be displayed (including links between the structures). This made the shape of data a LOT easier to determine.
Several people talked about adding reverse execution. That was actually done for Java by a tools company back in 1996 (maybe 1997). It didn't always work, but it was pretty neat. The problem is that the recording code was pretty expensive and there's a limit to people's patience and what they'll spend on hardware (although I grant that hardware is a lot less of an issue today than it was even in 1997, to say nothing of 1989). Anyway this feature is a lot less interesting if your debugger can find errors at the point they occur.
Anyway, back in 1995 when I started to work with Visual-C++ on a regular basis I was appalled by what Microsoft was calling a "state of the art" debugger. Feh! It was 1985 technology with a pretty face. It really has not improved much since then either, although they do at least have a pessimistic heap allocator now. With their ability to hook in with the compiler they could do a phenomenal job with heap allocation tracking and instrumentation of code with error checks but they don't bother (because, I suppose, they make their money even without it).
Now, using a language with runtime error checking built in - such as Java, C#, or Visual Basic, seriously undermines the amount of effort you have to put into the debugger. Still, I'd like to see a lot better heap debugging features built into compilers and runtime environments.
Hope this gives people some ideas, and that they lead to tools I can use.
[1] One thing that could be done to avoid the Purify patents would be to have the compiler emit the instrumentation. For the OSS world, where you always have the source code, that is eminently practical. Furthermore you can get all the benefits of an interpreter but at higher performance. A big problem with Purify's technology, and the technology we came up with, was that you're doing a lot of heuristics during code inspection. It's easy to miss code or misinterpret code, and of course the code inspector is VERY architecture and compiler dependent. What I would like to see is a gcc/glibc/gdb combination where the compiler, libraries, and debugger all worked together. That wasn't possible back when the vendor controlled the compiler and you didn't have access to library source. Today it is, at least for the OSS world.
Re:Write-time debugging (Score:2, Interesting)
Try using DDD. It's a graphical frontend to gdb. Also, it's open source.
Re:Perhaps.... (Score:3, Interesting)
I would call a bug a serious problem. I made no statement about abandoning unit and integration testing, only that the utility of a debugger is essential for the development process. And, to be precise, I should have said a radiologist rather than a surgeon since this would be a pre-screening before any manipulation is done to the body, I mean code.
I agree with Brian W. Kernigan (Score:4, Interesting)
Re:Debugger improvements (reversability) (Score:4, Interesting)
Yes, I could really use a debugger that runs backward for a problem I'm having recently. I wrote about it here [livejournal.com] last month:
There has been progress... (Score:2, Interesting)
For example, I used to work with a tool called BugTrapper from MuTek Solutions (it's now called AppSuite, and the company is now called Identify Software [identify.com]). It can record an application's runtime trace, replay it, rewind it and provide many more helpful debugging tools. Unfortunately it never caught on (maybe because of its price), and nobody has copied the idea to open-source tools
OO Debuggers (Score:2, Interesting)
This one single feature would greatly improve my productivity.
Also, simply generating a list of objects of a certain type would also be useful, so I can see how many there are in the system etc. And I'm sure there could be lots more OO-specific features other than these too. Come on debugger designers - please!