Slashdot Log In
Memory Checker Tools For C++?
Posted by
kdawson
on Wed Jun 06, 2007 04:40 AM
from the heaps-and-bounds dept.
from the heaps-and-bounds dept.
An anonymous reader writes "These newfangled memory-managed languages like Java and C# leave an old C++ dev like me feeling like I am missing the love. Are there any good C++ tools out there that do really good memory validation and heap checking? I have used BoundsChecker but I was looking for something a little faster. For my problem I happen to need something that will work on Windows XP 64. It's a legacy app so I can't just use Boosts' uber nifty shared_ptr. Thanks for any ideas."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
um (Score:5, Informative)
Re: (Score:3, Informative)
Re:um (Score:4, Insightful)
Regarding legacy applications, I think the point was that he can't go back through the app and rewrite everything to use smart_ptr.
Parent
Re:um (Score:5, Insightful)
I guess that's a long way of saying "I agree completely with what you just said."
Parent
Re:um (Score:4, Insightful)
I also have (unfortunately) written enough ugly stuff that when I go back later I say "I can't believe I actually did something that stupid."
You live, ideally you learn, and when you look at code you wrote 5 years ago you likely slap your forehead in embarrassment - that's how you know you're getting better. That, and when your coworkers aren't trying to slash their wrists when they get handed something you wrote...
Parent
Re:um (Score:5, Insightful)
You are confusing two aspects here. Ugliness does limit maintainability. But it does not limit "solidness". "Solidness" would mean that the code actually works, and has a proven track record, such as being used in production for over 20 years. Code that has been in production for over 20 years is usually both solid and ugly.
Or it could be a monument over "the world is a complex place, and if you change anything here, and it causes the program to fail in some weird special case, your company is going to loose umpteen zillion dollars". While the reality is probably somewhere in between, rewrites should still be avoided like the plague. However, if you really have taken the time to understand what some nasty bit of code does, there's nothing wrong about cleaning it up. But most of the time, the ugly code is there for a reason.
Parent
two points (Score:4, Interesting)
First of all, shared_ptr is going into the standard library as part of TR1. Does anyone know when common development environments, i.e. GCC and MSVC, will start including TR1?
Second, I believe that there are a number of garbage collectors available as libraries for C++. I've heard boehm's garbage collector mentioned numerous times. My question is, are any of these libraries any good? Are they really practical to use in real world applications? Do you have to modify all of your types to use it, or can primitive and stl types work with it?
Re:two points (Score:5, Informative)
Boudewijn
Parent
Re:two points (Score:4, Informative)
shared_ptr is a blessing and a curse. It saves you from manually destructing objects held in a collecton (good) but too many developers use it for lifetime management (bad).
Parent
Re:two points (Score:4, Interesting)
Parent
Re:two points (Score:5, Insightful)
It's not automatically bad, but using semi-automated memory management like this tends to reduce the emphasis on constructing things only when they're needed and destroying them immediately when you're done with them. This concern, known as "Java bloat syndrome" in honour of the language that first popularised it, can lead to major performance problems in applications that manipulate a lot of data, and is a favourite mistake made by the cult of "hardware is cheap, so optimisation doesn't matter".
The thing is, this sort of care-free programming philosophy is natural in languages like Java, so languages like Java have had to learn from their early mistakes and adapt. There have been dramatic improvements in GC technology since those early days, and today there isn't the same degree of performance penalty associated with relying on GC to clear everything up.
However, this sort of behind-the-scenes magic isn't really the "C++ way". You can do it, but tools like shared_ptr don't have the same level of sophistication as full-blown GC. Using them requires some care from programmers, and as the grandparent post said, this can lead to problems if the programmers come to rely on them more than they ought.
FWIW, I'm not sure I'd have described things in quite such black-and-white terms as the GP, but I can see the underlying point and I think it's a valid one.
Parent
Re:two points (Score:4, Insightful)
Parent
Re:two points (Score:5, Informative)
You'll be able to use C++-0x in gcc-4.3 with a switch.
I also heard that std::auto_ptr is being deprecated (not removed) I guess in favor of rvalue references.
Finally, there [open-std.org] is a motion to include garbage collection in the C++ language. This is sponsored by none other than Hans Boehm among others.
I realize this doesn't help immediately.
Parent
Re:two points (Score:4, Informative)
They don't want to remove the old ways (including malloc/free). And they don't want to penalize people who don't use garbage collection. In other words, if you don't want to use garbage collection you won't be paying for it in code size/runtime.
But for them that do... It might be nice to offer it.
Parent
Re: (Score:3, Informative)
In order to use it, you just have to include 'gc.h' in your files, which replaces 'new' with 'gcnew' using macros. Alternatively, you can bypass the standard library and provide a replacement for 'malloc' and 'free', but I did not manage to do that (due to the configuration bugs mentioned above).
Re:two points (Score:4, Interesting)
It's a pity there's no way to ensure compatibility between boost::shared_ptr and std::tr1::shared_ptr , nor a really attractive non-preprocessor-reliant mechanism to switch between them (since typedefs in C++ do not work on incomplete template types).
Parent
Fluid Studio's Memory Manager (MMGR) (Score:5, Interesting)
Re: (Score:3, Informative)
STLPort (Score:4, Informative)
Re:STLPort (Score:5, Informative)
Parent
Most tools I've tried are useless (Score:4, Insightful)
They might be useful for small apps but if you have a massive app they are almost more trouble than they are worth.
It's hard to say what you can do except foster safe coding practice and highlight the common pitfalls such as memory leaks, buffer overflows etc. Many compilers can help detect heap / memory overruns because the debug libs put guard bytes on the stack & heap that trigger exceptions when something bad happens. There are also 3rd party libs such as Boehm [hp.com] which help with memory leeak / garbage collection issues and dump stats. I'd say using STL & Boost is also a very good way of minimizing errors too simply because doing so avoids having to write your own implementations of arrays, strings etc. which are bound to be less stable.
Re: (Score:3, Interesting)
Most people can't understand Purify's output (Score:5, Insightful)
For example, this code has serious issues:
char *ptr;
.
.
.
ptr = method_that_returns_string_object();
.
.
And FWIW, I've used Purify on massive apps, and found huge problems that the developers didn't even know were there. On one project, they couldn't explain why their "perfect" app kept crashing, either. Worse for them, I had been hired as a consultant to fix their problems that they couldn't seem believe existed (HINT: your boss hired someone from the outside...), and after watching the team flail and spend literally almost a man-year trying to find one memory bug, I finally had enough of "advice giving" being ignored and got on their system, linked their app under Purify, ran it, and found the bug - a double delete of an object from two different threads. It all took me about fifteen minutes. I did that in front of their management. I made my point.
Purify (and like tools) are a great help. Not using them is like trying to build a house without power tools. Yeah, it can be done. But what would you think if hired a builder to make your house and his team showed up carrying hand saws? Oh, and you are paying that team to hand-saw all the lumber...
What would you think of that builder?
Yet, when a developer asks for tools like Purify, management often balks. Because 1) they're shortsighted, and 2) developers don't know how to use such tools.
Like I said - what would you think of a construction company where the workers don't know how to use modern power tools to help their productivity?
Well, you just put yourself in that category.
Yes, Purify is somewhat slower than running without Purify. But it's a lot faster than most other full-memory checking methods. If you're worried about speed, link against the Win32 debug libraries - they'll at least show problems with double free() calls, access of free()'d and deleted objects, etc. And without too much performance problems.
Parent
Re: (Score:3, Insightful)
Q) How do I deal with memory leaks?
A) By writing code that doesn't have any. (goes on to advocate vector & string)
And also: C++ Is my favorite garbage collected language because it generates so little garbage (http://www.research.att.com/~bs/bs_faq.html#reall y-say-that [att.com])
Over the past 6 months or so, I've really made an effort to better my usage of C++ (using Effective C++, Effective
Re:Most tools I've tried are useless (Score:5, Insightful)
A good example of what I'm talking about is a std::ifstream versus a java.io.FileInputStream. If you make an ifstream on the stack, you can be absolutely certain that when it goes out of scope, the destructor will be called and the file closed. You can be certain that it will happen, and you can also be certain when it happens; at the very point it goes out of scope.
With a heap based FileInputStream, you have no such gaurentee. You leak it, and you just hope that the finaliser gets called soon (if at all). I've had more than one occasion where I've been leaking FileInputStreams quicker than the garbage collector cares to clean them up, and sooner or later the OS says 'no' and you get an exception. And it's very difficult to reproduce, because it's all down to the whim of the garbage collector, and you always go slower when you're looking for a bug.
Of course the answer to this is to say "Well you should Close() your input stream beforehand". But that's just as bad as saying "You should delete your heap based objects" in C++. It's that situation of having to manually shut down objects that seems old fashioned to me.
Maybe there's a better way these days, I've been away from Java for a couple of years now.
(I do enjoy coding in either language though!)
Parent
Boost? Ugh (Score:3, Interesting)
Re:Boost? Ugh (Score:5, Interesting)
Getting it to compile was a bit of a nightmare too. It has its own native compilation management tool that you have to download as well. What the hell is wrong with using make like everybody else? It also uses a very complex template hierarchy that produces terrifying error messages.
I'm sure that once you become an expert, BGL is really powerful and efficient, but I found the learning curve too steep. I just want to get in and build a working prototype quickly so I can see what I'm doing, not spend hours wading through manuals and examples to build the simplest program.
I'll be with the parent post and get modded a troll by boost developers.
Peter
Parent
Re: (Score:3, Informative)
And Boost.Build is muuuuuch more powerful than makefiles. You can try to use Boost.Build v2 in your own projects - it's very very useful.
Re:Boost? Ugh (Score:5, Insightful)
I'm used to templates syntax (though I think its ugly and Stroustrup could have done a lot better) but Boost makes it worse by overloading operators and then using them in ways never intended that produce syntax that a plain C++ wouldn't even recognise, never mind understand what its doing.eg the gratiutous overload of () for matrix ops where a simple function call would have been much cleaner and easier to follow.
Parent
Re: (Score:3, Insightful)
"I suppose you like adding vector components manually, instead of doing v1 + v2?"
No , something like vectorAdd(v1,v2) would be a lot more readable and a damn site easier to grep for. Idiot.
Then probably we should remove the operators for built-in types as well. After all, you could use functions like doubleAdd(a, b). As a bonus, you'd not get nasty surprises when mixing unsigned and signed integers. intGreater(a, n) would always give you the expected answer, even if a is negative and n is unsigned. If you'd want to compare in unsigned arithmetics, you'd use uintGreater(a, b) instead. And what dereferenePointer(p) does is self-evident, unlike *p. Also, removing all operators would greatly sim
You are looking for PageHeap (Score:5, Informative)
For more information look here:
http://support.microsoft.com/kb/286470 [microsoft.com]
Valgrind (Score:5, Informative)
Re: (Score:3, Informative)
http://www.ibm.com/developerworks/java/library/j-
Re: (Score:3, Interesting)
http://www.idiom.com/~zilla/Computer/javaCbenchmar k.html [idiom.com]
Of course you can create your own "dynamic" allocator in C++... but that doesn't make allocation/deallocation automatically faster.
Maybe you should search for some numbers to support your argument... just by stating "I don't buy it, C++ is still faster" it doesn't become reality.
Re: (Score:3, Interesting)
Speed isn't everything. If your server application is network or database bound then stability and API richness is considerably more important than speed of execution. After all, does it matter that your app completes 2ms faster if it has to wait 500ms for the database to return? Or that it crashes more frequently? Or that you can't
A second vote for Valgrind (Score:3, Insightful)
Re: (Score:3, Informative)
It's been a few years since I last programmed in C++, but Valgrind indeed really saved the day on a regular basis. Also look into (KDE-)frontends if you think looking at text output is not very convenient. Couldn't agree more with this part, it's the best tool I've seen - and it's free software, too.
Re:Valgrind (Score:4, Informative)
Parent
eFence (Score:4, Interesting)
Unfortunately, what you prolly want is valgrind or purify.
Visual Leak Detector (Score:3, Interesting)
In any case, Visual Leak Detector is a free memory checking tool. It's only for Windows / Visual Studio, but if you are using that, VLD is awesome: http://www.codeproject.com/tools/visualleakdetect
It's super easy to set up, just #include "vld.h" somewhere in your program, and then run the debug mode. No need to rebuild everything in instrumented mode, and no false results (at least I haven't got any). Real memory leaks will be reported in the output window of the IDE.
Detailed article on memory usage (Score:4, Informative)
"Monitoring Your PC's Memory Usage For Game Development" [gamasutra.com]
While the title says it's for game development, I found that the meat of the article applies to any windows developer.
Don't allocate or free = no leaks = need no tools (Score:5, Insightful)
We pre-allocate pools of objects at startup and then re-use them. No other memory is allocated or freed while the process is running. Our pools of reusable objects are monitored very carefully as an object that isn't release back to its pool when the job is done is akin to a memory leak. Use of sentries to automatically release objects back to the pools when they fall out of scope is mandatory.
So my answer is to the problem is:
1. Use sentries (or some other mechanism) to guarantee memory is released.
2. Don't allocate except at startup.
3. No need for elaborate tools due to the above.
I'm sure that not all applications data usage would fit into this model, but it is surprising how many can.
We have seen some leaks in our applications. These were tracked down to STL internally leaking. They weren't generally very large and therefore we continue to live with them.
On the subject of garbage collectors, some of our colleagues use Java and
So don't think that a garbage collector is the solution. Perhaps in less demanding applications it is a potential answer.
Lastly, I strongly dislike anything from Rational. I find them overpriced unreliable bloatware (YMMV). Purify used to be good some time ago, but those days are long gone.
I echo what others have said above. You are a developer. You know your requirements. Build a simple tool to monitor and check your usage. For us it was managed pools of re-usable objects.
Memory Validator (Score:4, Informative)
Use a testing framework (Score:4, Informative)
Home brew tool for memory leaks with glibc (Score:4, Insightful)
One of our guys coded up a simple shared lib that can be loaded with LD_PRELOAD that sets simple hooks of printing memory locations for new/realloc/delete. He then wrote a perl script that kept track of these things and spit out anything that was malloc'ed and not realloc'ed or free'd.
I can't post it, because technically it's not my code it's my company's. But his shared lib code is just 300 lines long, and shouldn't be hard to duplicate. The perl log filter is even more straighforward. Each malloc gets saved. Each free removes the malloc. Each realloc removes the old malloc and adds a new one. Anything left over is a leak.
Override __malloc_initialize_hook with a pointer to your init_function. In your init_function, save the old functions at __malloc_hook __free_hook __memalign_hook and __realloc_hook and substitute your own. Now write your replacement functions, in it, do your logging and temporarily replce the old hooks and call the original functions, replace with your hook on the way out to get the next call. All of the hooks should be wrapped in a mutex to help re-entrancy problems.
It's not a full memory detector, just does leaks, but it's non-intrusive, requires no recompiles, and is the best way we have to leak detect our huge server long running code.
Re:Duh! (Score:5, Funny)
Parent
Re:I've used... (Score:5, Informative)
It is a very handy feature for finding leaks, buffer overflows, ect. The only other product I've used to find memory problems on recent incarnations of Windows is Purify. The MS solution is infinitely simpler because it's built into the environment and it's narrowly focused on memory problems. To state the obvious and in fairness to Purify, MSDev is infuriatingly fussy when it comes to building debug modules for IBM's Purify.
Parent
Re:I've used... (Score:5, Funny)
Memory checkers? GARBAGE COLLECTORS?! Have you no HONOR?!
We are KLINGON. We need no checkers. We need no garbage collectors. We need none of these weak HUMAN facilities. We write our CODE from the COMMAND LINE:
> COPY CON DECAPITATOR.EXE
[Alt+077] [ALT+090] . . .
Parent
I must agree - Purify is it. (Score:3, Informative)
Yes, it isn't cheap. Just do it. You'll thank me.
The productivity increase alone will make it worthwhile for management.
Re: (Score:3, Interesting)