Slashdot Log In
C# Memory Leak Torpedoed Princeton's DARPA Chances
Posted by
Zonk
on Sat Nov 17, 2007 06:22 AM
from the omg-a-tumbleweed dept.
from the omg-a-tumbleweed dept.
nil0lab writes "In a case of 20/20 hindsight, Princeton DARPA Grand Challenge team member Bryan Cattle
reflects on how their code failed to forget obstacles it had passed. It was written in Microsoft's C#, which isn't supposed to let you have memory leaks. 'We kept noticing that the computer would begin to bog down after extended periods of driving. This problem was pernicious because it only showed up after 40 minutes to an hour of driving around and collecting obstacles. The computer performance would just gradually slow down until the car just simply stopped responding, usually with the gas pedal down, and would just drive off into the bush until we pulled the plug. We looked through the code on paper, literally line by line, and just couldn't for the life of us imagine what the problem was.'"
Related Stories
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Well, there's your problem! (Score:5, Funny)
Re:Well, there's your problem! (Score:5, Funny)
Parent
Re:Well, there's your problem! (Score:5, Funny)
Parent
Re:Well, there's your problem! (Score:5, Informative)
there are very clear constructs in place in the language/runtime to allow any object to unregister itself from event registrations it initiated.
this was VERY MUCH a bug in the end-user software, not the runtime (i've written code almost IDENTICALLY to this and blew lots of time having made this same mistake).
the only thing the runtime could do to protect the idiot developer (myself included) is automagically make all event references WEAK references, but that has plenty of undesirable side-effects too... in clr, you can do this yourself if you're so inclined... (just like in a JVM)
cheers.
Peter
Parent
Re:Well, there's your problem! (Score:5, Informative)
Parent
Slashvertisement (Score:5, Informative)
Re:Slashvertisement (Score:5, Interesting)
Parent
Re:Slashvertisement (Score:5, Insightful)
But I do believe that articles written by companies pretending to be written by end-users are not terribly useful and probably shouldn't end up on
I mean, the article clearly states at the top "By Red Gate Software.".
So where did the "Bryan Cattle reflects on
Seriously.
"One of our team members downloaded the 14-day trial of ANTS Profiler"
"To our amazement, it was only minutes before we realized that our list of detected obstacles was never getting garbage collected"
"If Only We Had Used It Earlier..."
ANTS Profiler helped us fix a problem in minutes that would have taken us weeks to track down. If only we'd thought of it before the competition, we would most likely have finished the entire race and had a chance at the top prize money.
All this stuff sounds either very naive or very marketing. You choose.
Parent
Re:Slashvertisement (Score:5, Insightful)
Some old Visual Basic programmer jokes come to mind when I read this article. People use to make fun of Visual Basic programmers because it was to easy to write programs in VB. They thought it would produce sloppy code with errors, and other similar things. To some extent, it appears that same case could be made for C#, and to a lesser extent Java. In the end this is simply a case of not stress testing their event stack.
Parent
Re:Slashvertisement (Score:5, Insightful)
Agreed. I was a good VB programmer. But my VB experience was an eight-month interval between C++ jobs. I've knocked out minor MS Office applications when needed since then, but that's it. I'm sorry to say that most of the VB programmers I've worked with were very poor engineers. Admittedly I've not worked in VB for wealthy companies whilst I have in C++, so that colours things somewhat. I don't doubt that there are some good VB programmers out there. But in the cases of most of those I worked with, I could very clearly see how VB led them to be poor engineers. The amount of shortcuts and wizards and instances where they would start their program by dragging a form object onto the design panel and dumping form objects onto it was obviously a leading reason for their poor skills. VB *led* them to take this approach. It works for small Excel apps, barely for database front-ends and not well at all for large projects.
As is common with the lower end of Microsoft products, the selling point is that they make it very easy to do what they think you want to do. The ability of VB to knock out an interactive form with near-zero knowledge of programming has encouraged a lot of colleges to sell people the idea that a ten-week course of dragging and dropping text box objects is programming whilst a lot of cheap or ignorant employers have taken the graduates at their word and plunged them in over their heads.
VB is a poor language in many ways and not, imo, suited to a large or sophisticated project. But you can find good VB programmers (was one). It's just that it encourages bad ones.
Oh, the summary is also wrong. C# hasn't started springing leaks. The programmers missed a reference to objects that they were creating and the garbage collection therefore never triggered to unallocate the memory. I don't doubt it's not easy to automate a vehicle to drive any even 9 miles, but this could have been detected with more thorough debugging. At any rate, the article submitter and overseeing editor should be ashamed of twisting this into an anti-Microsoft jab. I'm a Linux programmer. I can tell you that Linux can compete happily without sinking to the level of lies and misinformation.
Parent
Stupid Slashdot headline (Score:5, Interesting)
It's not C#'s fault. The team had references to the obstacle list (event handlers), which prevented garbage collection. The
Re:Stupid Slashdot headline (Score:5, Insightful)
Decent programmers might understand that, but let's be honest, it's not like Java (and other GC languages) haven't been presented as if memory leaks were a thing of the past.
As a matter of fact, some people will probably still claim that it's technically not a memory leak, but instead an object life-span issue.
What surprises me is that outspoken proponents of managed languages use the garbage collection so often as a good thing, as if now you can be a sloppier programmer and get away with it.
In reality you have to identify/control the lifespan of objects anyway, so I personally never understood what the big deal is about freeing memory manually. Not to mention that memory leaks in say, C++ code, really aren't that hard to find. The tools have become pretty freakin decent.
And also not to mention that garbage collection might be handy for memory, but memory is only one of a plethora of resources that can be leaked. And since for many resources it isn't nearly as appropriate to 'lazy' free them, as a programmer you still have to be aware of the allocate/free paradigm. (as just one silly example, it would suck if you wouldn't be able to explicitly close a file, because you can't delete it before it's closed)
In other words, you are right. Of course you can have memory leaks in garbage collected languages. And I wish people would stop using GC as an argument why languages as Java are so much better to use than C++.
Parent
Re:Stupid Slashdot headline (Score:5, Interesting)
Parent
Re:Stupid Slashdot headline (Score:5, Insightful)
I think you're getting hung up on the method name. There is no standard "delete" function that marks something as unused (dispose on the other hand sort of gets there). The article itself is unclear but I would assume that they were simply deleting the collision objects from a collection of potential hazards. Whilst that would remove the object from the collection itself it is *not* a delete. As references to the object existed elsewhere the object still exists (look ma, no null pointer exceptions) no delete happens. You cannot specifically say to the GC "We're done with this, delete it", the GC sweeps on a regular basis looking for objects with no references.
Would you really want the GC deciding that just because an object is no longer part of a collection it's safe to unsubscribe it from events and delete it? I know I wouldn't.
Parent
This is not a C# memory leak! (Score:5, Informative)
Though we thought we had cleared all references to old entries in the list, because the objects were still registered as subscribers to an event, they were never getting deleted.
So references were held to the objects in two places - the list of encountered obstacles, and the list of event subscribers. They were being removed from the list of encountered obstacles, but not being unsubscribed from the event.
How do you think event subscription works? Something has to hold a reference to the objects that are subscribed to the event! That thing is going to hold a reference until you unsubscribe the object - it neither knows nor cares about any other list of references you may be maintaining separately, how could it?
This is a coding error. A subtle, non-obvious one perhaps, but a bug nevertheless. It is not an error in the CLR, and in fact the article never paints it as such. That particular bit of spin is wholly down to the submitter.
Re:Hard/weak references for event handlers (Score:5, Interesting)
The poster of the article was trolling, and not only trolled with the post, managed to get a troll posted to a slashvertisement which was not even trolling.
Impressive on the part of the person who submitted it, but disappointing considering Taco's comments a few weeks back about articles that are truly nothing but advertisements.
Parent
don't these kids learn anything anymore? (Score:5, Interesting)
(2) You are particularly supposed to test your software if you send $200k and 1 ton of hardware careening through the street on autonomous real-time control.
(3) Garbage collectors do not prevent memory leaks.
(4) Garbage collected systems can be good for building real-time systems, but you need a real-time garbage collector or you need to treat the system as if it didn't have a garbage collector at all.
What "ruined their chances" was not that they overlooked a memory leak, what ruined their chances was that they didn't know what they were doing.
Re:I'll show you mine if you.. (Score:5, Insightful)
This just tells us once again that our wonderful editors on /. don't even try to understand what's behind an article, but they just find some sensationalistic title (the more AntiMS, the better) and done. This results in more comments of the type... "See, M$ id teh SuCkS", or "thanks god for my Linuzzz."..., so they got more profit for their /. ads (oh, the irony often MS ads, BTW).
Yellow press..... yes, I know, /. is not supposed to have any credibility like any other parasite news sites, but anyway....
Parent
Re:I'll show you mine if you.. (Score:5, Insightful)
However I take exception to your use of the world "Editor". Slashdot does not have Editors. They have guys who accept submissions.
They don't read The Fucking Articles, They don't check links, The don't edit submissions...
Parent
Re:I'll show you mine if you.. (Score:5, Informative)
Parent
Perpetual motion machine vendors (Score:5, Funny)
I was working at a coal-fired power plant which needed a new pollution control device before 2010. There, I would dig through the literature, and try to find suitable products and operating conditions for this device. Anyway, this involved a lot of meetings, conference calls, and business lunches with the suppliers in question.
Then there was Joe.
Joe was our Alstom sales rep: portly, humorless, slow to speak and slower to understand. He was also a devote Utahnian.
Well, one day, we were killing time while waiting on a conference call, my supervisor left the room, and we started talking about universities. Then he dropped the bomb:
"In my Senior year, I worked on developing perpetual motion machines."
My supervisor then reentered the room, and we got back to work. I felt like I'd just seen a dancing frog.
Parent
Re:Friends do not let Friends use Windows and Driv (Score:5, Funny)
Parent
Re:As a C kernel programmer... (Score:5, Funny)
Parent
Re:C# Garbage... (Score:5, Informative)
The main problem with garbage collectors (I like GCs, so this isn't a diatribe against them) is that far too many mediocre programmers assume they have a magical ability to know precisely what they want their code to do. The reality of course is that they use algorithms to decide what should be collected, when it should be collected, and how it should be collected, and those who are unfamiliar with the particular strategies that their GC uses can therefore not only write code with more than a few memory leaks, but also code that results in the GC being used so inefficiently that it does vastly more work than would be necessary if the same functionality was implemented in a slightly different way.
There are plenty of articles about Java memory leaks that can be found by Googling "java memory leaks". Googling "java GC tuning" will produce some useful links to articles containing tips on ensuring that it's not used inefficiently.
Parent
Re:Reference counting (Score:5, Informative)
Actually, C# doesn't reference count at all, it 'Reference Traces' :)
Please, let me explain; it's quite sad how often people don't get this ...
.Net has its block of managed memory, called the Managed heap. It's separated into 3 'generations'. This heap has 2 areas, free space and reserved space, from top to bottom.
When you allocate and object to the heap, by using the new command (object o = new object();) there is a set of rules? that have to be enforced:
The GC manages Reference tracing, and this doesn't occur when the object goes out of scope, it actually happens when the Heap is full and you attempt to allocate a new object.
In something called 'the sweep', the GC goes through each object in the heap to see if it's reachable. To do this it starts with so-called 'roots'. It then traces to see which objects are referenced by these roots.
A root identifys a storage location, which referes to objects on the managed heap, or objects that are set to null. For example, all of an applications global and static objects are considered to be it's roots. (hence the reason that all C# apps have a static void main).
When the sweep starts, it assumes that all objects are garbage. So for each root object, it builds up a graph of the objects that root references, and marks them as being live.
However, if it finds an object that's already in the graph, it stops traversing that path. This is two (massively) increase performance by not scanning the same object twice, and more importantly, it stops you getting into an infinite loop by scanning a circular list.
The pinch is, it prevent the circumstance that you mentioned! :)
Because the strong reference to a linked circular list is gone, the circular list isn't attached to a root object, so it gets disposed. If you don't want it to get dropped, unless it theres a memory shortage, the C# GC also supports something called Weak References, but I'm not going to go into those here as it's headhurting
So once all the roots have been checked and we've got a nice graph of all the objects that are referenced by the live parts of the application somehow, the second stage of GC happens.
Any objects that haven't been touched by the walk are of course still marked as Garbage. The GC now walks up the heap linearly, looking for contigious groups of garbage which are now considered to be free space. The GC looks for the next live object and moves it to the start of this free space with a good old memcpy :)
This ofcourse invalidates all the root pointers, so the GC then updates the points in the root objects.
:)
So now, we've got rid of all the garbage and our heap is pleasantly compacted; Take that Heap Fragmentation, Kerpow!!
But, that's not all she wrote of course
Now we're free'd and compacted, the 'nextObjPtr' is moved to the top of the heap. At this point the new object creation that triggered the collection is performed and the new object appears at the top of the heap.
This is a dramatic over-simplification and I've not attempted to explain finalization or weak references, but it's still good to know this stuff, it helps us as .Net programmers to consider how to write our code properly :)
The other thing I've not explained is how the Generations work:
Parent