Slashdot Log In
Firefox Memory Leak is a Feature
Posted by
Zonk
on Tue Feb 14, 2006 05:42 PM
from the that's-what-they-all-say dept.
from the that's-what-they-all-say dept.
SenseOfHumor writes "The Firefox memory leak is not a bug. It's a feature! The 'feature' is how the pages are cached in a tabbed environment." From the article: "To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited less than 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Total cached page limit. (Score:5, Insightful)
Whoops!
Re:Total cached page limit. (Score:5, Insightful)
Parent
Re:Total cached page limit. (Score:5, Informative)
2. It does (try opening a huge Fark photohop thread, huge as in multiple hundreds of pictures, see Firefox ramp up to 600 or 700Mb ram consumption, close the fark tab, see firefox' ram usage drop dramatically to regular ram usage levels)
Parent
Re:Total cached page limit. (Score:5, Informative)
Parent
Re:Total cached page limit. (Score:5, Informative)
Parent
Re:Total cached page limit. (Score:5, Informative)
See here's what happens:
Firefox allocates memory for a rendered page. You've got 20MB allocated already, all in 3 chunks. None have enough room for the single large allocation it needs so the OS sets aside a new chunk of memory for the firefox process.
Now it's using say 28MB of memory. And only 22MB of that is used. Well, it does a couple more allocations, some fairly permanent ones, and these get put in the newest block of memory.
Then you close the tab. Firefox frees the associated memory. The OS changes it tables around for that block to indicate so. But it still has some stuff in that block. So guess what? Firefox' memory usage remains exactly the same.
The solution? Use a GC system. Some Garbage Collectors (most) actually move objects to condense them in memory. This is one of the things that makes garbage collections noticeable if a lot has happened since the last one (it's gotta move a lot of RAM and change a bunch of links to said RAM). It becomes especially bad when you move into swap space
The downside? While GC advocates will often amaze you with the fact that malloc is not an atomic operation (it has a lot of work to allocate, more or less depending on the current situation of your memory chunks and the free memory on the system), malloc is still not nearly as costly as a garbage collection cycle. And, free is atomic (at least, TMK all a good implementation does is remove something from a data structure, unless it's the last part in which case it also needs to mark that memory as free).
So, you see, no matter how few memory leaks firefox has, it still won't drop in RAM usage every time you click close.
If you want to prove memory leaks in firefox you can. Get yourself a memory debugger (such as valgrind) and run firefox under it. Now, I'll warn you that this is harder than it sounds:
1.) Memory debuggers are about 100x to 1000x slower than your machine natively.
2.) Firefox is a script, not a binary, it sets up a bunch of stuff for the binary to run.
3.) Everything you see on the memory debugger is not necessarily a leak. Some of the leaks aren't even really leaks (it's generally no big deal to leak when you're exitting because the kernel cleans that up for you).
4.) To get any useful information on the leaks (other than size) you'll need to have compiled with debug symbols and you'll need to have the source code.
Go ahead, post your list of firefox memory leaks. Then post your list of IE memory leaks. I bet both have some, but neither has anything major. And I bet it takes you a week to find them
Parent
Re:Total cached page limit. (Score:5, Interesting)
People have written garbage collectors for C++, and they work just fine. But they do not help with fragmentation, which is the problem you're describing. That requires a heap-compacting allocator (aka a "handle" allocator). Many languages with garbage collection also use a heap-compacting allocator. C++ does not, because of a low-level language "feature": pointers, and specifically, pointer arithmetic.
If an object moves in memory, then people have to be notified that it has moved, or they won't know where to access it. Languages like Java handle this behind-the-scenes; the system library tracks objects for you, and your program never knows (or cares) whre an individual object is.
C++ allows direct access to system memory, and it tells you precisely where your objects are located. Programmers are then free to do all kinds of things like compute distance to other objects, or convert the location to a number and do arbitrary math operations on it.
When an object moves, anything that refers to it needs to be updated. Well, good luck figuring that out in a language with pointer arithmetic! The system would need to magically determine whether or not a numeric value was actually a memory locations. And what if a program computed the distance between two objects, and later on used that distance to get from one object to the other? The system has no idea of what can be safely moved, and what has to stay put. So nothing can ever be moved.
There are workarounds of course -- if you write a program with heap-compaction in mind, then you can use a "handle" system, where every object has an ID. You remember the ID, and to access the object, you ask the system for a temporary memory location. And as soon as you're done, you "forget" the memory location and let the system shuffle things around in memory. The next time you give that ID to the system, you might get back a different memory location, but you were already expecting that so your program doesn't mind.
But handle allocation is slower, less efficient, and more annoying to use than a traditional fixed-location allocator. You have to start your project with it in mind; retrofitting existing code to use a handle allocator is a giant timesink and prone to conversion errors. And if you don't mind the loss of performance due to using a handle allocator, why are you using C++ in the first place?
Parent
Re:Total cached page limit. (Score:5, Informative)
Parent
Re:Total cached page limit. (Score:3, Interesting)
I browse with a lot of tabs in FireFox, and with FireFox 1.5 the performance when a lot of those tabs are loading has been beyond horrible. Like several seconds just to switch tabs, and then actually trying to scroll...
If you are feeling generous, perhaps you also know how to shutoff the new tab thumbnail "feature" when you've got images. 16x16 thumbnails of 4000x4000 images are nothing but a waste of CPU time and a vi
Re:Total cached page limit. (Score:5, Informative)
Specify the memory cache usage
Normally, Firefox determines the memory cache usage dynamically based on the amount of available memory. To specify a specific amount of memory cache, add the following code to your user.js file:
user_pref("browser.cache.memory.capacity", 4096);
To disable the memory cache completely, add the following code:
user_pref("browser.cache.memory.enable", false);
Parent
What a small world (Score:5, Funny)
My boss doesn't agree.. (Score:5, Funny)
In other news... (Score:5, Funny)
And in totally unrelated news, the Mozilla foundation recently announced that their flagship browser Firefox shall soon be renamed to Bigfoot, to reflect the software's large memory footprint.
More breaking news on these topics at 11.
Doesn't seem to be true (Score:4, Insightful)
releasing memory (Score:5, Interesting)
The heap, where dynamic allocations occur, is only allowed to grow or to be truncated. An application cannot release memory in the middle of the heap without also releasing the memory at the end of the heap.
So let's say Firefox makes 10 one-page allocations, and frees the first 9. The memory layout might look something like:
XXXXXXXXXU (X- unused, U- used)
Those 9 pages worth of memory aren't being used, but it's impossible to release them back to the OS.
Thankfully, there is some good news: when Firefox needs to allocate more memory, it can and will just reuse those 9 unused pages instead of allocating more memory from the OS and growing the heap.
The best solution to this problem is to use a compacting garbage collector. Which is something that Java and C# and other higher-level langauges can easily make use of (and many do use them), but which C and C++ can't really make use of given the complete lack of compiler support. That's one reason why a Java or C# app can actually out-perform a similar C/C++ app, especially with a good native-code compiler and an library implementation with a modern GC.
Parent
Re:releasing memory (Score:3, Insightful)
But you pointed out the flaw in the wording of the article - this IS NOT a memory leak, just inefficient use of the heap.
I thought the definition of a memory leak was an application that kept allocating memory from the OS as it ran, not an application that asked for a chunk of memory and just reused it inefficiently?
(If I'm wrong, someone please correct me).
Re:releasing memory (Score:5, Funny)
Maybe in theory, but in practice 99.9% of the world uses C++ browsers because the Java ones suck.
Parent
Re:releasing memory (Score:3, Interesting)
mmap() can do this, but on many systems [s]brk() cannot. brk() is also alot faster than mmap().
This is really moot on most systems; don't do a lot of little allocations that you're going to keep around for a while and DO use pooled al
Re:releasing memory (Score:5, Informative)
I wouldn't know about C, but this statement is utterly false as applied to C++.
No, its not false, its true.
Replacing the default new and delete routines is perhaps not for the inexperienced C++ programmer, but to say that there's an complete lack of compiler support is simply wrong.
And? What do you mean with compiler support for heap compacting (or GC)?
Q: What has replacing new and delete with your own implementations to do with garbage collection?
A: Nothing
Q: How would new and delete of class A be able to compact a heap by moving allocated instances of class B down?
A: Difficult!
Q: So if you now add a class C you like to rewrite A::new and B::delete to also cope with class C instances?
A: I assume you understand that EVERY delete of EVERY class needs to know EVERY other class to be able to compact the heap, yes?
It is true that out-of-the-box C++ does not have a compacting garbage collecter, but one can certainly be written (and used, of course) with any conformant compiler.
Indeed, but not by merly only by replacing operator new and delete.
Existing C++ garbage collectors are very limited to more or less conservative garbage collecting. See e.g. Boehms c++ / C garbage collector.
And, the mere point of garbage collecting if you want to start nitpicking is: you don't ever call delete.
angel'o'sphere
Parent
Re:releasing memory (Score:5, Insightful)
Only on a computer without virtual memory. In a PC (which *has* virtual memory), you just punch holes in the memory.
What happens is a process gets an "address space", into which pointers can point, but any given address may not map onto some real storage. The process asks the operating system to map a range of addresses onto real storage which the operating system will try to map to real fast memory when it thinks it will be used at any moment. When the OS figures the memory wont be needed for a while, and something else needs some memory, the OS copies the data to disk and redirects the mapping to a proxy that will pull the data back into memory when the process tries to use it again.
When a process knows that it won't need a section of that real storage, it can tell the operating system to unmap it from the address space.
There are various other things that go on, but that's the simple story. From a figure posted in an earlier message, it seems that opera does pretty damned well (in comparison to most modern programs) with just the simple story, not having to rely much on nasty unreliable heuristics. Of that I am impressed.
Parent
Re:releasing memory (Score:5, Informative)
I wouldn't recommend mixing anything resembling C pointer maths with compaction, since its incredibly difficult to tell what's a pointer and what isn't (in fact, without modifying the compiler, it can't be done in C or C++). For this reason, the Boehm collector (a collector that replaces new and delete) goes for the Mark-and-Sweep method instead of compaction. Because you dont move objects, you don't have to worry about figuring pointers. Boehm's collector is also called conservative, not only because it doesn't modify live objects, but also in that it treats any data on the stack or in the heap as a potential pointer. If the data points inside the heap, the object containing that address is marked. This can lead to false positives on occasion, but there's no helping that without any support from the compiler (again contradicting the grandparent). The good news is that a false positive isn't going to cause direct harm in mark and sweep. All that happens is that space that could be used isn't; Boehm claims this is irrelevant in today's operating systems with virtual memory, although I doubt you'd see an entire page's worth of false positives. Certainly, I can't do any better than him.
In language R&D labs where people are paid quite well to think hard and long about things, they tend to use both approaches in what's called a "generational" collector. Young objects can be copied or collected as needed, while older objects are mark/swept away as needed. This works because old objects much more likely to stay than new ones. Last I knew, both Java and C# use generational techniques, because it makes sense in most nearly every case. However, as I described above, C++ doesn't have that, and even those libraries that replace new and delete have conventions and costs associated with it. I certainly wouldn't try to take Boehm and pidgeonhole it into Mozilla. And even if you did, it still wouldn't solve the compaction problem. All you can do is hope the virtual memory manager is doing it's job well. Even though the application and garbage collector is more likely to know what's useful than the VM manager.
Parent
about:config (Score:3, Insightful)
So I'll be the first to say it.... (Score:5, Insightful)
Re:So I'll be the first to say it.... (Score:4, Informative)
Parent
Re:So I'll be the first to say it.... (Score:3, Interesting)
This is proof positive, I think, that OSS != the best option in all scenarios. Opera consistently beats FF out on features, secur
Re:So I'll be the first to say it.... (Score:4, Interesting)
Well, you just picked up the worst reason - opera is great when it comes to performance, but firefox + extensions consistently beat opera and IE when it comes to features. I can have the features I want, there're way more extensions that features than opera has, and if I don't want them, I don't need to keep the extra UI involved in those features.
Parent
Quick Fix (Score:4, Informative)
NOT per tab (Score:5, Informative)
Ben was mistaken, it's cached globally.
See this comment [mozillazine.org] by Boriz Zbarsky:
and this comment [mozillazine.org] by David Baron:
(Boris and David are back-end developers; they have much more working knowledge of this than Ben does.)
Also, there are actual memory leaks in Firefox. See this weblog post [squarefree.com] about progress on that. However, as that weblog post says as well, most excessive memory usage that people are seeing is entirely due to faulty extensions.
What I'd like Mozilla devs to do (Score:5, Insightful)
Before someone jumps at my throat, it's just a description what I'd like to see, but of course its all up to the developers, they decide what to code and do with their time. It is just simple user feedback.
Firefox is the most unstable program in common use (Score:5, Informative)
The Firefox CPU hogging bug makes a computer unusable until all Firefox windows and tabs are closed. Basically, Firefox uses first maybe 10%, then maybe 20% of the CPU, and, as Firefox windows and tabs are opened and closed, continues taking more of the CPU time until Firefox is closed. This CPU usage is with NO Firefox activity, or any activity of any program.
This bug is more than 3 years old. It is extremely difficult to characterize; no one has succeeded yet. Here are some clues:
Somehow Thunderbird and Mozilla share this bug. Sometimes when Firefox is taking say, 94% of the CPU, and Firefox is closed completely, Thunderbird or Mozilla will begin using a lot of CPU time. Very weird, but it often happens.
Firefox 1.5.0.1 is much worse than 1.5, which is worse than earlier versions. This suggests that there is some resource in Firefox that is being more overused as features are added.
The CPU hogging bug continues unchanged when Firefox 1.5.0.1 is installed with a clean profile and no extensions.
Too many mouse clicks too closely spaced will often increase Firefox's CPU usage, or sometimes cause it to crash.
--
Before, Saddam got Iraq oil profits & paid part to kill Iraqis. Now a few Americans share Iraq oil profits, & U.S. citizens pay to kill Iraqis. Improvement?
Why does Opera work well, and not Firefox? (Score:5, Informative)
- Opera has none of these problems. So, the quote from the Mozillazine blog
shown below, although it is typical, is not supported by the
facts.
- Whatever causes the CPU hogging bug is definitely associated with extreme
memory use. No doubt there are leaks, but this is not a leak, since it is not
necessarily associated with greater use of Firefox.
- Users often report that just leaving Firefox open overnight causes CPU
hogging and extreme memory use.
- The problems are the same in Mozilla browser.
- It's good to test Firefox with a laptop in a quiet environment. When you
hear the laptop fan begin to run while there is no activity, you know Firefox
has begun to suck CPU cycles.
- Putting a computer into standby or hibernation often makes the CPU hogging
bug much worse. That's why Firefox users sometimes just leave their computers
on.
- When a computer takes a long, long time to start from standby, you know Firefox
is taking CPU cycles. What about coming out of standby makes Firefox unstable? No
other program has that problem.
Quote from the blog linked in this Slashdot story About the Firefox "memory leak" [mozillazine.org]: "A lot of people complain about the Firefox "memory leak(s)". All versions of Firefox no doubt leak memory - it is a common problem with software this complicated."No other program in common use is so buggy. The problems in Firefox are not "common".
Another quote from the linked Mozillazine blog: "What I think many people are talking about however with Firefox 1.5 is not really a memory leak at all. It is in fact a feature."
That's not what the technical magazines [cmp.com], newsletters [scotsnewsletter.com], web sites entirely devoted to Firefox problems [slyerfox.com], and even the mainstream media say. They say it is a serious problem.
Mozilla developers have been denying that there is a serious problem for more than 3 years. It seems that it would be less work to fix the problem than to undertake a cottage industry of trying to convince people they aren't having problems. Mozilla developers have been impeding characterization by marking Bugzilla bug reports of these problems invalid.
However, it is clear that it would take a serious scientific investigation; this is not an easy bug to characterize.
Parent
Re:Firefox is the most unstable program in common (Score:3, Interesting)
seriously, the only thing i could think of is that if firefox ran out of ram and had to start using the pagefile, that would eat up tonnes of CPU. This would also effect other programs on the system. Are you sure you have enough ram in the machine? I assume you can replicate this bug on more than one machine right?
Ive had some sites crash firefox repeatidly but i cant think of any examples off hand.
seems snappier (Score:3, Informative)
On a side note, if anyone is like me and looks in about:config for browser.sessionhistory.max_total_viewers and doesn't see it, you have to actually add the line. Right click and choose "new" then type in "browser.sessionhistory.max_total_viewers" and then 0 (or whatever you like).
Huh? (Score:5, Insightful)
Don't bug me (Score:5, Interesting)
What is being described here sounds much more like a cache of recent pages, which in my opinion is perfectly sane for a browser. Sure, maybe the cache is a bit overzealous, but even if that's the case, just disable it - worse case scenario, you edit the source. But otherwise, this is definitely a feature - I can promise you it's much more programming effort to save old pages for a quick redraw than to free the old page and replace it with the new.
So I guess the discussion here is, "is it right for firefox to use so much memory?" My answer is yes. It is not a memory leak, it seems like a very valid design decision. But if you disagree, old versions of firefox still work great (I still haven't upgraded myself).
There is another serious Firefox bug as well (Score:5, Informative)
Firefox crashes when two browser windows are making synchronous XMLHttpRequests. I have experienced this under Linux - I have no idea whether it is the same under Windows. Basically under Lunux all Firefox windows are running in the same thread utilizing a scheme of cooperative multitasking.
So far so good. The bug appears when two separate Firefox windows are making periodic synchronous XMLHttpRequest-s. When such a potentially lengthy task has to be executed synchronously, Firefox creates a new "nested" event queue. If two (or more) browser windows are doing it at the same time, new event queues are created all the time and eventually (within 5 minutes) the application core-dumps.
I found this by recompiliging Firefox with debug information and debugging it. Even if my interpretation of what happens is not completely correct, the fact remains - a simple JavaScript can crash Firefox causing all open browse windows to be closed.
The solution is to always use asynchronous XMLHttpRequest (which is a better practice anyway) and to hope that the same problem doesn't appear in other places. Still, it is troublesome.
Firefox developers don't "get it" (Score:5, Insightful)
With the number of people complaining about this (and the number of people that don't even KNOW to complain) isn't it a safe bet that you've made a mistake in the amount of cached pages?
Re:Firefox developers don't "get it" (Score:4, Insightful)
Parent
R-RTFA (Score:4, Informative)
http://lxr.mozilla.org/seamonkey/source/docshell/
If you're unhappy with the memory usage with 50 tabs open, I advise the following workaround:
DON'T DO THAT.
Rewritting history. (Score:5, Informative)
The only problem is there were bugs filed for memory leaks long before Firefox 1.5 and the Back-Forward cache were implemented. Maybe this feature does contribute to Firefox's large memory footprint, but to say that this feature is the only reason and that there are no leaks is simply false.
Re:My pet peeve! (Score:3, Informative)
Tab Mix Plus also has an option to always open the current page in a new tab.
Re:My pet peeve! (Score:3, Informative)
Re:My pet peeve! (Score:4, Informative)
Usually the only time I use a browser under Windows is for Windows Update.
And just testing right now, middle-clicking on the Back button does nothing for me under Linux. It has a visual reaction but otherwise does nothing else. Maybe it is another one of those Firefox features not found in Mozilla?
Parent
Re:My pet peeve! (Score:3, Interesting)
Re:My pet peeve! (Score:4, Informative)
https://addons.mozilla.org/extensions/moreinfo.ph
Parent
Re:My pet peeve! (Score:5, Informative)
Parent
Clone Window (Score:3, Informative)
Look it up, dingus. There's no reason that every web browser should behave exactly like IE out of the box. That's what the extension feature is for. =)
Re:spyware with IE or memory leak with FF.. (Score:5, Funny)
i could never stand behind a company like that and refuse to use opera products untill he makes good on his word. You cant just throw statements like that around. Browsers designed by liars are dishonorable browsers.
Parent
Re:sounds good to me (Score:3, Interesting)
You've totally missed the point. People aren't bitching because the back and forward buttons are faster. They're bitching because the memory used for the fast back/forward is never released. Because Opera implements the sa
Re:bfd? (Score:5, Insightful)
Virtual memory is not a carte blanche for memory hogging. As you should know, memory hogging will result in degraded performance.
Assuming that users have unlimited resources is exactly how Mozilla is barely usable on Windows 95-ME - especially when you have Slashdot Moderator access.
In an ideal situation, that would be correct.
However, the operating system does not know which memory is currenly "in use" and which ones are "in cache" - in fact, it's quite easy for an "in use" to be physically sandwiched between two "in cache" entries. Because of this, you will have a sudden loading time if you do plenty of other tasks in the background and suddenly switch back to Mozilla.
Small applications, being small, do not generally have to wait 1/2 seconds to recover from being pages in or out. Since Mozilla allocates the cache in memory, it will have to wait those two seconds.
An OS with decent vmem support would allow you to map files to memory. This results in no swapping at all - only writing perodic output to the hard drive, and loading the file into memory as required. If another application needs more memory, the memory map is discarded with no need to write the contents of memory.
An application that doesn't exploit the usage of memory maps is as good as an OS with shoddy vmem support. (Of course, it can simply use it's disk cache for the same effect.)
Parent