Slashdot Log In
Firefox Memory Leak is a Feature
Posted by
Zonk
on Tue Feb 14, 2006 06:42 PM
from the that's-what-they-all-say dept.
from the that's-what-they-all-say dept.
SenseOfHumor writes "The Firefox memory leak is not a bug. It's a feature! The 'feature' is how the pages are cached in a tabbed environment." From the article: "To improve performance when navigating (studies show that 39% of all page navigations are renavigations to pages visited less than 10 pages ago, usually using the back button), Firefox 1.5 implements a Back-Forward cache that retains the rendered document for the last five session history entries for each tab. This is a lot of data. If you have a lot of tabs, Firefox's memory usage can climb dramatically. It's a trade-off. What you get out of it is faster performance as you navigate the web."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Total cached page limit. (Score:5, Insightful)
Whoops!
Re:Total cached page limit. (Score:5, Insightful)
Parent
Re:Total cached page limit. (Score:5, Informative)
2. It does (try opening a huge Fark photohop thread, huge as in multiple hundreds of pictures, see Firefox ramp up to 600 or 700Mb ram consumption, close the fark tab, see firefox' ram usage drop dramatically to regular ram usage levels)
Parent
Re:Total cached page limit. (Score:5, Informative)
See here's what happens:
Firefox allocates memory for a rendered page. You've got 20MB allocated already, all in 3 chunks. None have enough room for the single large allocation it needs so the OS sets aside a new chunk of memory for the firefox process.
Now it's using say 28MB of memory. And only 22MB of that is used. Well, it does a couple more allocations, some fairly permanent ones, and these get put in the newest block of memory.
Then you close the tab. Firefox frees the associated memory. The OS changes it tables around for that block to indicate so. But it still has some stuff in that block. So guess what? Firefox' memory usage remains exactly the same.
The solution? Use a GC system. Some Garbage Collectors (most) actually move objects to condense them in memory. This is one of the things that makes garbage collections noticeable if a lot has happened since the last one (it's gotta move a lot of RAM and change a bunch of links to said RAM). It becomes especially bad when you move into swap space
The downside? While GC advocates will often amaze you with the fact that malloc is not an atomic operation (it has a lot of work to allocate, more or less depending on the current situation of your memory chunks and the free memory on the system), malloc is still not nearly as costly as a garbage collection cycle. And, free is atomic (at least, TMK all a good implementation does is remove something from a data structure, unless it's the last part in which case it also needs to mark that memory as free).
So, you see, no matter how few memory leaks firefox has, it still won't drop in RAM usage every time you click close.
If you want to prove memory leaks in firefox you can. Get yourself a memory debugger (such as valgrind) and run firefox under it. Now, I'll warn you that this is harder than it sounds:
1.) Memory debuggers are about 100x to 1000x slower than your machine natively.
2.) Firefox is a script, not a binary, it sets up a bunch of stuff for the binary to run.
3.) Everything you see on the memory debugger is not necessarily a leak. Some of the leaks aren't even really leaks (it's generally no big deal to leak when you're exitting because the kernel cleans that up for you).
4.) To get any useful information on the leaks (other than size) you'll need to have compiled with debug symbols and you'll need to have the source code.
Go ahead, post your list of firefox memory leaks. Then post your list of IE memory leaks. I bet both have some, but neither has anything major. And I bet it takes you a week to find them
Parent
Re:Total cached page limit. (Score:5, Informative)
Parent
Re:Total cached page limit. (Score:5, Informative)
Specify the memory cache usage
Normally, Firefox determines the memory cache usage dynamically based on the amount of available memory. To specify a specific amount of memory cache, add the following code to your user.js file:
user_pref("browser.cache.memory.capacity", 4096);
To disable the memory cache completely, add the following code:
user_pref("browser.cache.memory.enable", false);
Parent
What a small world (Score:5, Funny)
My boss doesn't agree.. (Score:5, Funny)
In other news... (Score:5, Funny)
And in totally unrelated news, the Mozilla foundation recently announced that their flagship browser Firefox shall soon be renamed to Bigfoot, to reflect the software's large memory footprint.
More breaking news on these topics at 11.
So I'll be the first to say it.... (Score:5, Insightful)
NOT per tab (Score:5, Informative)
Ben was mistaken, it's cached globally.
See this comment [mozillazine.org] by Boriz Zbarsky:
and this comment [mozillazine.org] by David Baron:
(Boris and David are back-end developers; they have much more working knowledge of this than Ben does.)
Also, there are actual memory leaks in Firefox. See this weblog post [squarefree.com] about progress on that. However, as that weblog post says as well, most excessive memory usage that people are seeing is entirely due to faulty extensions.
What I'd like Mozilla devs to do (Score:5, Insightful)
Before someone jumps at my throat, it's just a description what I'd like to see, but of course its all up to the developers, they decide what to code and do with their time. It is just simple user feedback.
Firefox is the most unstable program in common use (Score:5, Informative)
The Firefox CPU hogging bug makes a computer unusable until all Firefox windows and tabs are closed. Basically, Firefox uses first maybe 10%, then maybe 20% of the CPU, and, as Firefox windows and tabs are opened and closed, continues taking more of the CPU time until Firefox is closed. This CPU usage is with NO Firefox activity, or any activity of any program.
This bug is more than 3 years old. It is extremely difficult to characterize; no one has succeeded yet. Here are some clues:
Somehow Thunderbird and Mozilla share this bug. Sometimes when Firefox is taking say, 94% of the CPU, and Firefox is closed completely, Thunderbird or Mozilla will begin using a lot of CPU time. Very weird, but it often happens.
Firefox 1.5.0.1 is much worse than 1.5, which is worse than earlier versions. This suggests that there is some resource in Firefox that is being more overused as features are added.
The CPU hogging bug continues unchanged when Firefox 1.5.0.1 is installed with a clean profile and no extensions.
Too many mouse clicks too closely spaced will often increase Firefox's CPU usage, or sometimes cause it to crash.
--
Before, Saddam got Iraq oil profits & paid part to kill Iraqis. Now a few Americans share Iraq oil profits, & U.S. citizens pay to kill Iraqis. Improvement?
Why does Opera work well, and not Firefox? (Score:5, Informative)
- Opera has none of these problems. So, the quote from the Mozillazine blog
shown below, although it is typical, is not supported by the
facts.
- Whatever causes the CPU hogging bug is definitely associated with extreme
memory use. No doubt there are leaks, but this is not a leak, since it is not
necessarily associated with greater use of Firefox.
- Users often report that just leaving Firefox open overnight causes CPU
hogging and extreme memory use.
- The problems are the same in Mozilla browser.
- It's good to test Firefox with a laptop in a quiet environment. When you
hear the laptop fan begin to run while there is no activity, you know Firefox
has begun to suck CPU cycles.
- Putting a computer into standby or hibernation often makes the CPU hogging
bug much worse. That's why Firefox users sometimes just leave their computers
on.
- When a computer takes a long, long time to start from standby, you know Firefox
is taking CPU cycles. What about coming out of standby makes Firefox unstable? No
other program has that problem.
Quote from the blog linked in this Slashdot story About the Firefox "memory leak" [mozillazine.org]: "A lot of people complain about the Firefox "memory leak(s)". All versions of Firefox no doubt leak memory - it is a common problem with software this complicated."No other program in common use is so buggy. The problems in Firefox are not "common".
Another quote from the linked Mozillazine blog: "What I think many people are talking about however with Firefox 1.5 is not really a memory leak at all. It is in fact a feature."
That's not what the technical magazines [cmp.com], newsletters [scotsnewsletter.com], web sites entirely devoted to Firefox problems [slyerfox.com], and even the mainstream media say. They say it is a serious problem.
Mozilla developers have been denying that there is a serious problem for more than 3 years. It seems that it would be less work to fix the problem than to undertake a cottage industry of trying to convince people they aren't having problems. Mozilla developers have been impeding characterization by marking Bugzilla bug reports of these problems invalid.
However, it is clear that it would take a serious scientific investigation; this is not an easy bug to characterize.
Parent
Huh? (Score:5, Insightful)
Don't bug me (Score:5, Interesting)
What is being described here sounds much more like a cache of recent pages, which in my opinion is perfectly sane for a browser. Sure, maybe the cache is a bit overzealous, but even if that's the case, just disable it - worse case scenario, you edit the source. But otherwise, this is definitely a feature - I can promise you it's much more programming effort to save old pages for a quick redraw than to free the old page and replace it with the new.
So I guess the discussion here is, "is it right for firefox to use so much memory?" My answer is yes. It is not a memory leak, it seems like a very valid design decision. But if you disagree, old versions of firefox still work great (I still haven't upgraded myself).
There is another serious Firefox bug as well (Score:5, Informative)
Firefox crashes when two browser windows are making synchronous XMLHttpRequests. I have experienced this under Linux - I have no idea whether it is the same under Windows. Basically under Lunux all Firefox windows are running in the same thread utilizing a scheme of cooperative multitasking.
So far so good. The bug appears when two separate Firefox windows are making periodic synchronous XMLHttpRequest-s. When such a potentially lengthy task has to be executed synchronously, Firefox creates a new "nested" event queue. If two (or more) browser windows are doing it at the same time, new event queues are created all the time and eventually (within 5 minutes) the application core-dumps.
I found this by recompiliging Firefox with debug information and debugging it. Even if my interpretation of what happens is not completely correct, the fact remains - a simple JavaScript can crash Firefox causing all open browse windows to be closed.
The solution is to always use asynchronous XMLHttpRequest (which is a better practice anyway) and to hope that the same problem doesn't appear in other places. Still, it is troublesome.
Firefox developers don't "get it" (Score:5, Insightful)
With the number of people complaining about this (and the number of people that don't even KNOW to complain) isn't it a safe bet that you've made a mistake in the amount of cached pages?
Re:My pet peeve! (Score:5, Informative)
Parent
releasing memory (Score:5, Interesting)
The heap, where dynamic allocations occur, is only allowed to grow or to be truncated. An application cannot release memory in the middle of the heap without also releasing the memory at the end of the heap.
So let's say Firefox makes 10 one-page allocations, and frees the first 9. The memory layout might look something like:
XXXXXXXXXU (X- unused, U- used)
Those 9 pages worth of memory aren't being used, but it's impossible to release them back to the OS.
Thankfully, there is some good news: when Firefox needs to allocate more memory, it can and will just reuse those 9 unused pages instead of allocating more memory from the OS and growing the heap.
The best solution to this problem is to use a compacting garbage collector. Which is something that Java and C# and other higher-level langauges can easily make use of (and many do use them), but which C and C++ can't really make use of given the complete lack of compiler support. That's one reason why a Java or C# app can actually out-perform a similar C/C++ app, especially with a good native-code compiler and an library implementation with a modern GC.
Parent
Re:releasing memory (Score:5, Funny)
Maybe in theory, but in practice 99.9% of the world uses C++ browsers because the Java ones suck.
Parent
Re:releasing memory (Score:5, Informative)
I wouldn't know about C, but this statement is utterly false as applied to C++.
No, its not false, its true.
Replacing the default new and delete routines is perhaps not for the inexperienced C++ programmer, but to say that there's an complete lack of compiler support is simply wrong.
And? What do you mean with compiler support for heap compacting (or GC)?
Q: What has replacing new and delete with your own implementations to do with garbage collection?
A: Nothing
Q: How would new and delete of class A be able to compact a heap by moving allocated instances of class B down?
A: Difficult!
Q: So if you now add a class C you like to rewrite A::new and B::delete to also cope with class C instances?
A: I assume you understand that EVERY delete of EVERY class needs to know EVERY other class to be able to compact the heap, yes?
It is true that out-of-the-box C++ does not have a compacting garbage collecter, but one can certainly be written (and used, of course) with any conformant compiler.
Indeed, but not by merly only by replacing operator new and delete.
Existing C++ garbage collectors are very limited to more or less conservative garbage collecting. See e.g. Boehms c++ / C garbage collector.
And, the mere point of garbage collecting if you want to start nitpicking is: you don't ever call delete.
angel'o'sphere
Parent
Re:releasing memory (Score:5, Insightful)
Only on a computer without virtual memory. In a PC (which *has* virtual memory), you just punch holes in the memory.
What happens is a process gets an "address space", into which pointers can point, but any given address may not map onto some real storage. The process asks the operating system to map a range of addresses onto real storage which the operating system will try to map to real fast memory when it thinks it will be used at any moment. When the OS figures the memory wont be needed for a while, and something else needs some memory, the OS copies the data to disk and redirects the mapping to a proxy that will pull the data back into memory when the process tries to use it again.
When a process knows that it won't need a section of that real storage, it can tell the operating system to unmap it from the address space.
There are various other things that go on, but that's the simple story. From a figure posted in an earlier message, it seems that opera does pretty damned well (in comparison to most modern programs) with just the simple story, not having to rely much on nasty unreliable heuristics. Of that I am impressed.
Parent
Re:releasing memory (Score:5, Informative)
I wouldn't recommend mixing anything resembling C pointer maths with compaction, since its incredibly difficult to tell what's a pointer and what isn't (in fact, without modifying the compiler, it can't be done in C or C++). For this reason, the Boehm collector (a collector that replaces new and delete) goes for the Mark-and-Sweep method instead of compaction. Because you dont move objects, you don't have to worry about figuring pointers. Boehm's collector is also called conservative, not only because it doesn't modify live objects, but also in that it treats any data on the stack or in the heap as a potential pointer. If the data points inside the heap, the object containing that address is marked. This can lead to false positives on occasion, but there's no helping that without any support from the compiler (again contradicting the grandparent). The good news is that a false positive isn't going to cause direct harm in mark and sweep. All that happens is that space that could be used isn't; Boehm claims this is irrelevant in today's operating systems with virtual memory, although I doubt you'd see an entire page's worth of false positives. Certainly, I can't do any better than him.
In language R&D labs where people are paid quite well to think hard and long about things, they tend to use both approaches in what's called a "generational" collector. Young objects can be copied or collected as needed, while older objects are mark/swept away as needed. This works because old objects much more likely to stay than new ones. Last I knew, both Java and C# use generational techniques, because it makes sense in most nearly every case. However, as I described above, C++ doesn't have that, and even those libraries that replace new and delete have conventions and costs associated with it. I certainly wouldn't try to take Boehm and pidgeonhole it into Mozilla. And even if you did, it still wouldn't solve the compaction problem. All you can do is hope the virtual memory manager is doing it's job well. Even though the application and garbage collector is more likely to know what's useful than the VM manager.
Parent
Re:spyware with IE or memory leak with FF.. (Score:5, Funny)
i could never stand behind a company like that and refuse to use opera products untill he makes good on his word. You cant just throw statements like that around. Browsers designed by liars are dishonorable browsers.
Parent