Firefox Too Big To Link On 32-bit Windows 753
An anonymous reader writes "Firefox has gotten so large that it cannot be compiled with PGO on a 32-bit linker anymore, due to the virtual memory limitation of 3 GB. This problem had happened last year with 2 GB, which was worked around by adding a/3GB switch to the Windows build servers. Now the problem is back, and things aren't quite that simple anymore."
This only affects the inbound branch, but from the looks of it new code is no longer being accepted until they can trim things from the build to make it work again. The long term solution is to build the 32-bit binaries on a 64-bit system.
interaction of two things (Score:5, Informative)
Size of the Firefox codebase is one factor of course, but the amount of RAM needed by Visual Studio to compile code with all optimizations turned on (especially PGO, which is extra RAM-intensive at the compilation stage) is also a major factor. Notice that this only happens in the 32-bit Visual Studio builds specifically.
Re:Wow (Score:1, Informative)
1080P video has nothing to do with the bitness of your operating system.
Re:The code gets larger, and yet things dissapear! (Score:5, Informative)
Re:whose bloat (Score:5, Informative)
Speaking of someone who regularly does large C++ builds, MSVS is nowhere near a bad culprit here. The linker is essentially doing code generation -- link time optimization. Why? Because LTO gives a substantial performance benefit. The profile-guided optimization mentioned in the summary gets them about 10% over even doing non-profile-guided LTO.
One project I've worked on has single files which cause GCC to take over 6 GB to compile when you compile with -O2. Who's bloated now?
(Takeaway: broadly speaking, MSVS is actually very competitive, at least compared with GCC, when comparing similar settings.)
Re:Last paragraph in the TFA is... confusing (Score:5, Informative)
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778(v=vs.85).aspx#memory_limits [microsoft.com]
The problem is VS's PGO architecture (Score:5, Informative)
Summary should read: Visual Studio is too teh suck to link Firefox on Windows with PGO.
Firefox links just fine with VS, if you don't use PGO. The problem is that Visual Studio's PGO routine loads all our object files in at once, then uses a ton of memory on top of that. And the linker for 32-bit programs is itself a 32-bit program; if there were a 64-bit x86-32 linker, we wouldn't have this problem. But so far, Microsoft has not given any indication that they will release a 64-bit to x86-32 cross-compiler.
Note that Chrome doesn't build with PGO at all, last time I checked.
http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/533e94237691e2f6 [google.com]
Note: Visual Studio 2010 seems to help a bit, but not much. We use VS 2005 because it's the last version whose CRT supports Windows 2000 and Windows XP before Service Pack 2.
Re:Last paragraph in the TFA is... confusing (Score:5, Informative)
Also MSVC only has 32-bit binaries
Clarification: the only linker for 32-bit targets is, itself, 32-bits.
The linker which targets 64-bit Windows is still 64 bits. If they had a 64-bit-to-32-bit cross compiler (they have the reverse, but not 64-to-32) that would solve Mozilla's problem. (Well, for some definition of "their problem.")
Compare Chrome. Is it a plug-in, app, or OS? (Score:5, Informative)
Re:Time to move on, perhaps? (Score:5, Informative)
No, the long-term solution involves freezing the 32-bit version as an eternal final-state "stable" branch, and moving on to the 64-bit world.
Um.
Some 90% of our users are on 32-bit Windows. Just because *you're* not one of them doesn't mean that they don't matter.
It's nice that you don't expect us to support your aging XP boxes, but I think you'd find you're the minority in this respect.
(Also, all phones are 32-bit, and will be for at least the next few years.)
Not FF's fault (Score:4, Informative)
The software product I work on also used PGO at one point. Our software is roughly 2-3 million lines of C++ code, and links with 40-50 external libraries. Under Window XP, VS2008 could not link our product with PGO turned on.
Before you complain about the bloat of FF, please understand that PGO uses many many times more memory than just compiling a regular release build. This is a Visual Studio linker problem, not FF problem.
Re:Last paragraph in the TFA is... confusing (Score:4, Informative)
32 bit programmes (with the large address aware flag set) get 4GB of address space on 64 bit Windows, compared to 2GB / 3GB on 32 bit Windows.
Re:Trying to do too much (Score:5, Informative)
Chrome doesn't even try to build with PGO, last time I checked.
http://groups.google.com/a/chromium.org/group/chromium-dev/browse_thread/thread/533e94237691e2f6 [google.com]
Re:Trying to do too much (Score:5, Informative)
An excellent takeaway from this article.
Unfortunately, it's completely incorrect. TFA is talking about the build process on a 32-bit host, specifically that VS builds using profile-guided optimization require more memory than is available in the address space *DURING THE BUILD PROCESS*, not an issue encountered by the resulting binary.
I know you want a chance to get in a quick dig at Firefox, but this isn't the article for that.
Re:The problem is VS's PGO architecture (Score:5, Informative)
Re:Wow (Score:2, Informative)
Re:Old timer chimes in (Score:5, Informative)
It might seem a bit strange now, but back in the good 'ol days we used to have to break up a project into separate components, just in order to compile it!
Yep, we used to do this. Then we merged them together, because it greatly improves the startup speed of Firefox.
We have a lot of smart people working on this stuff, believe it or not.
Re:VS 2005? (Score:5, Informative)
Point taken.
But FYI: We use VS2005 because it's the last version whose CRT supports Windows XP before SP2 and Windows 2000.
We would love to upgrade, and are in fact devoting a lot of engineering time towards figuring out if we can upgrade while maintaining compatibility.
Re:Big deal (Score:5, Informative)
Oh, also, it looks like he was trying to say that compiling Chromium with PGO would use at least 9GB of RAM but he hit Shift too early
Its the compiler, stupid. (Score:5, Informative)
Or just, yknow, stop running a bloated resource hog of an INTERNET BROWSER.
Read TFA agin. Oh, I know this is /.
So read the summary again.
it cannot be compiled with PGO on a 32-bit linker anymore, due to the virtual memory limitation of 3 GB
It is the compiler which is having ressource problems. The profile-guided optimiser needs more than 3GB to be able to do its optimisations. And apparently, the Windows its running on can't do PAE to use more than 3GB neither.
Re:Its the compiler, stupid. (Score:5, Informative)
It is the compiler which is having ressource problems. The profile-guided optimiser needs more than 3GB to be able to do its optimisations. And apparently, the Windows its running on can't do PAE to use more than 3GB neither.
PAE allows 32-Bit computers to use more than 4GB of ram, but it doesn't allow Windows to assign more than 3GB to any single process.
Re:Last paragraph in the TFA is... confusing (Score:4, Informative)
Re:I don't understand the issue (Score:5, Informative)
There is no 64-bit version of the MSVC linker
Clarification (I've posted this a few times): there's no 64-bit version of MSVC that targets 32 bits. There is a 64-bit version which targets 64 bits, but that doesn't help the current situation.
Re:Trying to do too much (Score:5, Informative)
Define "too much"?
It's been over a year since Chrome had to turn off PGO altogether and move to 64-bit builders even without PGO, because they ran into this same problem.
So maybe your issue is with the fact that all "browsers" as you call them are trying to do too much? They should drop the fast graphics and jits and video and audio support and all that jazz, right?
Re:interaction of two things (Score:3, Informative)
Re:Time to move on, perhaps? (Score:5, Informative)
No, you missed a fact.
Visual Studio 2005 is a 32-bit app on any Windows platform.
Visual Studio 2010 is a 32-bit app when running on a 32-bit platform.
Mozilla builds on 32-bit platforms can no longer support the PGO linker.
Visual Studio 2010 is a 64-bit app on a 64-bit platform.
Mozilla builds on 64-bit platforms can PGO link just fine.
When Visual Studio 64-bit compiles a 32-bit app, that app can run only on XP SP2 or later.
Mozilla has millions of users on pre-XP SP2 platforms.
So Mozilla has a choice: change nothing but stop PGO linking the 32-bit versions (sub-optimum for 32-bit users), go forward on a 64-bit only path and disenfranchise the old users from getting any new functions, abandon them completely (which is irresponsible in terms of security), cut back on new features for all, or take an axe to the existing code. Only one is an easy choice.
Re:Time to move on, perhaps? (Score:4, Informative)
Re:Linux PAE (Score:3, Informative)
You need to go re-read your source on PAE (or tell them to reread their source).
PAE does not increase the memory space available to a single process, so your statement that "So GCC *could* use 6GB on a 32bits machine" is absolutely false. What PAE allows is multiple processes to, in total, take more than 4 GB. (So you could have a 32-bit machine with 6 GB of RAM and have, e.g., two GCC processes, each taking 3 GB with no paging.)
Re:VS 2005? (Score:5, Informative)
One of the biggest problems with newer versions is the runtimes using EncodePointer/DecodePointer, which aren't available pre-SP2.
Try this:
* Install VS2010
* In your project configuration(s), under Configuration Properties->General, set "Platform Toolset" to "v90".
I can use this configuration with VS2010 and the latest Windows SDK and get binaries that work on XP pre-SP2.
Re:No PAE?! (Score:5, Informative)
Not only that, but apparently Windows cannot use PAE - Physical Address Extension [wikipedia.org] to address more than 4GB
Sure it can, you just have to either pay for a server edition or hack the restriction out of the kernel.
But more physical address space doesn't help here, the problem here is virtual address space for running an effective but memory hungry profile guided whole program optimisation process. Nromally 32-bit windows has a maximum of 2GB virtual memory per process (and this is one big process we are dealing with). This can be increased to 3GB at the cost of reducing the kernel address space to 1GB.
Going to a 64-bit OS (which allows 4GB of virtual address space for 32-bit processes) will buy them a little bit of time but it's not a long term soloution. Really they need a 64 to 32 cross toolchain (which according to other posts here MS do not offer) if they want to keep using profile guided optimisation as the codebase grows.
Re:Trying to do too much (Score:5, Informative)
Well you're right in that Firefox does need a hell rather large amount of RAM to build... but it's not just them; all browsers are trying to do a lot nowadays.
Chrome doesn't exactly have light build requirements either. In fact, the Chromium project already seems to have dropped 32-bit build environments:
(From "Build Instructions (Windows) - Build Environment" [chromium.org])
That's why I think that the parent poster's implication that it's due to Firefox becoming "bloated" is basically hogwash. Browsers are more complex than they were in the mid-90s. That's what happens when you add 10+ years of new formats and technologies that must be supported for a browser to be considered "usable". Directing one's ire at Firefox is unwarranted, IMHO.
Re:Trying to do too much (Score:5, Informative)
> And that is to make lots of parts be separately loaded
> modules
This actually leads to pretty serious performance penalties because of the way that web specs tend to interdepend on each other. It also loses you optimization opportunities.
Pieces of functionality that are not interdependent with other stuff (e.g. audio and video backends, webgl) are in fact either in separate libraries or being moved there.
For the other stuff, there did use to be more modules. It turned out that in practice most users needed them at startup (to show the browser itself and restore their tabs), so the only thing multiple modules got you was a slower startup and slower runtime code.
It's not an accident that both Firefox and Chrome ship almost all their code in a single library (binary in the case of Chrome). It turns out that for web browsers specifically this works somewhat better than the alternatives.
Oh, I just checked and looks like Opera also also links all its code into a single library, at least on Mac.
And Safari links all the core WebKit code into a single library.
Not sure what IE does nowadays, but last I checked mshtml.dll in fact included all the actual browser bits.
Now it may be that all the people involved in all these projects can't design worth anything. Or maybe they did some measurements that you haven't done and found that this approach works better....
Re:whose bloat (Score:2, Informative)
You're wrong, you can easily output 32-bit binaries with a 64-bit toolchain with MSVC - even from within the Visual Studio IDE if you set up the environment from the command line and then type "devenv /useenv". However you need to be running a 64-bit Windows which presumably the Firefox buildbots aren't.
Re:interaction of two things (Score:5, Informative)
Sure, except that (especially in C++ code with templates) VS uses FAR less memory than the GNU toolchain when compiling the same code. This isn't a VS problem, it's a Firefox problem.
Re:Wow (Score:2, Informative)
And we're talking about builds, not in-use resource consumption... where it behaves just like every other browser.
Nope. It doesn't even behave like the 3.x series (of the very same browser, Firefox!) before rampant version number inflation. A perfectly serviceable "Athlon XP" based system with 750MB of RAM went from just fine to useless (in terms of firefox usage) at the version number break, under both Windows XP and Fedora. Memory footprint was my first suspect, but throwing in more RAM (up to 2.5GB now) helped but didn't make it completely better. Ironically, it's almost ok now under XP and still atrocious under Fedora.
Something baaad happened. But who cares about old systems, right? Only users, it turns out, not people triaging bugs.
Re:VS 2005? (Score:4, Informative)
I think what he was trying to say -- in the meanest possible way -- is that the setting you chose tells VS to use the old compiler and linker. It doesn't switch out the CRT -- it switches out the whole toolchain. So using that setting is no different from where we are now, afaik.
Re:Time to move on, perhaps? (Score:5, Informative)
I am a Mozilla guy.
There's an official 64-bit version for Linux. We've been shipping that since before I can remember. There are also nightly builds for 64-bit Windows, but we're not shipping these even as Aurora at the moment.
64-bit Linux isn't listed on most of our download pages. I'd argue it should be there, but I'm not in charge. :)
Anyway, here are links to get all the builds we produce:
Nightly builds: http://nightly.mozilla.org/ [mozilla.org] (has win-64 builds)
Aurora builds: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora/ [mozilla.org]
Beta builds: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/9.0b6-candidates/build1/ [mozilla.org] (I don't know the directory for the latest beta build, unfortunately, so you'll have to update this URL each time you go looking.)
Release builds: http://releases.mozilla.org/pub/mozilla.org/firefox/releases/latest/ [mozilla.org]
Re:Trying to do too much (Score:5, Informative)
Re:Time to move on, perhaps? (Score:4, Informative)
The MSVC compiler is a 32-bit program
NO:
C:\vs10\VC\bin\amd64>link /dump /headers link.exe
Microsoft (R) COFF/PE Dumper Version 10.00.40219.01
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file link.exe
PE signature found
File Type: EXECUTABLE IMAGE
FILE HEADER VALUES
8664 machine (x64)
Re:Wow (Score:3, Informative)
Making the usage of Firefox fast is the whole point of PGO, a process that uses up orders of magnitude more when compiling and linking an executable than without PGO. So, in order to get the fastest speeds out of the Firefox binary, the Firefox build process optimizes and uses lots of RAM in the process. The problem is that the MSVC linker is 32-bits, so the *linker* is running out of memory ONLY WHEN FIREFOX IS COMPILING AND OPTIMIZING. The executable that is generated is not as large, does not use as much memory, and is faster than if Firefox was built with a "straight" boring old build process.
The whole point of this exercise is to save assholes like you more time by making your browser faster and more responsive so you can post snarky criticisms of Firefox (without understanding the problem) faster.
Your welcome, asshole.
Re:The code gets larger, and yet things dissapear! (Score:3, Informative)