Linux Kernel 2.4 Or 2.6 In Embedded System? 178
snikulin writes "My 6-year-old embedded software happily runs on kernel v2.4 on an XScale CPU. The software gets a bunch (tens of megabytes) of data from an FPGA over a PCI-X bus and pushes it out over GigE to data-processing equipment. The tool chain is based on the somewhat outdated gcc v2.95. Now, for certain technical reasons we want to jump from the ARM-based custom board to an Atom-based COM Express module. This implies that I'll need to re-create a Linux RAM disk from scratch along with the tool chain. The functionality of the software will be essentially the same. My question: is it worth it to jump to kernel 2.6, or better to stick with the old and proven 2.4? What will I gain and what will I lose if I stay at 2.4 (besides the modern gcc compiler and the other related dev tools)?"
testing? (Score:5, Informative)
if you're migrating, no doubt you're performing tests to ensure your product is still fit.
once you have your test plan ready, determining fitness against either kernel should be straight-forward.
Why Linux? (Score:5, Informative)
2.6 (Score:5, Informative)
I had the same question asked for an embedded project 3 years ago. And it was very clear cut then
2.6 you get (off the top of my head)
-Modern drivers (including USB/Network/etc)
-Various tick rates and tickless
-More support
-Several other improvements
So really don't bother w/ 2.4
This article was written upon 2.6 release (Score:5, Informative)
http://www.linuxdevices.com/articles/AT7751365763.html [linuxdevices.com]
Without knowing your exact parameters though, it's hard to debate any specific advantages.
GCC 4 & linux 2.6 (Score:4, Informative)
Re:Why Linux? (Score:5, Informative)
All of these work on FreeBSD 6 or 7 (aio is in a module that isn't loaded by default on 6, not sure about 7), and most of them work on Linux 2.6.
Better stick with the old and proven 2.6 (Score:1, Informative)
http://www.linuxdevices.com/articles/AT7751365763.html
Just say no to backporting (go with 2.6!!) (Score:3, Informative)
If you foresee needing to periodically update the firmware to along with a library or app, then I would say a definitive YES - use the 2.6 kernel (assuming your device is supported).
It might also be the case that the board you would like to use is not supported in the 2.4 kernel if it's new enough - kernel developers usually don't want to waste time backporting their code if they can avoid it.
Which introduces the most important issue - backporting is a PITA!! To make a long story short, if you need to track a library or app, such as an embedded JRE, or a hardware interface that requires a kernel module inserted, playing catchup and needing to backport at the same time is an awful game of one-step-forward two-steps-back. Avoid it at all costs. Backporting is not always guaranteed to work!
The 2.4 kernel has a slightly faster boot time, while the 2.6 kernel has so many improvements that it's hard to shy away from. Do yourself a favour and go with a stable 2.6 kernel.
Embedded in volume or just custom? (Score:3, Informative)
If you're doing embedded systems in mass market volume, it's a matter of hardware requirements and cost per unit. Then potentially staying with the 2.4 kernel may be a good choice. If what you're making is a small volume custom setup, I'd go with whatever is getting the most use and the most testing now, which is definately the 2.6 kernel.
Go 2.6 (Score:5, Informative)
Lots of the improvements to 2.6 have probably been added to 2.4, but many come "native" to 2.6 so no outside patches are required. For example, kernel pre-emption, better scheduler, etc.. There are other intangibles too such as development time, testing, new toolchain, etc.., but you're already moving to a new processor and you'd have to do that anyway.
Sometime last year I was rebuilding some antique MIPS-based Linux from a 2.4 to a 2.6. Almost everything in the userspace was effortless (though much of it was based on Busybox); the main issue was related to some in-line assembler that took a while to figure out what it was doing. Once I did, I googled it and realized someone else had already solved a year or so ago.
So in short, no real benefit to sticking with 2.4 IMHO.
Re:GCC 2.95? Seriously? (Score:3, Informative)
GCC 3.4 is quite outdated. 2.95 is just plain old. Why not code in Fortran while you're at it?
My development group is also stuck with gcc 2.9x series because it's only compiler our toolchain maker (WindRiver) supports for VxWorks 5.X. I'm guessing he's in a similar situation. I can't complain though -- we've never had an issue with it.
Re:2.6 (Score:4, Informative)
The only issue is size. 2.6 is "significantly" larger than 2.4
Really a non-issue when bigger is cheaper and smaller RAM/ROM chips are being phased out. Just as an example, we developed a product using 32MB RAM but that was phased out (really, you couldn't buy the chips anymore) in favour of 64MB RAM
How much is a 1GB usb drive today again??
I guess you need something like 2MB flash for a 2.6 system (if you really squeeze things). With 16MB/32MB you can do pretty much anything.
Re:If you are olready doing 90% of the work... (Score:5, Informative)
No clue what gp meant.
From all I heard (I was in embedded business only in 2.2/2.4 times) that 2.6 integrated some number of patches from embedded folks and generally can be customized to run on smaller number of resources. Also, the improved I/O (much lower latencies) and scheduler (interactivity; soft-real-time) would benefit in embedded too. 2.4 has number of problem related to memory management, when virtual memory subsystem can easily grab half of available RAM - only for supporting virtual memory. 2.6 solved the problem for most architectures.
Generally, many embedded folks moved to 2.6 already - mainly due to support for more new OTS hardware. 2.4 has this support only through vendor patches (e.g. I used in past BlueCat and MontaVista patches).
In my experience changing kernel on embedded system is quite easy task. Using development system within couple of days you can come up with suitable minimal .config (one needs development system since on target embedded systems might not have sufficient resources to run vanilla kernel). Generally it would either work or not. Normally it works.
Also note that H/W vendors started being more active in 2.6 times. In 2.4 times best shot at Linux driver was some crude port from e.g. LynxOS or VxWorks. From all I know, 2.6 now supports more PowerPC system than did patch from MontaVista for 2.4 I used three years ago.
Last, but not least, if you are looking at new modules, many hardware vendors supply Linux compatibility information. 2 years ago finding module with "Linux compatibility" chapter in documentation wasn't a problem at all.
Re:GCC 2.95? Seriously? (Score:2, Informative)
This system is based on an Intel Atom CPU, so it does have SSE and and FPU.
For the main architectures (x86, x86-64, ARM, and maybe PowerPC), GCC has been getting significantly better with each release. For less well-supported architectures (like Hitachi's SH series, an uncommon embedded CPU, or really old architectures like 68k), there's usually an older version of GCC that works better than the current ones.
Given that this is a new(ish) CPU, newer versions of GCC are going to support it much better than older ones.
From a kernel point of view, it's likely to be easier to get everything running on an Atom board under Linux 2.6 than 2.4, simply because 2.6 has newer drivers.
However, the application itself may not compile correctly with GCC 4, or may compile but have incorrect behaviour. Changing the C compiler is one of those things that's probably not worth the effort unless you have to, or you have a decent test suite and a good idea of how all the code works.
Of course, given that this is C, not C++, there's no reason you couldn't run Linux 2.6, compile everything except the app itself with GCC 4, and then compile the app with GCC 2.95.
Re:Go 2.6 (Score:1, Informative)
Granted, but I'd think that for an Atom-based system, a tickless kernel would be a significant improvement. Of the generic improvements, the new scheduler might make a difference.
Other improvements that benefit embedded systems in general, but may or may not benefit the OP's use case, are: asynchronous IO, pre-emption and real-time support (although not yet fully integrated in the main tree), MMU-less operation (obviously not relevant here).
New thread scheduler (Score:3, Informative)
If you want to minimize latency in your applications, chances are you'll like the new scheduler [wikipedia.org] implemented in 2.6.23 and following. In general, 2.6 has better support for realtime (low-latency) applications: http://www.ibm.com/developerworks/linux/library/l-real-time-linux/index.html [ibm.com]
Re:Embedded in volume or just custom? (Score:3, Informative)
What you are talking about???
2.6 supports more hardware that 2.4 - especially embedded hardware. Several architectures in 2.6 (ARM, PPC) went through restructuring to allow easily add another board/module support.
And more importantly - for "mass market" - with 2.6 you also get much much better support from hardware vendors. In 2.4 times market was only heating up. Now, in 2.6 times, the embedded Linux market is full swing. You would be hard pressed to find H/W vendor who doesn't support Linux now - but only few of them do support 2.4 now.
If you're doing embedded systems in mass market volume, it's a matter of hardware requirements and cost per unit.
If you are talking about memory consumption, then think again. In past years I haven't seen embedded system with less than 64MB RAM. When I asked "isn't it expensive? we can run on less RAM." I got a response that when buying in any kind of volumes, 32MB or less vs. 64MB make pretty much no difference. Cheap RAM is dirt cheap nowadays.
Add here fact that 2.4 has numerous problems in its virtual memory implementation, meaning that on less RAM 2.6 would run better than 2.4. And do not forget that in 2.6 kernel is truly modular - you can't remove I believe only PCI bus support, rest is optional - you can further decrease kernel image size and its memory footprint.
Times have changed... (Score:1, Informative)
My company also has an XScale board which interacts with an FPGA for data collection. When the project first started it ran 2.4 (along with GCC 2.95) and generally sounds very similar to what you're doing.
Things have come a long way in the embedded Linux world since that time. Besides the TONS of additional features present in the the 2.6 kernel, I'm fairly certain you'll find that the vast majority of device vendors are only going to be writing drivers for the 2.6 tree. If you're upgrading to a new board I doubt you have a choice. Be comforted though, 2.6 is great (even for embedded XScale processors).
Your comment "This implies that I'll need to re-create a Linux RAM disk from scratch along with the tool chain" seems reminiscent of the old days where building cross compiling toolchains was a marathon. I highly recommend checking out Crosstool-Next Generation [freshmeat.net] and OpenEmbedded [openembedded.org] if you're looking for the current state of the art in embedded Linux.
Re:Why Linux? (Score:5, Informative)
Who cares which came first. The important thing is that that kernel is small, and has the features. Linux 2.4 does not have the features, Linux 2.6 does not have the small.
2.6. See our Space Station project (Score:5, Informative)
Hi
We moved our project from 2.4 to 2.6 during development, because the maximum interrupt latency of 2.6 is so much better. We needed to handle UDP packets within 20 ms. max and occasionally on 2.4 we would have a 60 or more. Going to 2.6 solved our problems immediately, even with early versions.
See this Linux Journal article for more details on our project http://m.linuxjournal.com/article/7190 [linuxjournal.com]
Bart van Deenen
Re:Why Linux? (Score:3, Informative)
I've trimmed-down linux systems before and it was more work than my current method: start with an OpenBSD base, which is usually small enough for most purposes. Installing only the 'base' and 'etc' components results in a pretty damn tiny footprint and yet a full-featured Unix OS (and a quite stable and secure one at that).
Also, you end up with an actually supported OS that you can update every 6 months without a lot of patching and hacking around a custom forked/branched tree. I don't have time for that stuff.
And besides, OpenBSD is a joy to work with. It's so easy to admin, the system is clear and straightforward, with good (and current) documentation and easy to follow code.
Re:Stay with GCC 2.95 (Score:3, Informative)
If you want people to stay with old code to run on their boxes, please leave GCC out of the mix.
GCC's had two major improvements over the versions:
1. Better language compliance. It matters *a*lot*. And frankly, you're not going to win an argument against that on a techie forum :-) This included a new hand-written C++ parser in g++ ~3.4 that closed out over 100 bugs at once. You don't ignore hand-written C++ parsers, they're complete bitches to do.
2. Better optimizer.
As for any library bloat, the basic C/C++ libraries are shared by nearly everyone on your system. You pay for them exactly once per running system (minus templates, but I don't know of anyone using template-heavy code on a normal Linux desktop installation).
If you were running Solaris, I'd just tell you to dtrace the system and find out what's making it slow. But, no, *sigh*,
Re:New hardware new issues (Score:5, Informative)
Yes, 2.4.37 runs fine on an Asus EEE-Box (Atom, PCI-E, SATA, USB2, ...)
Willy
Re:linux2.4 is NOT proven on Atom (Score:3, Informative)
Atom is just plain x86 with HT. Nothing fancy, nothing new. Drivers for rare hardware might cause problems though, but that's true with any kernel. I would have worried if the guy wanted to switch to an exotic architecture but that's definitely not the case here.
Re:Why Linux? (Score:4, Informative)
Define small, I'm working towards a first release of a Linux 2.6 based project, the Kernel + bundled busybox based RFS is 2.1M. Just saying.
Re:Why Linux? (Score:2, Informative)
OK, found out that the tool is called crunchgen [gw.com], and that its main purpose is efficiently combining multiple binaries into one (although I've read it could be used for building efficient libs, too), so it is more like an alternative to BusyBox multi-app binaries.
Re:Why Linux? (Score:4, Informative)
And there is also CONFIG_EMBEDDED
Specific features that make 2.6 better than 2.4 (Score:3, Informative)
Your question feels a bit of strange question to ask as surely anyone who has looked would notice a huge difference between the latest 2.6 (2.6.28) and the latest 2.4 (2.4.37).
Preemptible kernel [lwn.net] (so lower latencies are possible)
Far more devices supported (both in terms of architectures and additional add on devices e.g. SATA support [lwn.net])
Better scheduler (initially made O(1) scales better under load [lwn.net] and then fairer with CFS [kernelnewbies.org])
Task Control Groups [kernelnewbies.org]
Better support for threads (schedules them in a more intelligent fashion)
Strict overcommit [kernel.org]
Massive VM changes
Tickless/dynticks support [kernelnewbies.org]
Asynchronous I/O support [lwn.net]
Introduction of different I/O schedulers [lwn.net] (deadline [lwn.net], cfq [lwn.net]
Network stack improvements (faster, better under load e.g. NAPI support [lwn.net])
epoll support [xmailserver.org]
Improved ACPI support [lwn.net]
Network filesystem improvements
Initramfs support [lwn.net]
There is a huge list of Linux kernel changes that happened between 2.4 and 2.5 [kniggit.net]. There is also a good Linux kernel 2.5 changes page on IBM's developerworks [ibm.com]. Kernelnewbies has an excellent summary of changes for each of the 2.6 kernels [kernelnewbies.org] and a 2.5 changes page [kernelnewbies.org]. LWN is also excellent for kernel news [lwn.net].
I hate it when people don't bother to state exactly the points they object to. What other changes (not listed above) do you think the question poster wouldn't benefit from? Follow the links to the full lists (don't just use the ones off the top of your head)...
Re:Why Linux? (Score:3, Informative)
Finally! I think it's amazing that we're discussing embedded systems, Linux, BSD... yet ignoring NetBSD, which is the flavor that most caters to embedded systems!
build.sh [netbsd.org] is a great example of one of those unheralded "little things". If I'm on my Mac OS X laptop and want to build a NetBSD ARM kernel or distribution for my embedded single board computer, I don't have to go fussing around with finding and downloading cross-compiling toolchains and figuring out how to make them go, etc. I just add a single flag to the normal build script that's part of the OS source code:
And that's it. It will build the toolchain and then the kernel. It's so simple, it's brilliant.
NetBSD tends to get ignored a lot, but IMHO undeservedly. It is on a par with Linux 2.6 and FreeBSD in terms of speed, is flexible and feature-rich, has a nice ports/packages system [netbsd.org] and doesn't suffer from a lot of "superfluous redesign" (Linux, I'm looking at you). It's a really nice clean system and it's easy to get the basics done. It deserves more attention than it gets.
That said, the original question was "Linux 2.4 or 2.6?" If you can strip 2.6 down to fit within the confines of your embedded system, by all means go for it. This is especially true for new small x86 devices, where 2.6 is typically the baselined kernel anyway. On other architectures, it varies. On many ARM flavors, 2.6 is getting shaken out and 2.4 is still often the default. I suspect that the transition will complete this year.
Re:Premption, responsiveness (Score:3, Informative)
The -rt patches do NOT a hard real time kernel make. Here's the difference. One says, 'I SHALL be called at EXACTLY this time', the other one says, 'make sure I'm called BY this time'.
network performance with atom (Score:1, Informative)
Hi,
I have been playing with some embedded systems as I have some personal projects with some colleagues to develop an embedded network based sflow/netflow collector. We have been testing a variety of architectures using development products and different tool chains etc. One of the common themes is that we have moved out entire system from 2.4 to 2.6 and newer versions of gcc etc which provided us with as much as 5% improvement in performance with no additional hardware. One of the observations we have found with our system is that the ATOM processors do not seem to be able to push network traffic nearly as quickly or efficiently as the MIPS or ARM based systems. This may well be because all of the ATOM based systems we have tested use realtek interfaces which don't seem to be able to handle the volumes of sflow/netflow we are pushing towards them. When we compare the performance to some of the ARM/MIPS combinations we have tested we get at least 20-40% higher network throughput when compared to the single core ATOM 1.6Ghz processors despite the ATOM having at least twice the raw Mhz. I would highly recommend testing the network throughput if your doing anything that requires bandwidth.
In the coming weeks I will update this post with a URL with all the information on the tests we have been conducting and the results. Its unfortunate that this post came soo soon before we had the information compiled.