Kernel Exploit Cause Of Debian Compromise

mbanck writes "The cause of the recent Debian Project server compromise has been published by the Debian security team: 'Forensics revealed a burneye encrypted exploit. Robert van der Meulen managed to decrypt the binary which revealed a kernel exploit. Study of the exploit by the RedHat and SuSE kernel and security teams quickly revealed that the exploit used an integer overflow in the brk system call. Using this bug it is possible for a userland program to trick the kernel into giving access to the full kernel address space'. This issue has been fixed in 2.4.23. Thus, the Linux kernel compromise was not Debian specific."

A shift of focus

[chrysalis (50680)]

It's fun to see how security research shifted from applications to kernels lately.

Re:A shift of focus

[by Anonymous Coward]

It's fun to see how security research shifted from applications to kernels lately.

Fun!? You must be Klingon.

Re:A shift of focus

[by Anonymous Coward]

"Fun!? You must be Klingon."

Today is a good day to get rooted.

Re:A shift of focus

[pclminion (145572)]

The "nice" thing about kernel exploits (from a cracker's perspective, of course), is that it doesn't matter what sort of userland software is running on the machine -- if you can get local access, by whatever means, it is very easy to boost yourself to root.

Traditional local root exploits are all based on overflowing a setuid application or server. But if the box doesn't have any vulnerable apps installed, the attacker is SOL. However, if the kernel itself is exploitable, it no longer matters whether those setuid programs are present. All you need is to somehow acquire local access, and wham, you are root.

To summarize, kernel exploits are very convenient for turning local user account compromises into full-blown root compromises.

Re:A shift of focus

[Frymaster (171343)]

what i want to know is...

does this code belong to sco?

Re:A shift of focus

[GlassHeart (579618)]

The "nice" thing about kernel exploits (from a cracker's perspective, of course), is that it doesn't matter what sort of userland software is running on the machine -- if you can get local access, by whatever means, it is very easy to boost yourself to root.

We should point out that you're talking about common kernels like Linux today, not that this must apply to any OS kernel. For example, if the kernel verifies the binaries it runs using digital signatures, then it can refuse to run unsigned binaries. The cracker's problem then becomes how to get an existing userland program (compiled by the trusted administrator) to issue the kernel call that results in the exploit, which is a much harder problem. A program may not ever make that system call, or may check its input so that you can't make it do the system call with (presumably) invalid parameters. In most cases, I expect that this means you have to insert a backdoor into a common program and wait for the trusted administrator to compile it for you.

This sort of security is probably overkill and too tedious for a desktop box, but not unreasonable for a server which should only have a few programs installed and running anyway.

Re:A shift of focus

[Sloppy (14984)]

That's why all the smart admins have been migrating their servers over to the best platform for the job: XBox.

Re:A shift of focus

[John Whitley (6067)]

That's a silly reason for plugging DRM. Simply mount all user-writable space with option "noexec" and you have the same level of security.

Life is more complicated than that. Regular binaries can be run as: /lib/ld-linux.so /noexeced_path/myprog

Even locking this out, any available program that is itself an interpreter (e.g. Perl, etc.) can be used to run code. Assume that any attacker (or their scripts) know this and will leverage it.

I'm seriously beginning to think that we won't be able to achieve secure systems that reliably push the security problem to the social boundary until ground-up designs such as the Extremely Reliable Operating System [eros-os.org] mature.

Re:A shift of focus

[John Whitley (6067)]

Seems like a stupid security hole.

This thread [debian.org] on the Debian mailing lists talks about this issue, and a poster in this thread notes that SE Linux is capable of closing this hole (with example). I don't recall offhand what tools are available in grsecurity (www.grsecurity.net) to address this issue; check out the grsec mailing lists for more info.

Kicking it up a notch.

[_Sprocket_ (42527)]

And they call Windows unsecure. How does crow taste, Slashdot?

Pretty good if you know how to spice it right. The trick is, knowing you've got crow to eat. How's that mystery meat you're chewing on?

(there's a joke about feeding trolls to be made in this somewhere)

Shows the dangers of C

[by Anonymous Coward]

If the kernel was coded in visual basic, this wouldn't be happening.

Well then they'd better get some help

[Hal The Computer (674045)]

CLIPPY:
You appear to be trying to write a kernel. Do you want to:

• Automatically make sure the Visula Basic DLL is included in your program?
• Answer some questions and have me generate a nice windows kernel for you?
• Straigten me, and turn me into a very attractive piece of modern art?

Re:Shows the dangers of C

[stefanlasiewski (63134)]

By 'this' do you mean the exploit wouldn't be happening? Or the Kernel?

Re:Shows the dangers of C

[Lussarn (105276)]

Why not Brainfuck [muppetlabs.com]

If you can't read your own code who else can..

what kind of person...

[potpie (706881)]

What kind of person spends that much time trying to find exploits in operating system kernels? Likewise, why do I spend so much time on www.thinkgeek.com/fortune.shtml? We are a sad people.

Re:what kind of person...

[Smallpond (221300)]

What kind of person?
spammers [bbc.co.uk]

People who expect to make money by hacking systems and using them to send millions of unsolicited emails.

Re:what kind of person...

[ashridah (72567)]

The kind who only has to read through LKML to realise that a potential root bug was discovered in september, but wasn't fixed in 2.4.22, and doesn't appear to have been backported to the kernel?

ashridah

Re:what kind of person...

[noda132 (531521)]

What kind of person spends that much time trying to find exploits in operating system kernels?

The kernel developers, i.e., Andrew Morton. Good for him, too.

There *was* a patch before the Debian systems were compromised. Hopefully in the future these things will be given more attention before they blow up.

What Crow?

[IBitOBear (410965)]

Exactly what crow is that?

Would that be that a legitamate error was found verified and fixed in public in just about two weeks with no hiding or spin?

If Windows had a memory allocation error (application memory space being the thing controled by brk) of this sort would they allow it to be publicized?

Once they made the "patch" available, would you be able to apply it to every past version of Windows?

Would you be able to verify that the patch was applied to windows?

If Debain's FTP server had been running IIS and windows, what kind of forensic annalysis would have been done, and how LIKELY is it that the *SINGLE* *INCIDENT* compromise would have lead to a complete and detailed report from Microsoft, and a fix?

The "linux is more secure" argument is not (truthfully) based on the idea that linux is inherently error free. It is based on the idea that the persons experiencing the problems can determine what those probems actually are; come up with a fix; have that fix reviewed [and installed] (by all who care) *WITHOUT* needing to waking some sleeping-bear corporations nacient interest in the suffering of their lowly surfs... er... "customers".

Re:Well, well, well...

[HeghmoH (13204)]

The worst Linux exploit of the year: an obscure kernel vulnerability that allowed one person to gain control of one box, disrupting one small OS group for a few days.

The worst Windows exploit of the year: a hole in the RPC services (which you can't turn off) that allowed a worm to gain control of millions of Windows boxes, disrupting the entire internet.

How does this make Linux equally bad as Windows, then?

Re:Well, well, well...

[Sri Ramkrishna (1856)]

Um no.

First the exploit compromised one of the largest linux distribution and potentially they could have put trojan horses in all our packages and we would really be up shit river when that happens.

Secondly, we are no longer getting package updates so they have successfully stopped Debian development while they patch all this.

Although it's not in the scale of windows, if GNU/Linux had larger marketshare this would have been a big deal.

sri

Re:Well, well, well...

[_Sprocket_ (42527)]

Perhapse you would like to add a point to copy-and-pasting the Linux Security advisory page? Maybe some context? Some sign of understanding what you're reading?

A couple of points...

1) Note that of the 15 listed advisories:

5 are the same BIND DOS vulnerability

2 (or 3 if you count Turbolinux's mega-update) are the same Ethereal vulnerability (DOS, possible arbitrary code)

2 are the same stunnel hijacking vulernability
2) None of these vulnerabilities lead to a remote exploit (although it could be argued one might be able to create a favorable condition with the ethereal issue)

Sure - Linux runs buggy code too. If that's your point, make it. But this hardly seems to be a suitable response to the parent's (semi-trollish) comment on MS' run of remote exploits.

Userland exploits

[Hayzeus (596826)]

The evidence mounts: users should be eliminated.

Yup

[ENOENT (25325)]

Just like Nancy Reagan said: Users are Losers.

Re:Userland exploits

[by Anonymous Coward]

No really, a user account is as good as root on almost all systems. If you need security, don't have user accounts on the system.

Re:Hmm, Methinks I've Heard this theme before

[RetroGeek (206522)]

Several million others that I missed, which courteous slashdotters will point out.

I'm sorry Dave, I can't do that...

Will redhat provide an rpm???

[cybrthng (22291)]

Just wondering if they will still support us lowly 7.3 and 8.0 users anymore with a fix for this.

Not yet

[dmeranda (120061)]

I just checked the current Red Hat 9 kernel source RPM and it does not have the patch yet [kernel-2.4.20-20.9.src.rpm]. I would expect a new kernel to show up soon though....I hope. The supposed patch which fixed this was in do_brk() [a /. comment further down provides the bk url]

--- 1.31/mm/mmap.c Fri Sep 12 06:44:06 2003
+++ 1.32/mm/mmap.c Thu Oct 2 01:18:19 2003
@@ -1041,6 +1041,9 @@
if (!len)
return addr;

+ if ((addr + len) > TASK_SIZE || (addr + len) < addr)
+ return -EINVAL;
+
/*
* mlock MCL_FUTURE?
*/

So it sounds like....

[satyap (670137)]

So it sounds like someone used a compromised user account to get in, ran a binary that exploited the bug, and got root that way. This is a local exploit, then.

Time for better security.

[Sheetrock (152993)]

It's obvious that with the gradual acceptance of Linux by the business community, it's time for a stricter security model to be adopted. While OpenBSD has not shared in the commercial success of Linux, it does have one area of technical superiority: its security review process has yet to permit a remote root compromise in a standard install.

Linux is a compelling choice in the Free Software world because of its pace of development and wide availability of software. However, it is this strength that is becoming a weakness. Perhaps it is time to slow down and review with more vigor to mimic the accomplishment of OpenBSD.

Re:Time for better security.

[chrysalis (50680)]

Well... yes... and no.

The strength of OpenBSD is that people continously audit the code and implement preventive stuff like privilege separation to reduce the risks in case of a vulnerability.

On the other hand, the code of BSD kernels is a real mess. Some parts are really tricky, with glue between historic and new code and a lot of ugly, possible unsafe macros everywhere. The Linux kernel framework is way cleaner and more robust. When something goes wrong in a kernel thread, it can almost always properly recover and not just go to panic().

And Linux has also some barriers like SELinux that theorically renders uncommon situations not exploitable. Theorically, because there can still be bugs in SELinux or other parts of the kernel that would bypass it.

The "barriers" approach, although described as useless by some people is, in a real world, very efficient. Grsecurity (or recent OpenBSD with PaX and co) and SELinux make it very difficult to write reliable exploits. Still if an exploit would work, it will only work after having filled gigabytes of log files, giving a change to system administrators to take an action on time.

The cons of the "barriers" approach is that it cures the implications of a problem, not the cause. The bug is still there, but instead of being exploitable to execute arbitrary code, it crashes the process (eventually immediately restarted with a tool like Monit or Supervise).

The OpenBSD auditing approach aims at curing the bug itself, thus not causing any crash.

Both approaches are actually complementary, but still not 100% efficient.

The only way to make reliable and secure (even from a theoric point of view) is to prove the code. Unfortunately it's not a trivial task and it can't be made upon an existing unix-like base.

But if you never heard about it, have a look at the very promizing EROS Project :
http://www.eros-os.org/

Re:Time for better security.

[bahamat (187909)]

I don't mean to burst your bubble, bash Theo or OpenBSD, but I read Bugtraq daily, and I can't count the number of exploitable bugs reported in the OpenBSD kernel over the past few weeks, but it would probably take both hands and at least one foot. Linux however, iirc, had somewhere about 3. Given this info, I wouldn't be so hot to boast about OpenBSD's QA process.

I repeat, I'm not trying to bash OpenBSD. Just trying to bring a little balance to the arguement. OpenBSD is an excellent choice for an operating system, but it's designed by humans. Humans make mistakes.

The real flaw in what happened with this exploit is that there were no backport patches created. When the ptrace vuln came out I was able to patch my 2.4.19 and 2.4.20 systems right away, I didn't have to wait for the next release to come out in order to get a working fix.

This should trun into a plea to the developers, if a bug is discovered through the normal course of development that is potentially serious enough for older kernels, it should be brought out into the open and the fix backported.

Re:Time for better security.

[jemfinch (94833)]

While OpenBSD has not shared in the commercial success of Linux, it does have one area of technical superiority: its security review process has yet to permit a remote root compromise in a standard install.


You know, when I install a computer, I consider the "install" to be the files installed on my hard drive. "/bin/ls" is part of the "install." "/sbin/ifconfig" is part of the "install." "/usr/libexec/ftpd" is part of the "install."

Apparently, however, the OpenBSD developers disagree. Something has to actually be running to be part of the "install." Thus, by their definition, none of those programs are part of the "install."

Good thing, too. Because /usr/libexec/ftpd has sure as hell [openbsd.net] had a remotely-exploitable root hole.

Jeremy

Re:Time for better security.

[wirelessbuzzers (552513)]

While I agree with your point, OpenBSD's numbers are a bit skewed. For one thing, there has been one remote root compromise, not none.

Second, that "standard install" has most of the features turned off... No Apache, etc... I don't even know if SSHD is on by default. I mean, they could have zero remote root compromises if their standard install didn't include network drivers.

I know that OpenBSD can't possible comb every line of apache and all the other contrib software ten times over, but this would be a problem for the Debian folks too.

Re:Time for better security.

[keesh (202812)]

This wasn't a remote root exploit. It was a local root exploit, and OpenBSD has had several of those.

The kernel patch...

[lurp (124527)]

Here's the kernel patch that fixed the overflow in brk():
http://linux.bkbits.net:8080/linux-2.4/diffs/mm/mm ap.c@1.32?nav=cset@1.1148.2.2 [bkbits.net].

The patch is from 9 weeks ago... I wonder if the exploit writer got the idea from looking at the kernel changelog...

Re:The kernel patch...

[pclminion (145572)]

It's only "gory detail" to those who are capable of reading the code: i.e., the crackers. The entry for that patch in the ChangeLog basically reads: "Bounds checking on do_brk()". Only a programmer will recognize that this is a security problem, and the ChangeLog entry is vague and doesn't explain the importance of the change.

If fixes are made which affect security, the ChangeLog should clearly spell out that it was a SECURITY fix. I guess people don't want to admit that they have found a security problem...

There goes my Saturday

[mariox19 (632969)]

I had just convinced myself there was no compelling reason to upgrade my kernel from 2.4.22.

Actually, there still isn't, since the likelihood of my machine "coming under attack" is slight. But, what's the point of running Linux if you're not going to get all worked up over things like this ;-)

Happy make menuconfig to all!

so are other distros possible infected?

[liquidpele (663430)]

The people at debian caught on, but what about at other distros? Have they made sure that their machines havn't been exploited and no trojan type code was introduced?

I'm expecting so, but just a "yes, it's been taken care of" would be nice...

Re:so are other distros possible infected?

[Nothinman (22765)]

This is a local exploit, it can't be used until you have a shell. Debian had the misfortune of having a developer's password stolen and used to get a shell, other distros would have to have similar problems with either passwords or network accessable services.

This isn't a special situation, everyone should be checking the integrity of their servers periodically anyway.

The real question remains....

[RedHat Rocky (94208)]

Any word on the parties behind the attack?

The question remains, who is targetting Open Source? With this being the latest in several high-profile attacks, the evidence would suggest a determined effort is under way to put egg on the Open Source face.

Who? Why?

Up 107 days...

[jehreg (120485)]

kc grub # uptime 17:21:06 up 107 days, 22:45, 1 user, load average: 0.35, 0.82, 0.47

Great..... there goes my uptime.....

If I have to reboot more than once per year, I'm switching to Windows.

Mysteries of good system administration

[Pflipp (130638)]

What I find intriguing is that those fine folks at Debian have come this far at detecting the exploit and tracking down the who's and why's (with the who's still being left undecided for the public, anyway).

Honestly, if I were smart enough to sniff a password, I'd also be smart enough not to let anyone know I've sniffed. Still, the folks at Debian were able to blame the unpriviledged account part on a sniffed password. Now how do you gain evidence for something like that?

Likewise, if I'd be smart enough to gain local root access by flipping the kernel, I would also be smart enough to ditch the binary with which I did that. Nevertheless, though after a thorough research, the Debian team has found the binary and managed to understand its potentials.

But still, what intrigues me the most is that they have found out that they were hijacked in the first place. Now I have a rock solid system for that at home, which is an 8 Mb RAM Sparc Classic, which starts to trash so hard at the least of activity, that I would well be alarmed if someone else than me was using that machine for whatever purposes. But as I may assume that those Debian machines weren't that low-end, how could you ever expect to know when you have been exploited?

RTFA

[Stradenko (160417)]

Recently multiple servers of the Debian project were compromised using a Debian developers account and an unknown root exploit. Forensics revealed a burneye encrypted exploit. Robert van der Meulen managed to decrypt the binary which revealed a kernel exploit. Study of the exploit by the RedHat and SuSE kernel and security teams quickly revealed that the exploit used an integer overflow in the brk system call. Using this bug it is possible for a userland program to trick the kernel into giving access to the full kernel address space. This problem was found in September by Andrew Morton, but unfortunately that was too late for the 2.4.22 kernel release.

This bug has been fixed in kernel version 2.4.23 for the 2.4 tree and 2.6.0-test6 kernel tree. For Debian it has been fixed in version 2.4.18-12 of the kernel source packages, version 2.4.18-14 of the i386 kernel images and version 2.4.18-11 of the alpha kernel images.

Re:How did they get in to run a userspace util?

[g1zmo (315166)]

I believe an earlier article said that it appeared that he sniffed a password to the box.

Or perhaps "she" sniffed a password?

I refuse to believe that the really hot, Debian-using, password-sniffing, root-exploiting geek girl is a myth.

Re:Hurray for the Debian Security Team!

[__past__ (542467)]

Hats off to the Debian Security Team.

And to the RedHat and SuSE security teams for helping them to track it down. In other words, hats off to the whole Free Software Community for collaborating when desaster strikes.

Re:Hurray for the Debian Security Team!

[Troed (102527)]

The bug had been found by Andrew Morton before, and was already fixed in 2.4.23. Thus, it wasn't unknown. It might even the because it was known that it was exploited aswell, I assume.

Quoting the Bugtraq post:

This problem was found in September by Andrew Morton, but unfortunately that was too late for the 2.4.22 kernel release.

Agreed

[DenOfEarth (162699)]

I agree with you totally. It's one thing to say that Linux is rock-solid secure, but in the real world this just might not always be true. It is however, a good thing to be able to say that the parties concerned with this particular security breach have been forthcoming to the community. A large part of security is just that. Hats off to the debian people.

Re:How does this compare...

[EverDense (575518)]

This story is about how great the Open Source Community is for fixing an exploit. The Microsoft story was about how incompetent Microsoft is for having an exploit.

Actually the Windows story was about how Microsoft had not patched an exploit they had known about for months.

This Linux exploit had ALREADY been patched.

Re:This smells like the work of...

[Overly Critical Guy (663429)]

Every time there's some sort of compromise on a high-profile Linux network, some idiot tries to implicate Microsoft on the basis of idle speculation.

Why it continues to be modded up as "Insightful" every time, I'll never know. Honestly, what insight was gleamed by this post?