Rust-Written Linux Scheduler Continues Showing Promising Results For Gaming (phoronix.com) 40
"A Canonical engineer has been experimenting with implementing a Linux scheduler within the Rust programming language..." Phoronix reported Monday, "that works via sched_ext for implementing a scheduler using eBPF that can be loaded during run-time."
The project was started "just for fun" over Christmas, according to a post on X by Canonical-based Linux kernel engineer Andrea Righi, adding "I'm pretty shocked to see that it doesn't just work, but it can even outperform the default Linux scheduler (EEVDF) with certain workloads (i.e., gaming)." Phoronix notes the a YouTube video accompanying the tweet shows "a game with the scx_rustland scheduler outperforming the default Linux kernel scheduler while running a parallel kernel build in the background."
"For sure the build takes longer," Righi acknowledged in a later post. "This scheduler doesn't magically makes everything run faster, it simply prioritizes more the interactive workloads vs CPU-intensive background jobs." Righi followed up by adding "And the whole point of this demo was to prove that, despite the overhead of running a scheduler in user-space, we can still achieve interesting performance, while having the advantages of being in user-space (ease of experimentation/testing, reboot-less updates, etc.)"
Wednesday Righi added some improvements, posting that "Only 19 lines of code (comments included) for ~2x performance improvement on SMT isn't bad... and I spent my lunch break playing Counter Strike 2 to test this patch..."
And work seems to be continuing, judging by a fresh post from Righi on Thursday. "I fixed virtme-ng to run inside Docker and used it to create a github CI workflow for sched-ext that clones the latest kernel, builds it and runs multiple VMs to test all the scx schedulers. And it does that in only ~20min. I'm pretty happy about virtme-ng now."
The project was started "just for fun" over Christmas, according to a post on X by Canonical-based Linux kernel engineer Andrea Righi, adding "I'm pretty shocked to see that it doesn't just work, but it can even outperform the default Linux scheduler (EEVDF) with certain workloads (i.e., gaming)." Phoronix notes the a YouTube video accompanying the tweet shows "a game with the scx_rustland scheduler outperforming the default Linux kernel scheduler while running a parallel kernel build in the background."
"For sure the build takes longer," Righi acknowledged in a later post. "This scheduler doesn't magically makes everything run faster, it simply prioritizes more the interactive workloads vs CPU-intensive background jobs." Righi followed up by adding "And the whole point of this demo was to prove that, despite the overhead of running a scheduler in user-space, we can still achieve interesting performance, while having the advantages of being in user-space (ease of experimentation/testing, reboot-less updates, etc.)"
Wednesday Righi added some improvements, posting that "Only 19 lines of code (comments included) for ~2x performance improvement on SMT isn't bad... and I spent my lunch break playing Counter Strike 2 to test this patch..."
And work seems to be continuing, judging by a fresh post from Righi on Thursday. "I fixed virtme-ng to run inside Docker and used it to create a github CI workflow for sched-ext that clones the latest kernel, builds it and runs multiple VMs to test all the scx schedulers. And it does that in only ~20min. I'm pretty happy about virtme-ng now."
Re:So for the 1% of Linux users... (Score:5, Insightful)
.. that actually use Linux for gaming, Rust is a big win!
The scheduler works for gaming because of the algorithm, not the implementation language.
it could've been implemented just as well in either Rust or C++.
Re: So for the 1% of Linux users... (Score:3)
I think something like this would be a hell of a lot easier to do in rust. Concurrency in C++ is very much an after-thought, and it shows. C++ makes it easy to do it the wrong way, and cumbersome to do it the right way. Rust is the exact opposite, and even if you do it wrong, the compiler will tell you exactly why it's wrong, which you fix until it can compile. C++ will just compile it anyways, you'll find out it's wrong at runtime, and then you'll have to whip out a debugger to step through it until you fi
Re: (Score:2)
Concurrency in Rust is fairly painful, actually, but like many things you do in Rust, once you get it working it's got a good chance of being reliable, efficient, and extensible.
Re: So for the 1% of Linux users... (Score:2)
Painful how? All you have to do is add the async keyword and yield where necessary. That's it. No callbacks or anything. If you've already got an intuitive understanding of the borrow checker, everything else will just fall right into place. I do this all the time.
One thing I've noticed about seasoned C++ developers is they never like to see a anything get mutated concurrently because they're so used to shit breaking. They tend to freak out and say to use a database if you need to do that. Database locks ar
Re: (Score:2)
One thing I've noticed about seasoned C++ developers is they never like to see a anything get mutated concurrently because they're so used to shit breaking
You meant to say "unskilled" C++ developers.
Re: (Score:3)
I am aware. But that is not what the story pushes.
Re: (Score:2)
The article is about a performance increase. Do you not like those?
Re: (Score:3)
Apparently you are easily confused. It was a thinly veiled article about how great Rust is.
Re: (Score:2)
it could've been implemented just as well in either Rust or C++
Well you're right except for C++. There's a funky middle ground between embedded and user-application that if you write it in C++, it'll suck ass. This is why Linux kernel development with C++ is mostly a non-existent thing except for a few masochist out there. And it's not that C++ is a bad language, it's just a bad standard that makes low level dev that has to interact with non-C++ stuff a pain that's never done correctly the first 183 times.
Re: (Score:2)
And what works well for gaming is likely to work well for other real time tasks such as video calls, sound synthesis and many others.
Re: (Score:3)
It's kind of a chicken or egg question. There aren't a lot of people using Linux for gaming because Linux has a pretty bad reputation as far as game compatibility and performance goes. If the reputation improves, more might move from Windows to Linux.
Re: (Score:3)
I don't know. People used to tell me that WOW ran faster on Wine than on native Windows. I didn't have an account to benchmark, but word of mouth makes me suspect that we're both hearing contradictory things. That hardly makes it a "reputation"
Re:So for the 1% of Linux users... (Score:5, Informative)
Steam Deck runs Linux, and performs very well running games. The reputation of Linux on gaming is improving rapidly, and deservedly.
The main issue with games on the Steam Deck is usually using a controller to play. Lots of Windows games do not handle that well, unfortunately.
Re: (Score:2)
Lots of Windows games do not handle that well, unfortunately.
That's just not the case, the overwhelming majority of games play perfectly on controller, which should come as no surprise since a majority of indie games are arcade style or simple enough to require few inputs, and the majority of AAA titles are cross platform with xbox and playstation.
Honestly I've not had a controller issue this decade and invested in a controller for windows gaming a long time ago. Sure I'm not playing a FPS with one, although I could, but it's amazing how well many things do work with
Re: (Score:2)
That is most definitely the case. I play mainly tactical and strategy games, and even the ones that have console ports which work fine with controls on console will often have really lousy controller support in Windows. As you note, indie titles are the common exception.
Just because a title is cross platform doesn't mean the control scheme has been well done in the Windows port.
Often the better choice on the Steam Deck is to use the Switch version in Yuzu instead of grappling with the control scheme of the
Re: (Score:3)
It's kind of a chicken or egg question. There aren't a lot of people using Linux for gaming because Linux has a pretty bad reputation as far as game compatibility and performance goes. If the reputation improves, more might move from Windows to Linux.
Linux makes up 2% of the gaming world now according to Steam's Hardware survey. For the record that roughly matches Windows 7, Windows 8 and all versions of MacOS combined. It doesn't sound like much, but it translates into literally several million users.
Your reputation is about 5 years out of date. I highly recommend you give Linux gaming a go. Between Wine and Proton you have a huge library available to you the majority of which runs perfectly.
Re: (Score:2)
Your numbers are incorrect, MacOS is just over 5.5% of Steam users, so over 3x the number of Linux users.
Source: https://www.statista.com/stati... [statista.com]
Re: (Score:2)
Your numbers are incorrect, MacOS is just over 5.5% of Steam users, so over 3x the number of Linux users.
Source: https://www.statista.com/stati... [statista.com]
Why would you post some second rate source like Statista when the actual numbers are available from the actual source? You think Statistia numbers form September somehow are more valid than the actual Steam hardware survey from December? The Steam Hardware Survey puts MacOS (all versions combined) at 1.63% as of a couple of days ago, feel free to go check it out yourself.
Incidentally the number you quoted was for MacOS 13.5.2 apparently a version that doesn't even show up in the Steam Hardware Survey in De
Re:So for the 1% of Linux users... (Score:4, Interesting)
I'm pretty sure there isn't a magic scheduler algorithm that only works for games. It's likely that other similar workloads would see improvement too.
Re: (Score:3)
I'm pretty sure there isn't a magic scheduler algorithm that only works for games. It's likely that other similar workloads would see improvement too.
This reminds me of OSX feeling snappier than Windows, even on much less powerful hardware. I'm sure part of that is cultural--Mac developers and users tolerate less latency--but part of that could be the scheduler. From the individual user perspective, it would be great if one could turn on a scheduler with improved interactive responsiveness, and when the computer needs to be used as a workstation a more traditional scheduler could be swapped in. Eventually it could be switched automatically with a meta-sc
Re:So for the 1% of Linux users... (Score:5, Insightful)
One day you just might learn Steam Deck is a thing...
Re: (Score:2)
One day gweihir may learn? Your comment isn't insightful, it's outright funny. :-)
Re: (Score:3)
I think you reacted a bit too fast. This scheduler was not particularly intended for gaming; it was a fun project over the Christmas break. The final benchmark showed it was comparable to C, even faster than C in one of the benchmark scenario, which was gaming. In any case the main lesson is that Rust performs adequately even for the most performance-critical parts of the kernel.
Re: (Score:2)
Since when is a _scheduler_ performance critical? It is critical for performance, but unless you go full queuing theory, it is hardly a performance intensive task.
Re: (Score:2)
It's not the scheduler itself that is performance critical, but it's how the scheduler selects the next task to run that's performance critical.
The scheduler is an extremely vital part of the kernel, and often the performance of it can help determine at what point does the system start to fall over and become unresponsible. Some schedulers are ama
Re: (Score:2)
Yeah sure. But the posting I responded to made speed claims about the scheduler itself, like that was an important thing.
Re: So for the 1% of Linux users... (Score:2)
There are scheduling algorithms that do a better job picking next task but spend so much time analyzing the available choices that they burn down the advantages of the better choice. So performance of the scheduler itself is something to at least check for when claiming an improvement in the performance of the tasks.
Re: (Score:2)
The default scheduler is not for gaming (Score:5, Interesting)
So, how does that special-interest user-space scheduler compare to a scheduler actually optimized for the same use case?
What about SteamOS? (Score:1)
I wonder what SteamOS uses? It's a Linux distribution where the primary task is gaming. Maybe SteamOS could benefit from a scheduler that is built with gaming in mind.
So is this a Rust story or a Linux Scheduler Story (Score:3)
Is there something inherent to Rust here, as opposed to other languages, that makes this scheduler possible? If not, then it isn't really a Rust story. The dev just happened to use Rust instead of C or C++.
Re: (Score:3)
I see it a a "Rust within linux kernel" story. As you point out, it does not seem there is anything particular about the Rust implementation; but there also does not seem to be anything special about this scheduling algorithm. The most notable point is that the Rust integration designed by the linux kernel team and provided by the compiler chain is now good enough to make this hybrid combination work and perform well. It's evidence that Rust is now usable beyond a hello world keyboard driver.
Re: (Score:2)
It uses sched_ext (I have several sched_ext projects ongoing for different network processing tasks).
The in-kernel portion is BPF (written in C via bpfc, or direct BPF assembler).
Presumably this has the BPF component receiving instructions form a user-space component written in Rust.
Re: (Score:2)
Re: (Score:1)
Yes, there is nothing special by using Rust here.
In fact I wonder how much faster and more efficient it would run if it were written in C or C++. They (Rust developers) probably don't want to know the answer to that because they would have to admit something they don't want to.
Re: (Score:2)