Python in 2024: Faster, More Powerful, and More Popular Than Ever (infoworld.com) 45
"Over the course of 2024, Python has proven again and again why it's one of the most popular, useful, and promising programming languages out there," writes InfoWorld:
The latest version of the language pushes the envelope further for speed and power, sheds many of Python's most decrepit elements, and broadens its appeal with developers worldwide. Here's a look back at the year in Python.
In the biggest news of the year, the core Python development team took a major step toward overcoming one of Python's longstanding drawbacks: the Global Interpreter Lock or "GIL," a mechanism for managing interpreter state. The GIL prevents data corruption across threads in Python programs, but it comes at the cost of making threads nearly useless for CPU-bound work. Over the years, various attempts to remove the GIL ended in tears, as they made single-threaded Python programs drastically slower. But the most recent no-GIL project goes a long way toward fixing that issue — enough that it's been made available for regular users to try out.
The no-GIL or "free-threaded" builds are still considered experimental, so they shouldn't be deployed in production yet. The Python team wants to alleviate as much of the single-threaded performance impact as possible, along with any other concerns, before giving the no-GIL builds the full green light. It's also entirely possible these builds may never make it to full-blown production-ready status, but the early signs are encouraging.
Another forward-looking feature introduced in Python 3.13 is the experimental just-in-time compiler or JIT. It expands on previous efforts to speed up the interpreter by generating machine code for certain operations at runtime. Right now, the speedup doesn't amount to much (maybe 5% for most programs), but future versions of Python will expand the JIT's functionality where it yields real-world payoffs.
Python is now more widely used than JavaScript on GitHub (thanks partly to its role in AI and data science code).
In the biggest news of the year, the core Python development team took a major step toward overcoming one of Python's longstanding drawbacks: the Global Interpreter Lock or "GIL," a mechanism for managing interpreter state. The GIL prevents data corruption across threads in Python programs, but it comes at the cost of making threads nearly useless for CPU-bound work. Over the years, various attempts to remove the GIL ended in tears, as they made single-threaded Python programs drastically slower. But the most recent no-GIL project goes a long way toward fixing that issue — enough that it's been made available for regular users to try out.
The no-GIL or "free-threaded" builds are still considered experimental, so they shouldn't be deployed in production yet. The Python team wants to alleviate as much of the single-threaded performance impact as possible, along with any other concerns, before giving the no-GIL builds the full green light. It's also entirely possible these builds may never make it to full-blown production-ready status, but the early signs are encouraging.
Another forward-looking feature introduced in Python 3.13 is the experimental just-in-time compiler or JIT. It expands on previous efforts to speed up the interpreter by generating machine code for certain operations at runtime. Right now, the speedup doesn't amount to much (maybe 5% for most programs), but future versions of Python will expand the JIT's functionality where it yields real-world payoffs.
Python is now more widely used than JavaScript on GitHub (thanks partly to its role in AI and data science code).
Pushes the envelope for speed * (Score:5, Insightful)
* Compared only to previous versions of python.
Re:Pushes the envelope for speed * (Score:5, Informative)
Yes? They aren't claiming it's a speed demon compared to anything else.
It's not news to anyone in the python community that python is quite slow.
I'm glad they are really working on the speed of it. The GIL is good. Speeding up python with multiple cores is currently not much fun. Also the jit.
Re: (Score:3)
I'm glad they are really working on the speed of it. The GIL is good. Speeding up python with multiple cores is currently not much fun. Also the jit.
Speed is good. What I'd really like to see is that the packaging story gets sorted out. That seems to maybe be being sorted out with the new UV tool [saaspegasus.com] which is getting towards both being complete and acceptably fast unlike past tools which were either one or the other. UV is written in Rust which is not a problem but you fear that some people might make it one (for good reasons, Python itself is mostly not written in Python). Seeing the Python Packaging Authority start to update their tutorials to match with
Re: (Score:3)
Ah yes uv, and it's precursor rye. And also ruff.
Having tried them, I love them compared to the previous tooling, and the high speed is a huge boon.
I don't have any strong opinion on rust, native code is good for fast things and rust appears to be a reasonable choice. I hope it sticks, there has in the past there has been a strong push for "pure python" in the python world. I appreciate the desire to eat ones own dog food, but it made a lot of the ecosystem glacially slow and for years there was very little
Re: (Score:2)
Steaming piece of shit language, IMO, lol. OK, not that bad, I just had a bad introduction and someone tried to write a production app in python 1, when it was still a shit programming language but not bad for scripting. I still hate white space significance, but 200 hours debugging a Makefile issue is entirely to blame (and yeah, modern editors would've found it in minutes, but vi on Solaris, not so easy, and no, emacs wasn't available, but don't get me going on emacs).
Re: (Score:1)
"...but vi on Solaris, not so easy, and no, emacs wasn't available, but don't get me going on emacs)."
Why is your solution the two oldest and shittiest editors on the planet?
Re: Pushes the envelope for speed * (Score:3)
Re: (Score:2)
Re: Pushes the envelope for speed * (Score:2)
Re: Pushes the envelope for speed * (Score:2)
Re: (Score:2)
"I have written bubble sorts for 10 to 20 items,..."
masturbates wildly.
"but if you are doing them for more than that you should probably be using something else."
So you think sorting 10-20 items is indicative of 95% of programming? Do you listen to yourself?
"In other words, a long running nested loop is probably indicative of you doing it won't"
Thanks for the other words, really cleared things up. Nice to hear your definition of a long running loop, not at all surprising.
Re: (Score:2)
For 10-20 items, I use the built-in sort function, and I don't concern myself about how it goes about doing it.
For millions of items, I dump them in a database, and I do a database query that returns the results sorted as required.
Re: (Score:2)
Re: (Score:2)
... Why not use the built-in function for millions of items?
Re: (Score:2)
One of the very first things they teach in interpreted programming is that if you're writing a loop, never mind a nested one, you're probably doing it wrong.
Re: (Score:2)
That's silly. If you have a set of data and you need to do something to each item, then you're going to have a loop of some sort somewhere. And if your data has multiple dimensions, you're going to have nested loops of some flavor in your code, even if they aren't the classic "for i in (level1) { for j in (i.level2) {} }" sort of thing. The looping always exists, even if it's hidden in a language's operator.
Re: (Score:2)
Yes. I didn't say the looping didn't exist. I said, very specifically, "if you're writing a loop, never mind a nested one, you're probably doing it wrong."
Emphasis added because apparently it was too subtle without.
You avoid writing loops in interpreted languages by:
(1) using a library (e.g. numpy) or a language feature (e.g. list comprehension, generator, map, etc.)
(2) writing your loop in a compiled language and calling it as a functi
Re: (Score:2)
Most programming is low level computing, you just don't know any better. And sped is a requirement for more than just that.
"In most real life cases, IO transfer speeds and latency is going to be your bottleneck anyway."
Complete bullshit. But at least we know that your programming is neither performant nor useful.
Re: Pushes the envelope for speed * (Score:2)
Re: (Score:2)
Re: Pushes the envelope for speed * (Score:1)
Re: (Score:2)
Because for them, that saves a lot of money. But the vast majority of programmers in the world do not work on Facebook. Or on any other company which has the kind of use case Facebook has. They're an outlier among a group of outliers, and what they need is not in any way representative for what the majority of companies need, and thus for what the majority of developers spend time doing.
Heck, even at Faceboook, work like this is an outlier. The majority of developers there won't ever touch this kind of code
Re: (Score:2)
Most programming is business logic, where speed is at best a distant fifth on the priority list. For every programmer doing low level work, there are dozens working at large corporations building and maintaining the internal systems that keep them running.
And for them, the times when speed matter, they will usually be IO bottlenecked.
Re: Pushes the envelope for speed * (Score:2)
Unless the c++ modules did exactly what the program needed there would be be a tonne of data marshalling back and forth. Often this was exacerbated by algorithms implemented by bolting together a long, convoluted, sequence of operations because the right operator wasn't available, or inevitably someone would loop through all the pixels directly.
My predecessor tried to solve that with a sort of visor pattern where
Re: Pushes the envelope for speed * (Score:3)
Re: (Score:2)
Re: (Score:2)
So you ended up using Python as a scripting language and C++ as your general purpose language.
Using Python as a general purpose language is like using pliers as a hammer, or a slot-headed screwdriver as a chisel. Effective, but not recommended.
Re: (Score:1)
That was true 30 or 40 years ago.
Re: Pushes the envelope for speed * (Score:3)
Compiling Python to machine code? (Score:2)
What are the options to compile Python to machine code? Wouldn't that be an obvious way to increase its performance? Or possibly translate it to C or something and then compiling the C?
Re: (Score:2)
Certainly a subset of python could be compiled into high speed code. Cython is an example. Other parts of the language are much harder to compile due to python's dynamic nature. There are projects to make self contained binaries but the speed up is minimal. Jit code generation in the interpreter is still the best ready to get a nice speed up.
Re: (Score:2)
Decently written Cython code can run as fast as hand written C, and efforts to improve the interpreter have more or less doubled Python's speed over the last few versions. Interpreter improvements aside, JIT is the most convenient, for the end user, way to get a speed up.
So nothing to fix performance (Score:2, Insightful)
"It's also entirely possible these builds may never make it to full-blown production-ready status, but the early signs are encouraging."
So they're experimenting and have no solution. Really great news?
"...JIT. It expands on previous efforts to speed up the interpreter..."
So you'll have the fastest version of shitty global lock single threaded crap.
Python's popularity is a mirage, the moment performance matters programmers must switch to a different language via a library.
Oh really? (Score:2)
The no-GIL or "free-threaded" builds are still considered experimental, so they shouldn't be deployed in production yet.
Who do you think you're talking to? You can't tell them what to do [imgur.com].
Specialized python compilers (Score:1)
There's also Mojo, for compiling python (and AI extensions) into high-speed binaries: https://www.modular.com/mojo [modular.com].
Re: Specialized python compilers (Score:2)
Yeah I haven't tried it yet, I wasn't sure how ready it was.
Re: Python is a poor language (Score:2)
If you don't like it, don't use it. I started on 6502 and 68k assembly, before C, and it doesn't take long to prefer white space over braces. And yes you can compile and distribute, I did it for HP recently.
Not impressed (Score:4, Interesting)
I've tested this against Python 3.12 with GIL, Python 3.13 noGIL, and Go 1.23.
TL;DR:
- Python 3.13 noGIL did better than Python 3.12 with GIL on 1 benchmark (pidigits) by about 30%
- 3.13 noGIL actually did a lot worse on binary-trees benchmark, and slightly worse on fasta benchmark
- Golang smoked it out of the water on every single test
Re: (Score:2)
I don't understand why you are surprised by this. The NoGIL version is expected to be slower, as is Golang.
NoGIL will probably never be faster than "regular" Python on most code. It is only when you have a lot of truly highly parallel (i.e.: little interaction between threads) that anyone is expecting an eventually speedup. The version that exists in the Python main-line now is mostly there to allow for the gradual shifting of some fundamental aspects of Python to allow for the eventual speed-up in that hig
I find the defendant (Score:2)
25 year career about to become useless (Score:2)
There are three times more Python jobs than PHP around here and nobody is going to hire a 45 year old for a Junior Python Developer position XD