Python is Getting Faster. How a Team at Microsoft is Helping (microsoft.com) 108
It's been one week since Python 3.11 was released — and it's "faster than ever!" So says Jay Miller, a Microsoft developer writing about Microsoft's six-person "Faster CPython" team (which includes Python creator Guido van Rossum, and offers assistance to other core developers). Miller cites the team's report that Python 3.11 has already seen speedups of 10-60% in some areas of the language -- and offers this inside look at their work.
First, how the team came together: In 2020, Core Developer Mark Shannon drafted an Implementation plan for speeding up CPython (the most common implementation) by five times. This plan proposed a 4-stage process that, as Python's creator Guido van Rossum says, "was an effort that was too much for one volunteer to accomplish".
"Right from the start, my thought was well, we should try to see if Microsoft can hire Mark and a small team of people to support him." In the previous year Van Rossum came out of retirement and joined Microsoft as a Distinguished Engineer. "It was an important effort and it was too much for one person." Microsoft was open to the idea and a team of 6 engineers, including Van Rossum were established. That team has assisted other core developers in acting on this plan.
But the blog post also looks at how the team functions: Every contributor that made the switch from part-time to full-time contribution mentioned being able to get deeper into their work on the language.... The team meets regularly to discuss these things. "All six of us meet every Monday," says Van Rossum. "There's always more than enough to talk about. That is very different than as a core dev community getting together for a Sprint twice a year, like one day after the conference. That is a very special event, of course, but it doesn't feed me throughout the year." Van Rossum believes that knowledge of one another and their collaborative work gave the team a "leg up" because everyone "knows what communication styles people have and what everybody's weaknesses and strengths are...."
Shannon's original 4 stage plan has continued to evolve to have continuous optimizations for the next several years. "To make that as smooth as possible, you have to think in terms of smaller steps, right?" says [team member] Michael Droettboom. Droettboom has worked on long-term projects in the scientific community including the Hubble Space Telescope and more recently the James Web Space Telescope.... "We hope we can bring some knowledge from really large proprietary systems into what we develop for the Community." says Droettboom. "I think that's really valuable because then you're not just doing it in the abstract. Not just imagining what's going to make Python faster for real use cases, but actually measuring it." [Team member] Brandt Bucher adds in that developers working with these teams can test the impact of changes, "getting useful insights and contributions from people who maintain large, diverse codebases...." Many of the team's meetings feature core developers from other teams and companies.
The blog post highlights specific activities of team members:
First, how the team came together: In 2020, Core Developer Mark Shannon drafted an Implementation plan for speeding up CPython (the most common implementation) by five times. This plan proposed a 4-stage process that, as Python's creator Guido van Rossum says, "was an effort that was too much for one volunteer to accomplish".
"Right from the start, my thought was well, we should try to see if Microsoft can hire Mark and a small team of people to support him." In the previous year Van Rossum came out of retirement and joined Microsoft as a Distinguished Engineer. "It was an important effort and it was too much for one person." Microsoft was open to the idea and a team of 6 engineers, including Van Rossum were established. That team has assisted other core developers in acting on this plan.
But the blog post also looks at how the team functions: Every contributor that made the switch from part-time to full-time contribution mentioned being able to get deeper into their work on the language.... The team meets regularly to discuss these things. "All six of us meet every Monday," says Van Rossum. "There's always more than enough to talk about. That is very different than as a core dev community getting together for a Sprint twice a year, like one day after the conference. That is a very special event, of course, but it doesn't feed me throughout the year." Van Rossum believes that knowledge of one another and their collaborative work gave the team a "leg up" because everyone "knows what communication styles people have and what everybody's weaknesses and strengths are...."
Shannon's original 4 stage plan has continued to evolve to have continuous optimizations for the next several years. "To make that as smooth as possible, you have to think in terms of smaller steps, right?" says [team member] Michael Droettboom. Droettboom has worked on long-term projects in the scientific community including the Hubble Space Telescope and more recently the James Web Space Telescope.... "We hope we can bring some knowledge from really large proprietary systems into what we develop for the Community." says Droettboom. "I think that's really valuable because then you're not just doing it in the abstract. Not just imagining what's going to make Python faster for real use cases, but actually measuring it." [Team member] Brandt Bucher adds in that developers working with these teams can test the impact of changes, "getting useful insights and contributions from people who maintain large, diverse codebases...." Many of the team's meetings feature core developers from other teams and companies.
The blog post highlights specific activities of team members:
- L Pereira is working on a change to how integers are represented inside Python, and "intends to change smaller integers to use native computation instead of the slower algorithms for arbitrarily large numbers."
- Irit Katriel implemented the new Exception groups and except* features in Python 3.11, and reports that "By simplifying the interpreter's internal representation of raised exceptions, I reduced the time it takes to raise and catch an exception by about 10%."
- Brandt Bucher (who helped create structural pattern matching for Python 3.10) is working on a Specialized Adaptive Interpreter (and tools like Specialist to help users move to Python).
- Long-time contributor Eric Snow is working on a per-interpreter Global Interpreter Lock.
And they've already begun working on features for future versions of Python. "You can also find out more about what the Faster CPython Team has in mind for 3.12 in their ideas repo on Github."
In before... (Score:5, Funny)
Re: (Score:1, Troll)
You trust MS now? They arguably suck less than Oracle, but that's not saying much.
Re:In before... (Score:4, Insightful)
Microsoft does stuff to benefit ... microsoft.
So you really have to ask yourself what's in it for them. Sometimes what benefits them may not benenfit the rest of us.
But i'm definitely in the WTF ? camp on this. I have no idea why M$ would be doing anything for Python, much less trying to make it faster.
So I'll go ahead and be suspicious as to why they are doing this. It has to benefit them, or they wouldn't, and I just don't see how this is useful for them.
Who knows, maybe some manager at microsoft will read about this and say "wtf ? we have people trying to improve python, who allowed that ?!", and pull them right off the project. stranger things have happened.
Re:In before... (Score:5, Informative)
Microsoft uses a LOT of Python internally.
Any improvement in performance results in millions in cost savings.
Re:In before... (Score:5, Informative)
Not just internally - MS are one of the bigger cloud providers with their Azure platform.
Re: (Score:2)
Making Python faster for users outside Microsoft does not save Microsoft money. Even if they are using it on Azure, making Python faster would mean customers using less compute and paying Microsoft less money. On the other hand if Python is faster Microsoft saves money because it uses it internally.
Re: (Score:3)
Even if they are using it on Azure, making Python faster would mean customers using less compute and paying Microsoft less money.
Not necessarily: if Python uses less compute, then potential customer projects that were previously not cost-effective (either directly because of the compute cost, or indirectly because they'd need to be written in a "harder" language than Python, costing more development time, in order to be efficient enough) can suddenly become viable, meaning customers spend more money with Microsoft.
To put it another way, making Python more efficient means that each "unit of CPU time" can do more useful work, and hence
Also slightly friendlier to run on Windows (Score:2)
or indirectly because they'd need to be written in a "harder" language than Python, costing more development time, in order to be efficient enough
Python has the advantage of being also one of the more Windows-friendly languages out-there. (e.g. most of the pypi dependencies install nicely on Windows, most of the dependencies management work nicely there too, python core libraries have enough abstration to run reliably on things which aren't POSIX, etc.)
Meaning that people running business app in Python on Azure are more likely to be developing it on a Windows Laptop.
Thus licence money for Microsoft for Windows, and any other Microsoft ecosystem they
Re: (Score:2)
Re: (Score:3, Informative)
Re: (Score:2)
It's a fair point that if you can't see the benefit for Microsoft, you'd need someone to fill in the blank because they aren't going to be utterly altruistic. People have stepped up to offer up why Microsoft interests are what they are and how they can be compatible with reasonable benefit for all to fill in that gap. So his gut reaction to be wary when he can't think of the Microsoft benefit is fine, so long as the responses are internalized.
Re:In before... (Score:5, Interesting)
So you really have to ask yourself what's in it for them.
Microsoft makes stupid amounts of money using open source both internally and in their product offering. Literally half of their largest revenue stream is selling people Linux in the Azure cloud.
When you have a WTF moment, remember that it's very easy to build on and sell someone's work than it is to re-invent your own. Even if the company isn't outwardly open source friendly.
Re: (Score:1)
Thanks! Now I understand what's in it for them.
I'm surprised they would use Python internally instead of C# or whatever it is we're all supposed to be used now.
Re: In before... (Score:1)
instead of C#
Just my guess: C# (and a lot of other stuff) was developed in an era of "first, make sure it helps us sell Windows." Insiders know this and steer clear of the built-in booby traps. Python started out being O/S agnostic.
The jury is still out on whether any Microsoft-specific hooks got slipped into the new Python, intentionally or otherwise.
Re: (Score:2)
I'm surprised they would use Python internally instead of C# or whatever it is we're all supposed to be used now.
I didn't know anyone selects a programming language for "internally" vs "externally". I always thought programmers select a language based on what it offers to solve the problem and conditions at hand. I learn something new every day.
Re: (Score:3)
If Microsoft is spending any appreciable amount of money for Microsoft on the project, then it is a bit confusing. What do they care if python is slow or not? In fact, isn't it a benefit if all this scientific computing which goes on is slow AF, since that sells more cloudy time?
But if they have only a few engineers on the project, then it's not a big surprise. Maybe Microsoft is beginning to see the OSS light, not in terms of benevolence but benefit. I wouldn't trust them not to practice further skulldugge
Re: (Score:2)
Re: (Score:2)
Well, I suppose if they've used a lot of it internally that's relevant, but otherwise who cares? They can just throw up their hands and say "eh, Python, what you gonna do? Pay me, bitches!"
So I suppose that does suggest that they in fact have done.
Why is Microsoft investing in the JVM too? (Score:2)
You mean besides making Azure more attractive? Why else would they invest in non-MS platforms and open source tech? Oh right... it's to make Azure more competitive with AWS.
Re: (Score:3)
Embrace/Extend/Extinguish only works if your code remains closed. That cannot happen here. If all we have is speculation, then I'd speculate that MS is doing this to remain compliant.
Re: (Score:2)
A lot of Azure has Python examples. And a lot of python is run in Azure (especially there machine learning and AI services.) So it is in their own best interest to make python faster.
Re: (Score:2)
Microsoft makes money selling Visual Studio and has an invested interest preventing Linux and MacOSX to be a development platform. Hence why WSL came to be as well as great Linux support in Azure and Hyper-V.
I hate Apple and tried a mac but if I could not use Linux as my desktop at work 4 years ago I would have to use Apple to test things like GitHub and testing stuff for Ubuntu and ssh integration. Fast forward today Windows 11 can run run Linux gui apps and terminal actually doesn't suck. So much so I can
Re: (Score:1)
Did I make it in before all the dreary tech dinosaurs chime in with "embraceextend exteenguissssh!!!
Oh shit you beat me! I was going to complain about Micro$oft or Micros~1 depending on my mood.
Anyhow
You didn't however make it in before the dreary tech dinosaurs mindlessly ragging on python because python si teh sux etc. It's almost like people here have endless hours to craft C code full of memory errors and don't have a product to ship.
Re: (Score:3)
All languages suck, so that's tautologically true.
Also, python isn't my favourite language. I don't love it but it's fine. I'd certainly rather program in Python than not program at all.
Re: In before... (Score:2)
You know what they say about ignoring the lessons of history? Trusting those proven to be untrustworthy is generally a mistake.
Re: (Score:2)
While I like the joke as FP, the discussion seemed to completely miss the point. At least I couldn't find the obvious explanation of what is going on here...
A focus on speed means cutting out the glue code that makes things more flexible but slower. Alternatively, you could describe this as Microsoft's ongoing attempt to glue "fast" Python to Microsoft's other software, especially to the Windows OS.
GIL (Score:1)
So when are they getting rid of GIL? Still find it hard to believe people actually try to justify this crap.
Re: (Score:1)
Just use Jython on JVM, no GIL, and all the fun of running a bloated JVM
Re: (Score:1)
running a bloated JVM
GraalVM fixes that. Seriously.
It's weird making stuff in Java with memory usage that looks like Go.
Re:GIL (Score:4, Informative)
Five years [backblaze.com], estimated. (Scroll to the bottom.)
Re: GIL (Score:2)
What's your problem with GIL? Python has dead simple multiprocessing and IO is multithreaded. I don't see much value in full end2end multithreading support.
So far we given up too much single-threaded performance for multithreaded features. The later of which is a niche use case. If/When nogil comes online, I fully expect it to have a on/off flag.
I would much rather efforts be spent on Pipe, Queue, SharedMemory performance for multiprocessing. Or maybe add other useful classes.
Re: GIL (Score:2)
It's not an excuse. You are dismissing 30 years of review on the topic as if all those people, many PhDs in Computing, are fools. Sorry if we take your bitter opinion with a grain of salt.
Your use case is really small... almost insignificant in the overall scheme. I doubt anyone is even working on a Python solution to your problem. And that's if I assume your use case isn't performant enough for SharedMemory or Pipes but still is for using an interpreted language.
And yes, compared to most other languages
Re: (Score:2)
That's not how "appeal to authority" works... It wasn't a "Because PhDs so you are wrong." counter. Anyway, my point was to say you are going up against an acamedic stack and you need more to support your side of the discussion rather than just dismissing it as some fool's mistake. Which is all you basically done so far with a "But I have a use case..." thrown in. See the other guy about C extensions for a good response.
You don't need to guess how the GIL came to be or why it is still around. You can re
Interpreted language == derp (Score:3, Interesting)
Hurray! A faster way to justify getting all that awful Python code into prod (and watch it blow up)!
We've all certainly learned a lot from the ye olde days, and Python has a lot of modern aspects that promote dev happiness, but it's still just another interpreted language. My team has to run 2 to 4 different linters against their Python source in CI, hoping to ensure it won't accidentally blow up once deployed, and it takes far longer to certify syntactically than any compiled language I've ever used. The Go compiler absolutely destroys Python linters in terms time to complete "compiling". Also, our Go CLIs have a much better user experience compared to the same functions in Python, in terms of execution speed.
Python does still have some utility, but I do think its been banging against the doors of maximum-effective-use-case for a while now. At least where I'm at, I feel like Python has been carrying more water than it should, and I think we should be spending our time on improving Go rather than shoring up Python.
Re: (Score:1)
You're confusing interpreted languages with dynamic languages. CPython has always compiled down to an IL at run-time. Its slowness comes from its dynamic nature.
Re: Interpreted language == derp (Score:1)
You undermine any argument you make with this ridiculous understatement.
Re: (Score:3, Insightful)
Python is just fugly. White space significance sucks. Just uses f'ing braces like any half decent language. Not to mention the syntax, and function names, are like something from VB 5.0.
I have to deal with Python on a regular basis and it's just plain ugly. Horrible abbreviated, inconsistent funciton names just encourages people to start using horrible abbreviated variable names and the whole thing becomes barely legible crap.
Yuck...
Re: (Score:1)
Compared to the so called "one true brace style" the python solution seems almost elegant. Allman got it right.
Re:Interpreted language == derp (Score:4, Insightful)
You're confusing the facilities offered by a language with the way people use it. Significant whitespace is an abomination and leads to context errors. Some brace styles might be ugly but having visible context block demarcators whether braces or words is a Very Good Idea which is why almost every mid and high level programming language ever invented uses them.
Re: (Score:2)
Re:Interpreted language == derp (Score:5, Insightful)
"Only thing that braces add is to make it possible to have misleading indentation without compile errors"
Better misleading indentation than bad logic. You can delete whitespace in python by mistake which moves the line into the outer scope and python will not say a word about it - your code just won't work and good luck finding the bug. Now try deleting a brace in a C program and see how well it compiles.
Re: (Score:2)
You can delete whitespace in python by mistake which moves the line into the outer scope and python will not say a word about it - your code just won't work and good luck finding the bug. Now try deleting a brace in a C program and see how well it compiles.
I don't see much difference. Most likely in python such an error would be easily detectable because different scopes have different local variables. And in C it would cause really long error cascade because missing brace would throw off the parser real good. Not much of a problem though if you know to check errors from
Re: (Score:2)
Re: Interpreted language == derp (Score:3)
Yup I used to joke the best way to ruin a python program was to quickly do a select-all, dedent,dedent,dedent, save. Then lose undo history.
Any real language can survive that.
Re:Interpreted language == derp (Score:4, Insightful)
> Most coding conventions make you indent code properly anyway.
That depends on the language. By convention some (most?) people DON'T (heavily) indent the preprocessor in C/C++. This "TWO COLUMN" mode helps make debug-only code MORE readable since you decide which "column" to focus on when reading:
e.g.
> You could as well consider proper indentation mandatory at which point having both indentation and braces/marker keywords becomes a violation of DRY principle.
No, for three reasons:
1. Define "proper" indentation because there are always edge cases where people will argue over what is "proper".
2. There are many operations which are "dual" operations. Push/Pop, Startup/Shutdown, Open/Close. Indentation helps provide visual flow control especially when there are nested operations. Python's retarded indentation dogma destroys this communication:
Pythons pretends everyone is an idiot and makes them work harder due to some bullshit dogma:
There ARE times to have braces, and there are times NOT to have braces. There ARE times to indent, there are times NOT to indent, and there are times to EXTRA indent. An expert knows when to follow and when to break Rules-of-Thumb.
3. Extra whitespace may help with readability to see the "flow" of variables. Consider this swap (ignore the inefficient ctor for tmp):
i.e.
Making whitespace change semantic meaning is absolutely retarded. We write code for people, not compilers. To every rule there is almost always an exception.
Re:Interpreted language == derp (Score:4, Insightful)
Your "dual operation" code could be nicely implemented as a context manager in Python, which would give you back your block indent of a group of operations, and even save you a line of code potentially, as the "close" part of the operation is handled in the context manager.
Re: (Score:2)
Says the guy confusing terrible hack with good idea.
Should block demarcators be visible? Why yes - python has them. It's called whitespace, and it's perfectly visible. Different blocks start in different columns. Very easy to spot.
Should a language have two visible block demarcators? No, that's silly. Redundancy creates confusion. Brace and whitespace should not both be allowed to demarcate blocks. Likewise, spaces and tabs should not both be allowed. Python avoids this with -tt to prevent mixing
Who the whitespace haters are (Score:2)
As someone who works a lot in Python, C/C++ and R (and so goes back and forth from bracket- to whitespace-delimited), I have never felt very strongly on the whitespace issue. On the whole I guess I kind of like it.
I have wondered what type of coder get's all riled up about whitespace-delimited blocks....
...The only knock against whitespace is silly programmers expecting to cut and paste code directly from whitespace-munching systems like the web. ...
..and I think you have identified a significant population of such coders.
Re: Interpreted language == derp (Score:3)
I used to hate white space significance. Then I kind of realized the white space it insisted on is exactly what I do anyway. It different, that doesn't make it bad. It does make it easier to learn.
Re: (Score:2)
Python is very good for stuff where performance isn't all that important. A great example is Ansible. There would be no gain whatsoever from converting that to Go or some other compiled language. It being Python has the added advantage that it's easy for people to add their own modules to it.
Re: (Score:2)
Ansible is maddeningly, frustratingly slow. Some of that slowness may be intrinsically necessary due to what it does, but I'm surprised gathering facts takes more than a second. But everything is slow. (I use it anyway when I can. It's worth the frustration.)
Re: (Score:2)
That's basically because the methodology tends to be a slow way of doing things. It's spending most of it's time twiddling it's thumbs compute wise. Just a lot of waiting and serialized execution of commands that in turn spend most of their time waiting. So a hypothetical C implementation of the exact same design would still be maddeningly slow. One difference is that a python developer *might* be more intimidated by C interfaces (they can use through one of several FFI facilities in python) and resort
Re: (Score:2)
Does speed really matter with Ansible? You start it off and go do something else while it works. It's not an interactive process.
Re: (Score:2)
You could say the same of compiling, rebooting, system updates, downloads, etc. Yes, speed matters.
Re: (Score:2)
Python should have been somewhere in between a shell script and JSON, and used solely to tie disparate things together. Because of its poor performance it's not even great at that job (perl is approximately four times faster at mangling data, which is something that you often want to do when connecting programs together) but its pretty indentation makes it a close cousin. Except, you know, JSON is basically comma-structured and can be regenerated on that basis, unlike Python :)
Instead people have developed
Re: (Score:2)
Sounds like you're lacking good system testing before pushing to production. And complaining that the issue they're fixing (performance) is a reason not to fix it.
I'm happy you like Go. I've heard good things about it, but there is a gravity/momentum to python that Go hasn't overcome. Maybe you can identify it and help change things? I don't think complaining that people use and work on python is the solve here.
I mean people love Rust too. And I'm sure many other languages. People love what they're go
How long... (Score:1)
Before Py# is released as a new Microsoft language?
Re: (Score:2)
They already did. It's called IronPython.
https://ironpython.net/ [ironpython.net]
They just brought in Jim Hugunin who wrote Jython. This was in 2006.
Re: (Score:2)
Is it fully compatible with CPython. Because I have tried a few of these "python but faster" projects. And in all cases I tried there were slight incompatibilities; sometimes that were not detected until regression test showed that the two "interpreters" were not doing the same thing.
Re: (Score:2)
> I have tried a few of these "python but faster" projects
The only "python but faster" project is PyPy. Jython and IronPython are meant to work within Java and .NET ecosystems. Speed was not the point for them, although they usually are, as long as you were not using CPython extensions. They all pass the respective Python test suites for the versions they target.
Java int (Score:2)
Sun knew why they didn't make int an object...
It could hardly get much slower (Score:3)
According to this guys tests NON optimised C code ran 45 times faster than the same code in python. Optimised? Well you check it out:
https://peter-jp-xie.medium.co... [medium.com]
Sure, it'll vary depending on algorithm but others have similar results and Daves Garage channel on youtube did a whole series on comparing language speeds and python didn't do too well overall compared to pretty much anything.
Re: (Score:1)
.L3:
addl $1, -8(%rbp)
addl $1, -4(%rbp)
.L2:
movl -4(%rbp), %eax
cmpl -12(%rbp), %eax
jl
Re: (Score:2)
And if you knew anything about Python, you'd know that for use cases where speed is the primary factor, you can stub out to C and back. That's exactly how some of the more powerful libraries run - I've done a lot of code for the Python Imaging Library for example where that's done, including true multi-threading support.
So - bash all you want, it just shows you don't know when to use a hammer versus a screwdriver.
Re: (Score:2)
If you're going to stub out to C you might as well write the whole lot in C or better yet C++ in the first place unless your clientelle is halfwit script kiddies who wouldn't know one end of a pointer from the other.
Re: (Score:2)
Sure.
And you can reinvent hashtables, rudimentary string handling and an object lifetime strategy in C for the 100,000,000th time while you're at it. It's so fun!
Then you can buy a bunch of expensive 3rd party tools to try to identify all of the security vulnerabilities you just authored. With all the performance you'll gain running one-time setup code 50X faster, surely it will pay for itself!
Re: (Score:2)
"And you can reinvent hashtables, rudimentary string handling and an object lifetime strategy in C for the 100,000,000th time while you're at it. It's so fun!"
Which is why I said C++. Though FWIW C has the hsearch library for decades.
"Then you can buy a bunch of expensive 3rd party tools to try to identify all of the security vulnerabilities you just authored"
Sure, or alternatively use the ones that come for free either with Visual Studio or on the linux command line.
I'm guessing you've never coded in eithe
Re: (Score:2)
I have decades of experience with C and C++.
Familiarity breeds contempt, especially with crap languages from 1990.
Re: It could hardly get much slower (Score:2)
Good thing C++ is updated every 3 years then. I suspect your decades of experience relate to spelling them, not using them.
Re: (Score:2)
You seem like one of those people who have never bothered to learn any language beyond C++ since 1990, so you think that it's just wonderful now.
Hint: They've papered over a few syntactic sore points, but it remains nearly as full of footguns as ever.
Re: It could hardly get much slower (Score:2)
I can program in python 2. It's fine for toy projects and quick and dirty stuff that would be too much hassle in shell. Tried java, was just a poor mans C++ and memory hog. Awk has its moments, perl is an abortion I havent used since the 90s.
I doubt your experience extends beyond what guido farts out however and you clearly know nothing about modern (post 98) C++
Re: (Score:2)
Would you stop trying to insult me about what you think I don't know about C++ or my experience in general? The resume of languages you posted sure seems to be still stuck in the 90s, BTW. I sure hope you're not making assertions about how awesome "modern" C++ is without having learned any of the other current alternatives that have come along since Perl and Java (and I'm NOT talking about Python).
Here's just one example: Crappy modern C++ still won't keep you from segfaulting if you don't religiously follo
Re: (Score:2)
"The resume of languages you posted sure seems to be still stuck in the 90"
LOL :) Oh dear. I assume you know what most embedded (my area) systems are written in not to mention almost all OS kernels and even the most popular Python interpreter?
"In fact, the whole language is a giant steaming pile chock full of features that you should never use,"
I'll agree there is some stuff in it thats pretty niche but sadly for script kiddies like you its speed will always be its killer feature and it leaves you guys in t
Re: (Score:2)
Funny... embedded systems is my area too, and I'm the OP on this thread. Hammer vs. screwdriver... know when to use which, but have both in your tool belt.
Re: (Score:2)
Never said otherwise. My point was however if you require to call out to a load of C in python you might as well use C or C++ for the lot.
Re: (Score:2)
Re: (Score:2)
That's a horrible comparison! I mean, I don't like python myself, and it is definitely slow, but he compares the runtime of a compiled C program with the time to launch the python interpreter, compile and run the python code!
And the test code is so trivial, of course the optimization makes it a thousand times faster - if a compiler is smart enough it would do whatever sane human would do and assign the sum directly instead of looping!
Benchmarking is a tricky business, but this is retarted!
Re: (Score:2)
it's very easy to get really slow python code without doing anything that would jump at you as being bad practice.
I do performance for a living. And I very often work on python codes written by scientist that is not so bad. But just doesn't get executed reasonnably. In one case, I pretty much copied the time sensitive part in a C function and got a 100 times speedup.
CPython is very slow. And python can never be THAT fast, mostly because the semantic of the language sometimes does not enable optimizations to
Good job (Score:3)
Make it faster at sucking!
Python walks hard towards its ultimate destruction (Score:1)
Distinguished Engineer? (Score:2)
Distinguished? Engineer? What makes this guy an engineer at all? And distinguished mostly in infamy for some of the worst design decisions and poorest performance ever.
They should say what they really did...surrounded to controller of Python with a few competent people so that the language could stop sucking after decades.
What about PyPy? (Score:3)
I am surprised to see no mention of PyPy [pypy.org]. I would have thought that if one wanted faster Python, that would be the place to start. Has there been any mention of why they didn't go that way, what similarities there are, and where they differ?
Re: (Score:2)
GvR is embarrassed by the fact that he doesn't understand PyPy, and at this point, they can't ever admit that PyPy's approach had any merits. It is no accident that PyPy's team produced like three working JIT frameworks while none of the CPython forks could match them: CPython is architecturally slow.
Source: I still remember chatting with GvR about this like a decade ago at a Chinese restaurant midway through a PyCon.
Re: (Score:2)
Pypy is not fully compatible with CPython. It's close but not fully compatible.
I ran into incompatibility issues in the past when I tried to use it. In some cases you get a proper error message. In some cases, you get a different result out of the same code.
So for me, pypy is out; I can't trust it.
Re: (Score:2)
> Pypy is not fully compatible with CPython. It's close but not fully compatible.
True, and that might well be the answer to my question. In fact, I expected that it would be something like that, but I didn't see any mention of it in the linked article, so I guess we're left to guess.
If compatibility is the reason, though, I do have two points to make about that. The first is that while we might wish for compatibility, CPython versions aren't always backwards compatible, either. Both in deliberate and obv
Julia? (Score:3)
Re: (Score:2)
From https://julialang.org/ [julialang.org], first paragraph:
> Julia was designed from the beginning for high performance. Julia programs compile to efficient native code for multiple platforms via LLVM.
Turns out, if you set out to make something fast, and you use the excellent optimization and code generation abilities of LLVM, it is easy to come up with something faster than something that wasn't designed or built to be fast.
Re: (Score:2)
"Interpreted" isn't really a feature of a language anymore, now that most "interpreted languages" are actually JIT compiled. Look at Javascript, which started out interpreted but has had a ton of work put into making it faster. Features of the language do affect performance, or at least make it much easier to optimize, such as being statically typed. And if you want to write super optimized code, it's essential to have features like precise control of memory layout and explicit vectorization. But you ca
Re: (Score:2)
There is still quite a difference in run times between interpreted and compiled languages https://benchmarksgame-team.pa... [debian.net]
Pypy is garbage collected so one must write code to be Pypy compatible. I agree it is not a big deal though.
Some slowdown also results from runtime type checking (when compared to statically typed languages) and from structural typing. A dynamically typed language will still need to do some type checking in runtime (not all of it can be eliminated by JIT).
Re: (Score:2)
Semantics are different.
In particular in Julia, you can specify types where it is important. That can really help the compiler/optimizer to get good code out.
Mostly python has semantics that prevent a ton of standard optimization that you think should be straight forward.
and yet.. (Score:1)
Re: (Score:2)
And yet there are many who find braces exceedingly ugly. I'm never quite sure exactly where to place them. Do I go for what looks best and is most readable, or do I try to make the matching braces line up on the same column?
The most popular style, K&R or derivative, has you place the opening brace on the same line as the statement which looks the best to me, but then you've lost the ability to quickly visually line up opening and closing braces with your eyes. So at that point a casual glance at the c
Re: (Score:3)
> so why not just ditch the braces and make it formal.
1. Indentation is NOT enough to tell intent. Given:
Was the original block:
O?
With braces the intent is unambiguous.
2. Indentation does NOT change functionality. Image if a Mathematician said "a = b + c" means something different then "a=b+c". They would be laughed
Re: (Score:2)
I don't understand your point. You show two versions of the code that are indented differently. They do different things, and in each case the indentation makes it totally clear what it does. So what's your point?
2. Indentation does NOT change functionality. Image if a Mathematician said "a = b + c" means something different then "a=b+c". They would be laughed out of the room for being an idiot.
You first said "indentation", then gave an example that doesn't involve indentation. So again, what's your point? Are you saying no whitespace character should ever affect meaning? Tryremovingallthewhitespacecharactersfromsometextandseeifitmatters.
Re: (Score:2)
> and in each case the indentation makes it totally clear what it does. So what's your point?
Good luck emailing / copy-pasting when whitespace is trashed and those latter two both turn into the first case.
With the always-use-braces style [c2.com] I can copy-paste and not worry IF the whitespace gets mangled / dropped / condensed down to just 1 consecutive space like in HTML.
> Are you saying no whitespace character should ever affect meaning?
You're assuming NO whitespace equivalent to SOME whitespace. While th
Re: (Score:2)
Good luck emailing / copy-pasting when whitespace is trashed and those latter two both turn into the first case.
The situation is just as bad in C++ or most other languages. If you email code in a way that trashes indentation, your code turns into an unreadable mess. It may look the same to the compiler, but it sure doesn't look the same to a reader.
Conclusion: don't do that! With any language.
But this is a theoretical objection, not a real one. I email code snippets all the time, and it comes through perfectly. I've never used a mail system that had any problem with indentation. That just isn't how email client
Re: (Score:2)
If you email code in a way that trashes indentation, your code turns into an unreadable mess. It may look the same to the compiler, but it sure doesn't look the same to a reader.
Actually that is not true in C++ or any language that is not indentation dependent. You can always reformat in your editor and get a code that looks decent and is functionning correct.
You can't do that in python whenever an intermediate tool somewhere screws up with indentation. That does happen; I would not say it happenes to me often. But it does happen.
Re: (Score:3)
How are Begin / end less readable?
Personally I like the way Visual Basic handles it with only the End If, End With, End Select etc.
But the braces I don't mind either, it's much more clear as having indentation only.
Re: (Score:2)
I don't have a problem with indenting code to make it more readable. Style guides are a good thing. What I don't like is the way the "one true way" has been baked into the language definition. It's not compliant with our companies standards and will probably keep Python from being adopted.
But there's nothing wrong with the white space by itself. What could throwing a few tabs and spaces in here and there hurt?
Re: (Score:2)