Python 'Language Summit' 2024: Security Workflows, Calendar Versioning, Transforms and Lightning Talks (blogspot.com) 19
Friday the Python Software Foundation published several blog posts about this year's "Python Language Summit" May 15th (before PyCon US), which featured talks and discussions by core developers, triagers, and Python implementation maintainers.
There were several lightning talks. One talk came from the maintainer of the PyO3 project, offering Rust bindings for the Python C API (which requires mapping Rust concepts to Python — leaving a question as to how to map Rust's error-handling panic! macro). There was a talk on formalizing the PEP prototype process, and a talk on whether the Python team should have a more official presence in the Apple App Store (and maybe the Google Play Store). One talk suggested changing the formatting of error messages for assert statements, and one covered a "highly experimental" project to support structured data sharing between Python subinterpreters. One talk covered Python's "unsupported build" warning and how it should behave on platforms beyond Python's officially supported list.
Python Foundation blog posts also covered some of the longer talks, including one on the idea of using type annotations as a mechanism for transformers. One talk covered the new interactive REPL interpreter coming to Python 3.13.
And one talk focused on Python's security model after the xz-utils backdoor: Pablo Galindo Salgado, Steering Council member and the release manager for Python 3.10 and 3.11, brought this topic to the Language Summit to discuss what could be done to improve Python's security model... Pablo noted the similarities shared between CPython and xz-utils, referencing the previous Language Summit's talk on core developer burnout, the number of modules in the standard library that have one or zero maintainers, the high ratio of maintainers to source code, and the use of autotools for configuration. Autotools was used by [xz's] Jia Tan as part of the backdoor, specifically to obscure the changes to tainted release artifacts. Pablo confirmed along with many nods of agreement that indeed, CPython could be vulnerable to a contributor or core developer getting secretly malicious changes merged into the project.
For multiple reasons like being able to fix bugs and single-maintainer modules, CPython doesn't require reviewers on the pull requests of core developers. This can lead to "unilateral action", meaning that a change is introduced into CPython without the review of someone besides the author. Other situations like release managers backporting fixes to other branches without review are common.
Much discussion ensued about the possibility of altering workflows (including pull request reviews), identity verification, and the importance of post-incident action plans. Guido van Rossum suggested a "higher bar" for granting write access, but in the end "Overall it was clear there is more discussion and work to be done in this rapidly changing area."
In another talk, Hugo van Kemenade, the newly announced Release Manager for Python 3.14 and 3.15, "started the Language Summit with a proposal to change Python's versioning scheme. The perception of Python using semantic versioning is a source of confusion for users who don't expect backwards incompatible changes when upgrading to new versions of Python. In reality almost all new feature releases of Python include backwards incompatible changes such as the removal of "dead batteries" where PEP 594 marked 19 modules for removal in Python 3.13. Calendar Versioning (CalVer) encompasses a wide array of different versioning schemes that have one property in common: using the release date as part of a release's version... Hugo offered multiple proposed versioning schemes, including:
- Using the release year as minor version (3.YY.micro, "3.26.0")
- Using the release year as major version (YY.0.micro, "26.0.0")
- Using the release year and month as major and minor version (YY.MM.micro, "26.10.0")
[...] Overall the proposal to use the current year as the minor version was well-received, Hugo mentioned that he'd be drafting up a PEP for this change.
There were several lightning talks. One talk came from the maintainer of the PyO3 project, offering Rust bindings for the Python C API (which requires mapping Rust concepts to Python — leaving a question as to how to map Rust's error-handling panic! macro). There was a talk on formalizing the PEP prototype process, and a talk on whether the Python team should have a more official presence in the Apple App Store (and maybe the Google Play Store). One talk suggested changing the formatting of error messages for assert statements, and one covered a "highly experimental" project to support structured data sharing between Python subinterpreters. One talk covered Python's "unsupported build" warning and how it should behave on platforms beyond Python's officially supported list.
Python Foundation blog posts also covered some of the longer talks, including one on the idea of using type annotations as a mechanism for transformers. One talk covered the new interactive REPL interpreter coming to Python 3.13.
And one talk focused on Python's security model after the xz-utils backdoor: Pablo Galindo Salgado, Steering Council member and the release manager for Python 3.10 and 3.11, brought this topic to the Language Summit to discuss what could be done to improve Python's security model... Pablo noted the similarities shared between CPython and xz-utils, referencing the previous Language Summit's talk on core developer burnout, the number of modules in the standard library that have one or zero maintainers, the high ratio of maintainers to source code, and the use of autotools for configuration. Autotools was used by [xz's] Jia Tan as part of the backdoor, specifically to obscure the changes to tainted release artifacts. Pablo confirmed along with many nods of agreement that indeed, CPython could be vulnerable to a contributor or core developer getting secretly malicious changes merged into the project.
For multiple reasons like being able to fix bugs and single-maintainer modules, CPython doesn't require reviewers on the pull requests of core developers. This can lead to "unilateral action", meaning that a change is introduced into CPython without the review of someone besides the author. Other situations like release managers backporting fixes to other branches without review are common.
Much discussion ensued about the possibility of altering workflows (including pull request reviews), identity verification, and the importance of post-incident action plans. Guido van Rossum suggested a "higher bar" for granting write access, but in the end "Overall it was clear there is more discussion and work to be done in this rapidly changing area."
In another talk, Hugo van Kemenade, the newly announced Release Manager for Python 3.14 and 3.15, "started the Language Summit with a proposal to change Python's versioning scheme. The perception of Python using semantic versioning is a source of confusion for users who don't expect backwards incompatible changes when upgrading to new versions of Python. In reality almost all new feature releases of Python include backwards incompatible changes such as the removal of "dead batteries" where PEP 594 marked 19 modules for removal in Python 3.13. Calendar Versioning (CalVer) encompasses a wide array of different versioning schemes that have one property in common: using the release date as part of a release's version... Hugo offered multiple proposed versioning schemes, including:
- Using the release year as minor version (3.YY.micro, "3.26.0")
- Using the release year as major version (YY.0.micro, "26.0.0")
- Using the release year and month as major and minor version (YY.MM.micro, "26.10.0")
[...] Overall the proposal to use the current year as the minor version was well-received, Hugo mentioned that he'd be drafting up a PEP for this change.
current year as the minor version (Score:2)
Wrong problem (Score:2)
The perception of Python using semantic versioning is a source of confusion for users who don't expect backwards incompatible changes when upgrading to new versions of Python.
The problem is the backwards incompatibility, not the false expectation of compatibility. Fix the culture of poor compatibility, not just abandon even the pretense of compatibility.
Python is incredibly frustrating as users expect a wide variety of interpreter support but the core implementation makes things required in 3.x forbidden in 3.y. if I build a 15 year old C program,, it pretty much just builds. If I have an 8 year old go project, the code at least still builds, though the build system has had so
Re: (Score:2)
In addition, I was unaware of this "CPython doesn't require reviewers on the pull requests of core developers". What? How is this ever a good idea? I don't care if you are core developer, a review process should always be in place.
Re: (Score:2)
Yes, that will definitely need a fix. Eventually. At the moment Python is still evolving too fast. But given that Python will probably be the standard "glue code" and "light lifting" language for the next few decades, they should probably think about an LTS branch or such, that will compile code written for it for the next 20 years or more.
Re: (Score:3, Insightful)
My wife was programming in Python in the 1990s. The Python developers (or Guido van Rossum at the time) probably should have thought about "an LTS branch or such" 20 years ago, but now they have decades of a "move fast and break things" culture. They explicitly rejected backwards compatibility with the whole Python 3 transition, and as mentioned elsewhere in this thread, they've continued that within the Python 3 release series. What makes you think they might change that now?
Re: (Score:2)
User pressure. More and more Python code is in productive long-term use. Python was niche for a long time (I have been using it for about 20 years now when I got sick of Perl). That time is over.
It does not even require a whole Python version. A converter would be enough.
Re: (Score:2)
User pressure has been there since Python 2 to 3 (the first time they *really* screwed people over with backwards incompatible changes). Their conclusion was not "gee, breaking compatibility is hard on the users, let's not do that" it was "we broke too much at one time, we need to *continually* break so that such a large group doesn't get hung up on a particular version change".
The issue is the people that care the most about not having to revisit old code are also the people who wouldn't have time to fool
Re: (Score:2)
Python 2.x EOL has only been 4 years ago. Obviously 2.x held back a lot of things and they are now busy adding those. Give it some time. The other factor is that as far as I can see, a lot of teaching how to code is transitioning from Java (an abomination of a language) to Python. Hence if you use Go or Rust you will have trouble finding people to code for you, but far less so with Python.
That said, a lot of code-base in Python is expanding and at some point it will not be who attends such summits, but "You
Re: (Score:2)
The clock started ticking on the 2->3 migration in 2008, when Python 3 released, not when Python 2 EOLed. Python 2.x existence in no way held them back during the time it wasn't EOL. They didn't promise backwards compatibility, they did very little on that front (allowing u and b prefixes to strings in both 2.7 and 3.3 so there's at least a common string syntax, for example.
Quite frankly, not sure you fare better with 'the default educational language'. At least since 'software engineer' became perceive
Re: (Score:2)
You clearly are angry. But I do not think you are rational here.
Re: (Score:2)
That's an excellent way to concede his points, although the last four words of your comment are extraneous.
Re: (Score:2)
That's an excellent way to concede his points
Only to the terminally stupid.
What is his point? Use obscure tech because non-obscure tech is not managed how he wants it to be managed? That is a noble sentiment, but not practical at all.
Re: (Score:2)
He had a lot of points, the first of which was that you were wrong about when the Python 2 vs 3 incompatibility issues became major issues. I'm not going to repeat them because his comment was pretty clear. That you don't understand those points shows again that you weren't thinking and your arguments were thereby wrong.
Re:Wrong problem (Score:5, Interesting)
The thing is Python isn't some new language, it's significantly older than a lot of platforms that have been more mindful about backwards compatibility. I might forgive an awkward thing like Python 2->3 to fix some growing pains, provided it didn't happen often, but after 2->3 the floodgates were open for breaking backwards compatibility every single time.
Generally speaking, the changes aren't even improvements. It's often just some part of the codebase where no changes have happened in a while, and better to 'purge' it than abide 'stale' code in glorious python. Much of the time it's another obnoxious concept in the Python world, arguments about what is 'more pythonic', with changes back and forth depending on who is talking at the moment and their interpretation. The key tenant is roughly "there should only be one right way to do it", so when two sets of people disagree, *obviously* only one of them can be right, and they are allowed to be right at different times, jolting around the developer community in the process. When other ecosystems hit similar problems, they generally carry on in some backwards compatible way, but not Python (anymore).
For example, during Python 2 era, they did 'range()' as just generating a big list. They realized that, of course, this was bad, and made 'xrange' and everyone knew 'use xrange, not range, but if you used range, you are golden'. Similar to how, in C, you are still free to use the ill-advised string functions, but better ways emerged. Python3 was irked by that 'x' and so changed range() *and* forbade xrange as a valid keyword.
After Python 3, they now pull this stuff every release. Whoops, we missed a part of the core runtime where we capitalized the name of a module, so we are going to break all code that imports it because it's not 'proper' to support an uppercase name (or vice-versa, in the case of a class that *isn't* upper case). Particularly painful if you dare to try to implement async (asyncio is here, you must pass a loop around because you manage context, no option.... ok, improvement, that's now optional, we'll default to grabbing 'the' event loop since that's what everyone is doing.... you know what, let's break all code that explicitly passed a loop because that was a dumb thing.... except sometimes you also have to deal with the async loop manually, here's 'get_event_loop', which will give you the 'default' loop, or create it if it doesn't exist... actually, that was too easy, we are removing that and now you have to call different functions depending on if the loop is started or not).
The end result is just frustrating. You want to scale? Tough, threads are expensive and don't really spread amongst the cores... you know what, here's some async great... except you can't reasonably mix and match with more typical python code so you have split your ecosystem in two flavors. I'm calling a function, is it going to return 'normally', or do I need to iterate over it because it's a generator, or do I need to await it because it is one sort of async, or do I need to 'async for' it because it is an async iterator'? Oh no, it's an async function and my codebase isn't async at the top... Ok, I've converting my codebase to asyncio and... they insist on an entirely different conceptualization of network sockets (e.g. either a twisted-style transport/protocol pair, or a pair of reader and writer when traditional socket is just passing around a single 'socket' that is bidirectional, and you have to remember that you should close one, but not the other because ultimately it is just a single thing....
They aren't converging on some idyllic 'pythonic' future, they just churn and spin and break stuff different ways at different times, which is maddening for anyone that went with python and now has to rework stuff after 'job done' to keep up with new platforms.
Question (Score:3)
Is it possible to develop in Python without being connected to the internet?
Re: Question (Score:2)
... yes...
Re: (Score:2)
Yes.
No battleplan survives contact with the enemy (Score:2, Funny)
Rust DOESN'T have exceptions as any rustacean will tell you.
It has return codes, and panic for when that doesn't cut it. Panic is like abort, except it's well behaved and unwinds the stack calling destructors. But also you intercept the unwind and continue from that point. Sound familiar? What if I said it uses the Itanium ABI?
What if I said that means you can call rust from C++, it panics, and you can catch it in C++.
Yeah Rust has exceptions.
The obvious thing to do is map panics to exceptions because that'
Did Y2K teach us nothing? (Score:2)
How the hell did nobody ask "Is using just two digits for the year a good idea?" again ?