New Python 3.9 'Brings Significant Changes' To Language Features (infoworld.com) 74
This week's release of Python 3.9 "brings forward significant changes to both the features of the language and to how the language is developed," writes InfoWorld — starting with a new yearly release schedule and performance-boosting parser improvements:
- Python makes it easy to manipulate common data types, and Python 3.9 extends this ease with new features for strings and dictionaries. For strings, there are new methods to remove prefixes and suffixes, operations that have long required a lot of manual work to pull off. [The methods are named .removeprefix() and .removesuffix() and their return value is the modified string]
- For dictionaries, there are now union operators, one to merge two dictionaries into a new dictionary and one to update the contents of one dictionary with another dictionary.
- Decorators let you wrap Python functions to alter their behaviors programmatically. Previously, decorators could only consist of the @ symbol, a name (e.g. func) or a dotted name (func.method) and optionally a single call (func.method(arg1, arg2)). With Python 3.9, decorators can now consist of any valid expression...provided it yields something that can function as a decorator...
- Two new features for type hinting and type annotations made their way into Python 3.9. In one, type hints for the contents of collections — e.g., lists and dictionaries — are now available in Python natively. This means you can for instance describe a list as list[int] — a list of integers — without needing the typing library to do it. The second addition to Python's typing mechanisms is flexible function and variable annotations. This allows the use of the Annotated type to describe a type using metadata that can be examined ahead of time (with linting tools) or at runtime...
- Python extension modules, written in C, may now use a new loading mechanism that makes them behave more like regular Python modules when imported.
- For dictionaries, there are now union operators, one to merge two dictionaries into a new dictionary and one to update the contents of one dictionary with another dictionary.
- Decorators let you wrap Python functions to alter their behaviors programmatically. Previously, decorators could only consist of the @ symbol, a name (e.g. func) or a dotted name (func.method) and optionally a single call (func.method(arg1, arg2)). With Python 3.9, decorators can now consist of any valid expression...provided it yields something that can function as a decorator...
- Two new features for type hinting and type annotations made their way into Python 3.9. In one, type hints for the contents of collections — e.g., lists and dictionaries — are now available in Python natively. This means you can for instance describe a list as list[int] — a list of integers — without needing the typing library to do it. The second addition to Python's typing mechanisms is flexible function and variable annotations. This allows the use of the Annotated type to describe a type using metadata that can be examined ahead of time (with linting tools) or at runtime...
- Python extension modules, written in C, may now use a new loading mechanism that makes them behave more like regular Python modules when imported.
But does it address some issues? (Score:5, Interesting)
Like ensuring that the complete code is consistent before starting to execute?
In the current Python you won't get any warning until you try to execute a function that don't exist if it's in a different file, then the program bombs. So this means that you need 100% test coverage of the code for every release.
Compared to languages like Java and C# where that kind of issue is discovered at compile time.
Re:But does it address some issues? (Score:5, Interesting)
Testing code for "consistency" is an NP complete problem. It's complicated by tendency of Python developers to use "pip install" to pull in arbitrary modules at run time, which is not deterministic, and tracing down the dependencies is itself an NP complete problem due to the dependency chains and deliberate re-wrapping of some modules by other modules.
Re: (Score:3)
The way you describe it makes me think of storing sodium in biodegradable plastic bags in a barrel of water.
One day you will get an interesting surprise.
Re: (Score:3)
I've already experienced numerous such surprises, thank you. Many languages that "dynamically install modules as needed" have created just such surprises when the production version winds up with different versions of the same modules than were installed at the time of the test installation.
Re: (Score:2)
Many languages that "dynamically install modules as needed [...]
Only thing, that's not a Python language feature.
Some people have a build setup where the CI 'pip installs' everything from scratch every time, or a test setup where they "pip install -r requirements.txt" into a virtualenv every time they run the test. But that's just their build setup. They could stop doing that in an instant, and their program would still run fine.
If someone deploys to production in a way that's automated along those lines, with a requirements file that pulls in the latest and greatest
Re: (Score:2)
Take a look in /usr/bin/ on any system with python. Can you find any python scripts that do not import python modules installed separately? Systems that attempt to package their python modules provide some help stabilizing them, how many of us suffer the same difficulties we encounter with ant, maven, and perl where the dependency chains are not necessarily stable? It's fundamental not only to the language, but to the packaging of modules at pypi.org. The problem is exacerbated by modules like "bs4", which
Re: (Score:2)
I'm kinda confused as to how your problems with pip are fundamental to the Python language, when apparently you have the same kind of problems with ant and maven, which are written in a language very much different from Python.
Take a look in /usr/bin/ on any system with python. Can you find any python scripts that do not import python modules installed separately?
Well, yes I can. I went looking, and the first Python script I found was smtpd.py, which imports sys, os, errno, getopt, time, socket, asyncore and asynchat. Standard library only.
Can you find any C programs that don't use dynamically-linked shared objects installed separately?
Re: (Score:2)
I neglected ruby and rubygems, which have the same issue.
> I went looking, and the first Python script I found was smtpd.py
smtpd.py is part of the basic Python 3 source code, Python-[version]/Lib/smtpd.py. I'm nto surprised that it has no other modular requirements. I _did_ say to look in /usr/bin/. I suspect that is not where you found it. If you did find it there, what Python installation and operating system are you working with?
The same problem _can_ exist for C programs. "Mint" Linux is an infamous
Re: (Score:2)
I _did_ say to look in /usr/bin/. I suspect that is not where you found it.
That's where I found it.
Re: (Score:2)
How did that wind up in /usr/bin/smtpd.py? That script is normally installed in the _library_ directory for python, /usr//lib/python[version]/smtpd.py. What operating system's python setup are you using that it got placed or linked to /usr/bin/smtpd.py ? Might someone have placed or linked it there manually?
Re: (Score:2)
Re: (Score:3)
No, that problem doesn't have anything to do with typing.
Consider the following, 2 python files:
f2.py
f1.py
This example will randomly give the following error:
Re:But does it address some issues? (Score:4, Informative)
The Python VM doesn't check that, but there are various static analysis tools that will, such as pyflakes, pylint and mypy. The problem from this example would be detected by pyflakes, which requires no annotations and runs super quick. More complex problems might require type annotations and mypy to detect. These days you can use Python as a dynamically typed language or a statically typed one or anywhere inbetween.
Re: (Score:2)
Re: (Score:2)
Raise a PEP suggesting it.
Re: (Score:2)
Re: (Score:2)
This isn't something a scripting/interpeted language should consider. Most complied languages don't either (ie: reflections). You are asking the runtime to trace through all paths and also all permutations. In Python, being dynamic, that namespace could be changed/side-effected at anytime, and the quack could be valid now.
But the point is the execution should be as fast as possible, it shouldn't waste time thinking of all the potential possiblities and error conditions for each and every run. Compiled l
Re: (Score:2)
This isn't something a scripting/interpeted language should consider. Most complied languages don't either (ie: reflections). You are asking the runtime to trace through all paths and also all permutations.
No, I'm asking the runtime to just check that all calls have an implementation before starting to execute the whole thing.
Compiling languages do that either during compilation like Java or C# or during linking like C/C++/Fortran etc.
This is why using languages like Python where no checks are done to verify that everything is connected should be seen as risky languages for large scale production.
The dynamically selected imports is something that's convenient for flexibility but really scary from a long term
Re: But does it address some issues? (Score:2)
What I am saying is that in reflection and scripting based languages, not all calls are defined at runtime. Some of them are defined as part of the execution. So depending on the execution path, not all calls have to be valid... only the one on the execution path. And to validate you would need to figure out all execution paths and validate the pre-requisites of each.
And on top, for scripting languages, it's a pretty big penalty considering it's done on _every_ run.
Re: (Score:2)
Re: (Score:2)
Guy who doesn't understand dynamic binding [wikipedia.org] complains about dynamic binding. Film at 11.
Have you ever taken a programming course in your life? Or did you just rummage through a dog-eared copy of Java for Dummies you found lying in the gutter?
Dynamic binding is there for a reason. Hint: it's not for you. Go back to your nice safe Java straitjacket an
Re: (Score:2)
Dynamic binding is there for a reason. Hint: it's not for you. Go back to your nice safe Java straitjacket and use the safety scissors so you don't stab your own eye out. Let the grown ups handle the nasty wasty Python.
Whenever I see that statement I see that Python is perfect for academic research into potential software solution but it's a solution that is dangerous in production environments where long term stability is essential and it's a big headache to figure out why something seemingly unrelated stopped working for no apparent reason. And as I have seen in another comment - if the dynamically loaded items are on a remote service then you are in a world of pain if that service goes offline or changes the behavior.
A
Re: (Score:2)
But yeah, all the problems you are describing I've run into in C, C++, Java, and even FORTRAN. The headac
Re: (Score:2)
Re: (Score:2)
But just to be clear for others, all languages that seem to be statically binded have their dynamic equivalents within. Java appears static in its interfaces and OOP model, but has reflections that bypass all that static stuff. C/C++ have the preprocessor that does dynamic code enablement and void* for "unknown"; it is a exercise to the programmer to validate all calls and avoid execution paths with invalid calls. Assembly is ALL just dynamic with a "If it fits, ok, else that's OK too!" mindset that make
Re: (Score:3, Informative)
I honestly do not understand what practical problem it is that you have with this.
It sounds like: "If I do not know that something has mechanically checked that every referenced resource is in fact present and available, then the language is insufficient for my need for security."
But honestly, man, how do you know that somebody hasn't replaced parts of the content of your executable with gibberish? How do you know that your program, referencing a critical text file, -- that the text file hasn't been delete
Re: (Score:3)
In a compiled language like C or Java, if you remove or rename a function, or change its arguments, you'll get compile errors for any uses of the function that you haven't updated. In Python or JavaScript, you need to exercise every single code path to make sure nothing is still expecting the old function signature. You can't use a grep for the function name to help, because you can easily take a bound reference to it and pass that around.
Why is it so hard for people to understand that having these kinds
Re: But does it address some issues? (Score:3)
Python has plenty of tools to enforce strictness. We use mypy and other tools to avoid the issues you mention as part of our CI pipelines on every pull request and the use cases you describe are pretty much a non issue for us, to some extent as I mention below.
Iâ(TM)ve had runtime issues with Java where monolithic projects running on tomcat would end up with conflicting versions of common third party libraries and they would fail at runtime because some developers will get out of their way to shade dep
Re: (Score:2)
But the bigger is
Re: (Score:2)
Compared to languages like Java and C# where that kind of issue is discovered at compile time.
Huh? Both those languages load classes dynamically at runtime.
Maybe you meant C++.
Re: (Score:2)
Just try to build your application in Java or C# without having the ends tied together and you'll see what happens.
Of course you can build one class individually, but that's often not feasible, you usually build the system in an IDE like Eclipse or Visual Studio and there you'll see when it's broken.
Re: (Score:2)
Re: But does it address some issues? (Score:2)
Maybe because Python is interpreted is why compile time doesnâ(TM)t do what you think it should
Re: (Score:2)
Re: (Score:1)
I don't do much python, but isn't python supposed to be a dynamic language, like Lisp, Javascript or Perl?
If yes, then what you're asking for is impossible. Use C++ / etc if you want strong compile time checks.
If no, then again use C++, because you won't lose anything in term of flexibility, and you will get not only compile time checking, but also faster and lighter programs ;-)
But does it have a case statement yet? (Score:5, Interesting)
Case, switch, don't care what you call it but 7 layer nested if statements are insane.
If they're a bad idea why does virtually EVERY other language have them?
When you're lacking control constructs the shell scripting language has there's a problem.
Re: (Score:3)
Not having case statements is a minor annoyance, but a seven part "if ... elif ... elif ... else" statement is not nested. It's a single statement, the main difference with "case" is just that you need to repeat the "foo == " part at each step. I don't really see that as a huge deal breaker.
Re: But does it have a case statement yet? (Score:4, Insightful)
Re: (Score:2)
I know, but with the restrictions on anonymous lambdas, constructing such a dictionary with a bunch of named functions is a hassle and doesn't make one cohesive statement.
Re: (Score:2)
This is the correct python way, closures and Lambdas are often confused with one another and so people defer to the old case or if/elif because thats was you had in in older popular languages like php.
Re: (Score:2)
And that's yet another example of Python doing things in an inefficient way. You need to build a hash table in the heap and then call through a bound function reference, and you can only get results out of the function by return value because they're another level of scope, so you can't just assign to variables. It can get you a similar result to a switch statement, but it requires ugly code and it's inefficient.
Re: (Score:2)
the main difference with "case" is just that you need to repeat the "foo == " part at each step.
The fun part will come when arguments over what methodologies switch/case would actually support surface.
Honestly it seems to me that first class Function Tables with first class Lambdas would be most versatile. In such a case (pun) switch/case semantics are but one way of walking and processing such tables, or pairs of tables (one table the conditions, the other the actions)
But on a language that uses fucking whitespace... yeah... never gonna have anything nice
Re: (Score:2)
How do you know what shell I'm using?
Why would you make a language a moving target? (Score:5, Interesting)
I can't be the only one who's tried to use Python code I found on the internet and had no end of trouble because there are so many mutually incompatible versions of the language, versions of libraries, etc.
IMHO, "significant changes" should be absolutely the last thing a language ever tries to do. When you reach the point of "significant changes", it's time to call it something different and stop pretending it's the same language, because it isn't.
Re: (Score:2)
Or at least to call it version 4.0 rather than 3.9, to indicate the breaking changes.
Re: (Score:3)
Or at least to call it version 4.0 rather than 3.9, to indicate the breaking changes.
If there were any...The list in TFS doesn't mention any backwards incompatibilities, just added features. Releasing a new minor version for it seems perfectly consistent with semantic versioning https://semver.org/ [semver.org] conventions to me.
Re: (Score:3)
Oh, but there are breaking changes. They've removed ".tostring()" from "struct" and its replacement ".tobytes()" was only introduced in 3.2, so if you want your code to work with both 3.1 and 3.9 you need to do stuff like this: offsdata.tobytes() if hasattr(offsdata, "tobytes") else offsdata.tostring()
Re: Why would you make a language a moving target? (Score:3)
Re: (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
On *nix systems you could just spell it out with a shebang line.
And on Windows, ditto. If you write python2 on the shebang line, that's what you get.
Just add that to your existing Python 2 programs, ensure that Python 3 is the default otherwise (which it probably is without you having to do anything), and you're all set. It's not like you're going to be writing new Python 2 programs, is it?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
I wish my proficiency was that high, but... (Score:2)
Too bad I can't understand the esoteric details of this story. Looking at the early comments it seems their understanding of Python is even lower than mine. For example, there's a major difference between interpreted and compiled languages...
However in terms of improving my own Python skills and understanding, maybe this is a good place to ask? I'm looking for a moderate-sized project to hack at. That's my favorite way to learn new languages these years. Take working code and rework it, naturally learning f
Re: (Score:2)
Too bad I can't understand the esoteric details of this story. Looking at the early comments it seems their understanding of Python is even lower than mine. For example, there's a major difference between interpreted and compiled languages...
See it the other way - the experience from compiled languages gives a lot of experience on what's wrong with interpreting languages. Each time I look at Python I see a language where there's a potential for a lot of problems in the long run. Python isn't alone, PHP suffers from similar problems as well so you can't update the version of the interpreter without breaking a lot of stuff on your web server.
Python is basically repeating some of the mistakes made by interpreting Basic in the 80's.
Re: (Score:2)
I think you're getting diverted into areas of language theory, but a lot of my experience was with JIT (Just In Time) Java compilers that were trying to split the hairs. I remember multi-level compilation and recompilation as the hot code was identified on the fly... But this was really going to an intermediate language that reminded me of the old p-code.
However I think Python was designed to avoid a lot of the interpretation problems and learned more lessons along the way. The more learned response earlier
Re: (Score:2)
See an example I made above on one of the issues I have with Python. It applies to other scripting languages as well though.
Re: I wish my proficiency was that high, but... (Score:2)
Now I think you were the source of the "learned response" that I referenced. Not certain, because I'm mobile and the mobile interface to Slashdot still reeks.
Re: (Score:2)
If you mean "learned response" through someone "told me so" you are wrong. My "learned response" is because this is what I have discovered while fiddling with a lot of various languages during 40 years and discovered what works long term and what doesn't. Python (and other scripting languages) is like building on a place where the land is flat because it's easy just do discover a few years later that it's on clay base and suddenly you end up in the software variant of this: https://upload.wikimedia.org/w... [wikimedia.org]
Re: (Score:2)
Well, now you've triggered my nostalgia module. It's been about that long since I studied computer science formally. In those days I got to fiddle with a lot of languages and I found it helpful in many ways. Most of the highlights are lost in the mists of time, though I remember getting to do APL on a machine with an actual APL keyboard with all the weird symbols on it. (The only time I got to use APL in the real world it was with a normal keyboard and I had to learn all the dot codes for the weird characte
Compiled vs Intertpreted is not a distinction (Score:2)
The idea that a language needs to be either difficult to use and compiled or easy to use, no static typing and interpreted comes from C/*nix. Shell/Perl vs C. It is plain wrong.
That is what Lisp, thence Java and .Net introduced. Languages that were easy to use and yet could be compiled efficiently. No dichotomy.
Modern JavaScript systems also compile to native in a strange sort of way. Start with a nasty language with no static typing, and then do incredibly clever analysis and compiler tricks to compil
Re: (Score:2)
Not another nostalgia trigger? Lisp was one of my favorite languages. I remember when I was supporting the development team for one of the last real Lisp machines. No OS, just Lisp running right out of the microcode. My most interesting bug turned out to be in the compiler... I spent most of a morning trying to figure out it if was broken before bothering the actual expert, who spent about 30 seconds confirming the bug and then told me who was responsible for fixing it. (Really nice guy to give me the credi
Re: (Score:1)
The way i see Python is, if you, a researcher for example, needs to hack together an application to process a bunch of data, Python is just perfectly adequate.
However, if i was was big-corp and would rely my code base to be built upon Python, in the long run i'll get what i deserve - a disaster. It's not even hypothetical, python 2 is now curated by private entities.
Or if i work in a big team on a complicated big project - a disaster is around the corner anyways so using Python should really be justified by
Re: I wish my proficiency was that high, but... (Score:2)
C++ is not really a beautiful language. It seems to be in the dark zone between C and Java/C#. And it has some parts that are quite cryptic.
For new development I'd limit the use of C++ to a minimum.
Re: (Score:2)
In other words (Score:5, Funny)
"Python extension modules, written in C, may now use a new loading mechanism that makes them behave more like regular Python modules when imported."
Behave more like regular python modules... so, that means they now run slower?
Re: (Score:2)
I'm not gonna label you a troll, because well... you're kind of right: Python is a bit of a moving target, and it can be really frustrating to maintain code as version after version of it break your code.
Having said that, it's not a shit language. No language is perfect, and no language is suited for all tasks. But the fact is, I get more done in Python than in any other language.
Just sayin'...
Re: (Score:3)
That still makes it at least half an order of magnitude simpler than .Net...
Oh great (Score:1)
Just what python needs .. big changes. How did that work out python 3.0? People are STILL on python 2.7.
Python 4.0 they should just become Fortran or something like that. No big deal to migrate from 3.9 to 4.0 just convert all your code into Fortran.
Re: (Score:2)
Part of the problem is half of the developers creating modules in python are not programmers by definition, they are data scientist that learned to code in python and have no intention of using the latest and greatest version of python.
There are other issues to, redhat and mac OS both used 2.7 by default for the os....and what would often happen is people would upgrade the os version possibly causing issues for the os and whatever packages were already installed. Again this is due to shoe-horned non-program
Re: (Score:2)