See the PyPy JIT In Action 109
derGoldstein writes "Project PyPy is an alternative implementation of Python, with the main advantage being a Just In Time (JIT) compiler which speeds up your code considerably. They've announced the first public release of jitviewer, which is a visualization tool that helps you understand how your code is being compiled by PyPy's JIT, all the way down to assembly. If you just want to see how it looks and play with it, they've set up an online demo — just select a file, and click 'Show Assembler.'"
Re: (Score:2)
Re: (Score:2)
When are they going to start calling these thing YAPython and YAPerl?
Re: (Score:1)
I think you mean "And they're gone"
Unless they actually a possess a "gone"...
...whatever that is...
Go Pypy! (Score:4, Informative)
I love python, and pypy makes dream about the day python could be used for anything, making java irrelevant.
The language is readable, flexible, succinct, expressive and beautiful.
Programming in python is more a joy than a core, but it has always suffered from a wide performance gap in cpu intensive tasks compared to traditional static languages.
Hopefully, this is all about to change with projects like this.
It's worth noting that Pypy is more than just an alternative python implementation. It is a framework for implementing jitted compiled dynamic languages.
There are currently several works in progress with implementations of javascript, scheme, logo, php and other languages with pypy.
Re: (Score:1)
Re: (Score:2)
Uh, both Google's V8 and Mozilla's Tracemonkey are JIT engines. There's nothing wrong about creating your own, but if it's a question of speed and not about learning, sticking with V8 is probably a better option.
Re: (Score:3)
Javascript is not a nice language, it's the unholy chimera of Self and Java.
Sure it's better than how most people describe it, but there are far better languages out there.
Re: (Score:1)
Alright, give me an alternative that does the same thing. I agree...JS is a dirty, problem-inducing (rather than problem solving) language. I just don't know what else I can use to move things around and cause the useful JS effects without a refresh without JS.
Re: (Score:2)
Self, Io-language, Smalltalk, Ruby and of course Python or even Lisp, but also Java, C# or Vala if you want static typing.
You mean something supported on todays browsers? Yes, Javascript. But as long as you are reinventing the browser you can always pick a better scripting language.
Re: (Score:1)
Well, if I have the luxury of reinventing the browser and choosing a new standard, I would vote for Smalltalk (I would say Lisp, but I foresee forcing too many square pegs in round holes to go there). But right now we are stuck with JS or running things through the Java JIT.
Re: (Score:2)
Thanks anon!
Somebody should mod this coward up.
Re: (Score:3, Insightful)
The language is readable, flexible, succinct, expressive and beautiful.
I hear it cures cancer too. Oh wait, that's Ruby. (Actually, I love Python, but I'm allergic to fanboism...)
Re: (Score:3)
I hear it cures cancer too. Oh wait, that's Ruby. (Actually, I love Python, but I'm allergic to fanboism...)
While it's true that Ruby cures (certain types of) cancer, it also has the unfortunate side effect of causing autism. Don't believe me? Try reading non-Ruby code written by Ruby fans.
Re: (Score:2)
Your just scared because it allows one to quickly migrate U235.XML to U238.XML.
Re: (Score:3, Insightful)
Python is not going to make Java irrelevant. Sorry. I know it hurts to be told that your religion will never become universal, but that's reality for you.
Certainly one reason for choosing Java over Python today is performance. Python is relatively slow, and sometimes throwing more hardware at the problem is not an option, and rewriting stuff in C (like Python fanboys always suggest) is not actually that easy (and brings with it all the problems of writing anything in C -- there's a reason we want to use
Re: (Score:3)
The whole Java *or* Python discussion is irrelevant. You can use Jython, and take advantage of all the Python and Java libraries.
Re: (Score:2)
The whole Java *or* Python discussion is irrelevant. You can use Jython, and take advantage of all the Python and Java libraries.
Did you not read a word that this guy wrote, or did you simply not understand any of his points?
The weaknesses that he pointed out did not deal with Python "lacking access to Java libraries". The weaknesses that were pointed out included Python's lack of compile-time type-checking, lack of real publication and enforcement of API contracts, and lack of lexical scope.
Jython addresses none of that. It does not magically transform Python from a hacky Perl-successor into a language broadly suitable for large s
Re: (Score:1)
Go comeback. Really. The height of wit.
Except, you know...all of his points are valid and until we address them Python is the new Perl. Maybe you aren't old enough, Mr. Coward, but I remember when Perl was the new toy and people thought it was the best thing since sliced bread (hell, it would slice the bread for you).
Re: (Score:2, Insightful)
You're argument only makes sense if you accept that the python language design requires dynamic typing. And this may (may) be true for the really advanced and esoteric uses of python, but anyone who's ever built a tiny compiler knows that 95% of python can easily be implemented statically typed. It would be a great exercise to try to fully inline the entire python runtime library to cover the remaining 5%.
Personally I'm convinced that a statically typed python supporting 90% of common use cases, and the IDE
Re: (Score:1)
Everything is an object is not a property of dynamically typed languages, nor is it a property of static languages. It can be part of language design, or it can not be. It won't force a language to be either static or dynamically typed. Imho, usually laziness of language designers is what makes languages dynamically typed. Some at least have the decency to limit that dynamic typing to the implementation,and not build it into the design.
Personally I find the one language that actually does the "everything is
Re: (Score:2)
Are you a Java coder? There is no need to have a common base class named "object" to define the fundamental behavior of objects.
Re: (Score:2)
The reason for static typing is not implementation difficulties but bug checking. When I need an int variable I can declare it as such and make sure I never accidentally stick anything else into that variable. With dynamic typing a function can return values of different types or maybe a programming mistake makes the variable contain "3" instead of 3 and the error only pops up when something assumes you give it an int and it finds something else. In something like Lua you can have functions that return a va
Re: (Score:1)
... until you do IO.
Re: (Score:1)
Yes, but even then you can still group them into large chunks, instead of small allocations.
Re: (Score:1)
Ding ding ding! We have a winner!
Re: (Score:2)
So you are claiming that duck typing is not used in practice other than in esoteric projects. That is a little bit naive, consider why would something so complex be left in the language design if it was not necessary? The entire language relies on duck typing being there to make some other design decisions easier to make, for example all containers are polymorphic collections rather than the monomorphic collections in almost every other language. Your fragment of code is a single instance where forward inte
Re: (Score:2)
You've missed the point: yes it is easy to do this dynamically at the cost of performance. The comment was in response to the claim that doing it statically is easy. It's not easy to analyse this statically, but when it can be done the performance increase is large. One optimisation that it allows straight away is unboxing which can easily increase the performance of numerical code by an order of magnitude.
Re: (Score:3)
[...] and rewriting stuff in C (like Python fanboys always suggest) is not actually that easy (and brings with it all the problems of writing anything in C -- there's a reason we want to use Python or Java in the first place, right?)
Actually, dismissing writing parts in C is not so simple. The only problem I'm aware of is the portability. And even that is handled by package/module management automatically.
Otherwise, most problem-free C programming I ever done (as C stuff goes) was Perl XS (*). Because you can leave to Perl all the mundane tasks and implement in C only the most performance-critical parts, which often have well defined input and well defined output. Even memory allocation stops being a problem: the objects one alloca
Re: (Score:2)
Is PyPy very different from Psyco? I've been using the latter for scripts that needed speedups.
Re: (Score:2)
From what I understand, Psyco is the father of PyPy. Psyco is a finished project, and the things learnt from it are being developed and refined in the PyPy project.
Re: (Score:2)
I used psyco for my thesis back in 2007. It improved my SimPy runtimes by a factor of 100, merely with a simple "import psyco" in the beginning of the project!
Unfortunately psyco only works on 32-bit architectures, and after I got my degree I finally upgraded to a 64-bit system and sat on my work waiting for something to happen. Now I guess I finally need to pull it out again and see what's up.
And by see you mean (Score:3, Funny)
the connection has timed out
Python, the new Java
It's Python. It's Java. It's Jython. (Score:1)
Python, the new Java
That's Jython [jython.org], not PyPy.
Woo hoo! (Score:4, Informative)
I'm glad to see it scales well!
Re: (Score:1)
Re: (Score:1)
Here's your reminder:
http://tinyurl.com/42jo3qv [tinyurl.com]
Re: (Score:3)
No, that was Unladen Swallow [wikimedia.org].
Re: (Score:2)
No, that is Unladen Swallow [wikipedia.org].
Re: (Score:2)
No. IIRC it can compile to LLVM (it's actually got a bunch of backends), but it isn't actually based on LLVM itself.
Re: (Score:2)
I think Rakudo Star is written in Perl 6 (or a subset of it). At least that was my understanding after listening to one of its devs talk about its history.
Re: (Score:2)
Re: (Score:1)
Javac is in java, yes, but the JVM is written in C++.
IBM's Jikes RVM is a JVM written in Java. They use it as a research platform (the R stands for "research").
Quora (Score:2, Interesting)
quora.com is now running on PyPy.
http://www.quora.com/Alex-Gaynor/Quora-product/Quora-is-now-running-on-PyPy
Faster than C? (Score:4, Informative)
Not just faster than CPython, but faster than C [blogspot.com] for some common tasks. Pretty amazing.
However, this project is not yet very useful to the people who might be most interested in a really fast python, as it does not work with numpy [blogspot.com]. But when they get that to work, wow.
Re: (Score:2)
Re: (Score:2)
Did you read the comments?
That's not a common task if he created a file with a meaningful output it would be a good benchmark.
This looks to either be a stumbled on case or something contrived. He used -O4 for crying out loud (it's treated as -O3) there could be a bug from excessive optimization and the main function sprintf cannot be optimized as its a pre complied library (i could be wrong here).
The result is best summed up by the comment:
PyPy does nothing 1.9 times faster than C.
Re: (Score:2)
Re: (Score:2)
So what they are really saying is pypy is faster than a function in glibc for a task that glibc would never be used for (i would think you chose a modern library if you wanted to do this in some real code).
This is benchmarking not against C but a bloated and not particularly optimized library.
Re: (Score:1)
Re: (Score:2)
If you were going to do this as a one off sure you would use libc if you are going to do the same thing 10000 times in a row as you entire program and care about performance you would look for something better (this would likely be libboost in C++).
For anything to do with strings python is far more expressive than c and this case could be the more elegant and robust solution (except for memory usage). For this example i guess you 'parse' (not sure if that's the correct term in C) the text before the looping
Re: (Score:1)
Re: (Score:2)
We are talking about pypy being better than native programs so if C++ benchmarks are also valid. If this is just a contest to convert an integer to a string and write it to arbitrary memory as fast as you can and not factoring in pythons order of magnitude(s) greater expressiveness and readability. You are taking a standard data type and returning an array of standard data there no issues with other libraries. Using a copy and pasted integer to string function (itoa(...)) that has some checking of valid inp
Re: (Score:1)
PyPy solves a very hard problem, but is still slow (Score:5, Informative)
It's an impressive effort that they were able to get this to work at all. Python is really tough to optimize. All bindings are changeable at any time, even from another thread. (The latter is silly, but since it doesn't cost anything to do that in CPython, which is an inefficient naive interpreter that can only run one thread at a time and spends much of its time doing dictionary lookups. Von Rossum defines the language by what CPython can do. There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++. The Google team tried to fix that with Unladen Swallow, but gave up when their JIT system was barely faster than CPython.)
PyPy's most effective optimization to date is that it figures out when numbers don't have to be boxed. This allows generating numeric machine code, rather than grinding through the object machinery for every number. They have to be prepared to discard code when something binding gets changed. This requires a very complex system, involving two interpreters (regular and backup) as well as the JIT compiler.
The PyPy crowd is at last starting on the tougher optimizations, like hoisting some operations out of loops. (FORTRAN compilers were doing this in the 1960s.) That's real progress, but it's very hard to do in such a dynamic language.
Many of the optimizations involve generating run-time code that checks to see if the normal case is occurring, and that no other code has patched the code or changed the data from the outside in a way that invalidates the fast path. Then there's code to unwind what the fast path was doing, and interpret or recompile. Most such code is never executed.
Re: (Score:2)
There's a huge speed penalty for allowing such extreme dynamism, about 60x over C/C++.
That number is surely made up; perhaps it applies to some specific set of benchmarks? In reality the factor is going to vary widely depending on the problem. And oh, you can't lump C and C++ together and call them C/C++.
Re: (Score:2)
It matches what I've seen. I've written compilers, static analysers and visualisation code in Python. In most cases there is about a 100x difference in speed between the Python code and C code to implement the same algorithm. Python is still useful for prototyping in as most of that code could be written 5x to 10x more quickly than the C code.
Re: (Score:2)
Higher-level languages produce more asm per line but express more operations per line too. Easier to (validly) compare at the level of functions or whole programs.
If you get 60 times faster than that without changing your algorithms, even with hand-coded assembly and intimate knowledge of the hardware, you're probably bypassing the entire CPU and running on moonjuice.
Or spotting that something can be evaluated to a constant during compilation. That's a huge win when you can do it.
Of course, the other thing is that it is ever-so possible to write poor C code that can be out-performed by code generated from another language. A classic example of this is in string handling, where the C standard library routines c
Re: (Score:2)
Actually he just completely misread my post. The 60x increase was Python -> C, not from C -> assembly.
Re: (Score:2)
I noticed that their progress seemed to have stopped, is there any official announcement?
Re: (Score:2)
I noticed that their (Unladen Swallow) progress seemed to have stopped, is there any official announcement?
Unladen Swallow is an ex-parrot. [google.com]. "Jeffrey and I have been pulled on to other projects of higher importance to Google.", writes the former project lead.
Re: (Score:2)
Lisp yawns at how easily impressed you are.
And now for ruby please (Score:2)
And now for ruby please. I love ruby, but the performance issues drive me crazy sometimes.
Re: (Score:1)
Get a Mac and have a look at MacRuby. Near Obj-C Performance and JIT/AOT through LLVM.
Horrible Name (Score:2)
I'm sorry...I don't want to see how it looks or play with your PyPy. I will most definitely NOT click on your online demo to show your "Assembler".
I don't see this working (Score:2)
Seams boxing isn't too much of an issue for vanilla python when summing 1000,000 integers.
$ pypy -m timeit "sum(range(1000000))"
10 loops, best of 3: 55.2 msec per loop
$ python -m timeit "sum(range(1000000))"
10 loops, best of 3: 71.5 msec per loop
$ python3 -m timeit "sum(range(1000000))"
10 loops, best of 3: 62.7 msec per loop
$ python -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 26.6 msec per loop
$ pypy -m timeit "sum(xrange(1000000))"
10 loops, best of 3: 132 msec per loop
$ pypy --version
Python 2.7.1
Re: (Score:2)
Re: (Score:1)
Re: (Score:1)
Good idea. I say we shall henceforth pronouce this language as Pee-thon.
Re: (Score:3)
sorry for the accidental drunken redundant moderation. I deeply apologize.
Re: (Score:2, Interesting)
I'm a python fan, but I think you are being a little bit unfair with the ruby community.
Python simply has more people working on making it fast (the pypy project started 8 years ago, and only recently it started to deliver on its promises).
By the way, pypy is a framework for creating jitted dynamic languages (it's written in python, but can be used to implement any other language).
So chances are that soon, someone will write a Ruby implementation with pypy, and this ruby vm will enjoy all the performance of