Slashdot Log In
Help crack the Java 1.6 Classfile Verifier
Posted by
Hemos
on Mon Oct 31, 2005 08:40 AM
from the crack-it-break-it-open-jump-on-it dept.
from the crack-it-break-it-open-jump-on-it dept.
pdoubleya writes "As part of the development of Mustang (Java 1.6), Sun is developing a new, smaller and faster classfile verifier which they want your help in trying to break. As Sun VP Graham Hamilton puts it in his blog entry, "As part of Mustang we will be delivering a whole new classfile verifier implementation based on an entirely new verification approach. The classfile verifier is the very heart of the whole Java sandbox model, so replacing both the implementation and the basic verification model is a Really Big Deal.... The new verifier is faster and smaller than the classic verifier, but at the same time it doesn't have the ten years of reassuring shakedown history that we have with the classic verifier." You can read about the new verifier on Gilad Bracha's blog, and join the new Crack the Verifier initiative to if you can break it. Read all about the Crack the Verifier - Challenge."
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Only an Attaboy? (Score:2, Insightful)
I guess it could lead to more pay in some cases.
Re:Only an Attaboy? (Score:3, Insightful)
Take Java seriously (Score:4, Interesting)
Before those who go on to dismiss Java for various reasons (no matter how ignorant they are), take a look at the presentation given by Google at this year's JavaZone conference on how Google is using Java internally at extreme scales [javalobby.org]. Among them are AdWords and GMail.
"Extreme Scales" ? (Score:2)
In any case, the tools that impress me most are those that scale up, but scale down as well too. Google has the money and the brains to make anything work, more or less. What's more impressive to me is technology that lets Joe Schmoe get things up and running easily, and then still scale up reasonably well. I wrote a bit about it here: http://www.dedasys.com/articles/scalable_systems.h tml [dedasys.com]. Java scores just a bit better on this front than C and C++, because of the big standard li
Re:Take Java seriously (Score:5, Insightful)
They're probably using Java because its not as slow as its reputation, and its becoming a commodity language in the enterprise lately. My corporate overlords have dictated the use of Java (IBMs WebSphere) for all current and future enterprise development, and most of us developers couldn't be happier. Everywhere I do contracting work for lately also uses Java. Java Is A Great Language(TM), especially since 5.0.
There used to be a time when I believed that all techies had agreed that Java was slow and bloated, but once I stopped reading Slashdot comments so religiously I began to see some truth. It isn't slow, it isn't bloated, and it isn't something I expect the Slashdot crowd (that I'm a founding member of) to understand anytime soon.
Parent
Re:Take Java seriously (Score:3, Interesting)
Now, I need to explain that "heap NEVER shrinks" bit, because people are going to hop in and start talking a
Re:Take Java seriously (Score:3, Informative)
The only difference in this regard, is that java might alloc extra memory from the OS, when it could have run the garbage collector and reused some memory instead.
Very few programs actually ever release memory back to the OS.
Re:Take Java seriously (Score:3, Informative)
Re:Take Java seriously (Score:5, Insightful)
The parent post was obviously talking about ordinary client applications running on PC under either Windows, Linux, Mac or something simelar.
On such a system, malloc cannot map directly to the OS API, because the OS will only allocate full pages at a time. So if you want 40 bytes of memory, the OS would give you 1 KB (or whatever the page size is).
This also means that if you allocate 2x40 bytes, then free the first 40 bytes block, malloc can not free the page. It is simply not possible, since it would have to free the whole page.
Some J2ME implementations can defragment the memory in this situation, to be able to release memory back to the OS. That is impossible with a C program, where direct pointers to memory is allowed.
For large blocks you can of course go directly to the OS.
Parent
Re:Take Java seriously (Score:5, Informative)
Java will ALWAYS keep the 64MB heap. It will NEVER shrink it.
Who cares? This is completely irrelevant, as long as the JVM marks the pages it's not using as discardable, which modern JVMs on modern OSes do.
You have to remember that all memory is virtual. I can allocate a 1GB array, but as long as I never actually touch the pages (making them "dirty"), no storage, whether RAM or disk, is ever used. When the JVM allocates its 64MB, that virtual memory is initially all "clean" and therefore consumes no RAM. As it's used, it gets dirtied and physical RAM gets mapped to it... but when a garbage allocation occurs, both the Sun and IBM JVMs mark the now-unused pages as clean, allowing the OS to reuse them at will. Effectively, they no longer consume any space.
Even if the JVMs didn't mark the pages as clean, the impact of the JVM holding onto the 64MB wouldn't be that significant. The allocated, dirtied but now-unused memory will simply get swapped out to disk, allowing those pages of RAM to be used by other applications.
With a decent OS, it really doesn't matter if an app holds onto some memory that it's not using, especially if it has the decency to tell the OS that it's not actually using it.
That said, there is a "problem" here, it's just not the one you're pointing out. The problem, if you want to call it that, arises from the generational, copying, compacting garbage collector used by modern VMs. Now, don't get me wrong, this GC is very cool. It's significantly more efficient than typical malloc/free memory management, *and* it eliminates some major classes of memory leaks (programmers can still leak memory with GC, but it's harder).
Without getting into the details, although GC is safer and more efficient in terms of CPU cycles, and although on average it doesn't use a great deal more memory than manual allocation would use (particularly since many performance-sensitive manual allocation apps end up managing their own memory pools in order to avoid the run-time cost of malloc() and free()), GC does tend to increase the number of memory pages that get touched over time.
Why? Two reasons. First, suppose the application allocates a small chunk of memory, uses it, frees it, allocates another small chunk (about the same size), and then uses and frees it. Most malloc()/free() implementations will tend to return the same chunk of memory for both allocations. Repeating the process a thousand times (which isn't all that uncommon) will probably only dirty a single page of memory. With the sort of garbage collector used by current JVMs, every one of those allocations will return a different chunk of memory, and many pages will therefore be dirtied. In terms of CPU cycles, GC wins big, because, rather than a thousand small frees, there is one big one. And allocation is up to two orders of magnitude faster. It may sound like the GC approach is going to use a lot more memory, on average, but it's not that bad because of the tendency of malloc/free-managed heaps to become fragmented. On average, GC implementations don't have many more pages in use than malloc/free implementations, and may actually have less, but the GC allocators tend to "roam" across the pages.
Second, the "copying" nature of the GC. The main thing that makes generational, copying GC-based memory management so fast it that it never has to deal with fragmentation. To describe it in an oversimplified way: Every now and then, the GC relocates all of the in-use objects into a nice, compact block. That makes allocation fast, and tends to reduce the number of pages in active use, but it increases the number of pages that get dirtied and therefore require actual RAM to be provided by the OS. The copying also has a cost in CPU cycles, but it's small relative to the cost of managing and searching free lists, which is what malloc/free implementations have to do.
Theoretically, as the GC copies objects and marks the pages where they used to live as discardable, the OS coul
Parent
Re:Take Java seriously (Score:4, Informative)
Parent
Re:Take Java seriously (Score:4, Interesting)
Both Java and
p.s. Not bashing, just saying results of my testing. If you can suggest some approach, do that, but so far not even one person suggested something I haven't tried yet. Both Java or
Parent
Re:Take Java seriously (Score:5, Informative)
This will allow the vm to do small amounts of gc whenever it has a chance, as opposed to wating until an allocation request will fail and then running through the entire heap.
Our app runs 24/7 and while the markets are open and 10meg+ of raw data is coming down the line every second, we can't allow the app to take a timeout for a gc run. The app runs in 256meg, too.
Parent
Re:Take Java seriously (Score:3, Insightful)
Re:Take Java seriously (Score:5, Funny)
Yeah, it is a shame, if only Sun would do something like writing a new faster classfile verifier.
Parent
Re:Take Java seriously (Score:3, Informative)
They do: since (1.)5.0, JRE classes come preloaded in a shareable file.
Re:Take Java seriously (Score:3, Funny)
Re:Take Java seriously (Score:3, Insightful)
To the first point. Do you develop software? I do and it sucks big time if you code for Windows. You currently have to worry about DLL issues and what service packs are installed and what the heck Microsoft is going to throw at you in the next "hotfix". You say that you want to code for Linux?
Re:Take Java seriously (Score:5, Interesting)
A Java VM does some things that are simply not possible with C. To inline a C function, you need the source for both - this leads to some really ugly things like putting simple functions in headers, which should be reserved for interface definitions, not implementation details. The Java VM will inline functions on the fly. This can potentially give a huge performance boost - I got almost a 50% speed increase on some C code I was recently writing by shuffling things around to allow the compiler to inline some common functions.
The other advantage of higher-level languages is that they provide more semantic information to the optimiser. Consider the trivial example of autovectorisation. In C, if you want to do an operation on a vector, you will usually iterate over every element and perform the operation. The compiler then needs to check that there are no dependencies between loop iterations, which can be non-trivial. In a language like FORTRAN or Smalltalk, you can simply perform an operation on a vector type. The compiler then just needs to check if the operation you are trying to perform corresponds to one or more vector unit instructions, and substitute these in to your code. This is much easier to do.
C is a fairly easy language to write optimised code in for any CPU up to and including a 386. For anything more modern, you will find yourself fighting a language which is simply not designed to deal with parallelism - and compiler writers find themselves fighting even harder.
Parent
Re:Take Java seriously (Score:3, Interesting)
*rolls eyes*
LISP weenies and C++ gurus can knock Java. They've earned the right to do so. Anyone who has written code for one of the "scripting languages" people position as competitors to J
Re:Take Java seriously (Score:3, Insightful)
Also, the language is easily picked up, simple to write and the API's are quite sane compared to a lot of other languages.
Re:Take Java seriously (Score:3, Insightful)
Cross-platform byte code enables you to deploy the same application on your PC or Mac workstation and have it function exactly the same as on a 64-processor Ultra server. It also means your application is "future-proof". Deploy it now on a 32-bits machine, later on a 64-bits machine without recompiling AND run it at speeds comparable to native code.
While this is theoretically true for byte code, in Java isn't not something you
Re:Take Java seriously (Score:3, Insightful)
Certainly. Being cross-platform is always a big plus, didn't you read the Slashdot article about the large British retail chain that switched over all their POS terminals to Linux? They could do that thanks to Java. Of course, it is not COMPLETE platform independence, you are of course tied to the Java platform. You gain hardware and OS independence though, good en
Re:Take Java seriously (Score:3, Interesting)
Yes you do. The advantages of being able to develop on my local linux notebook and deploy to a Solaris cluster should not be overlooked simploy because it's important at dev time, not production time. Recompiling on another platform means retesting on that other platform. I'd rather run my unit and integration tests off the production & staging environments, load test in staging and no testing in production. This way unit and integration can be part of my build process (http://maven.apache.org/ [apache.org] and not
Re:Take Java seriously (Score:5, Interesting)
1. Cross-platform capability - Many companies still prefer to deploy applications on large Sun, IBM, or Linux (name your brand) servers. However, these companies would also like to give their developers Windows desktops so they can interact with the rest of the company. (Who most likely uses MS Office/Outlook.) As long as you avoid explicit path names, it is quite easy (and common!) to develop on a Windows machines but deploy on a Unix or Unix-like machine.
2. Automatic Memory Management - So your server is running along, and suddenly someone generates an unexpected error. In Java you can sleep soundly because even the worst programmer would have a hard time doing anything to completely take down the application. If you use a language that allows direct memory management, you have a good chance of that new guy coding a General Protection Fault/Segfault. The result is that your entire system coredumps when you least expect it.
3. Security - While Java is able to control the Security of the ENTIRE JVM through its security framework, most companies are happy with the lack of buffer overruns, code injection techniques, and other common attacks. That's not to say that a poor programmer can't put a security hole in the application wide enough to drive a Mack truck through, but at least you can rely on the underlying system not to betray you.
4. Flexibility - The Java server side frameworks are exceedingly flexible in their designs. For example, the servlet framework allows you to plug in your own custom server page technology. I have seen many a programmer (including myself) implement something like Reports by simply linking the ".rpt" extension to a custom servlet. The servlet then loads the requested configuration file and executes it. Very nice.
Another example is servlet filters. Need a security framework added in a hurry? Just add a filter servlet! It will execute before the rest of the code, allowing you to check the variables and security permissions to ensure that the client isn't trying any funny business.
5. API - When Java was first introduced, it absolutely creamed all the competing languages in the richness of its bundled API. As time has worn on, this has changed. However, Java still enjoys a sizable lead over even C/C++ with features such as Type IV (tested cross-platform, pure Java) JDBC database drivers. Unlike ODBC, many of these drivers have been tuned for excellent performance. Similarly, there are free APIs for handling Office Documents, PDF Creation/Editing, SOAP/XML-RPC communications, Object-Relational mapping, Image Management/Creation/Editing, CORBA, XML Databases, XSL-T, etc. While these APIs are all available for C/C++, there are significant cross-platform issues with many of them, as well as a lack of common "pluggable" APIs that allow for one API to many implementations.
Other languages have a hit/miss score with these sorts of features, often not providing these features, providing only a small subset, or only being available in an expensive commercial package.
6. Dynamic Loading - While C/C++ can manage dynamic loading of shared objects, it's a very difficult thing to implement. Java does it out of the box, with a full reflection API and interface support, thus allowing such wonderful code as Beans, Servlets, Pluggable Drivers, self-organizing code, and a host of other features that other systems can't compete with.
(If you don't believe me, try adding support for a feature in PHP sometime. "It's so simple! Just install the SO and recompile PHP!" Meh.)
7. Performance - This may sound like an odd thing to say, but the performance of Java is a key selling feature. Java server applications may execute more slowly than one written in C/C++ (just as C/C++ may execute more slowly than
Parent
Re:Take Java seriously (Score:3, Insightful)
I'd like to add "type safety" in there, but then I'm a programming language geek at the best of times.
Re:Take Java seriously (Score:3, Insightful)
you have no clue (Score:4, Interesting)
* java is nearly as fast as C++ according to all the benchmarks I've seen. Yes, really. The perception of java as being "slow" is simply the legacy of the old awt apps. Yes, the awt gui was (and is) slow. Server-side java applications are not. The "much better performance" is simply not there, particularly for typical enterprise apps.
* *All* the enterprise apps (which is the area where java is particularly successful) store stuff in a database and/or talk to remote apps. Newsflash: a database query or a remote procedure call is *orders of magnitude slower* than an in-process procedure call. Once you include DB/RPC into the equation, whatever little speed advantage C++ has is wiped out completely.
* This is CS 101: performance of a program is largely determined by the algorithm used. You can write a linear search in assembly, and it will be very fast for small lists. But for large lists, a binary search written in shell script will beat it.
* In an enterprise application scalability is much more important than raw speed. So what if I can write a C++ app that's 20% faster than an equivalent java app? Java has frameworks that make it easy to write an app that you can scale horizontally (i.e. by adding more boxes). Easy being the keyword.
* Developer's time is much more expensive than runtime. It is *much faster* to write an app in java than in C++. And for all but the smallest/simplest apps it is faster to write the app in java than in PHP/perl/whatever.
If it's a safety/security issue then again you could build the same thing in a native compiled language, sandbox and all.
Uhhhm, yes. Safety and security are *big* issues in enterprise apps. Show me *one* native language and platform that does it. You are saying it like one can just wave a magic wand and have it built in no time. "You could build the same thing" is not "it's already built".
I mean really, is it just because Java provides a lot of easy to use API's?
Yes. among all the other things I've mentioned.
These are just a few reasons why java is so popular in enterprise apps. Sure, I wouldn't write a game in java, but for enterprise apps, it's perfect. Why java and not PHP/perl/? Simply because java is better. It has all the advantages of compiled laguages (type safety, variable declaration checking, syntax checking, etc.) without some of the disadvantages (manual memory management). Think of java compiler as a sanity checker for your code. It will catch common mistakes like typos, missing return statements, invalid function parameters, etc. A scripting language will not complain about that, but force you to spend hours tracking down the bug. That's why java is faster to develop in than any scripting language for large apps.
Parent
Cross-platform really is a big deal (Score:3, Interesting)
Actually, yes -- the cross-platform ability is extremely useful. Speaking personally: the two biggest projects I have worked on, both for one client, are deployed (production) on a IBM iSeries server (these used to be called AS/400s -- using the OS/400 operating system), and a Solaris server respectively. Both web apps are built on the same code base, and we deve
Microsoft should take this approach (Score:2, Insightful)
I wonder what would happen if they junked the whole exclusive beta thing (which might get some of the more privacy-concerned, tech-savvy people on board? dunno - just a guess), and then actively encourage people to try and break the security? Surely that would produce better results than product x coming out, and then massive security problems fo
MS Anti spyware beta (Score:2, Informative)
Re:MS Anti spyware beta (Score:3, Insightful)
The most damaging problems I see and hear about are related to Windows and Internet Explorer. An open beta of those (I know about the I
More on Gilad Bracha (Score:5, Informative)
It's a relief that JDK 1.6 won't include any language changes (as far as I know?). Updating various parsers and whatnot to work with all the JDK 1.5 language changes was a big job, although some of the new features certainly are quite handy [blogs.com].
Re:More on Gilad Bracha (Score:3, Informative)
Re:More on Gilad Bracha (Score:2)
On the other hand, most folks I know are still on JDK 1.4, so I wonder how many people will move to 1.6? I even still get PMD [sf.net] parsing bug reports and whatnot for JDK 1.3...
Java 1.5 (Score:3, Interesting)
I was forced to use Java 1.2 some time ago, and found it a horrific experience with my background in dynamic languages. Since then, I've learned C++ and got used to the pluses and minuses of static languages (both in the sens
Optimization and late binding (Score:5, Insightful)
Not only does inlining eliminate method call overhead, but it allows you to re-run the peephole optimizer, which can eliminate range checks, reduce redudant type checks, etc.
The ultimate performance promise of Java is that it can do optimization very, very late in the process. Native libraries are basically black boxes in C/C++, and it's very hard to do that sort of inlining because most of the type information has been lost. Java may, someday, with sufficient ingenuity, rival or even beat C++ in performance, and it already does in certain limited areas.
Of course C# has all of the same advantages, and even though it's more recent there are some areas where its performance beats Java. I'd love to see all the Microsoft reasearchers vs. all the Sun researchers coming up with increasingly brilliant ways to take advantage of the late binding to turn a performance hindrance into a benefit.
Re:Optimization and late binding (Score:3, Interesting)
For long-running throughput-bound server processes, doesn't it already?
Not execution speed, but predictability (Score:4, Insightful)
Predictability and execution control are why I use C (and to a lesser extent c++) not Java. That cannot change for languages that give me no control over the raw execution path.
Garbage Collection (Score:3, Informative)
I believe that there is a concurrent garbage collector in Sun's JVM that while not as effecient over-all but runs continously preventing pauses and bubbles associated with traditional garbage collectors.
I'm my Java in a Nutshell 4th Edition (p. 246) one of the java(the interpreter) arguments is:
Re:Condition (Score:2, Informative)
https://mustang.dev.java.net/ [java.net]
Re:Java is overrated (Score:2)
I can safely say that Java doesn't suck, but on the other hand, .NET is an extremely nice technology. Java is certainly lagging behind (and has been for some time) in terms of suitability to desktop applications, but it's a solid base and is extremely useful for just about everything else.
dotNET is overrated (Score:3, Informative)
Java doesn't look like win32 because it isn't even trying to. It's trying to look platform-independant and the same on all platforms, with the option to skin it to any GUI you want. dotNET IS windows. There's no wonder that it looks a lot more like window
Re:dotNET is overrated (Score:3, Interesting)
C# makes a distinction between virtual and non-virtual methods (which is largely used for optimisation which is not available otherwise, as I understand it). The distinction between override (which seems a little unnecessary, but it's arguably better than ambiguity) and new stems from this.
I wouldn't say there's a huge difference between C# and Java, certainly not of the kind you're trying to imply there is. C#'s syntax is a little closer to C++, not so much the Windows API.
Re:dotNET is overrated (Score:3, Funny)
I mean:
"It's a floor wax!"
"It's a desert topping!"
"No, it's both! New Schimmel is a floor wax and a desert topping!"
(This is from early Saturday Night Live for the humor impaired)
Re:dotNET is overrated (Score:3, Interesting)
Re:Aren't QA people supposed to get paid? (Score:5, Insightful)
Also it is an opportunity for someone to get recognition for breaking a new peice of software.
It is important to get extra scrutiny on newly designed peices of software, for it is the new designs that usually break in the least expected ways.
Parent
Re:Don't believe the hype! (Score:3, Interesting)
I agree that C is portable and fast, however I don't it can be called very complex.
The smallest programming language manual I have ever owned (and I've owned quite a number), has to be "The C Programming Language", often hailed as the One True Reference to the language. How can it be that complex if the manual is less than half the size of most of my other manuals? I think languages (in ge
Re:Why not prove it? (Score:5, Insightful)
Did you just walk out of an undergrad Computer Science class?
Popping in pre/postconditions and doing line-by-line proofs doesn't cut it for an application of this complexity. While that is an important part of a real process, it doesn't guarentee coverage. You still have to make assumptions about the environment, which is the gotcha. Testing and QA is all about the assumptions you make and the boundaries you set. With a complex application the number of factors grows so large, that you cannot have the resources to cover every possible test. You can grab the most common stuff, but really need to dump it to the community to get the real 'out of the box' thinking hitting it.
Parent
Re:Why not prove it? (Score:4, Interesting)
One thing's for sure: Improvements in software quality will be harder to come by if everybody's attitude continues to be "Bugs are inevitable. Formal proofs are beyond us. Let's keep doing it the old way."
Parent