Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Hardware Virtualization Slower Than Software?

Posted by Hemos on Sun Aug 13, 2006 03:17 AM
from the the-jury-is-still-out dept.
Operating Systems Software
Jim Buzbee writes "Those you keeping up with the latest virtualization techniques being offered by both Intel and AMD will be interested in a new white paper by VMWare that comes to the surprising conclusion that hardware-assisted x86 virtualization oftentimes fails to outperform software-assisted virtualization. My reading of the paper says that this counterintuitive result is often due to the fact that hardware-assisted virtualization relies on expensive traps to catch privileged instructions while software-assisted virtualization uses inexpensive software substitutions. One example given is compilation of a Linux kernel under a virtualized Linux OS. Native wall-clock time: 265 seconds. Software-assisted virtualization: 393 seconds. Hardware-assisted virtualization: 484 seconds. Ouch. It sounds to me like a hybrid approach may be the best answer to the virtualization problem. "
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Hardware Virtualization Slower Than Software? 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Sponsored by VMWare.. what do you expect? (Score:5, Insightful)

    by thegrassyknowl (762218) on Sunday August 13 2006, @03:21AM (#15897444)
    See title... VMWare make software virtualisation products. Of course they're going to try and find that software methods are better.

    • Even so, they may be at least partially right.

      Besides, if a hybrid approach is necessary, VMWare will need to adjust as well. Or am I missing something?

      • Re:Sponsored by VMWare.. what do you expect? (Score:5, Informative)

        by mnmn (145599) on Sunday August 13 2006, @03:48AM (#15897487) Homepage
        If you search back on Vmware vs Xensource, you'll see Vmware is doing everything to discredit Xen and hardware hypervisors. Instead of saying 'it doesnt work' its more effective to say it works, we have it too, it fails on its own so it needs our software too. From everything I've read about hypervisors including the Power CPU hypervisors from IBM (which have been functional for years) and the original Cambridge paper that created Xen, Hypervisors really outperform software solutions. You do need a software mini-OS as the root on top of which you'd install the OSes which is better than using Windows as the root OS.

        But Vmware's agitation is understandable. They're about to lose it all to an open source project. Where have I seen this before?

        • Re:Sponsored by VMWare.. what do you expect? (Score:5, Informative)

          by julesh (229690) on Sunday August 13 2006, @05:46AM (#15897673)
          From everything I've read about hypervisors including the Power CPU hypervisors from IBM (which have been functional for years) and the original Cambridge paper that created Xen, Hypervisors really outperform software solutions.

          Note that Xen's original hypervisor implementation *is* a software solution -- it relies on rewriting the guest operating system kernel so that the kind of hardware traps that VMware are talking about here are unnecessary. Note that it worked flawlessly before the virtualisation technology (eg. Intel VT) that VMware is testing was avialable.

          • This won't be the first time software beats hardware.

            The original Stacker product was a combination of a hardware card and software. Think of the hardware card as an accelerator for doing the comression/decompression.

            The hardware was faster on the oldest machines, but on anything above a 286/12 (I had a 286/20 at the time), or almost any 386, it ran faster without the hardware card. And on every 486, the card was useless.

            So, while you may want to "consider the source" of this news, this is only one f

        • Where have I seen this before?

          Citrix?
          Not an open source product and not lost it to an open source product, but they made a product that has been largely made superflouos because MS built it right into the OS.
        • Where have you seen VMware discrediting XenSource? I haven't seen that. Can you back this up with some links? Searching for "VMware vs Xensource" was fruitless for me. And searching for "VMware discredits XenSource" was also fruitless.

          But Vmware's agitation is understandable. They're about to lose it all to an open source project. Where have I seen this before?

          I'll let you in on a secret: if you consider all costs, and return on investment, using VMware is a competitive advantage over using Xen.

    • Haven't read it, but I wonder if they were using VT/Pacifica chipsets or no...

      It's like Apple's claim that their Intel jobbies are 5x faster - a bit silly and very, very specific...

      And yes, VMWare are hardly likely to mention that Xen-style virtualisation is going to be better now, are they?

    • Re:Sponsored by VMWare.. what do you expect? (Score:5, Insightful)

      by XMLsucks (993781) on Sunday August 13 2006, @03:53AM (#15897496) Journal
      VMware sells both hardware-accelerated and software virtualization products. They implemented full support for VT (how else would they benchmark it? Plus they were the first to support VT). If you run VMware on 64-bit Windows, then you use VMware's VT product. But because VMware's original software method is faster than the VT method on 32-bit, they continue to use the software approach.

      VMware's paper is a typical research paper, published at a peer-reviewed conference. This means that they have used the scientific method. The chances are 99.9999% that you will easily reproduce their results, even if changing the benchmarks.

      I, on the other hand, am smart enough to see that they are stating the obvious. If you read the Intel VT spec, you'll see that Intel does nothing for page table virtualization, nor anything for device virtualization. Both are extremely expensive, and besides sti/cli, are the prime candidates for hardware assists. Intel will likely solve this performance issue in future revs, but right now, VT isn't fast enough.

      Hmmm, virtualisation? Do you happen to work on Xen?

       
      • I see you mentioned most of what I was 'clarifying' in my post. Sorry, I didn't read the whole thing ;-). So about the only new info is that VMware runs both 32 and 64-bit AMD VMs in software mode.
      • If VMWare's solution still needs a host OS (I remember them using stripped-down Linux for their server offering), then no... they might use use a subset of VT, but its not a true hypervisor.

        And by the way... yes... device virtualization is still not there, but your page tables claim is bullshit. If you read the VT (and the SVM) docs, you would realize that you can implement shadow page tables RIGHT NOW. The hardware assists are there.

    • Re:Sponsored by VMWare.. what do you expect? (Score:5, Informative)

      by Anonymous Coward on Sunday August 13 2006, @04:02AM (#15897513)
      See title... VMWare make software virtualisation products. Of course they're going to try and find that software methods are better.

      Disclaimer: I work for VMware.

      1. VMware already supports VT, but it's not enabled by default because for normal workloads it's slower. If VT really were faster, do you really think we'd be choosing to use a slower approach and making customers unhappy?
      2. Even Intel admits the first generation of VT hardware wasn't so great and now claims that they were aiming for correctness instead of performance:

      • I designed h/w virtualization (Score:3, Interesting)

        by Anonymous Coward
        I designed one of the x86 h/w virtualization offerings. It's obvious that outside of device emulation, the biggest overhead of virtualization is the s/w emulation of what amounts to two levels of address translation (especially hairy in multiprocesor systems due to the brain-dead x86 page table semantics that do not require explicit invalidation). So clearly you want nested-paging support in h/w. However, that support is a little more complex than a few microcode changes to trap selected privileged instruct

        • Yes, AMD Pacifica seems to be far better (Score:4, Interesting)

          by Morgaine (4316) on Sunday August 13 2006, @09:27AM (#15898072)
          Is AMD's Pacifica virtualisation system any better?

          Apparently, yes, and by a good margin.

          There are several documents and articles out there which point out VT's problems and how Pacifica is quite dramatically better. Here's an excerpt from "AMD Pacifica turns the nested tables" [theinq.net], part 3 of an informative series of articles:

          • The basic architecture of the K8 gives AMD more toys to play with, the memory controller and directly connected devices. AMD can virtualise both of these items directly while Intel has to do so indirectly if it can do so at all.

            This should allow an otherwise identical VMM to do more things in hardware and have lower overhead than VT. AMD appears to have used the added capability wisely, giving them a faster and as far as memory goes, more secure virtualisation platform."

          So, it looks like AMD are ahead on hardware virtualization at the moment.

          If I read it correctly, this is because Intel's VT actually requires a lot of software intervention, so it's not actually a very strong hardware solution at all.

    • Re:Sponsored by VMWare.. what do you expect? (Score:5, Informative)

      by arivanov (12034) on Sunday August 13 2006, @04:09AM (#15897531) Homepage
      While they offer software virtualisation products, they are also interested in these products having hardware assistance. The AMD and Intel specs were designed with input from them (amidst other vendors).

      As far as the results there is nothing surprising here. This has happened before. Fault driven emulation of 80287 was nearly 50%+ slower than compiled in emulation. There were quite a few other examples x86 which all revolve around the fact that the x86 fault handling in protected mode is hideously slow. Last time I have had a look at it in asm was in the 386 days and the numbers were in the 300 clock cycle range for most faults (assuming no wait on memory accesses). While 486 and Pentium improved the things a bit in a few places, the overall order remains the same (or even worse, due to memory waits). Anything that relies on faults in x86 is bound to be hideously slow.

      Not that this matters, as none of the VM technologies is particularly caring about resources. They are deployed because there is an excess resource in the first place.
    • More importantly they sell virtualization products that do not support VT, and their primary competitor does.

  • The real question is what type of test was performed... It would make sense that different applications would function differently in a variety contexts. How about some variance? I dig VMWare, but come on...
    • So why don't you actually read the paper? It has quite a good explanation of what they did. FWIW, it wasn't a clear win for software; there were things the hardware implementation did better, but they're things that don't seem to be quite so important for real-world applications.
  • Software-assisted virtualization: 393 seconds. Hardware-assisted virtualization: 484 seconds. Ouch. It sounds to me like a hybrid approach may be the best answer to the virtualization problem.
    So, um, a hybrid approach is better because it will take 439* seconds? Why?

    * - I imagine in real life it's not a 1:1 ratio, but for the sake of argument, work with me.

    • I suppose there are certain things hardware virtualisation does better.

      The trick is, I'd guess, to find out which works better in which circumstances.

      You see that people suspect this white paper because of its origin; they are right in doing so at least because only one type of test has been performed; surely not all computing tasks perform the same way as a kernel compile.
      This suggests that VMWare have found the example which supports their claims the best; the question is, of course, whether this is th

    • It probably won't work that way at all. This could be more of an additive thing.

      For example, say you have a boat powered by 393 horsepower engine and a 484 horsepower engine. If you run them both at the same time, the net power is not going to be 439hp.

      Software+hardware won't add in nearly the same way, but I wouldn't be surprised if a hybrid approach was %50 faster than either method alone.
    • Because if you actually RTFA it shows that the hardware virtualization is faster for some benchmarks (e.g. processing system calls) and slower for others (e.g. performing I/O requests or page-table modifications); if you combine the best features of each you should be able to get a virtual machine that is faster than both.

  • The correct conclusion is more limited (Score:5, Insightful)

    by njdj (458173) on Sunday August 13 2006, @03:37AM (#15897469)

    The correct conclusion is not that virtualization is better done entirely in software, but that current hardware assists to virtualization are badly designed. As the complete article points out, the hardware features need to be designed to support the software - not in isolation.

    It reminds me of an influential paper in the RISC/CISC debate, about 20 years ago. Somebody wrote a C compiler for the VAX that output only a RISC-like subset of the VAX instruction set. The generated code ran faster than the output of the standard VAX compiler, which used the whole (CISC) VAX instruction set. The naive conclusion was that complex instructions are useless. The correct conclusion was that the original VAX compiler was a pile of manure.

    The similarity of the two situations is that it's a mistake to draw a general conclusion about the relative merits of two technologies, based on just one example of each. You have to consider the quality of the implementations - how the technology has been used.

    • The Intel processor design has been a pile of manure ever since the first 8086. On the other hand, the IBM zSeries range of computers has been doing virtualization since the 1960s, and presumably the hardware has been designed to get it right. Can anyone give comparable performance figures for programs running in a virtual machine or the bare metal for a zSeries machine?
      • You can't run on bare metal on zSeries. The whole architecture just isn't designed to work that way. It has to have the virtualisation layer, or at least something that provides nearly all of the functionality of what would traditionally be considered a virtualisation layer.

      • Re:The correct conclusion is more limited (Score:5, Interesting)

        by TheRaven64 (641858) on Sunday August 13 2006, @06:09AM (#15897712) Homepage Journal
        The easiest architecture to virtualise is the Alpha. It had a single privileged instruction, and all that did was shift to a higher privilege mode (which had a few shadow registers available) and then jump to an address in firmware. The firmware could be replaced by using one of these calls. If you wanted to virtualise it then you could do so trivially be replacing the firmware with something that would check or permute the arguments and then vector off into the original firmware.

        It also had a few other advantages. Since you were adding virtual instructions, they all completed atomically (you can't pre-empt a process in the middle of an instruction). This meant you could put things like thread locking instructions in the PALCode and not require any intervention from the OS to run them. The VMS PALCode, for example, had a series of instructions for appending numbers to queues. These could be used to implement very fast message passing between threads (process some data, store it somewhere, then atomically write the address to the end of a queue) with no need to perform a system call (which meant no saving and loading of the CPU state, just jumping cheaply into a mode that could access a few more registers).

      • The IBM iSeries (identical to the pSeries hardware) also have a hardware HyperVisor.

        Their entry models (10k US$) are slow as shit though. Can't say anything about the more expensive machine, but anything that requires around 12 hours to upgrade it's operating system can't be trusted.
    • >The naive conclusion was that complex instructions are useless. The correct conclusion was that the original VAX compiler was a pile of manure.

      Note that the 'naive conclusion' and the 'correct conclusion' are not contradictory: I remember an article recently where it was shown that the Alpha had three times the power of a correspondig VAX, which made nicely the point that CISC is shit.

      Now as Intel has shown, given enough efforts and money even x86 the poorest CISC ISA ever (VAX ISA was much nicer than x

    • The naive conclusion was that complex instructions are useless. The correct conclusion was that the original VAX compiler was a pile of manure.

      Perhaps the intended conclusion was that it was feasible to write an efficient compiler using only a small, intelligently chosen with compiler optimization in mind, subset of the instruction set. Perhaps the fact that the original compiler was (as you assert) "a pile of manure" was not unconnected to the fact that it tried to achieve speed by exploiting the entir

  • hardware v/s software (Score:3, Insightful)

    by toolz (2119) on Sunday August 13 2006, @03:44AM (#15897479) Homepage Journal
    When are people going to figure out that "hardware solutions" are really software running on hardware, just like any other solution?

    Sure, the instructions may be hardcoded, coming out of ROM, or whatever, but in the end its instrructions that tell the hardware what to do. And those instructions are called "software", no matter how the vendor tries to spin it. And if the solutions performs badly, it is because the software is designed badly. Period.

    • I don't see how this is a significant distinction. The question, in terms you might prefer, is how virtualization using specialized hardware compares to doing the same thing in general purpose hardware. There doesn't seem to be any semantic difference. Are you just pointing out that a hardware implementation's performance is predicated on its design?
    • In the end, the software instructions are actually executed on hardware, and that hardware imposes limits on what they do. In the case of virtualization the problem comes with privlidge levels. Intel processors see 4 levels of privlidge called Ring 0-3, of which two are used by nearly all OSes, 0 and 3. The kernel and associated code runs in Ring 0, everything else in Ring 3. Now the effect of what ring you are in controls what instructions the processor will allow you to execute, and what memory you can ac

        • Re:No not really (Score:4, Interesting)

          by Sycraft-fu (314770) on Sunday August 13 2006, @08:01AM (#15897876)
          I haven't read the results, and I doubt I have the technical knowledge to properly analyze them properly. However if I were to guess as to why this might be the case I'd say it's because they didn't do it right. This is a new and fairly complex technology, I somehow doubt it's easy to get right on the first try.

          I am not willing, based on a single datapoint, to make any conclusions. That's tanget to my point anyhow, my point was that doing something in hardware and software are quite different.
          • However if I were to guess as to why this might be the case I'd say it's because they didn't do it right.
            Holy crap, you just bloated "They're wrong." into 26 words. Do you work as a government advisor in your free time?
  • It sounds to me like a hybrid approach may be the best answer

    As so many times and so many cases before has it proven to be the optimal solution. What gives ? Good is that we have all these alternatives, and every vm company will try to evaluate, then optimize, which will lead to better performing software VMs, and because hw is slower to catch up, probably software VMs will be better for a while.

  • wrong (Score:4, Insightful)

    by m874t232 (973431) on Sunday August 13 2006, @04:04AM (#15897519)
    Hardware virtualization may be slower right now, but both the hardware and the software supporting it are new. Give it a few iterations and it will be equal to software virtualization.

    It may or may not be faster eventually, but that doesn't matter. What matters is that small changes in the hardware make it possible to stop having to depend on costly, proprietary, and complex software--like that sold by VMware.
    • Yes, let's move it to our costly, proprietary, and complex hardware instead!

      Not to say that you're wrong or that hardware should be free-as-in-freedom, but the irony was to great to resist.

    • Not just the CPU (Score:5, Interesting)

      by kripkenstein (913150) on Sunday August 13 2006, @04:55AM (#15897610) Homepage
      What matters is that small changes in the hardware make it possible to stop having to depend on costly, proprietary, and complex software--like that sold by VMware.

      I am 100% in favor of cheap and open solutions. But I don't agree that this will soon be the case for virtualization. VMWare and the few other major vendors do a lot more than software virtualization of a CPU (which is all TFA was talking about). To have a complete virtualization solution, you need to also virtualize the rest of the hardware: storage, graphics, input/output, etc. In particular graphics is a serious issue (attaining hardware acceleration in a virtual environment safely), which from last I heard VMWare were working hard on.

      Furthermore, Virtualization complements well with software that can migrate VMs (based on load or failure), and so forth. So, even if hardware CPU virtualization is to be desired - I agree with you on that - that won't suddenly make virtualization as a whole a simple task.
      • In particular graphics is a serious issue (attaining hardware acceleration in a virtual environment safely), which from last I heard VMWare were working hard on.

        Actually, the people who have made the most headway are Microsoft. The Vista driver model is designed for support for virtualisation in mind. This means that the OS has access to video driver commands for things like saving and restoring GPU state. As far as I know, other operating systems currently lack this; Linux has a problem even switchi

  • Use Paravirtualization (Score:4, Insightful)

    by graf0z (464763) on Sunday August 13 2006, @04:28AM (#15897560)
    Paravirtualization (running hypervisor-aware guest kernels, eg patched linux on xen) is faster than both, binary translation and "full" virtualization. And you don't need CPUs with VT extension.

    g

    • The whole point of virtualization is so you can run your favoriate OS most of the time, and only switch over to Windows when you want to run games, isn't it?

      Well, I suppose if choose to stay in the Windows world most of the time, the whole point of VM is to try to keep malware off your computer... But either way, you're not getting a FLOSS paravirtualized Windows kernel any time soon.

      • Eerrr ... tautologically true, yes: if there is no paravirtualized version of the OS you want to use, paravirtualization is not an option. But there are many scenarios where you are only interested in running lots of paravirtualizable unixish OSes, eg server farming.

        Your windows desktop is not the whole point.

        g

        • It just seems like many people who try to move away from Windows seem to want to at least have the option to use Windows once in a while.... The Mac-moving-to-Intel thing was met with a lot of excitement because of this, a lot of linux people seem to say this, and it seems like in a lot of companies employees must be productive with specific document formats. Certainly Windows isn't the only point of virtualization, but it seems like it's a really big one, especially for desktop users.

  • Look to IBM (Score:3, Informative)

    by dpilot (134227) on Sunday August 13 2006, @05:40AM (#15897665) Homepage Journal
    IBM has been shipping virtualization since before many of these newcomers were even born. What do you think the 'V' in MVS or VM stands for? I wonder how well IBM's expired patents compare to modern virtualization. Of course in this case it helps to own the hardware, instruction set, and operating system.

    • Re:Look to IBM (Score:4, Interesting)

      by pe1chl (90186) on Sunday August 13 2006, @06:02AM (#15897699)
      IBM's VM also started as a software product that had to cope with virtualisation problems in the hardware.
      Just like what is happening now, they added specific support to the hardware to make VM perform better.
      This all happened before the development of today's architectures, but in the early days of microcomputing, IBM had the position that Microsoft has today: they were the big company that had 90% of the market, and in the eyes of the newcomers all they did was by definition the wrong thing. So nobody would bother to look at 360 mainframes, VM and how it was done before designing their own processor.
      (this would be similar to telling a Linux geek to look at how certain problems are solved in Windows... it is Windows, it is Microsoft, so it has to be the wrong solution)

      • I think that's a little innacurate (Score:4, Insightful)

        by Sycraft-fu (314770) on Sunday August 13 2006, @08:16AM (#15897902)
        It's not that people don't look to old mainframe solutions for things, they do, it's that often what was feasable on those wasn't on normal hardware, until receantly. There was no reason for chip makers to waste silicon on virtualization hardware on desktops until fairly receantly, there just wasn't a big desktop virtualization market. Computers are finally powerful to the point that it's worth doing.

        It's no supprise that large, extremely expensive computers get technology before home computers do. You give me $20 million to build something with, I can make it do a lot. You give me $2000, it's going to have to be scaled way back, even with economies of scale.

        You see the same thing with 3D graphics. Most, perhaps even all, the features that come to 3D cards were done on high end visualizaiton systems first. It's not that the 3D companies didn't think of them, it's that they couldn't do it. The orignal Voodoo card wasn't amazing in that it did 3D, it was much more limited than other thigns on the market. It was amazing in that it did it at a price you could afford for a home system. 3dfx would have loved to have a hardware T&L engine, AA features, procedural textures, etc, there just wasn't the silicon budget for it. It's only with more developments that this kind of thing has become feasable.

        So I really doubt Intel didn't do something like VT because they thought IBM was wrong on the 360, I think rather they didn't do it because it wasn't feasable or marketable on desktop chips.
    • IBM contribute to Xen. I was at a talk last year by one of the IBM Xen guys. He made the point that IBM has a real advantage in virtualisation because, when they get stuck, they can pop along the hall to the grey-bearded mainframe guys and say 'hey, you remember this problem you had twenty years ago? How did you solve it?'

  • Parallels on Mac OS? (Score:3, Interesting)

    by akac (571059) on Sunday August 13 2006, @09:28AM (#15898080) Homepage
    Well OK. But it could also mean that VMWare doesn't know yet how to properly create a hardware virtualized vm.

    Parallels on OS X switches between software and hardware virtualization and using hardware virtualization its about 97% the speed all around of native hardware (consider that virtualization on current Yonah CPUs is equal to one core only). Software virt on Parallels is much slower - on par with running Windows Virtual PC on the same box using Windows XP (not Mac Virtual PC).

    • Re:Bias? (Score:4, Insightful)

      by RegularFry (137639) on Sunday August 13 2006, @03:56AM (#15897502)
      Insisting on third-party verification of results is hardly damning either of them... It's just scientific. You (and everyone else) are absolutely right to be sceptical, and not just because VMware have a vested interest in this case. They might just be wrong. Or not.
    • I don't see how that tracks. How is the %2 impact going to save me a bundle? Moving to linux suposedly will save me money if I virtualize or not, don't see how it being virtualization friendly improves things. Are you saying I'll spend less in hardware by switching to linux? Migrating to linux isn't free (man-hours wise), so the hardware savings better be pretty damn substantial to offset it.

      I should be sleeping.

      Rich

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.