Slashdot Log In
EC2 Vs. App Engine Vs. GoGrid Vs. AppNexus
Posted by
kdawson
on Wed Jul 23, 2008 03:04 PM
from the seeing-shapes-in-the-clouds dept.
from the seeing-shapes-in-the-clouds dept.
snydeq writes "InfoWorld's Peter Wayner delves into the ill-defined realm of 'cloud computing,' providing a deeper look at four shared services: Amazon EC2, Google App Engine, GoGrid, and AppNexus. Offering wildly divergent amounts of hand-holding at various layers in the stack, the services simplify your workload but force you into a set, 'ball-and-chain-computing' routine that you may not prefer. Sure, the services allow you to pull CPU cycles from thin air whenever you need to, but they can't solve the deepest problems that make it hard for applications to scale gracefully, Wayner writes. He describes these 'clouds' as an evolving experiment, rife with potential but 'far from clear winners over traditional shared Web hosting.' The sobering look at the trend includes a QuickTime tour of each service — EC2, App Engine, GoGrid, AppNexus (those links all .MOV)."
Related Stories
[+]
Technology: Multiple Experts Try Defining "Cloud Computing" 117 comments
jg21 writes "Even though IBM's Irving Wladawsky Berger reports a leading analyst as having said recently that 'There is a clear consensus that there is no real consensus on what cloud computing is,' here are no fewer than twenty attempts at a definition of the infrastructural paradigm shift that is sweeping across the Enterprise IT world — some of them really quite good. From the article: 'Cloud computing is...the user-friendly version of grid computing.' (Trevor Doerksen) and 'Cloud computing really is accessing resources and services needed to perform functions with dynamically changing needs. An application or service developer requests access from the cloud rather than a specific endpoint or named resource.' (Kevin Hartig)"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
The definition of cloud computing is still vague (Score:5, Interesting)
Re:The definition of cloud computing is still vagu (Score:4, Funny)
you mean the definition of cloud computing is still cloudy?
Parent
Re: (Score:2)
you mean the definition of cloud computing is still cloudy?
My Magic 8-Ball(tm) says the outlook of cloud computing is cloudy.
Re: (Score:2)
Re:The definition of cloud computing is still vagu (Score:2)
I would categorize cloud computing as derivative of grid computing, if you will. You throw some crap at the beast, but unlike grid computing, there can be many independent cells working completely disconnected from the rest, possibly even unaware of them or even unable to communicate between one another.
Like the clouds in the sky, they don't need to be connected or aware of each other for it to rain.
Re:The definition of cloud computing is still vagu (Score:3, Funny)
So you're saying you fail Rowell's Extension to Einstein's Test of Comprehension? Sad, sad day.
Einstein: You do not really understand something unless you can explain it to your grandmother.
Rowell's Extension: You totally don't understand something if you can't even explain it to a bunch of geeks. Also, they'll probably laugh at you.
Re:The definition of cloud computing is still vagu (Score:5, Insightful)
I think the best way i've heard it explained is:
"When details of implementation are sufficiently hidden away that you no longer have to think about them, people often draw a 'cloud' around it, just like you do with the internet where (most of us) don't have to worry about all the wires and the protocols but it's just there, and it just works.."
Cloud computing is trying to draw the same cloud around.. computing (resources), you don't have to worry about connectivity, electricity, how to make db's and file systems scale across systems.. it's an abstract cloud that's just there without having to worry about it.
Parent
Re:The definition of cloud computing is still vagu (Score:5, Informative)
'the cloud' is old networking/telephony terminology. Describing interconnection of two sites, you'd diagram the systems at either end, and their local links, but once the links enter the network you don't know or care how the routing happens (generally). This part of the network was 'the cloud' (and was diagrammed as a cloud).
By inference, cloud computing would be where you know the computation is happening somewhere on the network, but you neither know or care exactly where.
See this thread back in 1995 -
http://groups.google.co.uk/group/bit.listserv.techwr-l/browse_thread/thread/d6384bd640275c43/14da0963ed1c294a?hl=en%0Eda0963ed1c294a [google.co.uk]
Or the first diagram in RFC 1587 (1994):
http://rfc.dotsrc.org/rfc/rfc1587.html [dotsrc.org]
I joined a telecoms company the year before that and the term was in use there, can't vouch for earlier.
Parent
Re: (Score:2)
From the way cloud computing is implemented with EC2, my basic definition of cloud computing is that it is "On Demand" computing. Unlike the Internet, which is a series of tubes, cloud computing is like a bunch of trucks that you just dump stuff on. If your servers get too busy, just dump the work on more trucks.
For instance, it allows a small web site to survive the slashdot effect by starting up a dozen servers for a few hours or days, costing much less than having a dozen servers running 365.2425 days a
I prefer Google for Cloud Computing... (Score:5, Funny)
oh wait. I do know what google does - It makes the internet better... and it prints money (I guess...)
Re:I prefer Google for Cloud Computing... (Score:5, Informative)
Of the four, Google seems to be the most limiting, at least on the surface. If I understand correctly, Google's offering requires the app to be written in Python and it denies some Python functionality such as file writes.
Amazon's offering, sitting at the other end of the spectrum, allows you to run a full instance of Linux complete with root level access.
The other two are not as confining as Google but more restrictive than Amazon.
On a side note, spam raining from the cloud has become a problem for at least Amazon. Some blacklists are blocking IP addresses owned by Amazon's EC2. If you want to run a mail server in the cloud, it just might rain on your parade.
Parent
Re: (Score:2)
Re: (Score:3, Interesting)
"I've been looking into this a bit, and the amazon option seemed the best."
I also looked a bit into Google and Amazons offerings for at Python project. Google was definitely the cheapest and if I could squeeze my project into the limitations they have established I would have chosen it. Unfortunately it is not possible to install C and Fortran extensions to Python (due to security reasons, you can install pure Python modules). This was a showstopper for me.
The critique of not providing access to a local fil
Wow (Score:5, Interesting)
Finally, a burst of common sense on the latest hype. Hosted servers have offered many of the benefits you get out of "cloud" computing for years, without locking you into a particular vendor or platform. With virtualization, you should be able to build your own images and farm them out to hosting companies, using your technology and platform of choice. Clustered ESX and SANs already give us the resource scalability we need for most systems, partitioning finishes the job. You can just pay a hosted server company to host your vmware image on their ESX cluster and scale up your storage as needed on their SAN. The key is that YOU build a scalable design.
I highly doubt a majority of businesses are going to lock themselves into one hosting provider's specific development platform just to take advantage of hosted servers that push themselves into the next layer.
Re: (Score:3, Informative)
I highly doubt a majority of businesses are going to lock themselves into one hosting provider's specific development platform just to take advantage of hosted servers that push themselves into the next layer."
This depends on the service. Amazon's ec2 is basically just Xen virtual servers provisioned on the fly. s3 is a little weird but there are plenty of tools available to use it in whatever your application is running on. Code changes to support it are not all that difficult.
Re: (Score:2)
I agree, for the "cloud" systems like Amazon's EC2, but then isn't this just managed/hosted servers plus clustered virtualization plus their proprietary database system (S3)? You're still moving from fairly industry standard SQL (despite so many damned vendor extensions to SQL), to Amazon's S3 storage. Similar to how part of the "value" Google is trying to add is locking you into their non-SQL database storage.
I certainly understand that SQL has shortcomings, but is vendor lock-in, especially hosting prov
Re: (Score:2)
You are confusing S3 with Amazon's SimpleDB service. I currently don't use SimpleDB. We use a SQL relational database cluster running on ec2 instances. We backup/recover off of S3. Going to another host is a matter of redoing the backup scripts (easy) and changing the asset server settings from pointing to S3 to wherever the data is now being stored.
Re: (Score:3, Interesting)
You're right, I did have those confused.
So it sounds like you have hosted virtual servers and some hosted SAN storage. That's cool, and it's a smart way to do business.
It's only when people call it "cloud" and act like it's something crazy and new beyond a combination of virtualization and SAN storage, managed by someone else just doesn't seem like it's worth all this hype.
I have a Vmware ESX cluster. I have an EMC SAN. They're supported through contracts with my reseller. When there are problems, the h
Re: (Score:2)
Yeah in my last job which wasn't a web business we went the virtual server attached to SAN route. It's great tech and works well for HA and expansion. The key difficulty is that there is a high cost of entry. Yes you could also go with having someone host this for you but at least with the vendors I've talked to there is always a long term contract associated.
You are right that on the technical side there isn't much that's very new about "cloud" computing. The key difference in these offerings is that with
Re: (Score:2)
Oops missed an edit - I wanted to say if Amazon messes up (see the s3 outage last weekend) there isn't anything you can do about it. Which is one reason why I feel more hesitant about using their more specialized services (such as simple db).
Re:Wow (Score:4, Informative)
SQL's scalability is next to nil for any real web application these days if all you are going to do is partitions. If you shard you get significantly more mileage but you still don't come anywhere near the reaches of Amazon and Googles simple database solutions. They didn't put all the time and effort into making those solutions because SQL scales well, they put the time and effort into it because SQL (and relational databases in general) do not scale at all past a certain threshold. Relational, partitioned SQL is for small to medium sized companies. If you're one of the big boys good luck keeping SQL up to speed with any type of real usage/growth.
I'll let Oracle and their customers running Oracle RAC that the "big boys" can't run relational SQL databases and that it is only for small to medium sized companies.
The "big boys" have been solving this problem for a lot longer than Amazon and Google have, so to appeal to their authority and the "time and effort" they put into making this product as proof that SQL doesn't scale is ridiculous. On the low end, there's sharding, and on the high end, there's scalable clustered SQL systems like Oracle RAC and IBM DB2 ICE. Making broad statements about some overall lack of scalability of SQL speaking from your MySQL and/or Postgres experience makes you look a little underinformed, when there are enterprise class solutions to SQL server scalability problems from major vendors, on top of the roll your own solutions you can do by doing partitioning / sharding.
On top of that, consider that database servers are optimizing for multi-core systems with things like parallel index scans, breaking up single indexes and joins into sub-processes, dispatched to different cores. This kind of scale up only serves to complement the scale out provided by sharding.
People who say that SQL is dead are just bored. Get over it.
PS. Go check out the TPC benchmarks for the biggest and baddest SQL servers you can buy to see how far people are scaling SQL up, and then explain why a few shards of those "big boy" SQL servers can't handle the load of any "real web application". Or go read the MySpace case study.
Parent
Re: (Score:2)
Oracle are scrabbling to catch up - some of their latest offerings are quite interesting, but really it is just trying to gain market share lost to the likes of Netezza. Netezza is great, but limited in capacity and very expensive. There are few SQL databases that can scale *huge* - and those that can are typically shared-nothing designs and suffer from data ingest problems (or are shared all (Superdome etc) and limited by general architecture problems, eg the typical CPU heavy/IO limited problem).
Re: (Score:2)
For most people that are going to be using "cloud", the limitation is I/O bandwidth - something that SAN really doesn't give you at all.
Re: (Score:2)
To handle an increase in I/O bandwidth:
Add disks, thus adding spindles
Use a RAID technology that supports bandwidth over storage, like RAID 10
Upgrade your SAN heads for greater throughput and more ports (be they fibre channel or iSCSI)
Upgrade your SAN switch for greater throughput if you're using fibre channel
Add more paths to your clients and use smart load balancing SAN client software
Partition your load among different LUNs on different SAN heads
There's no reason SANs can't scale up through upgraded SAN
Re: (Score:2)
Re: (Score:2)
I would have though the same thing about J2EE, but every site ends up using proprietary extensions in Websphere, or whatever, and then has an terrible time migrating to another platform. It's called "vendor lock-in". I don't see why it wouldn't happen again in the cloud. Heck, there are still plen
Re: (Score:2)
Sure, but are you going to choose your vendor lock-in for development and database/storage technology in the same decision that you make regarding a choice in hosting providers? It would be like if I had to choose one colocation facility or hosted server provider based on whether I wanted to do J2EE or .NET. I'd rather not bundle my hosting purchasing decisions with my development platform purchasing decisions. These are obviously two of the most critical technology decisions a company can make, and maki
OK, but let's look at the big picture (Score:5, Interesting)
Comparisons are OK, but let's look at reliability. EC2 is not the same as S3, but the recent [readwriteweb.com] fiasco with S3 and SQS should give people pause before considering using any other Amazon cloud services. Two of my clients were hit with this over the weekend.
I don't know what kinds of volumes (traffic and hosting) Google AE is handling at this point, but at this point I think I would trust Google more than Amazon. One of the issues with the S3 downtime for many people was the fact that Amazon itself (and all its properties) continued to run perfectly while all the sites that hosted images and other content with them failed. Does Google use its own infrastructure to host AE? I don't know, but if they do I'd trust them a hell of a lot more than AWS.
At this point I'm thinking I'm not going to recommend AWS anymore.
Re: (Score:2)
Does Google use its own infrastructure to host AE? I don't know, but if they do I'd trust them a hell of a lot more than AWS.
is it their own hardware and network resources? i'm sure. if AE goes down does that mean that search goes down with it, so they'll have to fix both quickly? not a chance. i'm not sure what "use its own infrastructure" is supposed to mean in this argument, neither company is going to support their free-to-use service as well as their own makes-us-money website
Re: (Score:2)
While AE is free, anyone using it seriously would wait until Google finishes putting together a pricing model for it so they can pay and secure a formal SLA and some sort of support. They might have already done this, I don't know.
None of the AWS are free.
EC2 is pretty sweet (Score:4, Interesting)
Missing the point? (Score:5, Insightful)
AFAICT, they aren't intended to. The deepest problems are software problems for which there is no general solution, only problem-specific solutions for each particular task; what they are intended to deal with is the hardware problem that having a scalable software solution is of limited value if you have a fixed pool of hardware and have to go through disruptive upgrades when you expand that pool of hardware (and deal with the associated capital costs.)
Cloud computing services are, largely, tools to help dynamically "right-size" hardware, changing it from a capital investment that requires predicting the future well to plan right to an operating costs that can be quickly adjusted based on changing needs. Complaining that they don't solve the fundamental problems of software scalability seems to be missing the point.
Re: (Score:3, Insightful)
Yes, this whole article, I think, misses the point. The cloud, by its very nature, forces you to develop a solution that is intrinsically scalable. It doesn't develop it for you. Who the heck imagines that it is the responsibility of a hosting provider to make your platform scalable? Give me a break!
EC2 is not your typical co-hosting service, nor should you ever treat it like one. To properly implement a platform upon it, you, the programmer / admin need to implement machine images which have the ability to
Re: (Score:2)
No cloud service should be judged by how much engineering this requires, because it's not their responsibility.
Yes, I agree with you, but I don't think that everyone knows this yet. I interviewed several users who seemed very annoyed that they had to backup their MySQL databases themselves.
Why? Here's a quote from Google AppEngine's front page:
Amazon is a bit more g
Re: (Score:2)
There could (conceivably) be a cloud that behaved something like a single computer with an the total number of cores available in the cloud, the total RAM of cloud, and the total storage of the cloud (all minus necessary operating overhead), but that won't change the fact that, no matter how you slice it, N cores at speed Y don't run software identically to one core at speed N*Y, so you still
Amazon EC2 wins (Score:5, Informative)
I run a small startup in the Boston area and have been using Amazon EC2 (plus S3, SQS, and the rest of the AWS family) for the last year. It's worked for us like a champ. A little downtime in the beginning plus some S3 outages, but with the right backup, failover, and restore procedures in place we've gotten reasonable uptime.
The big requirements for us were the following:
1. Ability to move our website (and code base) elsewhere if needed. Could be in-house, to another cloud provider, etc.
2. Minimize up-front cost and allow for massive scaling if needed
3. Cost competitive servers/computing over time
4. Cost competitive storage/disk over time
App Engine fails the first criteria, since (at least currently) you can't build a BigTable application on anything but Google App Engine. "Cloud computing" in general beat out traditional hosting on the second, third, and fourth points. I hadn't checked out GoGrid or AppNexus at the time, but other competitors (Sun, etc.) couldn't match Amazon's price-performance specs.
So, with all of those requirements, Amazon EC2 won out and I'm a happy customer.
Re: (Score:2)
Amazon does not automatically scale (Score:4, Informative)
Moving to ec2 (Score:4, Interesting)
The cost analysis was really what did it versus our managed hosting plan (1/10th the cost per month). Auto scaling and healing of the application cluster was also a benefit. To scale with a traditional host meant getting locked into a contract for the added server(s).
One thing about ec2 is that it forces you to use best practices for disaster recovery. Instances don't commonly just "disappear" but you need to plan for it. Well tuned ec2 images can have your site up and restored from backup automatically within minutes.
ec2 / s3 is far from perfect and certainly won't meet everyone's needs. The downtime s3 has seen (like last weekend) would be devastating to some businesses. Of course even with a traditional host you may have downtime due to truck crash [datacenterknowledge.com] or other random act.
Re: (Score:2)
Re: (Score:2)
I'm thinking on any number of tools built around ec2 to support this behavior. Some are commercial offerings (like rightscale.com or scalr.net). You can however roll your own methods to do this if you like.
Basically you can use open source monitoring tools to check for whatever states you need to look for (server down, cpu load too heavy etc.) and trigger actions based on that (bring up another server). There is nothing really magical that the pay services are doing. It's just a cost question, roll your own
Re: (Score:2)
All the features they have with starting, stopping and adding servers can be done in a dedicated server environment which has a much better price / performance.
Sound familiar? (Score:5, Funny)
...any Web site filled with an endless stream of mostly forgettable comments trolling for reactions from the rival fans
I can't think of any site to fit that description...
AppEngine not working for me (Score:2)
I tried to set up an app on Google's AppEngine, and got an error saying that they're out of space. They'll email me when space is available.
That somewhat deflates the promise of great scalability, etc.
Buzz word of 2008 (Score:2)
First time i heard a vendor use it was a week ago.. Now its everywhere.
Choice of OSes (Score:2)
Re: (Score:2)
Re: (Score:2)
EC2 gives me almost none of that, without something like RightScale.
Ummm... maybe you should be using RightScale, then?
If you need hundreds of thousands of calls per second, you're going to need a lot of resources... I don't know what rightscale costs, but I don't think it's very much.
For your purposes, do you really need something like auto-scaling? If your load doesn't vary much, just spin up a bunch of EC2 instances and run a monitoring program. Overloaded? Spin up a few more instances.
Re: (Score:2)
There's an open source cloud engine. Eucalyptus, or something like that.
Re: (Score:2)