Google Deprecates SOAP API

Michi writes "Brady Forrest at O'Reilly Radar reports that Google has deprecated their SOAP API; they aren't giving out any new SOAP Search API keys. Nelson Minar (the original author of the Google SOAP API) argues that this move is motivated by business reasons rather than technical ones. Does this mark the beginning of the end for SOAP or for ubiquitous middleware in general?" Forrest's post quotes developer Paul Bausch: "This is such a bad move because the Google API was the canonical example of how web services work. Not only is Google Hacks based on this API, but hundreds of other books and online examples use the Google API to show how to incorporate content from another site into a 3rd party application."

Honeymoon is Over?

From TFA:

I remember the first web services summit we did, where a Microsoft developer who I won't name admitted that SOAP was made so complex partly because "we want our tools to read it, not people."

Just as I suspected: SOAP suffers from an artificial (read: gratuitous) complexity; what more do you need besides XML-RPC, anyway?

Google quietly shutting down services, on the other hand, reminds me of differentiating stem-cells: the honeymoon is over.

Re:Honeymoon is Over?

Any technology that requires a specific tool to write code for it is way too complex.

Without a Next-Next-Finish wizard, SOAP is a pain. With the tool it's mildly uncomfortable.

Re:Honeymoon is Over?

This isn't about deprecating SOAP in favor of something simpler, is it? Sounds to me like google wants people to visit their website to use their services. Which, once again, proves why the semantic web and web services will never fly. Companies don't want to make their data and services available to each other.

Re:Honeymoon is Over?

Companies don't want to make their data and services available to each other.

I don't think it's so much that companies don't want to as it is that there is no money in it.

If Amazon provided an API for buying stuff, I think it would stick around

If eBay provided an API for listing / searching, I think it would stick around.

Google, however, provides their product strictly for advertising revenue...it's wayyyy too easy for a consumer of the content to filter out the thing Google makes their money from.

It's very similar to the problem with Tivo's (PVR's) and commercial television. Luckily in that case, the television providers don't make their money directly from advertising revenue...

Re:Honeymoon is Over?

There is an eBay API. I've used it myself to create a few apps. It's a full featured XML over HTTP interface.

http://developer.ebay.com/common/api/ [ebay.com]

Re:Honeymoon is Over?

Which, once again, proves why the semantic web and web services will never fly. Companies don't want to make their data and services available to each other.

Well, I beg the differ, please bear with me.

SOAP is based on an idea of giving APIs to external parties for accessing information the way the information-owners want it. SOAP might be bad, but the idea is sound. Thinking about the traditional and dirty way to do the same thing: write scripts to 'post' webpages and extract the return pages. You can imagine slight changes in webpage layout could render the original extraction scripts useless, and that kind of information extraction might not be the owners' desire.

In short, things SHOULD be done this way, but Google doesn't like this implementation(SOAP). Google might want to adopt other implementation, that's what we'd like to know.

Re:Honeymoon is Over?

AJAX, SOAP and so on are all based on the premise that there are ways of abstracting out a lot of the complexity of service/user and service/service interactions. That assumption, I believe, is essentially correct. From that premise, these standards go on to assume that vendors are the right people to make such abstractions. Here is where the error lies. Vendors are notorious for producing crippled standards (such as SQL) that require vendor extensions to be usable. At a recent PCI-SIG conference, I was amazed at both the obviously stupid limitations of the standards and the gratuitous "vendor extension" options that very obviously existed so that vendors could provide proprietary solutions to those limitations. If a standard cannot stand, it ceases to be a standard and becomes little more than a cub scout badge.

(Another case: I cannot name a single well-designed W3C spec that was consortium-driven, and cannot name a single consortium-driven W3C spec that was well-designed.)

Web service standards cannot be driven by the very people who profit most from non-standard solutions. Even when they are designed well, they will STILL carry unacceptable flaws precisely because they are not driven by a collective itch but by a desire to stop someone else's scratch being the one that's used. The day a truly open federation of user-developers (you need a group of people where each person is both user AND developer) who have no ulterior motive beyond solving the service issue is formed will be the day that you see a protocol that requires no "perfect case study", proprietary extensions, overweight IDEs, etc. It will just work and be just used. Same as every other system developed that way has always just worked and just been used.

Re:Honeymoon is Over?

What i truly don't understand is why it has to be so complex even the much vaunted and pparently much simpler XML-RPC looks like attatching a nuclear bomb to the process:

<?xml version="1.0"?>
<methodCall>
    <methodName>namespace.getCountryCodeFromAbbr</meth odName>
    <params>
        <param>
                <value><string>UK</string></value>
        </param>
    </params>
</methodCall>

Browsers already have Javascript engines in that take C-syntaxey looking ascii and convert it into functions and objects, so why not just use a C-syntaxey plaintext to describe the service?

read: namespace{ int getCountryCodeFromAbbr( string ); };

send: namespace.getCountryCodeFromAbbr( "UK" );

get : 44

now, ok you might want to send comlext data structures back, but hey, you can just slap in the curly braces and be done

read: namespace{ personStruct{ string name, int age, char sex }; personStruct getPersonFromUserId( int ); };

send: namespace.getPersonFromUserId( 12 );

get : { "John Smith", 34, 'M' };

oh, but i forget: everthing has to be XML to be enterprisey, wether or not it's the best tool for the job, or if there's already a tool for the job that can do it for you with just a little tiny bit of effort. The "include this .js file" AJAX approach is essentially a wrapper for doing what i've just described, but taking the communication automation out of it.

Re:Honeymoon is Over?

Yes, and where XML really shines is in turning a 1M record database query from 30MB of text into 150MB of text...

Re:Honeymoon is Over?

Have you ever tried using this for nontrivial examples? I must confess to being quite fed up with the whole thing. Support for anything beyond the basics seems to vary greatly from library to library, platform to platform, language to language. Axis is probably the best choice for Java, but it's rather lacking when it comes to commercial support, which is important for some people. For C/C++ you're more or less screwed. Gsoap works (for the most part), but it produces the most god-awful stubs I've ever seen. The library that comes as part of Visual Studio (for .Net I believe) either doesn't support MIME or DIME attachments, I don't recall which. There just seem to be too many problems for me to actually bother to use it.

In my opinion, at this point it's just a mess, and for anything beyond the complexity of the stock-quote example I look to other technologies. I, for one, shed no tears at the end of this honeymoon.

(And am I the only one that cringes at using SOAP messages (or XML in general) for something that's supposed to be a machine-to-machine interaction? If you're going to write a new standard, why not write something more efficient?

You have it backwards i.e. Google != Hypocritcal

One could argue that if it weren't for Google and other search engines, no one would ever know of about 90% of the web content put out by businesses and individuals. People and businesses who wanted to get *their* word out would have shrivelled up and died on the vine since no-one would ever have heard them calling. Since Google provides a service to those web sites, your argument could be considered spurious and therefore moot. If anything, those web sites owe Google, not the other way around as you contend.

Power of SOAP

The real power of SOAP comes when you are using a language or framework that has support for it builtin. SOAP is complex simply because it does more than XML-RPC with type handling etc.

In PHP you can use NuSOAP (or in 5.x the built-in SOAP library), to simply register some functions and autogenerate the WSDL, or generate a proxy from a given WSDL - takes a couple of minutes tops and then looks like you are simply calling another function.

Anyone who uses ASP.NET regularly has it even better - create an ASMX file, define a class and functions like you would in any C# class, add some namespace arguments, a [WebMethod] over all your public methods and it can then be instantiated and called from any other ASP.NET website or .NET dekstop app seamlessly, like it was a local class. It's really cool just how transparent it all is. You can even throw exceptions and catch them on the other side, pass back objects - it's really slick.

Re:Honeymoon is Over?

Yes indeed it is. The cool kids see externally provided services and say "mashup! mashup! mashup!", the old timers see them and say "risk! risk! risk!".

Dave

bastards

Bastards, I wrote one of those books! Quick buy your copy today, it's practicaly a collectable now.

Re:bastards

He's not kidding [amazon.com] - it's a good book too, I'd highly recommend it if you're doing web programming.

Well it was 'just' a Beta

Well it was 'just' a Beta, but then again so is gmail, Google maps, and every other neat Google application. Of course as a gmail user it does give me pause, as to what I'm really doing.

Re:Well it was 'just' a Beta

...not every other neat Google application. Maps is no longer in Beta as well as Google Earth and some other neat applications.

http://en.wikipedia.org/wiki/List_of_Google_produc ts [wikipedia.org]

What about XMLRPC?

Does Google offer XMLRPC services?

If so, then I'd say it's fine to drop SOAP. XMLRPC is a bit cleaner anyways.

Re:What about XMLRPC?

Does Google offer XMLRPC services?

The AJAX API [google.com] might be what you're looking for.

Re:What about XMLRPC?

There's also the Data API [google.com].

Oh! Come On.

If so, then I'd say it's fine to drop SOAP. XMLRPC is a bit cleaner anyways.

Come on... Cleaner than SOAP? What could be cleaner than SOAP?

Re:What about XMLRPC?

You sir could not be more correct... We have servers that process tens of thousands of xmlrpc transactions per day and could not be happier with it. SOAP is a absolutley over engineered, complicated pain in the ass spec to deal with. The couple of features it does have over xmlrpc do are not worth the complication of dealing wih it.

Soap, what was that?

Soap, what was that....

Maybe something to do with:

UNIX Sex

{look;gawk;find;sed;talk;grep;touch;finger;find;fl ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje ct;umount;makeclean; zip;split;done;exit:xargs!!;)}

Re:Soap, what was that?

{look;gawk;find;sed;talk;grep;touch;finger;find;fl ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje ct;umount;makeclean; zip;split;done;exit:xargs!!;)}

So is that why nerds are getting acused of rape? (Not checking return codes...)
The semicolons should be double ampersands, so that execution will stop if a command fails.

Re:Soap, what was that?

So is that why nerds are getting acused of rape? (Not checking return codes...) The semicolons should be double ampersands, so that execution will stop if a command fails.

Yeah, but in the guy's defense, there wasn't a single argument to any of those commands.

Re:Soap, what was that?

The semicolons should be double ampersands, so that execution will stop if a command fails.

Who said anything about execution? Sounds like you're into some sick kind of sudo masochism.
 

Re:Soap, what was that?

So is that why nerds are getting acused of rape?

You'd think Google would have paid attention to all those prison horror stories about dropping the SOAP.

Google says;

No SOAP, Radio!

Did you mean...?

Google: _SOAP___

Did you mean: _soup?_

Don't be evil!

Wow. This is the first time I read an article about Google and, for a second, find myself thinking "Google appears to be Evil and driven by Greed. This isn't happening. This is just a mistake. Google will see the light. Google is Good."

Re:Don't be evil!

Yes. Google is evil. They've become drunk on their stock earnings and intend to use their massive computing capabilities to use up all of the Earth's energy in order to enslave mankind... it's all clear now that the SOAP Search API is gone.

Um... seriously though. Give them a bit of time and see if the make and announcement and publish an alternative (non-AJAX) API.

Re:Don't be evil!

It needn't even be "non-AJAX". There are plenty of other possibilities for a web service API besides SOAP. The one I'm particularly well-acquainted with, and perhaps the biggest contender out there, is REST (REpresentational State Transfer). In particular, I recall one web developer howto-type site speaking about Amazon's SOAP-related services, and how most people don't use them, because they're an order of magnitude slower than most REST services.

Uncleanliness is next to Satanliness

Without soap Google will become evil.

(SOAP is a WS) != (WS is SOAP)

Simpy put the Author of the article has it all wrong. SOAP is a type of Web Service but not all web services use SOAP.

Google branding?

So, skimming the Google AJAX Search API [google.com] example code pages, it looks like a big part of this is to attach Google's name and image to everything your web page or web app does with the data Google provides. Does that seem to anyone else like a fair assessment? If so, is it a fair practice?

Re:Google branding?

So, skimming the Google AJAX Search API example code pages, it looks like a big part of this is to attach Google's name and image to everything your web page or web app does with the data Google provides. Does that seem to anyone else like a fair assessment? If so, is it a fair practice?

Well, since Google is the one who aggregated it in the first place... and is paying for the processing power and bandwidth requirements that go along with that... what's unfair about the practice? (It's not like they're really preventing one from giving you similar data, or somehow stealing away value from any of the sites they've indexed, or...)

loss, opportunity, lesson or deja-vu?

Yawn... Google is a company. Regardless of the marketing bullshit they spew on, they aren't in the business of providing APIs (or whatever), they are in the business of making money. Anything else is subject to change.

Their responsibility is more towards their shareholders, not so much towards their users. So, if they think one of their products -- be that APIs, ajax apps or whatever are providing diminishing returns for some reason, they'll axe it unless it is popular enough so that too many users feel ripped off. APIs aren't in the category at all.

Also, the bigger they get, the more expensive the stock becomes, and the more their ownership sreads, the more clout the bean counters have over any other management ideology.

So, if one relies exclusively to Google for anything, better check your contract with Google carefully and assess all risks (including risk from expensive litigation) first.

So Google is starting to act like a real business?

Say it ain't so!

It would be interesting to know how many active API users there were, and at what rate new ones were signing up, if at all. It may well be that continuing to support that API wasn't considered a useful (read: profitable) part of their business.

Google is a publicly held corporation now. They have a responsibility to their shareholders to make decisions based on sound business practices. For a software company that means sending products into the end-of-life bin periodically.

In a fabulous dose of irony, I found that on Google's AJAX Search API page [google.com], their own embedded search example is showing a blog posting titled "Google quietly backrooms SOAP API for AJAX".

Screenshot here [davidmays.com] (Yeah, I'm using IE7, wannafightaboutit?)

That's unfortunate

I loved using the Google API as the basis of one of my data structures programming assignments. It's a great way to have my student tackle the use of another party's API, as well as a useful way to grab a ton of data and play with it (say in a binary search tree or hash table). Now I need to find something else that will let us do the same, or come up with something else.

And the replacement

The recommended replacement AJAX api [google.com] not only has limited applications, but also it promises [google.com] to show google ads beside the results.

Not that Google Search API has ever been very stable - it probably works only 80% of the time. Now even the support has been dropped and usage samples along with FAQ have been removed for SOAP api.

Obligatory questions/puns

• Does Google own any patent/copyright on SOAP? If Google is dropping SOAP I don't want to get fucked if I pick it up.
  
  
• No! Please don't let us run out of SOAP, Google! I feel so dirty using Microsoft's rival technology!
  
  
• This is quite a slippery situation. I hope Google will come clean on my they are depreciating the APIs.
  
  
• My reaction to this required me to use some SOAP - on my clothes.
  
  
• When the developers heard this we had to get some SOAP - for their mouths.
  
  
• I guess the "do no evil" bubble has finally burst...
  
  
• There must be a better solution that will allow the technology to continue while satisfying Google's business reasons. No reason to throw out the baby with the bath water...
  
  
• Why does Google have to play dirty like this?
  
  
• This doesn't smell like an Irish Spring to me!
  
  
• Hopefully they will introduce something even better for us to use, then the whole issue will be a wash.
  

SOAP Nazis

No SOAP for you!

Fight Club

Tyler Durden says: use SOAP?

Google != web

Not many were the people that appreciated SOAP before "Google did it" (tm).

After "Google did it" (tm), SOAP is suddenly a good thing. Now that they drop it, peple are discussing if it's again a bad thing. Google is not the whole of Internet though. People will use SOAP if it's better than other solutions for a given tasks.

And if it isn't, then it was a fad all along.

Same with "AJAX" by the way. JavaScript was out there for years before "Google did it" (tm) but there were not many people appreciating it. If Google drops JavaScript tomorrow, would this spell the end of AJAX?

Same logic applies.

No SOAP, Radio

How could this deprecation of a SOAP API mark the end of "ubiquitous middleware" (as if that even existed) when deprecation means an API change, not a feature shutdown ?

Google is replacing SOAP with an AJAX API. Maybe it is a blow to SOAP - which is long overdue for the 1990s graveyard. But how could that be bad for open-access middleware when there's a new, better API?

The small guys are paying for this..

A friend of mine from a 50 people shop called me to check what I thought of Amazon web services [amazon.com]. I frankly had no opnion since I am not much into web services, but I guess this is because, they were using Google SOAP APIs for themselves.
I am sure they would much rather look at another SOAP oriented service provider than change to AJAX, and I can understand their concern. They never did expect to be desupported (even on a beta product) since it was google they were discussing!
This does make me worry about the other google services I use (Gtalk, GMail etc.)

Drop-In Replacement Already Available

There's already a drop-in replacement [evilapi.com] for applications that are using Google's SOAP API. It scrapes Google's web results and returns them via a SOAP layer. The code behind it is free under the MIT License.

Rest in Peace

Over at Amazon they have continiously been reporting dimininshing numbers of users for their SOAP based services. I think the last count was around 90% Rest Based Access to 10% Soap.

SOAP Was Always An Eye Test

IMHO, I always thought SOAP was over engineered. Simple Web Service's are like a console program, but the input is in field form from the web, with input type checking. Output is an XML string with an embedded "xml-stylesheet" statement, (for the unwashed) [w3.org]. The security of a simple Web Service is; Simple. A Simple Web Service already has many locked doors to slow down uninvited guests, and the other doors that the uninvited enter can be easily closed. This act from Google will reduce their maintenance cost, and should increase their band width by reducing their output to smaller chucks with the same information. The following files would be cashed on the client side, HTML, CSS, JavaScript, XSLT, and XML; So a simple change to one file, and the Client Side will download only the changed file. The Server Side has only a Simple Web Service of one input entry point, and one exit point. The Client sees faster turn-around, the Server sees a bigger band-width.

"Slowly, one by one, the Penguins steal my sanity." - Unknown

It's probably because SOAP blows

Any serializing transport where, ultimately, figuring out what is going wrong - in normal use - involves using a packet sniffer to dump XML, is just broken. Nevermind that XML was never intended to be written by humans. Nevermind that the XML used in SOAP isn't intended to be human readable. It's just a layer of unnecessary crap that isn't even interoperable between different languages. Or, for that matter, different implementations on the same platform. "Web services" is a lovely addition that removes the last bit of comprehensibility and connection to reality while adding nothing except gaping security holes.

JSON and the like are, on the other hand, reasonable, and also fairly easy on the eyes of us tired old programmers.

"Does this..?"

Does this mark the beginning of the end for SOAP or for ubiquitous middleware in general?

Does this mean google has turned evil? Is Bob your uncle? Will we ever see the day when a slashdot submission doesn't include an "interesting" question by the submitter?

These questions - and many others - will be answered in the next episode of Soap.

Good riddance.

Seriously. Good riddance. SOAP is a mess. Google has gone the XMLRPC way, and they're providing access via that.

This isn't google being evil. This is google removing a piece of completely unnecessary junk from their offerings. SOAP should never have seen the light of day, and google is now making sure that they do their part of burying it.

What Business Reasons?

Does anyone know what the business reasons mentioned in the scoop would be?

Of course they dropped it...

Correct me if I'm wrong, but the SOAP search results didn't come back with ads embedded in them. The AJAX API tools are capable of putting ad content in their results, as can the maps API because Google is controlling the display elements.

Given that Google want to run their business off the back of ad revenues, it should come as no surprise they're getting rid of services that don't allow them to sell lots and lots of ads. I also imagine that the cost of providing the SOAP interface was higher than any subscription fees would have brought in due to a small market. What's more, it would directly help their competitors pull in results from Google and run their own ads alongside it. The API was neat, but from a business perspective it was always an experiment at best.

Personally, I'd rather they brought something RESTful like Yahoo's interface or xml-rpc to the table, and charged us all a couple of cents per 100 queries, but that isn't going to happen any time soon.

Base Data API

Eh, I'm a bit confused by the myriad of APIs available for google, but doesn't he Base data API provide the same features, just over their REST/ATOM feeds? With my key these still work quite nicely, don't use SOAP and aren't depricated: http://code.google.com/apis/base/samples/python/py thon-sample.html [google.com]

What?

Google has an API for Snakes On A Plane? No wonder they're going down the (series of) tubes.

SOAP SUX!

Glad its leaving. Lets have a parade.

Wow.. I was just saying...

.. this yesterday. I am trying to figure out this nightmare of a web service I've been handed with no documentation. I was just talking to a friend of my via IM and telling him no wonder web services haven't really taken off. I mean sure they are around but they aren't widespread. It reminds me of EJB. The concept is great but good luck implementing them. Too complex IMO.

The S Stands for Simple

I'm not overly depressed at the decision to get rid of the SOAP API. See: The S Stands for Simple [wanderingbarque.com].

Maybe Google will follow in Yahoo!'s footsteps and implement a REST API now. Maybe.

There's an alternative.

It's called "EvilAPI" and it's available at http://evilapi.com/ [evilapi.com] -- it re-implements the old SOAP API with page scraping.

It's not the end of SOAP because of Microsoft

Does this mark the beginning of the end for SOAP or for ubiquitous middleware in general?"

Microsoft's tools make it so easy to use SOAP that it would be foolish to say it's "the end of SOAP." While Microsoft only has 31% of web servers out there in the wild (according to Netcraft), I'm sure internal corporate web efforts are 50%-50% Microsoft or better. ASP.NET 2.0 and Visual Studio 2005 are an amazing combination for writing web services using SOAP (as other posters have noted)

So, no, SOAP is not going away. Besides, who was building businesses around Google's SOAP API? Certainly not Google.

AJAX API not the same functionality as the SOAP

The SOAP API is a real RPC API that can be called from Python, C++, or Java programs. The AJAX API is a bundle of obfuscated Javascript that can effectively only be used with Javascript interpreters running inside Web browsers. This is a big reduction in functionality.

The world does not revolve around Google

Just because Google decided to cut a service doesn't mean a protocol is doomed. Get over it people, Google isn't everything. Google could go away completely and it wouldn't change the world much.

It's about the time to let SOAP...

REST in peace!

SOAP technology sucks, too

I leave town for a couple of days and I miss my moment of fame on Slashdot. Ah well. My blog post doesn't exactly argue it's "business reasons rather than technical ones", although my post does dwell on the business reasons too. There are plenty of technical reasons to hate SOAP too. For my view on that, see my blog post about why SOAP sucks [somebits.com]. Bottom line is SOAP is too complex, doesn't work well in practice, and strong typing is the wrong choice for loosely coupled distributed systems.

Re:WTF is SOAP?

Spoken like a true Slashdotter... :)

Re:WTF is SOAP?

It's something you use to wash yourself with...

oops... sorry.. wrong forum...

Re:WTF is SOAP?

> WTF is SOAP?

Based on the slashdotters I know, you aren't likely to get an answer to this here.

Re:looks like its the beginning of the end...

...for google's free services. I bet they got a new young hotshot in house that wants to make lots of money (insert greedy pic).

Well, that young Google whippersnapper will ultimately find out the hard way that each of his company's services wants to be free.

Re:WTF is SOAP?

SOAP [engadget.com] ???

Re:WTF is SOAP?

According to WP [wikipedia.org], it stands for Simple Object Access Protocol. I know no more than that.

Re:WTF is SOAP?

SOAP (originally Simple Object Access Protocol) is a protocol for exchanging XML-based messages over computer network, normally using HTTP. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework that more abstract layers can build on. The original acronym was dropped with Version 1.2 of the standard, which became a W3C Recommendation on June 24, 2003, as it was considered to be misleading. - Wikipedia.org [wikipedia.org]

Re:who cares

They allegedly watch for robot scraping and will cut off your IP if they detect it... is this not an issue? Or it's not an issue if you keep it to a rate similar to what the SOAP interface allowed?

Re:WTF is SOAP?

Have we already forgotten about Snakes on a Plane??

Re:A JSON alternative...

This is very interesting. Is it a documented feature of SearchMash?

Re:WTF is SOAP?

No it's not!

But WCF (Windows Communcation Foundation) is SOAP based in some protocols.

I actually asked a couple of MS developers (this is a true story):

Me: You guys should do a "Windows Transaction Framework" and call it WTF.

Microsoft Devs: Gee, maybe we will one day.

I hope so, I really hope they do...

Re:WTF is SOAP?

Simple Object Access Protocol.
What the fuck is WTF?

Re:A JSON alternative...

$query = count($argv) > 1 ? $argv[1] : 'google';

bad programmer! no cookie!

$query = count($argv) > 1 ? raw_urlencode($argv[1]) : 'google';

(That's a nifty trick though!)