Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Programming IT Technology

Google Releases Web APIs 144

Posted by Hemos from the developing-with-google dept.
skunkeh writes "Google have released the first beta of their Web APIs package. Used in conjunction with a free license key this SOAP based web service allows developers to execute up to 1000 automated queries a day, but is currently available for non-commercial use only. The download comes with Java and .NET code examples and includes a WSDL description for use with other SOAP supporting languages." There's also a write up about uses on Userland.
This discussion has been archived. No new comments can be posted.

Google Releases Web APIs More

Google Releases Web APIs

Comments Filter:

  • Web API Implementations (Score:5, Informative)

    by skunkeh ( 410004 ) on Friday April 12, 2002 @07:00AM (#3328731)
    A list of implementations of the Google Web API can be found on SoapWare:

    http://www.soapware.org/directory/4/services/googl eApi/implementations [soapware.org]

    At the time of posting languages catered for were for AppleScript, Frontier/Radio, Perl, Python and Visual Basic. I've written a basic implementation in PHP which has yet to be added to the list - you can find it here:

    http://toys.incutio.com/php/php-google-web-api.htm l [incutio.com]

    This is a very cool toy.

  • This story refers to (Score:3, Informative)

    by sydneyfong ( 410107 ) on Friday April 12, 2002 @07:08AM (#3328748) Homepage Journal
    The old one here - "Google to Offer API" [slashdot.org]

    • So, you can execute 1,000 searches a day through their API, OR you can code your program to do a normal google search and parse the results out of the returned html (like people have had to do until now)...

      How many projects can't afford the overhead of a little html parsing but CAN afford to be limited to 1,000 searches/day? I'm sure they'll offer higher limits for a fee, but I think the DIY-html-parsing google "api" is going to keep on working just fine (and for free).
  • http://radio.weblogs.com/0100012/stories/2002/04/1 1/applescriptForGoogleApi.html

    has some Applescript for your use

  • w00t! (Score:1, Funny)

    by Anonymous Coward
    Now I can write programs to generate Google-whacks!

  • A great corporate move (Score:5, Insightful)

    by shankark ( 324928 ) on Friday April 12, 2002 @07:16AM (#3328770)
    Other than being a really cool idea, this is a great tactical move from Google. On the one hand, by restricting the number of queries made to Google, they ensure that their APIs aren't misused/compromised, it also gives companies an initiative to purchase Google products and deploy this API (probably an unrestricted-query API) on their own network. Furthermore, an API such as this will easily muscle out any sniff of a competition from other search engine wannabes. Google has managed to do all this and yet be as compliant
    to an Open Source initiative as possible. Remarkable.

  • Google groups may require a Google account.. (Score:1, Interesting)

    by Anonymous Coward
    "...The initial service available with your Google Account is:

    Google Web APIs - a tool for software developers to automatically query Google

    In the future, your Google account will provide access to all Google programs requiring sign in including: Google Groups, Google AdWords, Google Store, Google in Your Language program...."

    Does that mean that accessing the Google Groups is now going to need me to create an account? Hmm..

  • Example of use (Score:5, Informative)

    by dtr20 ( 442135 ) on Friday April 12, 2002 @07:23AM (#3328795)
    I just had a go with this and some example output is displayed below. Basically you can do a search of their main web pages, request a cached page or use their spellchecker.

    Dave

    $ java -cp googleapi.jar com.google.soap.search.GoogleAPIDemo XXmykeyXX search "british empire"
    Parameters:
    Client key = XXmykeyXX
    Directive = search
    Args = british empire
    Google Search Results:
    ======================
    {
    TM = 0.117071
    Q = "british empire"
    CT = ""
    TT = ""
    CATs =
    {
    {SE="", FVN="Top/Regional/Europe/United_Kingdom/Society_an d_Culture/History"}
    }
    Start Index = 1
    End Index = 10
    Estimated Total Results Number = 688000
    Document Filtering = true
    Estimate Correct = false
    Rs =
    {

    [
    URL = "http://www.btinternet.com/~britishempire/empire/e mpire.htm"
    Title = "The British Empire"
    Snippet = "| Introduction | Articles | Biographies | Timelines
    | Discussio
    n | Map Room | Armed Forces | Art ... "
    Directory Category = {SE="", FVN=""}
    Directory Title = ""
    Summary = ""
    Cached Size = "5k"
    Related information present = true
    Host Name = ""
    ],
    ...

  • Trying to /. google???? (Score:1, Funny)

    by Anonymous Coward
    Won't work, evil doers!!

  • O'Reilly has some good code and stuff (Score:5, Informative)

    by bodin ( 2097 ) on Friday April 12, 2002 @07:25AM (#3328799) Homepage
    O'Reilly has a good article here with some code as well in both Java and Perl.

    http://www.oreillynet.com/cs/weblog/view/wlg/1283 [oreillynet.com]

  • Google Terms of Service (Score:4, Funny)

    by AVee ( 557523 ) <slashdot@av[ ]org ['ee.' in gap]> on Friday April 12, 2002 @07:29AM (#3328809) Homepage
    To create an Google account [google.com] you have te agree with the Google Terms of Service [google.com]. These state the following:

    No Automated Querying

    You may not send automated queries of any sort to Google's system without express permission in advance from Google. Note that "sending automated queries" includes, among other things:
    • using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
    • "meta-searching" Google; and
    • performing "offline" searches on Google.
    Now, how can I use the web API?!
    Note that this is not in the Google Api TOS [google.com] wich you must agree to before downloading [google.com] the api. But in the Google Terms of Service [google.com] wich you must agree to before creating a Google account needed to use the Google Api.

    Still, it's fun and i'll play with it!
  • Here's a copy of the write up. My machine barely made it to the site :)
    Google is just the juice
    Thu, Apr 11, 2002; by Dave Winer.
    Good afternoon

    A very quick piece today, a story, a question, an answer and a pointer.

    The story -- 1995. A new release of Netscape. Can't get through to their servers. This thing is exploding. A mind bomb every minute. Wow. I love this. End of story.

    The question: Can it happen again?

    The answer..

    Yes!

    This afternoon Google opened a public SOAP 1.1 interface.

    Now, from scripts, we can call Google as if it were a script running locally.

    What comes back? Data.

    What questions should we ask?

    That's where the mind bombs will come from.

    In the loop

    We've been in the loop with Google, privately, for the last few weeks, so we've had a chance to play with ideas and actually have some.

    Yesterday, as a tease, I put a Google Box on Weblogs.Com. Every hour it recalcs, showing the top 10 hits on Google for the term weblog. To my surprise, it changes, it's not constant. And it took me to places I didn't know about. The serendipity of queries that run for a long time. That, imho, is where the juice is in the Google API; and probably many or most of the APIs that are sure to follow; because Google is so popular.

    Google hits the ball over the net, then we return the volley. Finally, once again, signs of life. Let's hope we learn from the past -- and keep the spark going -- welcoming competition and learning from it instead of snuffing it out. The intoxication of a new idea every day is too good to not want to be there once again.

    Maybe the dark ages are over? I hope so.

    Google is just the juice

    It's happening in real time. As I write this I'm waiting for the embargo to lift. As soon as that happens, we'll start releasing new parts and samples for Radio and Frontier users that connect to Google's SOAP interface, with simple but geekish instructions for getting started.

    Later today Google Boxes will start showing up on Radio weblogs, which you can follow through Weblogs.Com. You'll see SOAP developers, on all platforms, getting to work, creating and publishing the glue that turns the Internet, finally, into a fantastic scripting environment. Google is just the juice we need.

    Dave Winer

  • Staggering Potential (Score:5, Insightful)

    by Captain Large Face ( 559804 ) on Friday April 12, 2002 @07:36AM (#3328829) Homepage

    Whilst the potential of a regular Google search is large enough, when you consider the Google search modifiers, the potential becomes staggering. Imagine using the following features:

    • Business Address Lookup
    • File Type Specific Search (.PDF etc..) (filetype:)
    • Stock Quotes
    • Cached Links (/. Favourite) (cache:)
    • Similar Pages (related:)
    • Linked Sites (link:)
    • Site Specific (site:)
    • Maps

    Does anyone happen to know if you can use the other sections of Google (e.g. news, images etc.)?

    Is Google the best company ever or what?!

  • Well,

    I know what I'll be doing when I get to work today. Just my additional 2 cents to this marvelous addition to Google.

  • Pigeonrank anyone? (Score:1, Funny)

    by Anonymous Coward
    I wanna program some pigeons!

  • Anyone care to explain... (Score:1, Interesting)

    by Anonymous Coward
    how this is different than using a script to parse Google's output?

    White Hat Research.net [whitehatresearch.net]

    Geek Clothes - Including a shirt with the (in)famous Ben Franklin quote! [cafepress.com]
    • The old terms and conditions prohibited any automated 'page scraping' so I guess this a leagal and much cleaner way of processing google output.

    • Re:Anyone care to explain... (Score:5, Informative)

      by Software ( 179033 ) on Friday April 12, 2002 @11:50AM (#3330128) Journal
      OK, your script parses Google's HTML output today, but what about a year from now when Google changes its output, to say, XHTML or plain text or something. How well will your script work then? Although the Google API could change tommorow like some companies' [microsoft.com], in general APIs are more stable. I haven't looked at their API, but I'm guessing it's also easier to develop against their API, and it should be less processor- and network-intensive.

  • More Advanced Features? (Score:5, Interesting)

    by Captain Large Face ( 559804 ) on Friday April 12, 2002 @07:53AM (#3328871) Homepage

    I think I speak for most when I ask if you can have your results back in the "interesting" language sets:

  • why not just use plain http (Score:3, Insightful)

    by sanermind ( 512885 ) on Friday April 12, 2002 @07:54AM (#3328877)
    If they are going to limit you to only 1000 queries, I fail to see the point. It wouldn't be hard at all to write a simple API on your own to, say, a c++ class that spits out the necessary url's [like http://www.google.com/search?hl=en&q=example]
    or the like, dispatch them to google port 80, and then parse the results into easily program readable data sets/results? A third party could write this sort of thing easily enough if there was demand for it. I mean, esentially the google search API isn't going to be offering anything not available in the standard forms, is it? Except their spell checker, I believe. [Which you could use via html too, actually, "Did you mean: ______" ]

  • What about slashdot? (Score:3, Interesting)

    by aozilla ( 133143 ) on Friday April 12, 2002 @07:54AM (#3328878) Homepage
    How long until slashdot offers this service?

    • They already do... (Score:3, Interesting)

      by xtermz ( 234073 )
      ... sort of.. they have a XML file out there off of the main site that you can query to get the latest headlines...

      i've actually used it before with a simple VB app...

      email me if you want the code...

    • access to postings (Score:1, Interesting)

      by Anonymous Coward
      I would love a computer-friendly way to access postings. The limited UI offered by a web browser is not the right way to read these huge, nested discussions! I've been playing with better interfaces, and have kludgey code for parsing slashdot HTML pages, but it would be wonderful to have something cleaner and less brittle.

  • NNTP tunneling ? (Score:3, Insightful)

    by Bert Peers ( 120166 ) on Friday April 12, 2002 @07:57AM (#3328882) Homepage
    In case the engineers at google are bored now that it's released, here's an idea ;) Open up groups.google.com via a similar API so that an application can get the latest Usenet info even through proxies blocking NNTP and/or newsservers. Showing the latest threads/posts etc on a webpage could be useful too.


    It's not something you have to go to google for, but it'd be nice :)

    • Re:NNTP tunneling ? (Score:4, Interesting)

      by km790816 ( 78280 ) <wqhq3gx02@@@sneakemail...com> on Friday April 12, 2002 @08:58AM (#3329152)
      This is where things get interesting.

      Companies have become happy blocking ports to restrict no-nos: messaging, newsgroups, etc.

      I'm wondering how long it will be until we start seeing firewalls that can filter/block SOAP calls for the very reasons you mention. SOAP just forces network admins to move up from ports and protocals to sniffing HTTP requests to keep people from having too much fun.

      Enjoy it while it lasts.
      • SOAP standard makes it easy to filter/block SOAP calls. But the key is, you can do it per-interface and per-method.
        SOAP clients send their data using M-POST, which mandates the server understand the Interface URI header, and the method-name header.
        This should allow network admins to restrict/allow specifically what they desire, and not force them to have to turn off SOAP through a firewall as a whole.
      • There will always be a way around that kind of stuff (besides moving to a better company). Imagine encoding your request and sending it through a proxy decoder outside your firewall. Surf the web using email? Or get a wireless carrier? The persistent hackers will never be stopped, because they have more knowledge and interest than a network admin who's just working their 40.

        -Kevin
    • And don't forget to fix the multipart merger.
  • The 1000 searches a day is very nice....I know I would never need that many (if results were unlimited anyway).

    HOWEVER...you only get 10 results per search??
    • I thought this was strange at first until I realised this was based on their existing search.

      Perform a search on Google. By default you'll get a list of ten results. The "Next" link shows a parameter called "start" that on the first result is 10, the second 20, and so on. So you can get more than 10 results by using multiple queries. This means the maximum number of results per day is 10,000.

  • Question (Score:2, Funny)

    by loconet ( 415875 )
    Anyone else feels that if google ever dissapears, they will become very unproductive?
  • FYI:

    We like them Monday, Wednesday, and Friday.
    We love them Tuesday, Thursday, and Saturday.
    And we alternate Sundays.

    Get with the program.
    --
    ---

  • Synthesis (Score:5, Insightful)

    by Asprin ( 545477 ) <gsarnold@@@yahoo...com> on Friday April 12, 2002 @09:07AM (#3329189) Homepage Journal
    Ummmmm. Ok, check this out.

    This morning on /. we have an article about Google releasing their SOAP 1.1 API followed immediately by an article from a guy that set up a spambot trap on his web site, and in the margin a poll about giving spammers what they deserve. Putting 2 and 2 and 2 together, I got 4, popped open a google box and started playing.

    All I did was ask google to search for "mailto" and "@msn.com" and lo and behold, she spit back 111,000 hits - hits that contain what look like legit email addresses IN THE THREE LINE SUMMARIES.

    The point is, now that google can be automated, what's to stop spammers from SOAPing their way into Google to do their harvesting? Would there be any point over what they're doing now? It might be cheaper, because you only have to run over the google results not the whole sites and since Google caches pages, you can even grab addresses from the past, somewhat.

    IT ALSO DEFEATS SPAMBOT TRAPS.

    Doesn't this give spammers whole new avenues to exploit?

    Worse, are webmasters going to have to put a halt to Google crawls?

    • Re:Synthesis (Score:2, Informative)

      by Pvt_Waldo ( 459439 )
      &ltsarcasm&gt
      LOL maybe we should just dismantle the whole internet, as clearly the internet is the channel used by spammers! Oh wait. The internet has many many positive uses. Gee!
      &lt/sarcasm&gt


      LOL a 4 for Interesting? Oh come on, this is ignorance, not information.


      Horrors! Spammers can use this!


      Uh 'scuse me but I can write a 10 line perl script that does the same thing. All I have to do is craft a query to google, and put a bunch of work into parsing out the real content from the HTML that comes back. Kind of a pain, but nothing a few regexp can't handle. This API is nothing new, it's just something handy. I'm seriously thinking I can replace a component of a research project here at our research facility with this. Why reinvent the wheel after all?


      Worse, are webmasters going to have to put a halt to Google crawls?


      It's called robots.txt. Ever run a web server? All this API does is let you do searches to google. Google is google is always searching. That's what robots.txt is for. You are not going to get crawled by this! This is not a BOT, just a QUERY TOOL.

      • I hadn't considered that Google's already crawling you anyway (point taken - oops - someone mod me down!), but again, this defeats spambot traps. Further, some of the more useful tech-support-web-forum postings would be somewhat filtered out if webadmins restricted Google crawls by using robot.txt. (I think the bot trap article had some better policy-oriented ideas about how to accomplish this.)

        Regarding the PERL scripting, ten years ago, I actually joked with my friends about sending emails to people with batch/script/program attachments that deleted files with a message that sez "run this c00l program d00dez!" but it didn't occur to me that anyone would actually fall for it and that's what the human-engineering-virus "revolution" (Melissa, ILOVEYOU, et. al.) was all about.

        (I've also decided on rereading my comments and yours that I should have gone with my first instinct and posted this under the spambot traps article -- in retrospect, it would have been far more appropriate there. Oh, well I had a 50-50 shot and lost - thanks for keeping me honest!)
        • ...ten years ago, I actually joked with my friends about sending emails to people with batch/script/program attachments that deleted files with a message that sez "run this c00l program d00dez!" but it didn't occur to me that anyone would actually fall for it and that's what the human-engineering-virus "revolution" (Melissa, ILOVEYOU, et. al.) was all about.
          That's the problem with systems/applications that think they are smarter than the user and hide things from the user. Not showing file extensions, even the DOS batch file @ECHO OFF is a bad idea.

  • PyGoogle [diveintomark.org] allows you to access the web API from Python. Download here [diveintomark.org]. Python has no SOAP support in the standard library, but a working SOAP library is included with PyGoogle.

    -Mark

    Dive Into Python [diveintopython.org] - a free Python book for experienced programmers

    • Hi,
      tried using the code. The page mentions that I need SOAP.py 0.9.7.1, but the included version is 0.9.7

      On trying a google search SOAP.py gives me a traceback

      Any suggestions?
  • It seems there is still time to enter the Google Programming Contest [google.com] and although I have neither the time nor the skill to do it, I do have an interesting idea if someone else wants to take a shot at it.

    Years ago, The Hollywood Stock Exchange [hsx.com] was a somewhat popular game (maybe it still is, but it doesn't really interest me). The general idea being that you could "Buy shares of your favorite actors, movies, and music artists and watch their values rise or fall based on the success of their careers and personal life."

    It would be interesting to see a similar game based on the popularity of queries. It's clear from the Google Zeitgeist [google.com] that certain search terms do gain and lose popularity on a regular basis, and for someone tapped in to mainstream culture, it may not be too hard to predict.

    I suppose you could do the same thing with the other info there (Browsers, OSs, Current Events, etc.) but I don't think it would be as interesting. Although... Anime searches might be neat.

    Anyhow, just an idea I'd love to see someone run with.

    -Tommy

  • Google is singular. Even though the term represents an organization of many people, it is just one organization, and so the word is singular. You don't say, "The class have learned the material from lesson 5"; you say, "The class has learned the material from lesson 5."

    Trust me on this one. It's not like the word "data" where we monkeyed around and changed the semantics.

    So it's not, "Google Release Web APIs," it's "Google Releases Web APIs"; and it's not "Google have released ... ," it's "Google has released ... ." I know, it doesn't matter. That doesn't keep it from bothering me.

    • Except maybe in Europe. And some other places, too. I know when I was in London I noticed that collective nouns were treated as plural: IBM have released... and so on.

      Same's true if you watch enough Britcoms or other British imports (damn do they make good crime dramas!).

      My 2 cents...

      GTRacer
      - Should be returning to England in a year or so...

  • Maybe now Kuro5hin can redo their Interactive Sucks-Rules-O-Meter [kuro5hin.org].
  • <sarcasm>
    LOL maybe we should just dismantle the whole internet, as clearly the internet is the channel used by spammers! Oh wait. The internet has many many positive uses. Gee!
    </sarcasm>
    LOL a 4 for Interesting? Oh come on, this is ignorance, not information.


    Horrors! Spammers can use this!


    Uh 'scuse me but I can write a 10 line perl script that does the same thing. All I have to do is craft a query to google, and put a bunch of work into parsing out the real content from the HTML that comes back. Kind of a pain, but nothing a few regexp can't handle. This API is nothing new, it's just something handy. I'm seriously thinking I can replace a component of a research project here at our research facility with this. Why reinvent the wheel after all?


    Worse, are webmasters going to have to put a halt to Google crawls?
    Huh? What does this API have to do with being crawled by google? All this API does is let you do searches to google. Google is google is always searching. That's what robots.txt is for. You are not going to get crawled by this! This is not a BOT, just a QUERY TOOL.

  • The Google Rights include rights to the following:......(3) the search results and spell checking you obtain when you use Google Web APIs.

    I never thought I'd read the words "Google Rights" in a legal document, but anyway, how can Google own the rights to "spell checking".. what exactly do they own? The words that come back? The association of misspelled words to spelled words? How could you abuse that??

    I must say this is incredibly cool though.. however I would much rather see a generic "Search Engine API" that isn't owned by Google, and can be implemented by anyone.

    • I must say this is incredibly cool though.. however I would much rather see a generic "Search Engine API" that isn't owned by Google, and can be implemented by anyone.

      It could well become a defacto standard, as they are the first major search engine to do this, and other search engines will want to at the minimum provide a tool for migrating from Google powered applications to thier own, which of course means that one should be able to write to the google API and then use thier tools to port it to whatever one wants.

  • Notice how Google avoided use of CORBA (Score:1, Insightful)

    by Anonymous Coward
    Notice how Google avoided use of CORBA for these APIs. That speaks volumes in favour of SOAP's robustness over CORBA knowing how these Google dudes are perfectionists.

  • Not needed (Score:3, Interesting)

    by NineNine ( 235196 ) on Friday April 12, 2002 @12:12PM (#3330263)
    Why is this needed? I've been using Google programatically for a while now. What does this offer that I can't use on my own?

    • Re:Not needed (Score:3, Informative)

      by skunkeh ( 410004 )
      Legality [google.com] for one thing:
      No Automated Querying

      You may not send automated queries of any sort to Google's system without express permission in advance from Google. Note that "sending automated queries" includes, among other things:

      • using any software which sends queries to Google to determine how a website or webpage "ranks" on Google for various queries;
      • "meta-searching" Google; and
      • performing "offline" searches on Google.
      It also stops your scripts from breaking every time Google redesign their results page.
      • That's really pretty unenforceable. How is Google supposed to know if a query is automated? How is a query defined to be "automated"?

        For the latter, we have the following scenarios which could be interpreted as being automated:

        • You are using a computer to query Google. You're not actually twiddling the electrons in Google's servers with your fingers to perform the search.
        • If your browser has a built-in "Web Search" or "Google Search" function, the browser is automatically sending the query to Google and parsing the results before displaying them to you. You might also be using some standalone program that does this.
        • Some IRC bots (particularly infobot) have a Google search function, wherein someone (on a channel or in private) asks the bot to do a Google search. The bot does the search and displays the results to the user.
        • A user instructs their computer to do a Google search and save the results when a (dialup) Internet connection is established. Thus, a delayed search is performed. (This could be very useful for those who have to pay per minute for phone time to connect, and get lower rates during evenings or such. The connection would be established at a time when rates are low, the search performed, and the connection broken.)

    • I was going to moderate you down, but decided to just respond to you bluntly instead.

      You're a fucking idiot.

      -Russ
    • There's a difference between a "hack" and a solid implementation.

    • HTML is great to make pages readable for users.

      SOAP is great to get stuff from web in format programs understand.

      Yes, it's possible to use HTML parsing and stuff for any site (I'm using such tools in Everything2.com), but they all need atrocitious amount of HTML parsing. Thank God for visual-regexp. =)

      However, if E2 would use SOAP, I'd just ask them "Give me IDs of writeups matching title "don't force your gray philosophy on me" and it gives them, and I have them in an @array. Period.

      (Praying the security bug in SOAP::Lite will be fixed and Nate will make good use of mod_soap =)

  • Does anyone know of any consumer retail sites that have a similar api to their online catalogs?

    I am a grad student looking to avoid html scrapping for one of my projects.

    Thanks
  • So, Google limits automated queries, but allows unlimited interactive queries, even though the automated queries consume vastly less bandwidth and CPU than the interactive queries? Does this seem just a tad stupid to anyone else?

    Also, the limit on results per query severely limits the usefulness of this API.

    Finally, the requirement for a license key sounds a little Microsoftish to me. Since Google is not Microsoft, this is unlikely to work in their favor.

    For these reasons, I suspect that the release of this API may hurt Google more than it helps them.

Slashdot Top Deals

The game of life is a game of boomerangs. Our thoughts, deeds and words return to us sooner or later with astounding accuracy.

Close