Using Google Search API from Java

April 25th, 2008 vira Posted in Computer science | 14 Comments »

Recently I’ve discovered ability to search in Google from Java program in a way different from brute “crawling”. And I’ve found out two options:

With this API you can issue search requests to Google’s index of billions of web pages and receive results as structured data, access information in the Google cache, and check the spelling of words. Google SOAP Search API is implemented as a web service and all you need is just get WSDL and call remote methods through SOAP protocol. Usage of this service is limited to 1000 requests per day.

But as you can read on SOAP API site

As of December 5, 2006, we are no longer issuing new API keys for the SOAP Search API. Developers with existing SOAP Search API keys will not be affected.

So you aren’t able to get a new API key (I found the old one in Internet). And Google encourages you to use the AJAX Search API, which is described below.

Example of code that perform search request and show obtained results:

import com.google.soap.search.*;
import java.io.*;
public class SimpleGoogleDemo {
	public static void main(String[] args) {
		// Create a Google Search object, set our authorization key
		GoogleSearch s = new GoogleSearch();
		String clientKey="xxxxxxxxxxxxxxxxxxxxxxxxxxxx";
		s.setKey(clientKey);
		s.setQueryString("vera.org.ua");
		GoogleSearchResult result = null;
		// Depending on user input, do search or cache query, then print out result
		try {
			result = s.doSearch();
		}
		catch (GoogleSearchFault f) {
			System.out.println("The call to the Google Web APIs failed:");
			System.out.println(f.toString());
		}    // if we made it here, the search went through
		System.out.println("Google Search Results:");
		System.out.println("======================");
		if (result!= null)
			System.out.println(result.toString());
	}
}

To run this code you need to have googleapi.jar in your classpath. This jar file contains very useful and easy-to-use class com.google.soap.search.GoogleSearch, which does all search work.

Google AJAX API

Usage of this unlimited and you can make as many request as you want. Despite the fact that Google in this API FAQ say “The Google AJAX Search API is currently available only for websites.” ability to use it exists :-)

Example of code that perform search request and show obtained results.

import java.io.*;
import java.net.*;
 
public class GoogleAJAXSearchAPI {    
 
	private static String endpointURL = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=vera";
 
	public static void main(String[] args) throws Exception {
		URLConnection uc = new URL(endpointURL).openConnection();
		HttpURLConnection connection = (HttpURLConnection) uc;
		connection.setDoOutput(true);
		connection.setRequestMethod("GET");
		connection.connect();
		String line;
		InputStream inputStream = null;
		try {
			inputStream = connection.getInputStream();
		} catch (IOException e) {
			inputStream = connection.getErrorStream();
		}
		BufferedReader rd = new BufferedReader(new InputStreamReader(inputStream));
		while ((line = rd.readLine()) != null) {
			System.out.println(line);
		}
	}
}

To run this code you need to can get API key here.

So, have fun :)

PS Take a look at this too – Java OO wrapper for Google AJAX API.


You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

14 Responses to “Using Google Search API from Java”

  1. ai Says:
    April 25th, 2008 at 5:15 pm

    I’ve recently looked for API to some giant search engine.

    And stopped on Yahoo’s API. Reason is simple – they’re provide SOAP and REST access to their engine. More handy approach, eh?

  2. vira Says:
    April 25th, 2008 at 5:58 pm

    But Google is Google :-)

  3. harsha Says:
    August 12th, 2008 at 9:13 am

    helloo.. could anybody help me..
    here in the above general code of google soap protocol.. i need google api jar file
    in the above link i am not able to download, there was an internal error with webserver..

    so could anybody forward to my mail account pleasee..

    harsha.siri@gmail.com

    thanking you forever…. harsha

  4. vira Says:
    August 12th, 2008 at 11:19 am

    >>in the above link i am not able to download, there was an internal error with webserver..

    Sorry for inconvenience. This problem was fixed.

  5. kulpster Says:
    December 7th, 2008 at 11:52 pm

    Vira,

    Can you explain the endpointURL string? I am trying to use my own google api key in the string but I am getting this back:

    {”responseData”: null, “responseDetails”: “invalid version”, “responseStatus”: 400}

    thanks.

  6. vira Says:
    December 8th, 2008 at 3:12 am

    kulpster,

    Checked this example again and it doesn’t works anymore :( Try to use this url “http://ajax.googleapis.com/ajax/services/search/web?v=1.0&q=query_str” . It’s much simpler and works for me. It returns 4 result at a time, to get next 4 use “start” parameter.

    BTW, according to google docs api key is optional for now. If you provide key and some error occurs while searching Google can help work it out :)

  7. Damian Says:
    January 19th, 2009 at 4:17 pm

    I get different results using Google.com and your code. Do you know the default preferences of ajax.googleapis.com?
    What I want to get is to obtain the same result with your code and google.com search.

  8. @ver Says:
    January 19th, 2009 at 5:41 pm

    2 Damian

    google ajax api and real google have different results (and mau be different index base).
    For example yandex.xml api and yandex.ru have different indexes (index yandex.xml more new).

    So, if you whant real google.com result – parse it ;-)

  9. Damian Says:
    January 20th, 2009 at 12:24 pm

    But Google sends back 503 (Forbidden).
    I heard that I have to send Http-Agent header, but I don’t know how to send it. Do you?

  10. vira Says:
    January 20th, 2009 at 11:03 pm

    @Damian
    You just need to set corresponding system property
    e.g. System.setProperty(”http.agent”, “Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; ru-ru) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1″);

  11. Damian Says:
    January 21st, 2009 at 10:04 am

    Thank you :)

  12. kevin Says:
    November 5th, 2009 at 6:47 pm

    How to parse?

    I want to get the same search number by using API as the real google search. Is there any java code example?

    2 Damian

    google ajax api and real google have different results (and mau be different index base).
    For example yandex.xml api and yandex.ru have different indexes (index yandex.xml more new).

    So, if you whant real google.com result – parse it ;-)

  13. leo Says:
    February 4th, 2010 at 3:56 pm

    hi,

    can u pls tell me how to run the code for ajax api? i got a ajax api key as required..

  14. vira Says:
    February 7th, 2010 at 12:21 am

    hi there!

    the code for ajax I provided here is very basic: it just prints Google response to console and query is statically defined :( . I’d recommend you to use my wrapper located here. In that post I placed an example of using Googler class. Currently, it doesn’t support API key, but this functionality could be easily added – just specify API key as value of “key” HTTP request parameter.

    BTW I’m going to add this some other functionality to the Googler in the distant future.

Leave a Reply

« Fun with Macbook’s accelerometer :)
True MacCat »