Yos Riady software craftsman 🌱

Consuming Web APIs with Python

Python provides a few built-in libraries to help with the process of consuming third-party web APIs. In this post I’d like to briefly outline how to retrieve and process web APIs with Python, using the Google Maps Geocoding API as an example.

Here’s a rundown of the libraries we will be using for this demonstration:

  • urllib2, which defines functions and classes which help in opening URLs
  • json, which defines helpful functions for dealing with the JSON format

You’ll also want to checkout Requests, an HTTP library with a much cleaner API, written in Python to replace urllib2.

Another library which may be helpful if you are following along is:

  • pprint, provides a capability to “pretty-print” arbitrary Python data structures, helpful for debugging

Geocoding is the process of converting addresses (like “1600 Amphitheatre Parkway, Mountain View, CA”) into geographic coordinates (like latitude 37.423021 and longitude -122.083739), which you can use to place markers or position the map. The Google Geocoding API provides a direct way to access a these services via an HTTP request.

We will interface with the Google Maps Geocoding API and write a method that allows us to geocode addresses. Let’s start by importing all our libraries:

import urllib2
import pprint
import json

Next, we read from the Google Maps Geocoding API documentation, that a Geocoding API request must be of the following form:

https://maps.googleapis.com/maps/api/geocode/json?parameters

The required parameters for this API are address — the address that you want to geocode — and sensor — which indicates whether or not the geocoding request comes from a device with a location sensor (boolean.) The API provides other optional parameters, such as region, language, and components; however I will leave them out for simplicity.

In this example, the Geocoding API requests a json response for a query on “1600 Amphitheatre Parkway, Mountain View, CA”:

https://maps.googleapis.com/maps/api/geocode/json?address=1600+Amphitheatre+Parkway,+Mountain+View,+CA&sensor=false

Let’s continue by opening the URL in urllib2:

import urllib2
import pprint
import json

address_str = '1600 Amphitheatre Parkway, Mountain View, CA'
address = urllib2.quote(address_str) #1
geocode_url = "https://maps.googleapis.com/maps/api/geocode/json?address=%s&sensor=false" % address
res = urllib2.urlopen(geocode_url) #2
res.read()

A few things of note:

  • We need to escape special characters in strings, such as spaces before we can pass it as a parameter. (#1)
  • We then make an HTTP GET request with urlopen and save the HTTP response (#2)

Reading the response outputs this mess we get:

'{\n   "results" : [\n      {\n         "address_components" : [\n            {\n               "long_name" : "1600",\n               "short_name" : "1600",\n               "types" : [ "street_number" ]\n            },\n            {\n               "long_name" : "Amphitheatre Parkway",\n               "short_name" : "Amphitheatre Pkwy",\n               "types" : [ "route" ]\n            },\n            {\n               "long_name" : "Mountain View",\n               "short_name" : "Mountain View",\n               "types" : [ "locality", "political" ]\n            },\n            {\n               "long_name" : "Santa Clara",\n               "short_name" : "Santa Clara",\n               "types" : [ "administrative_area_level_2", "political" ]\n            },\n            {\n               "long_name" : "California",\n               "short_name" : "CA",\n               "types" : [ "administrative_area_level_1", "political" ]\n            },\n            {\n               "long_name" : "United States",\n               "short_name" : "US",\n               "types" : [ "country", "political" ]\n            },\n            {\n               "long_name" : "94043",\n               "short_name" : "94043",\n               "types" : [ "postal_code" ]\n            }\n         ],\n         "formatted_address" : "1600 Amphitheatre Parkway, Mountain View, CA 94043, USA",\n         "geometry" : {\n            "location" : {\n               "lat" : 37.42141110,\n               "lng" : -122.08403720\n            },\n            "location_type" : "ROOFTOP",\n            "viewport" : {\n               "northeast" : {\n                  "lat" : 37.42276008029150,\n                  "lng" : -122.0826882197085\n               },\n               "southwest" : {\n                  "lat" : 37.42006211970850,\n                  "lng" : -122.0853861802915\n               }\n            }\n         },\n         "types" : [ "street_address" ]\n      }\n   ],\n   "status" : "OK"\n}\n'

Using the pprint library, we can format our response:

import pprint

pp = pprint.PrettyPrinter(indent=4)
pp.pprint(res.read())
 # Much better!
    {
   "results" : [
      {
         "address_components" : [
            {
               "long_name" : "1600",
               "short_name" : "1600",
               "types" : [ "street_number" ]
            },
            {
               "long_name" : "Amphitheatre Parkway",
               "short_name" : "Amphitheatre Pkwy",
               "types" : [ "route" ]
            },
            {
               "long_name" : "Mountain View",
               "short_name" : "Mountain View",
               "types" : [ "locality", "political" ]
            },
            {
               "long_name" : "Santa Clara",
               "short_name" : "Santa Clara",
               "types" : [ "administrative_area_level_2", "political" ]
            },
            {
               "long_name" : "California",
               "short_name" : "CA",
               "types" : [ "administrative_area_level_1", "political" ]
            },
            {
               "long_name" : "United States",
               "short_name" : "US",
               "types" : [ "country", "political" ]
            },
            {
               "long_name" : "94043",
               "short_name" : "94043",
               "types" : [ "postal_code" ]
            }
         ],
         "formatted_address" : "1600 Amphitheatre Parkway, Mountain View, CA 94043, USA",
         "geometry" : {
            "location" : {
               "lat" : 37.42141110,
               "lng" : -122.08403720
            },
            "location_type" : "ROOFTOP",
            "viewport" : {
               "northeast" : {
                  "lat" : 37.42276008029150,
                  "lng" : -122.0826882197085
               },
               "southwest" : {
                  "lat" : 37.42006211970850,
                  "lng" : -122.0853861802915
               }
            }
         },
         "types" : [ "street_address" ]
      }
   ],
   "status" : "OK"
}

Next, let’s deserialize our JSON response (usually a str or unicode instance) to a Python dictionary so we can extract and process it:

import pprint
import urllib2
import pprint
import json

address_str = '1600 Amphitheatre Parkway, Mountain View, CA'
address = urllib2.quote(address_str) #1
geocode_url = "https://maps.googleapis.com/maps/api/geocode/json?address=%s&sensor=false" % address
res = urllib2.urlopen(geocode_url) #2
jsonResponse = json.loads(res.read())

Now, it’s just a matter of finding the lat/lng values in the dictionary:

lat = jsonResponse['results'][0]['geometry']['location']['lat']
lng = jsonResponse['results'][0]['geometry']['location']['lng']
print lat,lng

Reorganizing the code into a function, we have:

import urllib2
import pprint
import json
import sys

def geocode(address_str):
    address = urllib2.quote(address_str)
    geocode_url = "https://maps.googleapis.com/maps/api/geocode/json?address=%s&sensor=false" % address
    req = urllib2.urlopen(geocode_url)
    jsonResponse = json.loads(req.read())

    status_code = jsonResponse['status']
    if status_code != 'OK':
        print status_code
    else:
        lat = jsonResponse['results'][0]['geometry']['location']['lat']
        lng = jsonResponse['results'][0]['geometry']['location']['lng']
        return lat,lng

lat,lng = geocode("1600 Amphitheatre Parkway, Mountain View, CA")
# returns 37.42141110, -122.08403720

And we’re done! We now have a method which can geocode address strings into a tuple of latitude/longitude values! You can use them to place markers on a map, calculate their great-circle distance from each other using the haversine formula, or pretty much anything under the sun!

Author

Yos is a software craftsman based in Singapore.

📬 Subscribe to my newsletter

Get notified of my latest articles by providing your email below.


Going Serverless book

Interested to find out more about serverless? Going Serverless teaches you how to build scalable applications with the Serverless framework and AWS Lambda. You'll learn how to design, develop, test, deploy, and secure Serverless applications from planning to production.

Learn More →