I already have an access_token and refresh_token, but I can't figure out a way to create an authorized gdata client without going through the entire token generation workflow in gdata.
So I got this working finally. Here's how I did it:
client = gdata.contacts.client.ContactsClient()
credentials = gdata.gauth.OAuth2Token(client_id = 'client_id',
client_secret = 'client_secret',
scope = 'https://www.google.com/m8/feeds/',
user_agent = auth.user_agent, # This is from the headers sent to google when getting your access token (they don't return it)
access_token = auth.access_token,
refresh_token = auth.refresh_token)
credentials.authorize(client)
contacts = client.get_contacts()