Calling annotate_text from IBM Watson Concept Insights with Python

Question

I wrote a set of Python functions to interact with the Bluemix/Watson Concept Insights API. I am able to generate a token and use it to get a result from the server, but the result stinks: it's nowhere near as good as what I get when I plug the same information into their Swagger testing utility.

I suspect that something is wrong with the way I am sending my request, but I don't know quite what. The code follows. First, from event_insight_lib.py:

def importCredentials(filename='credentials.json'):
    if filename in [f for f in os.listdir('.') if os.path.isfile(f)]:
        data = json.load(open(filename))['concept_insights'][0]['credentials']
        return data

def generateToken(filename='credentials.json'):
    credentials = importCredentials(filename)
    r = requests.get("https://gateway.watsonplatform.net/authorization/api/v1/token\?url=https://stream.watsonplatform.net/concept-insights/api", auth=(credentials['username'], credentials['password']))
    if r.status_code == requests.codes.ok:
        return r.text

def annotateText(text, token, content_type = 'text/plain'):
    base_url='https://watson-api-explorer.mybluemix.net/concept-insights/api/v2/graphs/wikipedia/en-20120601/annotate_text'
    headers = {'X-Watson-Authorization-Token': token, 'Content-Type': content_type}
    r = requests.post(base_url, headers=headers, data={'body': text})
    return r.text

These methods are executed by event_insight.py:

token = event_insight_lib.generateToken()
ret = event_insight_lib.annotateText("""long string being concept-analyzed...""", token)
    print(ret)

A full demonstration of the difference in output is here. The full codebase is here. I'm not very experienced with the Requests library: is there a subtle mistake somewhere on the Pythonic end?

The relevant part of IBM's documentation is here.

Answer 1

As @engineerc suggested you are sending a dict() as data. Quoting your comment data=text.encode(encoding='UTF-8', errors='ignore') is the solution for your problem.

---

On the other hand, please don't use https://watson-api-explorer.mybluemix.net, it's a proxy application we use to host the swagger documentation.
The service url is: https://gateway.watsonplatform.net/concept-insights/api

Also, we have a python-sdk that supports ConceptInsights and the annotate_text call.

It's a pip module so you will do:

pip install watson-developer-cloud

Calling annotate_text it's as simple as:

import json
from watson_developer_cloud import ConceptInsightsV2 as ConceptInsights


concept_insights = ConceptInsights(
    username='YOUR SERVICE USERNAME',
    password='YOUR SERVICE PASSWORD')

annotations = concept_insights.annotate_text('IBM Watson won the Jeopardy television show hosted by Alex Trebek')
print(json.dumps(annotations, indent=2))