comprehend.start_topics_detection_job Fails with Silent Error?

Question

I have Amazon sample code for running comprehend.start_topics_detection_job. Here is the code with the variables filled in for my job:

import re
import csv
import pytz
import boto3
import json

# https://docs.aws.amazon.com/code-samples/latest/catalog/python-comprehend-TopicModeling.py.html
# https://docs.aws.amazon.com/comprehend/latest/dg/API_InputDataConfig.html

# Set these values before running the program
input_s3_url = "s3://comprehend-topic-modelling-bucket/input_800_cleaned_articles/"
input_doc_format = "ONE_DOC_PER_LINE"
output_s3_url = "s3://comprehend-topic-modelling-bucket/output"
data_access_role_arn = "arn:aws:iam::372656143103:role/access-aws-services-from-sagemaker"
number_of_topics = 30

# Set up job configuration
input_data_config = {"S3Uri": input_s3_url, "InputFormat": input_doc_format}
output_data_config = {"S3Uri": output_s3_url}

# Begin a job to detect the topics in the document collection
comprehend = boto3.client('comprehend')
start_result = comprehend.start_topics_detection_job(
    NumberOfTopics=number_of_topics,
    InputDataConfig=input_data_config,
    OutputDataConfig=output_data_config,
    DataAccessRoleArn=data_access_role_arn)

# Output the results
print('Start Topic Detection Job: ' + json.dumps(start_result))
job_id = start_result['JobId']
print(f'job_id: {job_id}')

# Retrieve and output information about the job
describe_result = comprehend.describe_topics_detection_job(JobId=job_id)
print('Describe Job: ' + json.dumps(describe_result)) . #<===LINE 36

# List and output information about current jobs
list_result = comprehend.list_topics_detection_jobs()
print('list_topics_detection_jobs_result: ' + json.dumps(list_result))

It's failing with the error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-840a7ee043d4> in <module>()
     34 # Retrieve and output information about the job
     35 describe_result = comprehend.describe_topics_detection_job(JobId=job_id)
---> 36 print('Describe Job: ' + json.dumps(describe_result))
     37 
     38 # List and output information about current jobs

~/anaconda3/envs/python3/lib/python3.6/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    229         cls is None and indent is None and separators is None and
    230         default is None and not sort_keys and not kw):
--> 231         return _default_encoder.encode(obj)
    232     if cls is None:
    233         cls = JSONEncoder

~/anaconda3/envs/python3/lib/python3.6/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

~/anaconda3/envs/python3/lib/python3.6/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

~/anaconda3/envs/python3/lib/python3.6/json/encoder.py in default(self, o)
    178         """
    179         raise TypeError("Object of type '%s' is not JSON serializable" %
--> 180                         o.__class__.__name__)
    181 
    182     def encode(self, o):

TypeError: Object of type 'datetime' is not JSON serializable

It fails instantly, the second I pus "run". It seems to me that the call to comprehend.start_topics_detection_job may be failing, leading to an error line 36, print('Describe Job: ' + json.dumps(describe_result)).

What am I missing?

UPDATE

The same IAM role is being used for the notebook, as well as in the above code. Here are the permissions currently assigned to that IAM role:

Answer 1

It turns out that there was nothing wrong with the call to comprehend.describe_topics_detection_job -- it was just returning, in describe_result, something that could not be json serialized, so json.dumps(describe_result)) was throwing an error.