Eval on tricky string forms of defaultdict

Question

Somehow, I'm handed a text files of defaultdict saved using the str() method and the utf8 string reads:

defaultdict(<class 'set'>, {'protection': {'1058c_204062v_00:39:16->00:39:18_ko'}, 'protect': {'50c_45523v_00:01:22->00:01:24_ko', '5457c_150765v_00:08:34->00:08:37_ko', '5457c_144739v_00:34:25->00:34:28_ko', '1058c_204062v_00:39:36->00:39:39_ko', '504c_68856v_00:15:47->00:15:49_ko'}})

When I use eval(), it throws:

Traceback (most recent call last):
  File "consolidate.py", line 9, in <module>
    print (eval(translation_counter), ast.literal_eval(location))
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ast.py", line 46, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    defaultdict(<class 'set'>, {'protection': {'1058c_204062v_00:39:16->00:39:18_ko'}, 'protect': {'50c_45523v_00:01:22->00:01:24_ko', '5457c_150765v_00:08:34->00:08:37_ko', '5457c_144739v_00:34:25->00:34:28_ko', '1058c_204062v_00:39:36->00:39:39_ko', '504c_68856v_00:15:47->00:15:49_ko'}})
                ^
SyntaxError: invalid syntax

According to https://stackoverflow.com/a/4020564/610569 , I've also tried ast.literal_eval() and it throws the same error as above.

Then I tried to somehow escape it using `.replace('_', '_') and it threw this:

Traceback (most recent call last):
  File "consolidate.py", line 9, in <module>
    print (eval(translation_counter), ast.literal_eval(location.replace('_', r'\_')))
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ast.py", line 46, in literal_eval
    node_or_string = parse(node_or_string, mode='eval')
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/ast.py", line 35, in parse
    return compile(source, filename, mode, PyCF_ONLY_AST)
  File "<unknown>", line 1
    defaultdict(<class 'set'>, {'protection': {'1058c\_204062v\_00:39:16->00:39:18\_ko'}, 'protect': {'50c\_45523v\_00:01:22->00:01:24\_ko', '5457c\_150765v\_00:08:34->00:08:37\_ko', '5457c\_144739v\_00:34:25->00:34:28\_ko', '1058c\_204062v\_00:39:36->00:39:39\_ko', '504c\_68856v\_00:15:47->00:15:49\_ko'}})
                ^
SyntaxError: invalid syntax

The full code:

# -*- coding: utf-8 -*-

from collections import defaultdict, Counter
import ast

with open('related.txt', 'r', encoding='utf8') as fin:
    for line in fin:
        location = line.strip()
        print (eval(location))

And the head -n1 related.txt looks like this:

defaultdict(<class 'set'>, {'protection': {'1058c_204062v_00:39:16->00:39:18_ko'}, 'protect': {'50c_45523v_00:01:22->00:01:24_ko', '5457c_150765v_00:08:34->00:08:37_ko', '5457c_144739v_00:34:25->00:34:28_ko', '1058c_204062v_00:39:36->00:39:39_ko', '504c_68856v_00:15:47->00:15:49_ko'}})

Answer 1

That's because <class 'set'> can not be eval'd. You will need to extract the class from it

p = re.compile(r"^defaultdict\(<class '(\w+)'>")
c = p.findall(s)[0]

and then replace it with the name of the class

new_s = s.replace("<class '%s'>"% c, c)

That string should be able to eval with this result

defaultdict(set,
        {'protect': {'1058c_204062v_00:39:36->00:39:39_ko',
          '504c_68856v_00:15:47->00:15:49_ko',
          '50c_45523v_00:01:22->00:01:24_ko',
          '5457c_144739v_00:34:25->00:34:28_ko',
          '5457c_150765v_00:08:34->00:08:37_ko'},
         'protection': {'1058c_204062v_00:39:16->00:39:18_ko'}})