About 5 hours ago, version 4.1.0 was released. It is breaking my unit tests. Here is a clean MVCE displaying this:
Version 3.12:
>>> import numpy as np
>>> import yaml
>>> x = np.int64(2)
>>> yaml.dump(x, Dumper=yaml.Dumper)
'!!python/object/apply:numpy.core.multiarray.scalar\n- !!python/object/apply:numpy.dtype\n args: [i8, 0, 1]\n state: !!python/tuple [3, <, null, null, null, -1, -1, 0]\n- !!binary |\n AgAAAAAAAAA=\n'
Version 4.1.0:
>>> import numpy as np
>>> import yaml
>>> x = np.int64(2)
>>> yaml.dump(x, Dumper=yaml.Dumper)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/foo/anaconda3/envs/bar/lib/python3.6/site-packages/yaml/__init__.py", line 217, in dump
return dump_all([data], stream, Dumper=Dumper, **kwds)
File "/foo/anaconda3/envs/bar/lib/python3.6/site-packages/yaml/__init__.py", line 196, in dump_all
dumper.represent(data)
File "/foo/anaconda3/envs/bar/lib/python3.6/site-packages/yaml/representer.py", line 26, in represent
node = self.represent_data(data)
File "/foo/anaconda3/envs/bar/lib/python3.6/site-packages/yaml/representer.py", line 57, in represent_data
node = self.yaml_representers[None](self, data)
File "/foo/anaconda3/envs/bar/lib/python3.6/site-packages/yaml/representer.py", line 229, in represent_undefined
raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', 2)
Is there a clear reason for why PyYAML no longer supports these object types?
The breaking changes in 4.x were an attempt to resolve CVE-2017-18342, which is the possibility of arbitrary code execution for untrusted inputs.
In PyYAML 4.x, dump is an alias for safe_dump, which won't handle arbitrary objects:
>>> yaml.dump is yaml.safe_dump
True
Use danger_dump for the old 3.x behaviour.
>>> yaml.danger_dump(x)
'!!python/object/apply:numpy.core.multiarray.scalar\n- !!python/object/apply:numpy.dtype\n args: [i8, 0, 1]\n state: !!python/tuple [3, <, null, null, null, -1, -1, 0]\n- !!binary |\n AgAAAAAAAAA=\n'
The same goes for load/safe_load. Can't find any docs or release notes for 4.1.0, I only found out by digging through the commits (here).
Is there a clear reason for why PyYAML no longer supports these object types?
Yes. yaml.load was allowing arbitrary code execution, and such a dangerous feature should be opt-in only, not possible to use by accident. Arguably, it should have been this way from the beginning.
In current PyYAML 5.x: instead of using different functions, you can specify the loader/dumper class as an argument:
yaml.dump(x, Dumper=yaml.Dumper) # like "danger dump"
yaml.dump(x, Dumper=yaml.SafeDumper) # like "safe_dump", won't dump python objs
As with 3.x, the "danger" dump is still the default in 5.x:
>>> yaml.dump(sys)
"!!python/module:sys ''\n"
>>> yaml.dump(sys, Dumper=yaml.SafeDumper)
RepresenterError: ('cannot represent an object', <module 'sys' (built-in)>)