I'm using anaconda to select only some fields from my json.gz file using-
shodan parse --fields port,isp,hostnames,location,timestamp,org,domain,transport,vulns,ssl,http.server,ip_str --separator , shodan-export.json.gz > shodan-export-proc.json.gz
I keep getting this error though-
(base) C:\Users\me>shodan parse --fields ip,port,isp,hostnames,location,timestamp,org,domain,transport,vulns,ssl,http.server,ip_str --separator , shodan-export.json.gz > shodan-export-proc.json.gz
Traceback (most recent call last):
File "D:\Software\Anaconda\Scripts\shodan-script.py", line 11, in <module>
load_entry_point('shodan==1.11.1', 'console_scripts', 'shodan')()
File "D:\Software\Anaconda\lib\site-packages\click\core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "D:\Software\Anaconda\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "D:\Software\Anaconda\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "D:\Software\Anaconda\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "D:\Software\Anaconda\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "D:\Software\Anaconda\lib\site-packages\shodan-1.11.1-py3.7.egg\shodan\__main__.py", line 360, in parse
File "D:\Software\Anaconda\lib\site-packages\click\utils.py", line 260, in echo
file.write(message)
File "D:\Software\Anaconda\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 3627-3628: character maps to <undefined>
I came across this GitHub issue, and believe I need to use utf-8 encoding while converting my file; but how do I do so for Shodan/Anaconda?
In Anaconda- type the following and it will work-
chcp 65001
set PYTHONIOENCODING=utf-8