python angular angular6 xlsxwriter angular-httpclient

Angular6 HttpClient POST and flask - content difference on backend between httpclient and curl request

I'm using HttpClient to do a post:

return this.http.post(`reporting/report/generate/${code}`, {'mimetype': mimetype, 'input': value}, {responseType: mimetype});

against my API server using the following mimetype: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, so I'm getting xlsx data.

On server side (through nginx and uwsgi), I'm running flask with the following code (cropped):

    outputStream = io.BytesIO()

    workbook = xlsxwriter.Workbook(outputStream)
    worksheet = workbook.add_worksheet(Report.name())

    [...]

    worksheet.write(0, 0, 'YY', 'XX')

    workbook.close()

    output = outputStream.getvalue()
    outputStream.close()

    return Response(output, mimetype=mimetype)

(using the same mimetype for my response).

Back in angular, I'm using blob url and so on to download my file, but this is not important. The problem is that my Excel file is broken when I make the Http Request from angular and is OK when I use curl to execute the exact same request (copied from Chrome Network Tab).

BUT: I write a temp xlsx file on server side and both files are ok.

I printed the bytes of output and I see that there are some bytes that are different:

b'PK\x03\x04\x14\x00\x00\x00\x08\x00\x13}\x15M\xf5\x8e\x0eH\xb9\x01\x00\x00I\x04\x00\x00\x18\...docProps/app.xmlPK\x01\x02\x14\x03\x14\x00\x00\x00\x08\x00\x13}\x15M\x9c\xed\xd2_%\x01\x00\x00P\x02\x00\x00\x11\x00\x00\x00\x00\x00\x00\x00\x00

vs.

b'PK\x03\x04\x14\x00\x00\x00\x08\x003}\x15M\xf5\x8e\x0eH\xb9\x01\x00\x00I\x04\x00\x00\...docProps/app.xmlPK\x01\x02\x14\x03\x14\x00\x00\x00\x08\x003}\x15M\xa5\xc4\xa4$%\x01\x00\x00P\x02\x00\x00\x11\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x80\x81E\x06\x00\x00docProps/core.xmlPK\x01\x02\x14\x03\

90% of the bytes are the same, but some are different.

I'm searching for hours and I can't find the problem. Strange is that the temp file is ok in both situations, but the representation shows a little difference.

I also analysed the Requests and I see a small difference:

<Request 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1' [POST]>
{'environ': {'QUERY_STRING': ''
REQUEST_METHOD': 'POST'
CONTENT_TYPE': 'application/json'
CONTENT_LENGTH': '126'
REQUEST_URI': '/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1'
PATH_INFO': '/report/generate/DEMO_TECH_DEMO1'
DOCUMENT_ROOT': '/usr/share/nginx/html'
SERVER_PROTOCOL': 'HTTP/1.1'
REQUEST_SCHEME': 'http'
REMOTE_ADDR': '192.168.69.16'
REMOTE_PORT': '54436'
SERVER_PORT': '80'
SERVER_NAME': 'lsdev01'
HTTP_HOST': 'lsdev01'
HTTP_CONNECTION': 'keep-alive'
HTTP_CONTENT_LENGTH': '126'
HTTP_PRAGMA': 'no-cache'
HTTP_CACHE_CONTROL': 'no-cache'
HTTP_ACCEPT': 'application/json, text/plain, */*'

HTTP_ORIGIN': 'http://localhost:4213'

HTTP_AUTHORIZATION': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1MzQ0OTM4MDksInN1YiI6eyIkJFVTUklETE9HSU4iOiIxNDQ5IiwiJCRVU1JOQU1FTE9HSU4iOiI5OSAtIHdlaXRlcmUgRmFjaGdydXBwZW4iLCIkJFVTUkNPREVMT0dJTiI6IkRXRSIsIiQkU0VTU0lPTkxPR0lEIjoxLCIkJFNJVEVJRCI6IjkwMDAwMDMyNzAiLCIkJFNJVEVHUlAiOiJMQUJPIiwiJCRURVNURkVBVFVSRUxJU1QiOiJURVNUUkVGTkVXXHUwMDFiTVVMVElERUJcdTAwMWJSRVRSSUVWRU5FV1x1MDAxYlJFU1JFRlx1MDAxYkxYUkJBQzJcdTAwMWJGQ1RURVNUXHUwMDFiQ09NVEVTVFx1MDAxYk5PTkVVTklRVUVCQ1x1MDAxYlBFUkZfTFhURVhUIiwiJCRMQU5HVUFHRSI6IiIsIkxBTkdVQUdFIjpudWxsfSwiZXhwIjoxNTM0NTgwMjA5LCJpbnN0YW5jZSI6ImRldiJ9.33Pid_GDPP6LUgoGzv_uU26QisBJNqoqsc7uTczdLvU'
HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36'
HTTP_CONTENT_TYPE': 'application/json'
HTTP_REFERER': 'http://localhost:4213/login'
HTTP_ACCEPT_ENCODING': 'gzip, deflate'
HTTP_ACCEPT_LANGUAGE': 'de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7,fr;q=0.6'
SCRIPT_NAME': '/lx/ui_api/dev/dwe/dwe/reporting'
wsgi.input': <uwsgi._Input object at 0x7fd792236300>
wsgi.version': (1, 0)
wsgi.errors': <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.run_once': False
wsgi.multithread': True
wsgi.multiprocess': False
wsgi.url_scheme': 'http'
uwsgi.version': b'2.0.15'
uwsgi.core': 3
uwsgi.node': b'lsdev01'
werkzeug.request': <Request 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1' [POST]>}
shallow': False
view_args': {'reportCode': 'DEMO_TECH_DEMO1'}
url_rule': <Rule '/report/generate/<reportCode>' (OPTIONS, POST) -> report_generate>
_parsed_content_type': ('application/json', {})
stream': <werkzeug.wsgi.LimitedStream object at 0x7fd78f7d7438>
_cached_data': b'{"mimetype":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet","input":{"header":"Test 1","footer":"Test 2"}}'
_cached_json': {'mimetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
input': {'header': 'Test 1'
footer': 'Test 2'}}
url': 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1'}

vs.

<Request 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1' [POST]>
{'environ': {'QUERY_STRING': ''
REQUEST_METHOD': 'POST'
CONTENT_TYPE': 'application/json'
CONTENT_LENGTH': '126'
REQUEST_URI': '/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1'
PATH_INFO': '/report/generate/DEMO_TECH_DEMO1'
DOCUMENT_ROOT': '/usr/share/nginx/html'
SERVER_PROTOCOL': 'HTTP/1.1'
REQUEST_SCHEME': 'http'
REMOTE_ADDR': '192.168.69.18'
REMOTE_PORT': '55154'
SERVER_PORT': '80'
SERVER_NAME': 'lsdev01'
HTTP_HOST': 'lsdev01'
HTTP_ACCEPT_ENCODING': 'deflate, gzip'



HTTP_ACCEPT': 'application/json, text/plain, */*'
HTTP_REFERER': 'http://localhost:4213/login'
HTTP_ORIGIN': 'http://localhost:4213'
HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36'
HTTP_AUTHORIZATION': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE1MzQ0OTM4MDksInN1YiI6eyIkJFVTUklETE9HSU4iOiIxNDQ5IiwiJCRVU1JOQU1FTE9HSU4iOiI5OSAtIHdlaXRlcmUgRmFjaGdydXBwZW4iLCIkJFVTUkNPREVMT0dJTiI6IkRXRSIsIiQkU0VTU0lPTkxPR0lEIjoxLCIkJFNJVEVJRCI6IjkwMDAwMDMyNzAiLCIkJFNJVEVHUlAiOiJMQUJPIiwiJCRURVNURkVBVFVSRUxJU1QiOiJURVNUUkVGTkVXXHUwMDFiTVVMVElERUJcdTAwMWJSRVRSSUVWRU5FV1x1MDAxYlJFU1JFRlx1MDAxYkxYUkJBQzJcdTAwMWJGQ1RURVNUXHUwMDFiQ09NVEVTVFx1MDAxYk5PTkVVTklRVUVCQ1x1MDAxYlBFUkZfTFhURVhUIiwiJCRMQU5HVUFHRSI6IiIsIkxBTkdVQUdFIjpudWxsfSwiZXhwIjoxNTM0NTgwMjA5LCJpbnN0YW5jZSI6ImRldiJ9.33Pid_GDPP6LUgoGzv_uU26QisBJNqoqsc7uTczdLvU'

HTTP_CONTENT_TYPE': 'application/json'
HTTP_CONTENT_LENGTH': '126'


SCRIPT_NAME': '/lx/ui_api/dev/dwe/dwe/reporting'
wsgi.input': <uwsgi._Input object at 0x7fd792236300>
wsgi.version': (1, 0)
wsgi.errors': <_io.TextIOWrapper name=2 mode='w' encoding='UTF-8'>
wsgi.run_once': False
wsgi.multithread': True
wsgi.multiprocess': False
wsgi.url_scheme': 'http'
uwsgi.version': b'2.0.15'
uwsgi.core': 0
uwsgi.node': b'lsdev01'
werkzeug.request': <Request 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1' [POST]>}
shallow': False
view_args': {'reportCode': 'DEMO_TECH_DEMO1'}
url_rule': <Rule '/report/generate/<reportCode>' (OPTIONS, POST) -> report_generate>
_parsed_content_type': ('application/json', {})
stream': <werkzeug.wsgi.LimitedStream object at 0x7fd78f7bcf98>
_cached_data': b'{"mimetype":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet","input":{"header":"Test 1","footer":"Test 2"}}'
_cached_json': {'mimetype': 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet'
input': {'header': 'Test 1'
footer': 'Test 2'}}
url': 'http://lsdev01/lx/ui_api/dev/dwe/dwe/reporting/report/generate/DEMO_TECH_DEMO1'}

but there is nothing special...

Does someone know what the problem could be?

EDIT: I also tried send_files, but I have the same problem.

Solution

DAMMMMNNN!! I got it!

I encoded as base64 on server side:

return Response(base64.b64encode(output), mimetype=mimetype)

and in angular I combined some functions that I found on so:

  download(mimetype, data) {

    function s2ab(s) {
      var buf = new ArrayBuffer(s.length);
      var view = new Uint8Array(buf);
      for (var i=0; i!=s.length; ++i) view[i] = s.charCodeAt(i) & 0xFF;
      return buf;
    }

    var blob = new Blob([s2ab(atob(data))], {type: mimetype});
    var objectUrl = URL.createObjectURL(blob);
    window.open(objectUrl);
  }

atob = decode base64 encoded string s2ab = string to array buffer for the blob

I think the origin of all this is an encoding problem between the client and the server. But I have no explanation why!