MicroPython client not receive text file larger than 4kb (4096 bytes) from Python Server

Question

I have an micropython client on esp32 board, and Python on linux server. I am trying send 5.5kb text file from Python Server to MicroPython client. It sends successfully but MicroPython client does not receive all data. Codes as follows;

Python Server:

with open('downloads/%s' % (request_path), 'rb') as f:
    data = f.read()
self.wfile.write(data) #data is 5.5kb

MicroPython Client

recvData = sock.read(4096).decode('utf-8').split("\r\n")
print("Response_Received:: %s" % recvData) 
sock.close()

Response_Received:: ['HTTP/1.0 200 OK', 'Server: SimpleHTTP/0.6 Python/3.5.3', 'Date: Sat, 09 Jun 2018 09:29:41 GMT', '', '# Ity: asdasd\n# ksduygfkhsgdkjfksjdhfg\n                kjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy98\n                47y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhs\n                gdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349rio\n                t34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r3\n                49riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkv\n                nvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\n                oiruy9847y397r349riot34jt;o\n                giji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbs\n                djkvbjcxbvhweioufhoiweuoiruy9847y397\n                r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufh\n                oiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n                fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhwei\n                oufhoiweuoiruy9847y397r349riot\n                34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n                fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhw\n                eioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjd\n                hfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiwe\n                uoiruy9847y397r349riot34jt;ogiji4vuij\n                o4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhwe\n                ioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfk\n                hsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;o\n                giji4vuijo4vjlkvnvl;kksduygfkhsgdk\n                jfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvh\n                weioufhoiweuoiruy9847y397r349riot34jt;ogiji\n                4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiru\n                y9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;k4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduyg\n                fkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcx\n                bvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhs\n                gdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdj\n                nvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9bfkjcbsdjkvbjcxbvhweioufhoi847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\nnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweu\nnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogijiksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufhoiweuoiruy9847y397r349riot34jt;ogiji4vuijo4vjlkvnvl;kksduygfkhsgdkjfksjdhfgkjdhsbfkjdhsbfkjcbsdjkvbjcxbvhweioufnvl;k']

Client receives only 4140 bytes of the array data in due to buffer size(4096), 4th element of the recvData is lost. MicroPython does not accept over this Buffer size. How can i receive all my data (5.5kb) in 4th element of recvData array without any loss?

I have tried to fragment the received data, but it was not successful.

while True: 
    chunck = s.recv(4096)
    if not chunck: 
        break
    fragments.append(chunck)

Answer 1

Since your goal is to write the file to the filesystem, the simplest solution is to stop trying to hold the entire file in memory. Instead of building up your fragments array, just write the received chunks to a file:

with open('datafile', 'w') as fd:
  while True: 
      chunk = s.recv(4096)
      if not chunk: 
          break
      fd.write(chunk)

This requires a constant amount of memory and can be used to receive files of arbitrary size.