Search code examples
node.jsstreamreadable

readable stream on data event losing first/last character from chunk


I am using ssh2-sftp-client for nodeJS to connect to sftp server and get a file But I've meet a problem, when are more chunks than one, the data is not received correctly, it loses one character between chunks:
ex.: file on sftp has 'some string from sftp file'
and if it is splitted in chunks, the received data will be like:
first chunk : 'some string f'
second chink: 'om sftp file'
in this example 'r' is lost

const getFile = readable => new Promise((resolve, reject) => {
  let file = '';
   readable.on('data', (chunk) => { file += chunk; }); 
  readable.on('end', () => resolve(file));
  readable.on('error', reject);
});

const readable = await sftp.get(fileName, false);

sftp.get() return NodeJS.ReadableStream

Does someone meet same problem?


Solution

  • After a long research I found the problem, in module sftp-stream, for readable stream highWaterMark is seted to 64*1024, and the bug is that if a chunk has 64*1024 bites then one bite is lost. And I just setted watermark to 64*1024-1 .