Uploading chunked file using Ionic4 and Angular HttpClient fails after some successful uploads with net::ERR_FILE_NOT_FOUND
i'm trying to upload a large file (500+Mb, but could be even bigger) to our php server, using an app written in Ionic4+Angular+Cordova, on an emulator with Android 10. I set up a system to upload the file in chunks. It reads the file choosen by the user using THIS PLUGIN, chunk by chunk (5Mb per chunk). Then it proceeds to send it to our server performing a POST request with Content-type multipart/form-data. The file goes to server, server saves it, says "OK", then the app proceeds to send the following chunk. Everything works fine, for the first 25/29 chunks. Then, the POST request fails with
POST http://192.168.1.2/work/path/to/webservices/uploadChunks.php net::ERR_FILE_NOT_FOUND
I tried:
- starting at another point in the file instead of byte 0 - got the same error
- reading the file chunk by chunk, without making any POST request- could cycle the whole 500Mb file
- reading the file chunk by chunk and making the POST requests, but not sending the chunks with them - could execute every single call without any error, through the end of the file
- reading the file chunk by chunk and sending them to ANOTHER webservice - got the same error
- reading the file chunk by chunk and performing a POST request to another webservice, with content-type application/json and putting the formData object into the request body (not sure this is a valid test tho) - could execute every single call without any error, through the end of the file
Checking out memory snapshots taken in chrome inspector during different chunks upload did not show any sign of memory leak.
The case was tested on a rather old device, where the same procedure caused the app to exit, without signaling any error (not even in logcat apparently).
Here is the piece of code used to chunk and send the file:
const generatedName = 'some_name_for_file';
// Path obtained from fileChooser plugin
let path_to_file = 'content://com.android.externalstorage.documents/document/primary%3ADownload%2Ffilename.avi'
const min_chunk_size = (5 * 1024 * 1024);
// Converting path to file:// path
this.filePath.resolveNativePath(path_to_file).then((resolvedPath) => {
return this.fileAPI.resolveLocalFilesystemUrl(resolvedPath);
}, (error) => {
console.log('ERROR FILEPATH');
console.log(error);
return Promise.reject('Can not access file.<br>Code : F - ' + error);
}).then(
(entry) => {
path_to_file = entry.toURL();
console.log(path_to_file);
(entry as FileEntry).file((file) => {
//Getting back to the zone
this.ngZone.run(() => {
// Re-computing chunk size to be sure we do not get more than 10k chunks (very remote case)
let file_chunk_size = file.size / 10000;
if (file_chunk_size < min_chunk_size) {
file_chunk_size = min_chunk_size;
}
//Total number of chunks
const tot_chunk = Math.ceil(file.size / file_chunk_size);
const reader = new FileReader();
let retry_count = 0; //Counter to check on retries
const readFile = (nr_part: number, part_start: number, length: number) => {
// Computing end of chunk
const part_end = Math.min(part_start + length, file.size);
// Slicing file to get desired chunk
const blob = file.slice(part_start, part_end);
reader.onload = (event: any) => {
if (event.target.readyState === FileReader.DONE) {
let formData = new FormData();
//Creating blob
let fileBlob = new Blob([reader.result], {
type: file.type
});
formData.append('file', fileBlob, generatedName || file.name);
formData.append('tot_chunk', tot_chunk.toString());
formData.append('nr_chunk', nr_part.toString());
// UPLOAD
const sub = this.http.post('http://192.168.1.2/path/to/webservice/uploadChunk.php', formData).subscribe({
next: (response: any) => {
console.log('UPLOAD completed');
console.log(response);
retry_count = 0;
if (response && response.status === 'OK') {
//Emptying form and blob to be sure memory is clean
formData = null;
fileBlob = null;
// Checking if this was the last chunk
if (part_end >= file.size) {
// END
callback({
status: 'OK'
});
} else {
// Go to next chunk
readFile(nr_part + 1, part_end, length);
}
//Clearing post call subscription
sub.unsubscribe();
} else {
//There was an error server-side
callback(response);
}
},
error: (err) => {
console.log('POST CALL ERROR');
console.log(err);
if (retry_count < 5) {
setTimeout(() => {
retry_count++;
console.log('RETRY (' + (retry_count + 1) + ')');
readFile(nr_part, part_start, length);
}, 1000);
} else {
console.log('STOP RETRYING');
callback({status:'ERROR'});
}
}
});
}
};
//If for some reason the start point is after the end point, we exit with success...
if (part_start < part_end) {
reader.readAsArrayBuffer(blob);
} else {
callback({
status: 'OK'
});
}
};
//Start reading chunks
readFile(1, 0, file_chunk_size);
});
}, (error) => {
console.log('DEBUG - ERROR 3 ');
console.log(error);
callback({
status: 'ERROR',
code: error.code,
message: 'Can not read file<br>(Code: 3-' + error.code + ')'
});
});
}, (error) => {
console.log('ERROR 3');
console.log(error);
return Promise.reject('Can not access file.<br>Code : 3 - ' + error);
}
);
I can not figure out what is going wrong. Can someone help me debug this, or knows what could be going on?
Thank you very much.
I still do not know what caused this issue, but i resolved using a PUT request instead of a POST request, sending the raw chunk, and putting additional data in custom headers (something like "X-nr-chunk" or "X-tot-chunk"). Upload completed fine without the error message.
I also used the cordova-advanced-http plugin, but i do not think it made a difference here, since it did not work with the POST request, like the other method (httpClient).
This has been tested on android only for now, not on iOS. I'll report if there is any problem. For now i consider this solved, but if you know what may have caused this problem, please share your thoughts.
Thanks everyone.