Why does this curl command return Unexpected EOF

Question

I am somewhat new to Bash and cURL and cannot understand out why this Bash file does not run without throwing an Unexpected EOF error.

This cURL command should upload a large file (in the example script below, somewhere between 300 and 400 MB) in 20 MB chunks to a storage service. Once all MB are uploaded, a second command "completes" the upload. Both commands use the same GUID.

Inside upload-bits.sh:

#!/bin/sh
for i in {0..20}; do
curl -X POST \
  https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
  -H 'Rest-User-Token: 12345' \
  -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
  -F FileName='file.zip' \
  -F TotalChunks=20 \
  -F CurrentChunk=$i \
  -F 'file=@file.zip'
done

The Bash script throws the Unexpected EOF error. I have tried the cURL command alone without the Bash portion of the script and replaced CurrentChunk with 0 and 1 without success. I also used a script validator, which confirmed there were no problems in the script. I also ran dos2unix on it in a desire to eliminate end-of-line issues.

I have not been able to use this second script yet, as the first script has not worked, but I am posting it for context if I am not explaining the desired overall process well.

complete-upload.sh:

curl -X POST \
  https://community.<company>.com/api.ashx/v2/media/371/files.json \
  -H 'Rest-User-Token: 12345' \
  -F 'Name=file.zip' \
  -F ContentType=application/zip \
  -F FileName='file.zip' \
  -F FileUploadContext=21f23109-aac2-44ef-8b89-c0f62e67da4d

I would be grateful for any tips or insights. Thank you.

Answer 1

Judging by the parameters passed to curl, the server expects chunked data.

However the curl command sends the whole file 20 times.

Looking at definition of CurrentChunk at https://community.telligent.com/community/10/w/api-documentation/61481/upload-cfs-rest-endpoint , perhaps a modification like this would work:

#!/bin/bash

# using GNU split options will make arithmetic simpler
# with -d, we may get numbers like 09 which are invalid octal
# start from 101 if CurrentChunk is one-based
# start from 100 if CurrentChunk is zero-based
split -b20M -a3 --numeric-suffixes=101 file.zip part.

partlist=( part.* )
numparts=${#partlist[@]}

for part in ${partlist[@]}; do
  i=$(( ${part##*.}-100 ))
  curl -X POST \
    https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
    -H 'Rest-User-Token: 12345' \
    -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
    -F FileName='file.zip' \
    -F TotalChunks=$numparts \
    -F CurrentChunk=$i \
    -F 'file=@'$part
done

rm ${partlist[@]}