Search code examples
bashshellcurltelligent

Why does this curl command return Unexpected EOF


I am somewhat new to Bash and cURL and cannot understand out why this Bash file does not run without throwing an Unexpected EOF error.

This cURL command should upload a large file (in the example script below, somewhere between 300 and 400 MB) in 20 MB chunks to a storage service. Once all MB are uploaded, a second command "completes" the upload. Both commands use the same GUID.

Inside upload-bits.sh:

#!/bin/sh
for i in {0..20}; do
curl -X POST \
  https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
  -H 'Rest-User-Token: 12345' \
  -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
  -F FileName='file.zip' \
  -F TotalChunks=20 \
  -F CurrentChunk=$i \
  -F '[email protected]'
done

The Bash script throws the Unexpected EOF error. I have tried the cURL command alone without the Bash portion of the script and replaced CurrentChunk with 0 and 1 without success. I also used a script validator, which confirmed there were no problems in the script. I also ran dos2unix on it in a desire to eliminate end-of-line issues.

I have not been able to use this second script yet, as the first script has not worked, but I am posting it for context if I am not explaining the desired overall process well.

complete-upload.sh:

curl -X POST \
  https://community.<company>.com/api.ashx/v2/media/371/files.json \
  -H 'Rest-User-Token: 12345' \
  -F 'Name=file.zip' \
  -F ContentType=application/zip \
  -F FileName='file.zip' \
  -F FileUploadContext=21f23109-aac2-44ef-8b89-c0f62e67da4d

I would be grateful for any tips or insights. Thank you.


Solution

  • Judging by the parameters passed to curl, the server expects chunked data.

    However the curl command sends the whole file 20 times.

    Looking at definition of CurrentChunk at https://community.telligent.com/community/10/w/api-documentation/61481/upload-cfs-rest-endpoint , perhaps a modification like this would work:

    #!/bin/bash
    
    # using GNU split options will make arithmetic simpler
    # with -d, we may get numbers like 09 which are invalid octal
    # start from 101 if CurrentChunk is one-based
    # start from 100 if CurrentChunk is zero-based
    split -b20M -a3 --numeric-suffixes=101 file.zip part.
    
    partlist=( part.* )
    numparts=${#partlist[@]}
    
    for part in ${partlist[@]}; do
      i=$(( ${part##*.}-100 ))
      curl -X POST \
        https://community.<company>.com/api.ashx/v2/cfs/temporary.json \
        -H 'Rest-User-Token: 12345' \
        -F UploadContextId=21f23109-aac2-44ef-8b89-c0f62e67da4d \
        -F FileName='file.zip' \
        -F TotalChunks=$numparts \
        -F CurrentChunk=$i \
        -F 'file=@'$part
    done
    
    rm ${partlist[@]}