Search code examples
bashshellcurlsendy

Shell script - CURL post form data with another CURL captured html text in parameters

I need to capture a html content and pass it to another curl as form data.

My solution:

curl http://example.com/campaigns/create.php \
    -F api_key=myKey \
    -F subject="My Subject" \
    -F list_ids=12345 \
    -F html_text="'$(curl -s -L https://somedomain.com?feed=sports)'" &>/dev/null

Note: Yes, I have tried html_text="$(curl -s -L https://somedomain.com?feed=sports)". But server can't resolve isset($_POST['html_text']) then.

My output:

'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head>
    .....
    </center> </body> </html>'

This above example works fine except html_text wrapped with an unnecessary single quote ('). Any suggestion how can get I rid of this single quote around the html_text?

Solution

  • You are putting the single quotes there. Don't do that.

        -F html_text="'$(curl ... -)'"

    The double quotes collect the value into a single string as far as the shell is concerned, and the literal single quotes are part of that literal string.

    Fixed code:

    curl http://example.com/campaigns/create.php \
    -F api_key=myKey \
    -F subject="My Subject" \
    -F list_ids=12345 \
    -F html_text="$(curl -s -L https://somedomain.com?feed=sports)" >/dev/null 2>&1

    Notice also the POSIX-compatible redirection; I see no reason to use the Bash-specific notation &>

    If the server cannot handle this value, perhaps it expects it in some specific format or encoding; but without access to information about your server, we can't help troubleshoot that part.

    Update:

    If the HTML starts with a < character, it will look to the top-level curl like you are saying -F<!DOCTYPE ... where the text after < should be a file name.

    You can work around this by using this construct explicitly:

    curl -s -L https://somedomain.com?feed=sports |
curl http://example.com/campaigns/create.php \
    -F api_key=myKey \
    -F subject="My Subject" \
    -F list_ids=12345 \
    -F html_text="<-"