Search code examples
bashsecuritycurlenvironment-variablessh

Forcing cURL to get a password from the environment

This question about using cURL with a username and password has suboptimal answers for me:

  1. curl -u "user:pw" https://example.com puts the pw in the process list
  2. curl "https://user:[email protected]" puts the pw in the process list
  3. curl -u "user:$(cat ~/.passwd)" https://example.com puts the pw in the process list
  4. curl -u user https://example.com prompts for the pw
  5. curl --netrc-file ~/.netrc https://example.com requires a file

#4 is secure, but I might run this command hundreds of times a day, so it's tedious. #5 is close to secure, but that file could be read by somebody with root access.

The cURL man page says (note the bold text):

-u/--user <user:password>

Specify the user name and password to use for server authentication. Overrides -n/--netrc and --netrc-optional.

If you just give the user name (without entering a colon) curl will prompt for a password.

If you use an SSPI-enabled curl binary and do NTLM authentication, you can force curl to pick up the user name and password from your environment by simply specifying a single colon with this option: -u :.

I've tried setting $USER and $PASSWORD (and $CURLOPT_PASSWORD and others) in the environment, but cURL doesn't pick up either of them when invoked as curl -u : https://example.com (nor does it work without the -u :).

I'm not doing NTLM, so this doesn't work. Unless I'm missing something.

 

Is there a way to pass credentials to curl solely through the environment?

 

(Workaround moved to an answer)

Solution

  • This bash solution appears to best fit my needs. It's decently secure, portable, and fast.

    #!/bin/bash
SRV="example.com"
URL="https://$SRV/path"
curl --netrc-file <(cat <<<"machine $SRV login $USER password $PASSWORD") "$URL"

    This uses process substitution (<( command ) runs command in a sub-shell to populate a file descriptor to be handed as a "file" to the parent command, which in this case is curl). The process substitution contains a here-string (cat <<< text, a variant of echo text that won't put anything into your process list), creating a file descriptor for the netrc file in order to pass credentials to the remote web server.

    The security afforded by process substitution is actually pretty sound: its file descriptor is not a temporary file and is unavailable from even other calls in the same shell instance, so this appears secure in this context; an adversary would have to dig through memory or launch a complicated attack to find its contents. Since the $PASSWORD environment variable is also in memory, this should not increase the attack surface.

    As long as you haven't used export PASSWORD, a trick like ps ewwp $$ shouldn't reveal the password (as noted in this comment). It'd also be wise to use some less obvious variable name.

    Here is a simplified insecure version of the above code that may help explain how it works:

    #!/bin/sh
# INSECURE VERSION, DO NOT USE
SRV=example.com
URL="https://$SRV/path"
TMP=$(mktemp)
printf "machine %s login %s password %s\n" "$SRV" "$USER" "$PASSWORD" > "$TMP"
curl --netrc-file "$TMP" "$URL"
rm -f "$TMP"

    This insecure version has lots of flaws, all of which are solved in the previous version:

    • It stores the password in a file (though that file is only readable to you)
    • It very briefly has the password in a command line
    • The temporary file remains until after curl exits
    • Ctrl+c will quit without removing the temporary file

    Some of that could be solved by:

    #!/bin/sh
SRV=example.com
URL="https://$SRV/path"
TMP=$(mktemp /dev/shm/.XXXXX)  # assumes /dev/shm is a ramdisk
trap "rm -f $TMP" 0 18
cat << EOF > "$TMP"
machine $SRV login $USER password $PASSWORD
EOF
(sleep 0.1; rm -f "$TMP") &  # queue removing temp file in 0.1 seconds
curl --netrc-file "$TMP" "$URL"

    I consider this version to be messy, suboptimal, and possibly less secure (though it is more portable). It also requires a version of sleep that understands decimals (and 0.1 seconds may be too fast if the system is heavily loaded).

     

    I had originally posted a workaround that included a perl one-liner in my question, then (with help from Etan Reisner) I worked through a few better methods before settling on this here-string method, which is both lighter-weight (faster) and more portable.

    At this point, it's elegant enough that I'd consider it the "answer" rather than an "ugly workaround," so I've migrated it to be this official answer. I've given @ghoti a +1 for his answer, which correctly states that cURL's command line program is incapable of doing what I want on its own, but I'm not "accepting" that answer because it doesn't help solve the issue.