When dealing with file streams using the AWS CLI (see "Downloading an S3 object as a local file stream" in the docs here), does any processing performed on the stream occur AWS server side, or on the machine which executed the command?
To provide an example of what I mean, assume some_process below is a perl script or something like that which processes streaming data.
If I would execute:
aws s3 cp s3://mybucket/unprocessed_stream.txt - | \
some_process | \
aws s3 cp - s3://mybucket/processed_stream.txt
Should the some_process perl script be executed AWS server side (without downloading the data locally) or locally (implying the - character in the aws s3 cp command performs a download).
Ignoring the pipes, you are running three commands:
aws s3 cp s3://mybucket/unprocessed_stream.txt -
some_process
aws s3 cp - s3://mybucket/processed_stream.txt
Each of those three commands run on your local machine. The first and second obviously use network resources to some extent, but still, aws is running your local machine. Just as some_process, the only thing the | operator adds is that the output from the first command is feed to the second as input, and so on.
In other words, some_process is running locally. It may, of course, reach out and do things with network resources, but that would be work initiated by your local machine.