Goal I need to effectively run a copy (cp) command but have explicit quote symbols preserved. This is needed so that the z/OS Unix System Services Korn shell properly recognizes the target of the copy as a traditional MVS dataset.
The complexity is that this step is part of an automated process. The command is generated by Perl. That Perl is executed on a separate Docker container via ssh. This adds another layer of escaping that needs to be addressed, in addition to the escaping needed by Perl.
Basically, docker is doing something like
perl myprogram.perl
which generates the necessary SSH commands, sending them to the mainframe which tries to run them. When I run the Perl script, it generates the command
sshpass -p passwd ssh woodsmn@bldbmsb.boulder.mycompany.com export _UNIX03=NO;cp -P "RECFM=FB,LRECL=287,BLKSIZE=6027,SPACE=\(TRACK,\(1,1\)\)" /u/woodsmn/SSC.D051721.T200335.S90.CP037 "//'WOODSMN.SSC.D051721.T200335.S90.CP037'"
and the mainframe returns an error:
cp: target "//'WOODSMN.SSC.D051721.T200335.S90.CP037'" is not a directory
The sshpass is needed because my sysadmin refuses to turn on authorized users, so my only option is to run sshpass and shove a password in. The password exposure is contained and we're not worried about this.
The first command
export _UNIX03=NO
tells z/OS to treat the -P option as an indicator for MVS dataset control blocks. That is, this is where we tell the system, hey this is a fixed length of 287 characters, allocate in tracks, etc. The dataset will be assumed to be new.
For the copy command, I'm wanting z/OS to copy the HFS file (basically a normal UNIX file)
/u/woodsmn/SSC.D051721.T200335.S90.CP037
into the fully qualifed MVS dataset
WOODSMN.SSC.D051721.T200335.S90.CP037
Sometimes MVS commands assume a high level qualifier of basically the users userid and allow the user to omit this. In this case, I've explicitly specified this.
To get z/OS to treat the target as a dataset, one needs to prefix it with two slashes (/), so // to use a fully qualified dataset, the name needs to be surrounded by an apostrophe (')
But, to avoid confusion within Korn shell, the target needs to be surrounded by double quotes (").
So, somehow between Perl, the shell running my SSH command inside the Docker container (likely bash) and the receiving Korn shell on z/OS, it's not being properly interpreted.
My scaled down Perl looks like:
use strict;
use warnings;
sub putMvsFileByHfs;
use IO::Socket::IP;
use IO::Socket::SSL;
use IPC::Run3;
use Net::SCP;
my $SSCJCL_SOURCE_DIRECTORY = "/home/bluecost/";
my $SSCJCL_STORAGE_UNIT = "TRACK";
my $SSCJCL_PRIMARY_EXTENTS = "1";
my $SSCJCL_SECONDARY_EXTENTS = "1";
my $SSCJCL_HFS_LOCATION="/u/woodsmn";
my $SSCJCL_STAGING_HLQ = "WOODSMN";
my $COST_FILE="SSC.D051721.T200335.S90.CP037";
my $SSCJCL_USER_PW="mypass";
my $SCJCL_USER_ID="woodsmn";
my $SSCJCL_HOST_NAME="bldbmsb.boulder.mycompany.com";
my $MVS_FORMAT_OPTIONS="-P ".qq(")."RECFM=FB,LRECL=287,BLKSIZE=6027,SPACE=\\("
.${SSCJCL_STORAGE_UNIT}
.",\\("
.${SSCJCL_PRIMARY_EXTENTS}
.","
.${SSCJCL_SECONDARY_EXTENTS}
."\\)\\)".qq(");
putMvsFileByHfs(${MVS_FORMAT_OPTIONS}." ",
$SSCJCL_SOURCE_DIRECTORY.'/'.$COST_FILE,
${SSCJCL_HFS_LOCATION}.'/'.$COST_FILE,
${SSCJCL_STAGING_HLQ}.'.'.$COST_FILE);
# This function copys the file first from my local volume mounted to the Docker container
# to my mainframe ZFS volume. Then it attempts to copy it from ZFS to a traditional MVS
# dataset. This second part is the failinmg part.
sub putMvsFileByHfs
{
#
# First copy the file from the local file system to my the mainframe in HFS form (copy to USS)
# This part works.
#
my $OPTIONS = shift;
my $FULLY_QUALIFIED_LOCAL_FILE = shift;
my $FULLY_QUALIFIED_HFS_FILE = shift;
my $FULLY_QUALIFIED_MVS_FILE = shift;
RunScpCommand($FULLY_QUALIFIED_LOCAL_FILE,$FULLY_QUALIFIED_HFS_FILE);
#
# I am doing something wrong here
# Attempt to build the target dataset name.
#
my $dsnPrefix = qq(\"//');
my $dsnSuffix = qq('\");
my $FULLY_QUALIFIED_MVS_ARGUMENT = ${dsnPrefix}.${FULLY_QUALIFIED_MVS_FILE}.${dsnSuffix};
RunSshCommand("export _UNIX03=NO;cp ${OPTIONS}".${FULLY_QUALIFIED_HFS_FILE}." ".${FULLY_QUALIFIED_MVS_ARGUMENT});
}
# This function marshals whatever command I want to run and mostly does it. I'm not having
# any connectivity issues. My command at least reaches the server and SSH will try to run it.
sub RunScpCommand()
{
my $ssh_source= $_[0];
my $ssh_target= $_[1];
my ($out,$err);
my $in = "${SSCJCL_USER_PW}\n";
my $full_command = "sshpass -p ".${SSCJCL_USER_PW}." scp ".${ssh_source}." ".${SSCJCL_USER_ID}."@".${SSCJCL_HOST_NAME}.":".${ssh_target};
print ($full_command."\n");
run3 $full_command,\$in,\$out,\$err;
print ($out."\n");
print ($err."\n");
return ($out,$err);
}
# This function marshals whatever command I want to run and mostly does it. I'm not having
# any connectivity issues. My command at least reaches the server and SSH will try to run it.
sub RunSshCommand
{
my $ssh_command = $_[0];
my $in = "${SSCJCL_USER_PW}\n";
my ($out,$err);
my $full_command = "sshpass -p ".${SSCJCL_USER_PW}." ssh ".${SSCJCL_USER_ID}."@".${SSCJCL_HOST_NAME}." ".${ssh_command};
print ($full_command."\n");
run3 $full_command,\$in,\$out,\$err;
print ($out."\n");
print ($err."\n");
return ($out,$err);
}
Please forgive any Perl malpractices above as I'm new to Perl, though kind constructive pointers are appreciated.
First, let's build the values we want to pass to the program. We'll worry about building shell commands later.
my @OPTIONS = (
-P => join(',',
"RECFM=FB",
"LRECL=287",
"BLKSIZE=6027",
"SPACE=($SSCJCL_STORAGE_UNIT,($SSCJCL_PRIMARY_EXTENTS,$SSCJCL_SECONDARY_EXTENTS))",
),
);
my $FULLY_QUALIFIED_LOCAL_FILE = "$SSCJCL_SOURCE_DIRECTORY/$COST_FILE";
my $FULLY_QUALIFIED_HFS_FILE = "$SSCJCL_HFS_LOCATION/$COST_FILE";
my $FULLY_QUALIFIED_MVS_FILE = "$SSCJCL_STAGING_HLQ.$COST_FILE";
my $FULLY_QUALIFIED_MVS_ARGUMENT = "//'$FULLY_QUALIFIED_MVS_FILE'";
Easy peasy.
Now it's time to build the commands to execute. The key is to avoid trying to do multiple levels of escaping at once. First build the remote command, and then build the local command.
use String::ShellQuote qw( shell_quote );
my $scp_cmd = shell_quote(
"sshpass",
-p => $SSCJCL_USER_PW,
"scp",
$FULLY_QUALIFIED_LOCAL_FILE,
"$SSCJCL_USER_ID\@$SSCJCL_HOST_NAME:$FULLY_QUALIFIED_HFS_FILE",
);
run3 $scp_cmd, ...;
my $remote_cmd =
'_UNIX03=NO ' .
shell_quote(
"cp",
@OPTIONS,
$FULLY_QUALIFIED_HFS_FILE,
$FULLY_QUALIFIED_MVS_ARGUMENT,
);
my $ssh_cmd = shell_quote(
"sshpass",
-p => $SSCJCL_USER_PW,
"ssh", $remote_cmd,
);
run3 $ssh_cmd, ...;
But there's a much better solution since you're using run3. You can entirely avoid creating a shell on the local host, and thus entirely avoid having to create a command for it! This is done by passing a reference to an array containing the program and its args instead of passing a shell command.
use String::ShellQuote qw( shell_quote );
my @scp_cmd = (
"sshpass",
-p => $SSCJCL_USER_PW,
"scp",
$FULLY_QUALIFIED_LOCAL_FILE,
"$SSCJCL_USER_ID\@$SSCJCL_HOST_NAME:$FULLY_QUALIFIED_HFS_FILE",
);
run3 \@scp_cmd, ...;
my $remote_cmd =
'_UNIX03=NO ' .
shell_quote(
"cp",
@OPTIONS,
$FULLY_QUALIFIED_HFS_FILE,
$FULLY_QUALIFIED_MVS_ARGUMENT,
);
my @ssh_cmd = (
"sshpass",
-p => $SSCJCL_USER_PW,
"ssh", $remote_cmd,
);
run3 \@ssh_cmd, ...;
By the way, it's insecure to pass passwords on the command line; other users on the machine can see them.
By the way, VAR=VAL cmd (as a single command) sets the env var for cmd. I used that shorthand above.