The previous pipeline artifact is loaded for checking (file name is cache_deploy_.properties). The file contains every module and the last changed timestamp. A file is updated with timestamps (hashcode) if the module has changed from the previous run (yes similar to a Git commit). After this file has been updated, it is published as a pipeline artifact for the next run.
File contents example
#Tue Aug 01 11:09:53 AEST 2023
D\:\\agent_work\\agent-2\\168\\s\\PRIE\\Programs\\IEW340P3.NSP=ec656d00d2e5e12ae1eb3aa9771af464\!\:\!null
D\:\\agent_work\\agent-2\\168\\s\\PRMR\\Maps\\MRJ400M1.NSM=c4b10568d2dd5f88f255cf0192b0945d\!\:\!null
What is happening is, next time a pipleline runs, the azure "user agent" is different, So the modules being checked in the file are a thought to be a different location
echo $(System.DefaultWorkingDirectory)
D\:\\agent_work\\agent-1\\217\\s\\ ----> (compared to previous run as above D\:\\agent_work\\agent-2\\168\\s\\)
So when it goes to check this file, the recorded information will no longer match (as different "user agent" in the file). And the pipeline will try to create new entries for EVERY module. I cannot control the output to the file... otherwise I would remove the user agent (propriatry JAR from a vendor and Ant Script). It uses the complete file location in it's analysis to determine what to do with the changed modules.
What this is trying to achieve... need to rename ALL the lines in the file with the previous "user agent" to match the current "user agent" Use a bash script in a pipeline build to change the user agent in the artifact loaded file. I am trying the following, but might need some help please.
Script example
echo '#Tue Aug 01 14:16:13 AEST 2023
D\:\\agent_work\\agent-2\\168\\s\\PRIE\\Programs\\IEW340P3.NSP=ec656d00d2e5e12ae1eb3aa9771af464\!\:\!null
D\:\\agent_work\\agent-2\\168\\s\\PRUT\\Maps\\UTRESM06.NSM=eb0e29846516c74be5763a8cd9ab22d8\!\:\!null
D\:\\agent_work\\agent-2\\168\\s\\PRMR\\Maps\\MRJ400M1.NSM=c4b10568d2dd5f88f255cf0192b0945d\!\:\!null' > myconfig.txt
file="myconfig.txt"
# userAgent='D\:\\agent_work\\agent-2\\168\\s\\' #This is user agent is in the file no need for fixing!
userAgent='D\:\\agent_work\\agent-1\\170\\s\\'
echo "Current User agent:" $userAgent
if grep -F $userAgent $file; then
echo "Found user agent, no need to replace"
else
echo "User Agent not found in file, need to replace with current"
var=$(sed '2q;d' $file) # just need to check for prev agent on 2nd line
substr="s" # this 's' is the end position of the agent
prefix=#${var%%$substr*} #gives us the index nuumber of 's'
index=$((${#prefix}+2)) # add 2 to cater for the // after s
replaceUser=${var:0:index} #finally have the string to search the file for to replace
echo "Previous user agent:" $replaceUser
sed -i --debug "s/'$replaceUser'/$userAgent/g" $file # <-- This is not working
fi
Output:
Current User agent: D\:\\agent_work\\agent-1\\170\\s\\
User Agent not found in file, need to replace with current
Previous user agent: D\:\\agent_work\\agent-2\\168\\s\\P
SED PROGRAM:
s/'D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\P'/D:\agent_work\agent-1\170\s\/g
INPUT: 'myconfig.txt' line 1
PATTERN: #Tue Aug 01 14:16:13 AEST 2023
COMMAND: s/'D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\'/D:\agent_work\agent-1\170\s\/g
PATTERN: #Tue Aug 01 14:16:13 AEST 2023
END-OF-CYCLE:
INPUT: 'myconfig.txt' line 2
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRIE\\\\Programs\\\\IEW340P3.NSP=ec656d00d2e5e12ae1eb3aa9771af464\\!\\:\\!null
COMMAND: s/'D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\'/D:\agent_work\agent-1\170\s\/g
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRIE\\\\Programs\\\\IEW340P3.NSP=ec656d00d2e5e12ae1eb3aa9771af464\\!\\:\\!null
END-OF-CYCLE:
INPUT: 'myconfig.txt' line 3
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRUT\\\\Maps\\\\UTRESM06.NSM=eb0e29846516c74be5763a8cd9ab22d8\\!\\:\\!null
COMMAND: s/'D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\'/D:\agent_work\agent-1\170\s\/g
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRUT\\\\Maps\\\\UTRESM06.NSM=eb0e29846516c74be5763a8cd9ab22d8\\!\\:\\!null
END-OF-CYCLE:
INPUT: 'myconfig.txt' line 4
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRMR\\\\Maps\\\\MRJ400M1.NSM=c4b10568d2dd5f88f255cf0192b0945d\\!\\:\\!null
COMMAND: s/'D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\'/D:\agent_work\agent-1\170\s\/g
PATTERN: D\\:\\\\agent_work\\\\agent-2\\\\168\\\\s\\\\PRMR\\\\Maps\\\\MRJ400M1.NSM=c4b10568d2dd5f88f255cf0192b0945d\\!\\:\\!null
END-OF-CYCLE:
The full backslashes are not being used in the replacement string and it looks like sed is not matching the pattern for string replacement.
Final result should be the file is replaced with the correct string including matching and retaining double backslashes using bash
The full backslashes are not being used in the replacement string and it looks like sed is not matching the pattern for string replacement.
In particular:
sed -i --debug "s/'$replaceUser'/$userAgent/g" $file # <-- This is not working
I'm sure it isn't working. At minimum, the single quotes in that command are inside double quotes, and therefore represent themselves. Your file does not contain corresponding single-quote characters, so no lines will match that regex. Additionally, the backslashes in the expansion of userAgent will not be understood literally by sed, so you would get an incorrect replacement. (Also, my sed, at least, does not understand --debug.)
The backslash is an escape character for sed. To express a literal backslash, us a double backslash (\\).
The backslash is also an escape character for the shell, but that significance is suppressed by quoting with single quotes, which you appear to be using effectively for that purpose.
Your original code determines the old user agent from the file via a mix of a command substitution with sed and several bash parameter expansions, then tries to construct a sed expression to replace the old agent string with the new one. But you are already using sed, and the user-agent identification is right in its wheelhouse, so why are you making it so complicated?
Possibly you have simplified a more complex real-world scenario, but with what you've actually presented, I would probably do something more like this:
#!/bin/bash
file=myconfig.txt
userAgent='D\:\\agent_work\\agent-1\\170\\s\\'
echo "Current User agent: $userAgent"
if grep -q -F "$userAgent" "$file"; then
echo "Found user agent, no need to replace"
else
echo "User Agent not found in file, need to replace with current"
sed -i -e '2,$s/^[^s]*s\\\\/'"${userAgent//\\/\\\\}/" "$file"
fi
In fact, I might even take out the user-agent test, and just do the substitution unconditionally:
#!/bin/bash
file=myconfig.txt
userAgent='D\:\\agent_work\\agent-1\\170\\s\\'
sed -i -e '2,$s/^[^s]*s\\\\/'"${userAgent//\\/\\\\}/" "$file"
That uses about the same approach as your original to select the currect user-agent string (everything from the beginning of a line to the first 's' followed by a double backslash, as opposed to everything up to the first 's' regardless of backslashes), and it substitutes the chosen user agent string for that on the second and all subsequent lines.
The main trick there is doubling the backslashes in the sed expression, both the literal part and the expansion of $userAgent. The literal part is handled literally, and the $userAgent part with the pattern replacement feature of shell parameter expansion. ${userAgent//\\/\\\\} says the value of shell parameter userAgent, with all appearances of the pattern \\ (which, being unquoted, represents a single backslash character) replaced by \\\\ (which, being unquoted, represents two backslash characters).
Pay attention to the quoting details, too, however, as your original script is somewhat under-quoted.
If you need, for reasons not evident in the question, to stick closer to your original code, then you should be able to use the same kind of substitutions in the original code. Maybe with something like this:
sed -i --debug "s/${replaceUser//\\/\\\\}/${userAgent//\\/\\\\}/" "$file"