JFrog Artifactory: 5.8.4 Professional.
JFrog CLI: 1.12.1
I'm using the official documentation of JFrog CLI to set few properties on an artifact that's available in Artifactory repo.
Artifactory AQL for now and trying to get this working using jfrog cli.I successfully configured the CLI client (the file ~/.jfrog/jfrog-cli.conf with entries for authentication using either username/password and with an API Key was created successfully).
For setting properties on an artifact, I referred the following example:
jfrog rt sp "generic-local/*.zip" "a=1;b=2,3"
and ran the following command (in a free-style Jenkins job):
$ jfrog rt sp --server-id="artifactory-dev-instance" --url=${ARTIFACTORY_URL} ${REPO}/${FILE_PATH}/${FILE_NAME} -- props="release=${RELEASE};VERSION=${VERSION};PIPELINE_VERSION=${RELEASE}_${VERSION}_${BUILD_NUMBER};fileType=automated-file;PROJECT=${PROJECT}"
This command exited successfully without giving an error and applied all the properties with expected values except the first property release=${RELEASE}.
Question: Why jfrog rt sp is not applying the release property on the artifact while all others properties got successfully applied with correct values.
Actual variables values don't matter as I can see other properties getting applied successfully in the given Artifactory instance (ARTIFACTORY_URL), on the target file (sitting in a repo/path/file) and respective properties with their values.
So, this proves it actually (bug part in jfrog cli). re-add that first property i.e. add the missing first property twice and it works for all properties (including the first one) while giving properties to be added.
Now release property is getting applied as well.
$ jfrog rt sp --server-id="artifactory-dev-instance" --url=${ARTIFACTORY_URL} ${REPO}/${FILE_PATH}/${FILE_NAME} -- props="release=${RELEASE};release=${RELEASE};VERSION=${VERSION};PIPELINE_VERSION=${RELEASE}_${VERSION}_${BUILD_NUMBER};fileType=automated-file;PROJECT=${PROJECT}"
or
$ jfrog rt sp --server-id="artifactory-dev-instance" --url=${ARTIFACTORY_URL} ${REPO}/${FILE_PATH}/${FILE_NAME} -- props="release=${RELEASE};VERSION=${VERSION};PIPELINE_VERSION=${RELEASE}_${VERSION}_${BUILD_NUMBER};fileType=automated-file;PROJECT=${PROJECT};release=${RELEASE}"
NOTE: While setting properties jfrog rt sp, you can specify multiple values for the same property name using , i.e. --props="prop1=value1;prop2=value22,prop2=value23;prop3=value3" but during jfrog rt s (search), you can't use , (if you use , or ; at all for matching a property for more than 1 possible values, jfrog rt s --prop="..." is not going to give you all the results as it'll pick only the last value and override all other previous values for that given property).
ALSO, using jfrog cli --server-id="..." and --url="Artifactory URL" (as listed above) can be used in a more secure way if you use SSH key.
Example of using JFROG with SSH key (for better authentication) doesn't require username/password or server-id at command line (which requires ~/.jfrog/jfrog.conf file):
Just create your SSH pub/private key. Add pub key for the user (running the command in Artifactory wide/repo wide settings), then set JFROG_URL variable as:
JFROG_URL="ssh://artifactory-develop:1339"
and then simply run:
For setting properties on an artifact:
jfrog rt sp --ssh-key-path=/home/${USER}/.ssh/id_rsa --url=${JFROG_URL} ...same command as above for sp command
or for searching
jfrog rt s --ssh-key-path=/home/${USER}/.ssh/id_rsa --url=${JFROG_URL} ...same command as above for s searching artifacts (see JFROG cli help on their documentation site).
JFROG cli does NOT support 'AND' and 'OR' within --props="...." section for a given property i.e. if you want to search for an artifact where a property prop1 contains more than one value, then jfrog cli as shown above which uses --props="....." won't work.
For that, you have the use JFrog file search schema and use --spec ~/somefile.json option or curl command with -T file.json.
NOTE: The json file (syntax) for opening { and } is different when you use jfrog (with --spec file.json) and when you use curl with -T file.json
Let's see how to use with curl command first.
Create a curl-aql.json file which contains the following (as an example):
items.find(
{ "repo": { "$eq" : "libs-snapshot-local" }},
{ "@yourCustomProperty1": { "$match" : "someValue1" }},
{ "@yourCustomProp2": { "$match" : "anotherValue2" }},
{ "@moreCustomProps3": { "$match" : "string_or_number_or_alnum" }},
{ "$or":[ { "@customPropertyWhichHasMultipleValues" : "PASS", "@customPropertyWhichHasMultipleValues" : "UNSTABLE" } ]}
).include("*", "@customProperty5iWantAQLToReturnToMeBack", "@yourCustomProperty1","@yourCustomXProp","@yourCustomYProp")
NOTE: All properties which you create (i.e. which Artifactory doesn't give you for free like 'name', 'repo') MUST be prefixed with an @ character in AQL for you'll find a weird error mesg about something wrong with format at character # 74 or 75.
Then run this curl command (user id is: 123456 and password is contained in $p):
curl -u123456:$p -H 'content-type: text/plain' -X POST http://artifactory-develop:8081/artifactory/api/search/aql -T ~/curl-aql.json
This is less secure but works well (if you define and MASK the password variables in Jenkins and call your script). Still, I'd rather use jfrog --props="..." for simple searches or --spec jsonfile (for more complex searches or other operations what jfrog CLI supports) using MORE secure --ssh-key-path=/your/home/.ssh/your_private_key_id_rsa way using --url=$JFROG_URL (SSH url).
In jfrog if you want to use Artifactory's AQL capabilities, then try this:
Create a file jfrog-aql.json containing:
{
"files": [
{
"aql":
{
"items.find":
{
"repo": "libs-snapshot-local" ,
"@yourCustomProperty1": { "$match" : "ABC1.22.33" },
"@yourCustomProperty2": { "$match" : "Project1" },
"name": { "$match" : "ArtifactX-*.json" },
"@yourCustomProperty3": { "$match" : "2.*" },
"@yourCustomProperty4": { "$match" : "some-manifest-file" }
}
},
"sortBy" : ["created"],
"sortOrder" : "desc",
"limit" : 1
}
]
}
NOTE: this json file has jfrog cli's --spec jsonfile requirements and have a little different syntax than what we used in the json file (during curl run).
Now you can run it using jfrog via:
jfrog rt s --ssh-key-path=/home/${USER}/.ssh/id_rsa --url=${JFROG_URL} --spec ~/jfrog-aql.json
Finally found a better AQL where you can use $and or $or for multi-search on values of a single property (you can add more by tweaking it). See AQL file here:
(NOTE: It's better to use $qa than using $match when the value is known i.e. hard-coded and it's better to use simple form (see how repo is used below. Use or $match makes sense when your value has * in it, see how name property is used below):
{
"files": [
{
"aql":
{
"items.find":
{
"repo": "libs-snapshot-local" ,
"@yourCustomProperty1": "ABC1.22.33",
"@yourCustomProperty2": "Artifact1",
"name": { "$match" : "ArtifactX-*.json" },
"@yourCustomProperty3": { "$match" : "2.*" },
"@yourCustomProperty4": "some-manifest-file",
"$or": [
{
"@customProperty5": "Hard-Coded-Known-Value1",
"@customProperty5": "Hard-Coded-Known-Value2"
}
]
}
},
"sortBy" : ["created"],
"sortOrder" : "desc",
"limit" : 1
}
]
}
NOTE: See related post Chef Berks Install (Berkshelf) Timesout contacting Artifactory repository (Faraday Timeout / Actor Crashed) if you use Chef and berks install (for finding and locking cookbooks versions for stable repetitive deployments, using berks apply (lock-in versions of cookbooks) and using knife create <Chef-Env> -c ... for creating Chef Environments / Files).