download a directory from a nexus repo

Question

I'm trying to download a directory that exists in a nexus repo to my local computer. I've tried using the nexus API, but I can't seem to copy the folder to my local computer and have it keep the same folder hierarchy as it appears on the nexus repo..

I need it to have on my PC the same folder strucutre of the folder in the nexus repo.

any help would be highly appreciated :)

I tried using the GUI, but size downloads are limited, and the folder I want to download is too big.

I also tried making a bash script that uses the nexus API to download files, but it doesn't keep the same folder structure as it appears on nexus, it just recursively downloads all files in a the nexus directory and puts them in one big local directory.

Answer 1

I wrote this simple script, which you can use to download the whole repository or just the one specific directory you want.
Firstly, it send request to the nexus API, and get all assets from the repository. Iterates through them and download the asset\file if it matches your criteria

#!/bin/bash

USER=$1
PASSWORD=$2

REPO_URL="https://repo.test.com.ua"
REPO_NAME="repository_name"
OUTPUT_DIR="downloaded_from_nexus/" # '/' at the end is mandatory
REPO="$REPO_URL/repository/$REPO_NAME/" # '/' at the end is mandatory

DIRECTORY_TO_DOWNLOAD=$3
DOWNLOAD_ALL=1

if [[ -z "${USER}" ]] || [[ -z "${PASSWORD}" ]]; then
    echo "Username or password were not passed"
    exit 1
fi

if [[ -z "${DIRECTORY_TO_DOWNLOAD}" ]]; then
    DOWNLOAD_ALL=1
    echo "Downloading the whole repo"
else
    DOWNLOAD_ALL=0
    echo "Downloading only ${REPO}${DIRECTORY_TO_DOWNLOAD}"
fi

CONTINUATION_TOKEN=""
URLS_LIST=()

function download_assets() {
    local TOKEN="$1"
    local API_URL="${REPO_URL}/service/rest/v1/search/assets?repository=${REPO_NAME}"

    if [[ ! -z "$TOKEN" ]] && [[ ! "$TOKEN" == "null" ]]; then
        API_URL="${API_URL}&continuationToken=${TOKEN}"
    fi

    echo "Sending request to ${API_URL}" >&2
    local assets;
    assets=$(curl -s -f --user ${USER}:${PASSWORD} -X GET "${API_URL}")
    res=$?
    if [[ "$res" -ne 0 ]]; then
        echo "Api request failure. Making another one with verbose:" >&2
        curl -v --user ${USER}:${PASSWORD} -X GET "${API_URL}" >&2
        exit $res
    fi

    while read item; do
        url=$(echo "$item" | jq -r '.downloadUrl')
        filename=$(basename $url)
        structure=${url%$filename}
        structure=${structure##*$REPO}

        if [[ $DOWNLOAD_ALL -eq 0 ]] && [[ "${url}" != "${REPO}${DIRECTORY_TO_DOWNLOAD}"* ]]; then
            # If that's not the directory we want - skip it
            continue
        fi

        echo "Downloading ${OUTPUT_DIR}${structure}${filename}" >&2
        mkdir -p "${OUTPUT_DIR}$structure"
        wget --user ${USER} --password ${PASSWORD} ${url} -O "${OUTPUT_DIR}${structure}${filename}"
    done <<< "$(echo "$assets" | jq -c '.items[]')"

    # Extract continuation token from the response
    TOKEN=$(echo "$assets" | jq -r '.continuationToken')

    echo "$TOKEN"
}

while true; do
    TOKEN=$(download_assets "${CONTINUATION_TOKEN}")

    # Break the loop if there's no continuation token
    if [[ "$TOKEN" == "null" ]]; then
        break
    fi
    CONTINUATION_TOKEN=${TOKEN}
done

Save it for example, with the name nexus_downloader.sh. After that change REPO_URL and REPO_NAME variables in the script corresponding with your needs.
And finally you can use the script like this:
./nexus_downloader.sh username_from_nexus password_from_nexus directory_to_download
First two params are mandatory, they are used to access to your nexus repository. The last parameter is optional. If it was passed - the script will download all files only from the directory you provided. If it wasn't passed - the script will download the whole repository
By default it downloads all data into separate directory OUTPUT_DIR

Script uses dependencies

1. curl
2. wget
3. jq

I rewrote it - now it does not require any intermediate files to store links. It just downloads what you need