Search code examples
linuxarraysbashunique

How can I get unique values from an array in Bash?

I've got almost the same question as here.

I have an array which contains aa ab aa ac aa ad, etc. Now I want to select all unique elements from this array. Thought, this would be simple with sort | uniq or with sort -u as they mentioned in that other question, but nothing changed in the array... The code is:

echo `echo "${ids[@]}" | sort | uniq`

What am I doing wrong?

Solution

  • A bit hacky, but this should do it:

    echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '

    To save the sorted unique results back into an array, do Array assignment:

    sorted_unique_ids=($(echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))

    If your shell supports herestrings (bash should), you can spare an echo process by altering it to:

    tr ' ' '\n' <<< "${ids[@]}" | sort -u | tr '\n' ' '

    A note as of Aug 28 2021:

    According to ShellCheck wiki 2207 a read -a pipe should be used to avoid splitting. Thus, in bash the command would be:

    IFS=" " read -r -a ids <<< "$(echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')"

    or

    IFS=" " read -r -a ids <<< "$(tr ' ' '\n' <<< "${ids[@]}" | sort -u | tr '\n' ' ')"

    Input:

    ids=(aa ab aa ac aa ad)

    Output:

    aa ab ac ad

    Explanation:

    • "${ids[@]}" - Syntax for working with shell arrays, whether used as part of echo or a herestring. The @ part means "all elements in the array"
    • tr ' ' '\n' - Convert all spaces to newlines. Because your array is seen by shell as elements on a single line, separated by spaces; and because sort expects input to be on separate lines.
    • sort -u - sort and retain only unique elements
    • tr '\n' ' ' - convert the newlines we added in earlier back to spaces.
    • $(...) - Command Substitution
    • Aside: tr ' ' '\n' <<< "${ids[@]}" is a more efficient way of doing: echo "${ids[@]}" | tr ' ' '\n'