In the manpage for bash, under the "Compound Commands" section, there are the following two entries:
(list) list is executed in a subshell environment (see COMMAND EXECUTION ENVIRONMENT below). Variable assignments and builtin commands that affect the shell's environment do not remain in effect after the command completes. The return status is the exit status of list.
( expression ) Returns the value of expression. This may be used to override the normal precedence of operators.
The only difference I can see is that in one, the parentheses have no spaces next to them, and in the other, they do. Is that what actually differentiates grouping vs a subshell, or is it dependent on the context?
In other words, if I run
if ! [ 2 -eq 2 ] || ( [ 2 -eq 2 ] && [ 4 -eq 4 ] ); then echo "hello"; fi
is this just grouping the conditions or running in a subshell?
The context in question for those entries is relevant.
The latter is in the documentation for the [[ construct and is documenting the behavior of that construct on its arguments.
The former is discussing a top-level shell compound command construct (like the [[ construct itself) and introduces a sub-shell.
This comes up again in the description of the test/[ command later in the man page (but that is essentially just the same as the [[ discussion).
To do grouping within the current shell, you can use curly braces:
if ! [ 2 -eq 2 ] || { [ 2 -eq 2 ] && [ 4 -eq 4 ]; }; then
...
fi
(Note the spaces around the inside of the braces and the extra semicolon, which are both necessary.)