What is the difference between square brackets and parentheses in regard to lists?

Question

As the title implies, I am unsure about the difference between square brackets and parentheses in regard to lists.

Two versions of Haskell's insert defined, one using square brackets, the other using parentheses:

insert' :: Int -> [Int] -> [Int]
insert' y [] = [y]
insert' y (x:xs) | y <= x = y:x:xs | otherwise = x : insert' y xs
----
insert' y [x:xs] | y <= x = y:x:xs | otherwise = x : insert' y xs

What is the reason the second definition of insert' doesn't work?

The error message it gives, for anyone wondering:

test.hs:3:12: error:
    • Couldn't match expected type ‘Int’ with actual type ‘[Int]’
    • In the pattern: x : xs
      In the pattern: [x : xs]
      In an equation for ‘insert'’:
          insert' y [x : xs]
            | y <= x = y : x : xs
            | otherwise = x : insert' y xs
  |
3 | insert' y [x:xs] | y <= x = y:x:xs | otherwise = x : insert' y xs
  |  

Answer 1

(x:xs) as a pattern will match any non-empty list with x its head and xs its tail.

[x:xs] as a pattern will match a singleton list -- a list containing exactly one item -- which is a non-empty list, matching the pattern (x:xs). The pattern [x:xs] is in fact equivalent to the pattern [(x:xs)]. It is a nested pattern, and Haskell allows that. The outer pattern matches a singleton list, and the inner pattern matches a non-empty list. The parentheses in this position are optional.

That's why your second definition implies the type of the second argument is [[a]], but you've declared it to be [Int]. And Int can't match [a]. (That a is also determined to be an Int, since you compare x and y, and y the first argument is declared to be an Int, but that doesn't change anything).

     [ [a] ]
     [ Int ]
 ----------------
  ***mismatch***