I'm Trying to expand my understanding about symbols in Haskell :
$ : Function Application operator (Allow you to apply arguments over a function)
& : flipped version of Function Application Operator? (&) = flip ($)
<> : associative operator (You'll find it in Semigroups and Monoids)
<$> : function application ($) lifted over a Functor structure
<&> : flipped functor map
<*> : applicative operator
Can we make a link between <> and this family <*>,<$>,<&>? I made a quick conclusion when only looking at <*>,<$>,<&> that <..> was related to something over a structure, but then what is the link between structure and associative operator ?
Those names didn't come out of some overarching conceptual scheme. The best way to see that is by tracing their histories:
McBride and Paterson's Applicative programming with effects uses an asterisk in a circle, ⊛, as the binary operator of Applicative (note that there are theoretical reasons to pick a *-like symbol that suggests a product). When Control.Applicative made it to base (that is, in base-2.1/GHC 6.6/October 2006), that became <*>, which is, as far as I can see, the closest ASCII approximation to that.
The first version of Control.Applicative already featured <$>, and the final version of Applicative programming with effects I linked to above also mentions it (with the minor difference that the <$> there has an Applicative constraint). The point of picking a mashup of $ and <*> as the fmap operator was presumably allowing us to write nice-looking applicative style expressions (f <$> u <*> v <*> w) that could be acceptable substitutes for the idiom brackets mentioned in that paper (which, rendered in ASCII, look like [| f u v w |]).
The Monoid class came into being even earlier in the history of base (it already existed as of GHC 5.04.2, in a Control.Monad.Monoid module); however, there wasn't an infix version of mappend in base until version 4.5 (GHC 7.4, early 2012). Applicative programming with effects also mentions monoids, and suggests a circled plus, ⊕, as a binary operator for mappend. As far as I can tell, the <> name was first suggested by Ross Paterson in a Libraries mailing list thread from 2009, and made its way into a preexisting GHC proposal, and presumably also to Edward Kmett's semigroups package, whose Data.Semigroup module was eventually adopted by base. Paterson chose <> for it being a neutral name, which wouldn't suggest any specific monoid (see also: Why is the mappend infix alias <> instead of +?).