My intuition and assumption is whenever we can't use greedy, then A* will be the way to go, but I am not 100% sure. I need some more examples and patterns on how to recognize and spot A* algorithm.
Can someone give some special extreme cases that when you first see it and you know this wouldn't be greedy or it has to be A* without even bother trying.
Normally the term greedy is used to describe algorithms that do not backtrack. These are algorithms that make a choice by maximizing a heuristic (typically a fairly "local" one), and then do not ever revisit that choice. Think of a greedy algorithm as someone who picks a cake and then eats it right away, instead of putting it to one side and investigating whether another cake might be better.
By contrast, A* is a backtracking algorithm: it explores choices, possibly in some depth, but then is capable of abandoning these choices later and trying other possibilities.
So if your problem has "dead ends" (local maxima) where no further progress towards a solution is possible, then greedy algorithms are most likely not suitable. But that doesn't necessarily mean that A* and its variants are your only alternatives. There are many other types of search algorithm that use different techniques to escape from dead ends: simulated annealing, Monte Carlo tree search, tabu search, particle swarm optimization, ...