In a game loop everyone uses System.nanoTime() but for something like animations, the speed of something, etc some people use System.currentTimeMillis() and some use System.nanoTime(). Which one should I be using?
You should be using whichever is appropriate to your needs.
The two examples you gave are fundamentally different:
For animations, you simply need to meet the demands of the screen refresh rate which will be less that 100 times per second. For that a millisecond clock is fine. (Beyond 50 or so refreshes a second, a typical human can't tell the difference.)
For game loops, you could make an argument that "fairness" means you should make the clocks as accurate as possible. (And I'm sure some game players would make that argument.) If you accept that argument, then nanosecond clocks are more appropriate.
Nobody is likely to actually1 tell the difference between using millisecond or nanosecond clocks in games and other applications where the primary goal is smooth human-computer interaction.
In short: there is no "right" way.
1 - They may think that they can tell, but it wouldn't stand up to rigorous scientific testing.