In my application, I have a std::list<TreeNode> that is the owner of my tree nodes. Now I want to be able to access the left and right neighbor inside a TreeNode, so which smart pointer should I use to store the neighbors inside a TreeNode? The only pointer suitable for this is a raw pointer because a TreeNode is not the owner of its neighbors, so I hardly see how a smart pointer could be useful here.
Also, the TreeNodes have to be store in other containers for fast access (to have the highest-priority one), so I can't just store unique_ptr of neighbors, because I also need to store them in fast containers (priority_queue)
so which (smart) pointer should I use to store the neighbors inside a TreeNode?
You should use a suitable pointer type.
The only pointer suitable for this is a raw pointer
Well, there you go. If the raw pointer is the only suitable one, then using an unsuitable smart pointer would be counter-productive.
If you have trouble establishing correct lifetime for the nodes, then shared ownership is something that may be considered. An easy choice is to use shared pointers throughout. If you instead want to keep the ownership primarily in a list, then you could use weak pointers elsewhere.
But I want to use smart pointers because they are the way to go in modern C++
You've misunderstood the point of smart pointers.