I have used Rider in the past on Unity projects, there was a button to start Unity Play mode from the IDE, and attach the debugger. I am using 2019.2 version of Unity and latest of Rider right now.
I should have some icons like in the documentation here https://www.jetbrains.com/help/rider/Unity.html#.
Nothing show up, and the best I get, is Rider proposing to add a custom configuration... Rider is correctly set as the IDE for Unity in Unity preferences. I do not understand what is going on.
Installed Rider again, while checking every delete checkbox (cache and settings and more), did not re-use the config from previous version (see https://rider-support.jetbrains.com/hc/en-us/community/posts/360005280960-Cannot-attach-Unity-debugger), and also made sure I have Unity support checked. The button is back.