What's currently the best way to access an on-premise TFS installation via C#/.NET? Is the SDK still the best option to choose or should all new applications use the TFS API?
New applications should use the TFS REST API when possible. There are .NET client libraries you can use to access the both the old API and the new API. From the documentation, note that "REST-based clients only work with Visual Studio Team Services and TFS 2015 (not previous versions of TFS)."