I installed Chocolatey using the NuGet Console in Visual Studio which reported it was successfully installed in C:\ProgramData and I could use it in cmd or Powershell. I tried 'choco install git' which also completed successfully, it created a folder in C:\ProgramData but it has no binaries, just a config file. Also choco isn't included in the path and neither is git after installation.
Turns out there was an extra character(") in my Path because of which C:\ProgramData\chocolatey\bin was not included.