If I put up my signed apk file onto my google drive and download it directly to my android phone I can install and run it.
Can I do something similar with an iPhone? Or must I get the .ipa installed on TestFlight or something similar (HockeyApp, FlightFairy etc.) first?
Yes, you can do it via ad hoc distribution or in house distribution.
By distributing your app via ad hoc distribution you need to add all the devices that on which can be installed the app to the provisioning profile. Here you can use different devices like crashlytics, hockeyApp, diawi, etc.
If you distribute your app via in house distribution you will need to have an enterprise account but you can install the app on any device (is its important to mention that the device users needs to trust your developer profile in order to run the app).