I know of two ways of deleting an app under development from the emulator:
I may have discovered a third way, using 'adb shell':
rm /data/app/<package>.apk
It seems, however, that this isn't really a good way to delete apps because there may be additional information associated with it (registration?).
What is that information and where can it be found?
It's interesting you mention this. I ran a quick home made test to shed some light onto your question.
Generally, when you install a .apk file, Android creates an internal storage area for it located at /data/data/< package name of launching activity>. This is mainly used as an internal caching area that cant be accessed by other apps or the phone user. You can read up about that in a little bit more detail in the Internal storage chapter of Androids data storage section. It is an area exclusively used by your app and you can write private data there.
Once you uninstall an app theoretically, this internal storage area is also deleted. The first 2 ways which you outlined indeed does that: the .apk file in /data/app/ is deleted aswell as the internal storage area in /data/data/.
However if you used adb shell and run the rm command, all that is removed is the .apk file in /data/app/. The internal storage area in in /data/data/ is not deleted. So in essence you are correct that additional information with the app is not necessarily deleted. But on the flip side, if you reinstall the app after running the command, then the existing internal storage area gets overwritten as a fresh copy of it is being installed.