The documentation for git-rm contains this short description:
git-rm - Remove files from the working tree and from the index
What exactly is meant by the working tree and the index, and which local or remote files will be removed?
The "working tree" is your checkout of the files sitting on disk.
The "index", "staging area", or "cache" (you'll see it referred to as all three) is internal to Git. It's the space you prepare the next commit. When you git add you're copying the files from the working tree to the staging area. When you git commit you're committing what's in the staging area.
git rm removes files from both the working tree and the staging area (unless you tell it to just remove from the staging area with --cached).
This cheat sheet might help you understand the relationship between the working tree, staging area, and HEAD (the currently checked out commit). More importantly, it tells you how to manipulate them because the commands are really not-intuitive.