I am trying to understand how I can implement security using a restful client for an authenticated user. The scenario I am having trouble is how to stop a user from updating a purchase that's not his own since the restful client passes a purchase id back. The product id can be easily tampered with for a skilled user. I am not really interested in being caught in the middle since using https prevents that or mitigates it. I am really interested in a user trying to update what's not theirs.
How do you prevent such attacks in the rest world? Web Parameter Tampering
The problem you are facing is called authorization. Once the user is authenticated it is authorization that grants him permissions to access particular resources. Authorization in a REST scenario should be implemented on the server side.
Let's say user Bob tries to modify a purchase resource by sending an authenticated (e.g. using Basic HTTP Auth, authorized session cookie, etc.) POST request to /purchases/1 endpoint (supplied with appropriate payload). It is the server's duty to validate if Bob is allowed to modify the entity (e.g. by checking if it was really Bob who made the purchase). If the permission is granted the server proceeds with the operation and responds with 2xx success HTTP status code. Otherwise 403 error code will be returned informing that the user is not authorized to modify the given purchase.
Once the authorization mechanisms are established another problem arises: users posting malicious input to try to trick and overcome the authorization mechanisms. This touches a very broad topic of web application security. There are a lot of known attacks on webapps (e.g. injection attacks) and even more ways of protecting against them. Testing applications for security vulnerabilities is called penetration testing. It's worth mentioning that there are automated tools that perform such tests (and also automated tools for performing such attacks).
In general you've touched a very broad topic and there's no way for an SO answer to explain a millionth of it. Treat this answer as a starting point for your own investigation in the area.
API is stateless when no application state is stored on the server (as opposed to resource state). Click here for a nice explanation of the difference between these two. There's a lot of discussions out there about statelessness (particularly in context of authentication) - look up SO or Google.
In a nutshell stateless authentication in REST is very important given today's large distributed systems. The server-side application state in such environments might cause scalability problems when it comes to sharing it across many nodes in clustered environments. This is why it's advised to make the application state completely client side. I know it might be confusing for you at first, especially after what you've read in my answer that server should authorize user actions. Here is an example of a stateless authentication implementation (digitally signed self-contained session token).
But fear not - in practice most of the systems out there store at least some part of the application state on the server (AFAIK Google does this in their systems). So like stated in this answer:
"REST is not a religion (...), you should only follow the tenets of REST as far as they make sense for your application"