If an item is removed using the remove endpoint, does that make the access_token not valid anymore? is that effectively the same as destroying the token?
Basically yes.
As stated in Plaid docs :
The /item/remove endpoint allows you to remove an Item. Once removed, the access_token associated with the Item is no longer valid and cannot be used to access any data that was associated with the Item.