The question is a bit wordy, but it is exactly what I'm looking to do. The question is relative to an existing (currently open) question I have here. I believe that understanding this concept is key to answering my existing question, but distinct enough to warrant a whole new one.
given these keys in an s3 bucket named "my-permtest":
/1/
/1/a
/1/2/b
/1/3/c
How can I use prefix and delimiter correctly to get the objects that don't end in "/", (IE: "files").
The ultimate goal is to apply this knowledge to an IAM group policy granting ListBucket and getObject on /1/a while denying getObject or ListBucket to /1/2/, 1/2/*, 1/3 and 1/3/*
I'm effectively looking to mimic traditional file system permissions that let a user access all "files" in a "folder" but restrict access to "subfolders".
Currently, I'm using s3api calls and different values of prefix and delimiter options to get a feel for usage of these things. I've been reading and practicing with these resources, but going is slow and assist would be greatly appreciated.
This is actually pretty straightforward once it's understood that folders don't exist, per se. s3 objects are key/value pairs. the key is the "filename" which represents a full path containing "/" characters. the value for a key is the actual file content. so an s3 object with the key s3://my-bucket/1/2/a is NOT a file named a in a subfolder of 1 named 2. It's an object with a key that looks like a full path.
Knowing this, and applying the understanding that the * wildcard can be used to match key names in an apply or deny statement when writing a policy effectively answers the question. Additionally, it's important to include tightly scoped object or bucket allow/deny statements for specific actions.
So, basically to allow get/put access to the /1/2 "folder" but not the 1/2/a "file", you would need to allow list actions on s3://bucket/1/2 and allow object get/put actions on s3://bucket/1/2/*. Note that it's key to understand the importance of separating out your s3 object actions and s3 bucket list actions in to distinct statements within your policy.
If you wanted to deny access to s3://bucket/1/2/3/*, you would add 2 statements to the same policy. The first denying list access to s3://bucekt/1/2/3, the second denying get/put object access to s3://bucket/1/2/3*.
Now, should you want to allow some people access to s3://bucket/1/2/3/a you would be in a bind if you tried to use this policy, because s3://1/2/3/* has been explicitly denied. Any policy granting access will be ignored because of that explicit deny. The only option would be to have two policies that are nearly identical. One containing the original allow to s3://bucket/1/2/3/* and other containing the original plus the deny for list access to s3://bucket/1/2/3/ and object get/put access for s3://bucket/1/2/3/*. The folks with no access to s3://bucket/1/2/3/* would be in the group with the explicit deny, and the folks with access there would be in the first group that just has the allow.
This quickly gets out of control when there are many "subfolders" with different groups having access to each. Updating a policy every time a new nested "subfolder" is created is an untenable paradigm. For this reason, when using an IAM group policy based approach to securing s3 resources, you should take care to organize the data in your buckets in such a way that you don't have to do this kind of maintenance.
See my related answer here for detail on what I mean, but basically avoid creating subfolders with arbitrary restrictions of who can/can't access them. You're going to have a hard time saying "joe can access 1/3/5 and 1/3/7 but not 1/2/4 or 1/2/6. It's a lot easier to move /1/ /3/ and /5/ under /odd/ and move /2/ /4/ and /6/ under /even/ then just grant allow him to access /odd/*. you don't even have to specify deny to /even/ because it's implicit.