User with assigned policy can't access secrets in KV version 2

Question

I have created a kv (version 2) secrets engine, mounted on /secret:

$ vault secrets list
Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_915b3383    per-token private secret storage
identity/     identity     identity_9736df92     identity store
secret/       kv           kv_8ba16621           n/a
sys/          system       system_357a0e34       system endpoints used for control, policy and debugging

I have created a policy that should give admin access to everything in myproject:

$ vault policy read myproject
path "secret/myproject/*" {
  capabilities = ["create","read","update","delete","list"]
}

I have created a secret in the appropriate path (with root token):

$ vault kv put secret/myproject/entry1 pass=pass
Key              Value
---              -----
created_time     2022-05-11T15:06:49.658185443Z
deletion_time    n/a
destroyed        false
version          1

I have created a user that has been assigned the given policy:

$ vault token lookup
Key                 Value
---                 -----
accessor            CBnMF4i2cgadYoMNAX1YHaX6
creation_time       1652281774
creation_ttl        168h
display_name        userpass-myproject
entity_id           ad07640c-9440-c4a1-b668-ab0b8d07fe93
expire_time         2022-05-18T15:09:34.799969629Z
explicit_max_ttl    0s
id                  s.FO7PrOBdvC3KB85N46E05msi
issue_time          2022-05-11T15:09:34.799982017Z
meta                map[username:myproject]
num_uses            0
orphan              true
path                auth/userpass/login/myproject
policies            [default myproject]
renewable           true
ttl                 167h53m36s
type                service

However when I try to access anything (list,get), I get a 403 error:

$ vault kv list secret/myproject
Error listing secret/metadata/myproject: Error making API request.

URL: GET https://example.vault/v1/secret/metadata/myproject?list=true
Code: 403. Errors:

* 1 error occurred:
    * permission denied

$ vault kv get secret/myproject/entry1
Error reading secret/data/myproject/entry1: Error making API request.

URL: GET https://vault.private.gsd.sparkers.io/v1/secret/data/myproject/entry1
Code: 403. Errors:

* 1 error occurred:
    * permission denied

When I change the policy to this (change path to secret/*), I get access to everything:

$ vault policy read myproject
path "secret/*" {
  capabilities = ["create","read","update","delete","list"]
}
$ vault kv get secret/myproject/entry1
====== Metadata ======
Key              Value
---              -----
created_time     2022-05-11T15:06:49.658185443Z
deletion_time    n/a
destroyed        false
version          1

==== Data ====
Key     Value
---     -----
pass    pass

What am I doing wrong?

Answer 1

It turns out that you need to define your policy like this:

path "secret/metadata/myproject/*" {
 capabilities = ["list"]
}

path "secret/data/myproject/*" {
 capabilities = ["create","read","update","delete"]
}

because with engine v2 kv list prepends metadata to your path, and kv get prepends data to your path.

No idea how I missed the documentation on this here: https://www.vaultproject.io/docs/secrets/kv/kv-v2:

Writing and reading versions are prefixed with the data/ path.

Thank you @Matt Schuchard