Heroku webpacker with assets on s3 - asset not found in manifesst

I deployed a rails 6 app with webpacker on heroku. The public/packs are sync to an s3 bucket, same as the public/assets. Delivering assets from public/assets works. But when trying to open anything dependent on public/packs I get.

2020-06-04T05:33:03.622234+00:00 app[web.1]: [58f2d221-761b-44ec-807c-6b922342c10b] method=GET path=/teach/organizations format=html controller=OrganizationsController action=index status=500 error='ActionView::Template::Error: Webpacker can't find admin_unify_style.css in /app/public/packs/manifest.json. Possible causes:
2020-06-04T05:33:03.622242+00:00 app[web.1]: 1. You want to set webpacker.yml value of compile to true for your environment
2020-06-04T05:33:03.622242+00:00 app[web.1]: unless you are using the `webpack -w` or the webpack-dev-server.
2020-06-04T05:33:03.622243+00:00 app[web.1]: 2. webpack has not yet re-run to reflect updates.
2020-06-04T05:33:03.622243+00:00 app[web.1]: 3. You have misconfigured Webpacker's config/webpacker.yml file.
2020-06-04T05:33:03.622244+00:00 app[web.1]: 4. Your webpack configuration is not creating a manifest.
2020-06-04T05:33:03.622244+00:00 app[web.1]: Your manifest contains:
2020-06-04T05:33:03.622245+00:00 app[web.1]: {
2020-06-04T05:33:03.622245+00:00 app[web.1]: }

How is heroku expected to find about the manifest on heroku. The manifest contains

"entrypoints": {

    "admin_unify_style": {
        "css": [
            "/packs/css/admin_unify_style-c2ad729a.css"
        ],
        "js": [
            "/packs/js/admin_unify_style-62270c86710162620f91.js"
        ],
        "js.map": [
            "/packs/js/admin_unify_style-62270c86710162620f91.js.map"
        ]

Update 1: It is working in production env outside of heroku and without assets on cloudfront

Solution

Long story short - the manifest should be on heroku, while the assets could be on s3

Here is how I got it working - Heroku, S3, Cloudfront, Webpacker, Rails 6, Sprockets.

Assets are precompiled and uploaded to s3. public/assets and public/packs are ignored from the repo.

public/assets/*
!public/assets/.sprockets-manifest*
public/packs/*
!public/packs/manifest.json*

But as you could see the manifest are not.

Cloudfront is configured for production with

  config.action_controller.asset_host = "https://blabla.cloudfront.net"

public/assets and public/fonts are synced on s3 with a s3cmd command. Custom script.
After the are synced to s3 we are commiting the manifests

if git ls-files -m | grep "public/assets/.sprockets-manifest\|public/packs/manifest.json*"; then
  git add --all public/assets/.sprockets-manifest* -f
  git add --all public/packs/manifest.json* -f
  git commit -m "Autocompiling assets from jenkins"
  git push
fi

In this way we only have the two manifests in the repo with all the assets on S3. 5. Push to heroku and it works. For now, like charm