Adding public => false in register_post_type()'s arguments would solve the Google problem but then I have no access to the CPT posts in admin. A redirect like this prevents from viewing the pages but they still show up in Google.
Is what I want even possible at all?
Have a look at the documentation (https://codex.wordpress.org/Function_Reference/register_post_type) - it says the following:
public (boolean) (optional) Controls how the type is visible to authors (show_in_nav_menus, show_ui) and readers (exclude_from_search, publicly_queryable).
Default: false
'true' - Implies exclude_from_search: false, publicly_queryable: true, show_in_nav_menus: true, and show_ui:true. The built-in types
attachment, page, and post are similar to this.'false' - Implies exclude_from_search: true, publicly_queryable: false, show_in_nav_menus: false, and show_ui: false. The built-in
types nav_menu_item and revision are similar to this. Best used if
you'll provide your own editing and viewing interfaces (or none at
all).
As you can see public is some kind of meta property. Set public to false but set show_ui to true. This should work.