Will this anti cheat technique work for a multiplayer game using private servers (publicly unknown executable):
When the client starts the game it will auto update itself daily (using a launcher). The Servers will update themselves, too. Unless there is a real patch, the update only consists of changes in the gameobjects memory layout, netcode, and shaders. This is done using an automated system that auto generates and randomizes (C++) classes. Maybe it could also add fake objects to the hierarchy to make identifiyng objects harder.
This way I hope to update the game faster than a cracker can and will reverse engineer, update and publish/update a new cheat.
Will this work or can hackers somehow work around this mechanism? Will they do this work daily or can they automate it at some point? What can I do to improve this system?
It seems randomizing memory layout does not help in the long term because the layout can more or less easily be extracted by following function calls in the executable and extracting pointer offsets from that code. So to efficiently prevent this, the structure of calls and the code itself needs to be randomized also.
Are there good ways to do that? Is that working at all against automated cracking?
Client-side technological arms race is a completely wrong way to do this. You will never be better or faster than a bunch of kids that have too much time to spare. You cannot compete with a horde of attackers that have no costs (other than not doing their homework) while your actions cost you both time and money. This is a race you will lose, both on the way financially and in the end result as well.
There is two ways people can cheat:
There are exactly two ways you can keep people from doing this:
Remember the old saying: "The client is in the hands of the enemy."