One of the actions I want the agent to do needs to have a delay between every action. For context, in pygame I have the following code for shooting a bullet:
if keys[pygame.K_SPACE]:
current_time = pygame.time.get_ticks()
# ready to fire when 600 ms have passed.
if current_time - previous_time > 600:
previous_time = current_time
bullets.append([x + 25, y + 24])
I've set a timer to prevent bullet spamming, how would I construct this to work with the step() method? My other actions are moving up, down, left, right.
This is my first time creating a project with OpenAI-gym so I'm not sure what the capabilities of the toolkit are, any help would be greatly appreciated.
You can use whatever method of tracking time you like (other than pygame.time.get_ticks() I suppose), and use a similar approach as in that pygame code. You'd want to store previous_time as a member of the environment instead of just a local variable, because you want it to persist across function calls.
It's not easy to actually prevent your Reinforcement Learning agent (assuming you're using gym for RL) from selecting the fire action altogether, but you can simply implement the step() function in such a way that the agent does not do anything at all if they select the fire action too quickly.
As for measuring time, you could measure wall clock time, but then the power of your CPU is going to influence how often your agent is allowed to shoot (it might be able to shoot a new bullet every step on very old hardware, but only be allowed to shoot one bullet every 100 steps on powerful hardware), that's probably a bad idea. Instead, I'd recommend measuring "time" simply by counting the step() calls. For example, using only the code from your question above, the step() function could look like:
def step(action):
self.step_counter += 1
# other step() code here
if action == FIRE:
if self.step_counter - self.previous_time > 10: # or any other number
self.previous_time = self.step_counter
bullets.append([x + 25, y + 24])
# other step() code here
Don't forget that you'll also want to reset your newly added member variables in reset():
def reset():
self.step_counter = 0
self.previous_time = -100 # some negative number such that your agent can fire at start
# other reset() code here