[Introduction] I have a custom made Python game which uses 'w' 's' keys for moving and 'space' key for shooting as inputs. I've found a reinforcement learning algorithm which I would like to try implement into the game.
However, the RL algorithm uses openAI's atari games as the environment with command 'gym.make (env_name)'. I'm on Windows OS so can't experiment on the code as gym[atari] does not work for me.
class Agent:
def __init__(self, env_name, training, render=False, use_logging=True):
self.env = gym.make(env_name)
[Question] Is there another command I could use instead of 'gym.make()' in this class to implement the RL algorithm to train on my custom made game, or is the only option to make my own gym environment? Would 'pygame.surfarray.array2d()' return something similar to 'gym.make()'?
Please tell me if additional information is required, I'm new to gym and tensorflow so my understanding may be flawed.
[Edit] I made my game using functions, if I was to convert the game into a gym environment, would the only option be to convert the functions to classes? As an example of how my code looks like, here is the game loop: (I can't post the whole code as it's a controlled assessment for end of year grade so would like to avoid any plagiarisation problems)
def game_loop():
global pause
x = (display_width * 0.08)
y = (display_height * 0.2)
x_change = 0
y_change = 0
blob_speed = 2
velocity = [2, 2]
score = 0
lives = 3
pos_x = display_width/1.2
pos_y = display_height/1.2
previous_time = pygame.time.get_ticks()
previous_time2 = pygame.time.get_ticks()
gameExit = False
while not gameExit:
for event in pygame.event.get():#monitors hardware movement/ clicks
if event.type == pygame.QUIT:
pygame.quit()
quit()
pos_x += velocity[0]
pos_y += velocity[1]
if pos_x + blob_width > display_width or pos_x < 601:
velocity[0] = -velocity[0]
if pos_y + blob_height > display_height or pos_y < 0:
velocity[1] = -velocity[1]
for b in range(len(bullets2)):
bullets2[b][0] -= 6
for bullet in bullets2:
if bullet[0] < 0:
bullets2.remove(bullet)
current_time2 = pygame.time.get_ticks()
#ready to fire when 500 ms have passed.
if current_time2 - previous_time2 > 500:
previous_time2 = current_time2
bullets2.append([pos_x+25, pos_y+24])
keys = pygame.key.get_pressed()
for b in range(len(bullets)):
bullets[b][0] += 6
for bullet in bullets:
if bullet[0] > 1005:
bullets.remove(bullet)
if keys[pygame.K_SPACE]:
current_time = pygame.time.get_ticks()
#ready to fire when 500 ms have passed.
if current_time - previous_time > 600:
previous_time = current_time
bullets.append([x+25, y+24])
if x < 0:
x = 0
if keys[pygame.K_a]:
x_change = -blob_speed
if x > 401 - blob_width:
x = 401 - blob_width
if keys[pygame.K_d]:
x_change = blob_speed
if keys[pygame.K_p]:
pause = True
paused()
if keys[pygame.K_a] and keys[pygame.K_d]:
x_change = 0
if not keys[pygame.K_a] and not keys[pygame.K_d]:
x_change = 0
if y < 0:
y = 0
if keys[pygame.K_w]:
y_change = -blob_speed
if y > display_height - blob_height:
y = display_height - blob_height
if keys[pygame.K_s]:
y_change = blob_speed
if keys[pygame.K_w] and keys[pygame.K_s]:
y_change = 0
if not keys[pygame.K_w] and not keys[pygame.K_s]:
y_change = 0
#print(event)
# Reset x and y to new position
x += x_change
y += y_change
gameDisplay.fill(blue) #changes background surface
bullets_hit(score)
player_lives(lives)
pygame.draw.line(gameDisplay, black, (601, display_height), (601, 0), 3)
pygame.draw.line(gameDisplay, black, (401, display_height), (401, 0), 3)
blob(pos_x, pos_y)
blob(x, y)
for bullet in bullets:
gameDisplay.blit(bulletpicture, pygame.Rect(bullet[0], bullet[1], 0, 0))
if bullet[0] > pos_x and bullet[0] < pos_x + blob_width:
if bullet[1] > pos_y and bullet[1] < pos_y + blob_height or bullet[1] + bullet_height > pos_y and bullet[1] + bullet_height < pos_y + blob_height:
bullets.remove(bullet)
score+=1
for bullet in bullets2:
gameDisplay.blit(bulletpicture, pygame.Rect(bullet[0], bullet[1], 0, 0))
if bullet[0] + bullet_width < x + blob_width and bullet[0] > x:
if bullet[1] > y and bullet[1] < y + blob_height or bullet[1] + bullet_height > y and bullet[1] + bullet_height < y + blob_height:
bullets2.remove(bullet)
lives-=1
if lives == 0:
game_over()
pygame.display.update() #update screen
clock.tick(120)#moves frame on (fps in parameters)
The best option would indeed by to simply implement your own custom environment. You can find some instructions on implementing custom environments in the gym repository on github.
Some of those instructions are probably only relevant if you also intend to share your environment with others, not so much if you just want to use it yourself. I suspect the most important parts for you (under the assumption you just want to it yourself and not upload as a package that others can use) are (copied from link above):
gym-foo/gym_foo/envs/foo_env.py should look something like:
import gym
from gym import error, spaces, utils
from gym.utils import seeding
class FooEnv(gym.Env):
metadata = {'render.modes': ['human']}
def __init__(self):
...
def step(self, action):
...
def reset(self):
...
def render(self, mode='human', close=False):
...
gym-foo/gym_foo/__init__.py should have:
from gym.envs.registration import register
register(
id='foo-v0',
entry_point='gym_foo.envs:FooEnv',
)
register(
id='foo-extrahard-v0',
entry_point='gym_foo.envs:FooExtraHardEnv',
)
gym-foo/gym_foo/envs/__init__.py should have:
from gym_foo.envs.foo_env import FooEnv
from gym_foo.envs.foo_extrahard_env import FooExtraHardEnv
The first block is the implementation of the environment itself. You hopefully shouldn't have to implement a whole lot in there, if you already have a game implemented. This subclass of gym.Env should just be a "wrapper" around the already-existing game, forming the bridge between RL agents that expect the gym API (step(), reset(), etc.) and the game itself. You could take inspiration from the atari_env implementation in gym, which is itself also only a wrapper around already-existing Atari games, and does not directly contain the complete game logic of these games.
The second and third blocks are required to make sure that you can start creating instances of your custom environment using the gym.make() function.
You will indeed have to make a class that has the gym.Env class as base class, and make sure you implement all of its important functions (like step and reset). That is, assuming you want to use an RL algorithm that has already been implemented and expects those functions to exist. Of course, an alternative would be to throw gym out of the window entirely and implement everything from scratch, but you'd very likely end up just doing more work and ending up with a similar API anyway.