Python – OpenAI/Tensorflow Customize the game environment instead of using ‘gym.make()’

[Introduction]
I have a custom Python game that uses the ‘w’ ‘s’ keys to move and the ‘space’ key to shoot as input. I found a reinforcement learning algorithm and I want to try to apply it to games.

However, the

RL algorithm uses openAI’s atari game as the environment, with the command “gym.make (env_name)”. I’m using a Windows operating system, so I can’t experiment with the code because gym[atari] doesn’t work for me.

class Agent:
    def __init__(self, env_name, training, render=False, use_logging=True):

self.env = gym.make(env_name)

[Question]
Is there another command in place of “gym.make()” in this class to implement the RL algorithm to train my custom game, or is it the only option for making my own gym environment?
Does “pygame.surfarray.array2d()” return something similar to “gym.make()”?

Please let me know if you need additional information, I’m new to gym and tensorflow, so my understanding may be flawed.

[edit].
I make a game using functions, if I want to convert the game to a gym environment, is the only option to convert the function to a class? As an example of my code, here is the game loop: (I can’t post the whole code because it’s a controlled assessment of year-end grades, so want to avoid any plagiarism issues).

def game_loop():
global pause
x = (display_width * 0.08)
y = (display_height * 0.2)

x_change = 0
y_change = 0

blob_speed = 2

velocity = [2, 2]

score = 0
lives = 3

pos_x = display_width/1.2
pos_y = display_height/1.2

previous_time = pygame.time.get_ticks()
previous_time2 = pygame.time.get_ticks()

gameExit = False
while not gameExit:
    for event in pygame.event.get():#monitors hardware movement/ clicks
        if event.type == pygame. QUIT:
            pygame.quit()
            quit()

pos_x += velocity[0]
    pos_y += velocity[1]

if pos_x + blob_width > display_width or pos_x < 601:
        velocity[0] = -velocity[0]

if pos_y + blob_height > display_height or pos_y < 0:
        velocity[1] = -velocity[1]

for b in range(len(bullets2)):
        bullets2[b][0] -= 6

for bullet in bullets2:
        if bullet[0] < 0:
            bullets2.remove(bullet)

current_time2 = pygame.time.get_ticks()
    #ready to fire when 500 ms have passed.
    if current_time2 - previous_time2 > 500:
        previous_time2 = current_time2
        bullets2.append([pos_x+25, pos_y+24])

keys = pygame.key.get_pressed()

for b in range(len(bullets)):
        bullets[b][0] += 6

for bullet in bullets:
        if bullet[0] > 1005:
            bullets.remove(bullet)

if keys[pygame. K_SPACE]:
        current_time = pygame.time.get_ticks()
        #ready to fire when 500 ms have passed.
        if current_time - previous_time > 600:
            previous_time = current_time
            bullets.append([x+25, y+24])

if x < 0:
        x = 0
    if keys[pygame. K_a]:
        x_change = -blob_speed
    if x > 401 - blob_width:
        x = 401 - blob_width
    if keys[pygame. K_d]:
        x_change = blob_speed
    if keys[pygame. K_p]:
        pause = True
        paused()

if keys[pygame. K_a] and keys[pygame. K_d]:
        x_change = 0
    if not keys[pygame. K_a] and not keys[pygame. K_d]:
        x_change = 0

if y < 0:
        y = 0
    if keys[pygame. K_w]:
        y_change = -blob_speed
    if y > display_height - blob_height:
        y = display_height - blob_height
    if keys[pygame. K_s]:
        y_change = blob_speed

if keys[pygame. K_w] and keys[pygame. K_s]:
        y_change = 0
    if not keys[pygame. K_w] and not keys[pygame. K_s]:
        y_change = 0

#print(event)
    # Reset x and y to new position
    x += x_change
    y += y_change

gameDisplay.fill(blue)  #changes background surface
    bullets_hit(score)
    player_lives(lives)
    pygame.draw.line(gameDisplay, black, (601, display_height), (601, 0), 3)
    pygame.draw.line(gameDisplay, black, (401, display_height), (401, 0), 3)
    blob(pos_x, pos_y)
    blob(x, y)

for bullet in bullets:
        gameDisplay.blit(bulletpicture, pygame. Rect(bullet[0], bullet[1], 0, 0))
        if bullet[0] > pos_x and bullet[0] < pos_x + blob_width:
            if bullet[1] > pos_y and bullet[1] < pos_y + blob_height or bullet[1] + bullet_height > pos_y and bullet[1] + bullet_height < pos_y + blob_height:
                bullets.remove(bullet)
                score+=1

for bullet in bullets2:
        gameDisplay.blit(bulletpicture, pygame. Rect(bullet[0], bullet[1], 0, 0))
        if bullet[0] + bullet_width < x + blob_width and bullet[0] > x:
            if bullet[1] > y and bullet[1] < y + blob_height or bullet[1] + bullet_height > y and bullet[1] + bullet_height < y + blob_height:
                bullets2.remove(bullet)
                lives-=1

if lives == 0:
        game_over()

pygame.display.update() #update screen
    clock.tick(120)#moves frame on (fps in parameters)

The best option is indeed to simply implement your own custom environment. You can find some instructions for implementing a custom environment in gym repository on GitHub.

Some of these instructions may only be relevant if you also intend to share your environment with others, not so much if you just want to use it yourself. I suspect the most important part for you (assuming you just want to do it yourself and not upload it as a package that others can use) is (copied from the link above):

gym-foo/gym_foo/envs/foo_env.py should look similar to:

import gym
from gym import error, spaces, utils
from gym.utils import seeding

class FooEnv(gym. Env):
  metadata = {'render.modes': ['human']}

def __init__(self):
    ...
  def step(self, action):
    ...
  def reset(self):
    ...
  def render(self, mode='human', close=False):
    ...

gym-foo/gym_foo/__init__.py should have:

from gym.envs.registration import register

register(
    id='foo-v0',
    entry_point='gym_foo.envs:FooEnv',
)
register(
    id='foo-extrahard-v0',
    entry_point='gym_foo.envs:FooExtraHardEnv',
)

gym-foo/gym_foo/envs/__init__.py should have:

from gym_foo.envs.foo_env import FooEnv
from gym_foo.envs.foo_extrahard_env import FooExtraHardEnv

The first block is the implementation of the environment itself. If you’ve already implemented a game, you don’t want to have to implement a lot there. gym. This subclass of Env should just be a “wrapper” for existing games, forming bridges between RL proxies that require gym APIs (step(), reset(, etc.) and the game itself. You can take inspiration from the atari_env implementation in gym, which is itself just a wrapper for existing Atari games, and does not directly contain the full game logic of those games.

The second and third blocks are needed to ensure that you can start creating instances of the custom environment using the gym.make() function.

You do have to create a gym. The Env class acts as a class of the base class and ensures that you implement all its important functions (such as step and reset). That is, suppose you want to use an already implemented RL algorithm and expect these functions to exist. Of course, the alternative is to throw the gym completely out the window and implement everything from scratch, but most likely you’ll just end up doing more work and ending up with a similar API.