jatinror/ppo-pixelopter

PixelCopter RL Agent

Overview

PixelCopter is a side-scrolling reinforcement learning environment inspired by the classic helicopter game. The agent:

• Controls a pixel copter navigating through a scrolling cavern.
• Avoids colliding with walls that move from right to left.
• Gains positive rewards for surviving timesteps; receives negative rewards on collision.

This repository contains:

• train_pixelcopter.py – Training script
• test_pixelcopter.py – Test script with Pygame visualization
• ppo_pixelcopter_cert.zip – Pre-trained PPO agent

---

Environment Details

• Observation Space: [copter_y, wall_gap_positions...]
• Action Space:
  • 0: Do nothing (gravity pulls the copter down)
  • 1: Apply lift (move upwards)
• Rewards:
  • +1 for surviving a timestep
  • -5 for collision
• Medium Difficulty Settings:
  • Gap size: 6
  • Gravity: 0.25
  • Max timesteps per episode: 500

---

Installation

pip install stable-baselines3 gymnasium pygame numpy