TESLIDER

Vision-Based Autonomous Driving in KartRider

2022 · Creative Convergence Project · Team TESLIDER

TESLIDER is a team project exploring how computer vision can connect game-screen observations to driving actions. Built with Python and OpenCV, the prototype captures the KartRider client window, detects lane markings, and generates keyboard inputs to steer the in-game vehicle.

Four demonstration frames showing KartRider running beside the TESLIDER lane-monitoring interface
Demonstration frames from the team's final presentation.

System Overview

The system combines screen capture, lane detection, driving control, and a monitoring interface. Python’s win32gui locates the game window, ImageGrab captures its contents, and ctypes sends keyboard inputs. A Tkinter interface displays the perception and control outputs.

The implementation separates lane detection and keyboard input into modules and uses threads to run monitoring and control concurrently.

TESLIDER architecture showing lane detection and keyboard-input modules feeding driving control and the Tkinter interface
Module organization and the main components of the application.

Lane Detection and Steering

The perception pipeline turns the captured game image into line-slope estimates for steering:

  1. Select regions of interest. Focus processing on relevant road regions in the original image and a bird’s-eye view.
  2. Preprocess the image. Apply Gaussian blur and convert the image to HSV color space.
  3. Extract lane colors. Use HSV ranges intended to isolate yellow, white, and gray lane markings.
  4. Detect lane lines. Apply Canny edge detection and a Hough transform to identify straight-line segments.
  5. Estimate slopes and steer. Use line-slope estimates, with linear regression explored and applied during development, to select forward, left, or right keyboard inputs.

The controller uses threshold-based decisions and short steering-key presses. Its behavior depends on the quality of the detected lines and the selected slope thresholds.

Straight and curved KartRider road scenes paired with bird's-eye views and highlighted processing regions
Original game views and bird's-eye regions used for driving control.

Monitoring Interface

The Tkinter GUI shows left and right lane-monitoring views, the estimated driving state, and keyboard-input logs. The interface also includes Run, Off, and Reset buttons and monitoring checkboxes, although some button and checkbox bindings remained incomplete at the final presentation.

Annotated Tkinter interface with lane views, driving-state logs, keyboard-input logs, and application controls
Monitoring interface and its intended controls.

Demonstration Videos

Four short clips from the project presentation show the prototype in operation alongside its monitoring interface.

Demonstration 1
Demonstration 2
Demonstration 3
Demonstration 4

Contribution

I developed the main lane-detection algorithm in Python and OpenCV. My work covered ROI selection, HSV color segmentation, Canny edge detection, and Hough-transform-based line detection to provide lane estimates for steering.