TESLIDER
Vision-Based Autonomous Driving in KartRider
2022 · Creative Convergence Project · Team TESLIDER
TESLIDER is a team project exploring how computer vision can connect game-screen observations to driving actions. Built with Python and OpenCV, the prototype captures the KartRider client window, detects lane markings, and generates keyboard inputs to steer the in-game vehicle.
System Overview
The system combines screen capture, lane detection, driving control, and a monitoring interface. Python’s win32gui locates the game window, ImageGrab captures its contents, and ctypes sends keyboard inputs. A Tkinter interface displays the perception and control outputs.
The implementation separates lane detection and keyboard input into modules and uses threads to run monitoring and control concurrently.
Lane Detection and Steering
The perception pipeline turns the captured game image into line-slope estimates for steering:
- Select regions of interest. Focus processing on relevant road regions in the original image and a bird’s-eye view.
- Preprocess the image. Apply Gaussian blur and convert the image to HSV color space.
- Extract lane colors. Use HSV ranges intended to isolate yellow, white, and gray lane markings.
- Detect lane lines. Apply Canny edge detection and a Hough transform to identify straight-line segments.
- Estimate slopes and steer. Use line-slope estimates, with linear regression explored and applied during development, to select forward, left, or right keyboard inputs.
The controller uses threshold-based decisions and short steering-key presses. Its behavior depends on the quality of the detected lines and the selected slope thresholds.
Monitoring Interface
The Tkinter GUI shows left and right lane-monitoring views, the estimated driving state, and keyboard-input logs. The interface also includes Run, Off, and Reset buttons and monitoring checkboxes, although some button and checkbox bindings remained incomplete at the final presentation.
Demonstration Videos
Four short clips from the project presentation show the prototype in operation alongside its monitoring interface.
Contribution
I developed the main lane-detection algorithm in Python and OpenCV. My work covered ROI selection, HSV color segmentation, Canny edge detection, and Hough-transform-based line detection to provide lane estimates for steering.