Robot Vision System

Vision-Based Autonomous Driving on the ERICA Campus

2023 · Robot Vision System · Hanyang University ERICA

This vision-system design project developed a camera-based autonomous driving system for a Unity model of the Hanyang University ERICA campus. The simulated vehicle used its front camera to follow lane markings, read traffic lights, and detect stop lines. A ROS 2 controller combined those observations with the vehicle’s simulated position to decide when to steer, slow down, or brake.

Red simulated vehicle approaching a curve on a road in the Unity model of the ERICA campus
The simulated vehicle approaching a bend on an ERICA campus road.

Driving Demonstration

Onboard footage of lane following and intersection approaches in the Unity campus model.

Campus Route and Simulation

The assignment defined a four-waypoint loop through the campus, with a shared start and finish near Way 0. The Unity road network reproduced the campus setting with curves, lane markings, and intersections. Between intersections, the vehicle followed the visible lane boundaries. At selected junctions, position ranges identified where the controller should apply the route-specific rules.

Aerial view of the Unity campus model showing the road network, intersections, and campus buildings
The ERICA campus road network in Unity.
ERICA campus map with a red loop linking Way 0, Way 1, Way 2, and Way 3
The planned campus loop with four waypoints and a shared start and finish.

Unity and ROS 2 Integration

The ROS TCP Endpoint connected Unity to the Python ROS 2 nodes. Unity supplied front-camera frames and vehicle state. The lane, traffic-light, and stop-line detectors ran as separate nodes and published their results to the driving node, which returned steering, motor-speed, and brake commands to the simulator.

Simulated front-camera view showing the vehicle hood, a yellow lane marking, and a right-turn road arrow
The front-camera view used by the three perception nodes.

Lane Detection and Steering

The OpenCV lane detector converted each front-camera frame into a steering reference.

  1. Isolate lane markings. Use HSV color filtering to retain yellow and white road markings.
  2. Extract edges. Apply Gaussian smoothing and Canny edge detection, then keep the road region of interest.
  3. Detect line segments. Use the probabilistic Hough transform and group candidate segments by slope.
  4. Fit the lane boundaries. Fit a line to each group to estimate the left and right boundary angles.
  5. Publish a steering reference. Average and scale the fitted angles and send the result to the driving node.
Six lane-detection stages from a camera frame through HSV filtering, Canny edges, road masking, Hough segments, and two fitted lane boundaries
Lane detection from the raw camera frame to fitted left and right boundaries.

Traffic Lights and Stop Lines

The traffic-light detector estimated the active signal color.

  1. Select the search region. Keep the upper portion of the frame where traffic lights are expected.
  2. Locate the signal. Apply a Haar cascade and extract the detected region.
  3. Classify the color. Compare pixel counts in red, yellow, and green HSV masks and publish the dominant signal color.
Traffic light localized by the Haar cascade and classified as yellow based on the pixel counts in its color masks
Traffic-light localization and HSV classification of a yellow signal.

The stop-line detector checked for a road marking directly ahead of the vehicle.

  1. Filter road markings. Use HSV masks for gray and white pixels, then apply Canny edge detection.
  2. Isolate the road. Limit processing to the road region of interest and apply a perspective transform to produce a bird’s-eye view.
  3. Detect a nearby stop line. Search the lower half of the transformed image with the Hough transform and publish a detection when a candidate line is found.
Six stop-line detection stages including color filtering, road masking, perspective transformation, and a final red line overlay
Stop-line detection with a perspective transform and a near-field Hough search.

PID Steering and Intersection Control

The driving node used a PID controller for lane following and applied intersection rules based on the detected signal, stop line, and vehicle position.

  1. Compute the steering error. Compare the lane-derived steering reference with a target of zero.
  2. Calculate the steering command. Apply the PID controller to the error. The integral gain was set to zero, so the implementation used the proportional and derivative terms.
  3. Apply the intersection rules. Brake when a red light and a stop line are detected in a configured intersection zone. At selected junctions, a yellow light reduces speed and a green light triggers the configured straight or turning maneuver.
  4. Send the vehicle commands. Publish steering, motor speed, and brake state back to Unity through ROS 2.

Contribution

I implemented the lane, traffic-light, and stop-line perception nodes and integrated them with the ROS 2 driving controller. I connected the simulator’s camera and state inputs to steering, speed, and brake outputs, so the same system handled lane following and intersection decisions on the planned campus route.