Technical Overview

This project is a real-time robotic system built to predict where a fluid stream will land and move a platform underneath it before impact. The system combines computer vision, depth sensing, encoder feedback, flow sensing, and physics-based prediction to calculate landing position and command motion in real time.

Computer vision view

Computer Vision & Trajectory Prediction

Using OpenCV and an Intel RealSense depth camera, the system tracks the source position and orientation in 3D space. Retroreflective markers improve detection reliability and allow the system to estimate position from camera input.

A flow sensor provides initial velocity data, and projectile motion equations are used to predict the landing position before impact. These calculations determine where the platform must move and how quickly it must respond.

Arduino and electronics

Hardware & Control System

The platform uses a custom CoreXY gantry driven by DC motors and controlled through Arduino-based motor logic. Rotary encoders provide rapid position feedback, while the PC handles higher-level vision and prediction.

A closed-loop PID control system is used to move the platform accurately and quickly. Serial communication separates high-level trajectory computation from low-level motor execution.

Robot system overview

Key Challenges & Optimization

One of the biggest engineering challenges was achieving useful response time in a real-world physical system. The project required synchronizing camera input, encoder feedback, and flow sensor data while maintaining stable and accurate movement.

A major optimization was replacing slow camera-based platform tracking with encoder-based estimation, reducing response time from roughly 0.1 seconds to 0.001 seconds and making real-time interception practical.

Skills Used

  • Python
  • OpenCV
  • Arduino / C++
  • Computer Vision
  • Sensor Fusion
  • PID Control Systems
  • Embedded Systems
  • Serial Communication
  • Intel RealSense Depth Camera
  • Linear Algebra
  • Kinematic Physics
  • CoreXY Motion Systems

Achievements

  • Designed and built a real-time robotic system for trajectory prediction and interception
  • Integrated computer vision, sensor data, and physics modeling into a unified system
  • Reduced system latency from 0.1s to 0.001s using encoder-based position estimation
  • Implemented closed-loop PID control for precise and responsive motion