Computer Vision

These computer vision projects cover fundamentals of image formation, camera imaging geometry, feature detection and matching, stereo, camera calibration, motion estimation and probabilistic tracking, image classification, and scene understanding.

Detecting Traffic Lights and Signs

For this Python project, I was tasked with writing a program to detect traffic lights and traffic signs in both simulated images and real-world photos.

The images are represented using 2D and 3D arrays with 3 color channels. I applied the Hough algorithms to detect prevalent circles and lines. Additionally, the project addressed the presence of distortion and noise in an image.

Output Images

Detecting Traffic Lights and Signs

The program found the center of the traffic light, marked it with a black dot, and listed the coordinates & light color.

The program found the center of the traffic light, marked it with a black dot, and listed the coordinates & light color.

The program found the center of the traffic light, marked it with a black dot, and listed the coordinates & light color.

The program found the center of the sign, marked it with a dot, and listed the coordinates in the white textbox.

The program found the center of the sign, marked it with a dot, and listed the coordinates in the white textbox.

The program found the center of the sign, marked it with a dot, and listed the coordinates in the white textbox.

The program found the center of the sign, marked it with a dot, and listed the coordinates in the white textbox.

The program found the center of the sign, marked it with a dot, and listed the coordinates in the white textbox.

The program found the center of the signs, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a noisy scene, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a noisy scene, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

The program found the center of the signs in a real-world photo, marked them with dots, and listed the coordinates & sign type in the white textbox.

Object Tracking and Pedestrian Detection

In this project, I implemented the Kalman and Particle filters to track objects and pedestrians in videos. These algorithms relied on measurements, prior states, and expectations between frames. The main challenge was continuing to locate the object when occlusions occured.

Output Frames

Object Tracking and Pedestrian Detection

The program used the Kalman filter to track the pink circle, and marked its location with a blue ring.

The program used the Kalman filter to track the pink circle, and marked its location with a blue ring.

The program used the Kalman filter to track the pink circle, and marked its location with a blue ring.

The program used the Kalman filter to track the pink circle, and marked its location with a blue ring.

The program used the Kalman filter to track a pedestrian, and marked his location with a blue ring.

The program used the Kalman filter to track a pedestrian, and marked his location with a blue ring.

The program used the Kalman filter to track a pedestrian, and marked his location with a blue ring.

The program used the Kalman filter to track a pedestrian, and marked his location with a blue ring.

The program used the Particle filter to track the pink circle. Then it drew a pink bounding box around the circle, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pink circle. Then it drew a pink bounding box around the circle, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pink circle. Then it drew a pink bounding box around the circle, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pink circle. Then it drew a pink bounding box around the circle, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's head. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's head. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's head. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's head. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's hand. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's hand. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the man's hand. Then it drew a pink bounding box around it, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian in the white coat. Then it drew a pink bounding box around her, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian in the white coat. Then it drew a pink bounding box around her, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian in the white coat. Then it drew a pink bounding box around her, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian in the white coat. Then it drew a pink bounding box around her, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used Particle filters to track the 3 pedestrians. Then it drew bounding boxes around them, marked the particle locations with yellow dots and the mean particle locations with a white ring.

The program used Particle filters to track the 3 pedestrians. Then it drew bounding boxes around them, marked the particle locations with yellow dots and the mean particle locations with a white ring.

The program used Particle filters to track the 3 pedestrians. Then it drew bounding boxes around them, marked the particle locations with yellow dots and the mean particle locations with a white ring.

The program used the Particle filter to track the pedestrian with the white bag, even when he was occluded behind another object. Then it drew a pink bounding box around him, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian with the white bag, even when he was occluded behind another object. Then it drew a pink bounding box around him, marked the particle locations with yellow dots and the mean particle location with a white ring.

The program used the Particle filter to track the pedestrian with the white bag, even when he was occluded behind another object. Then it drew a pink bounding box around him, marked the particle locations with yellow dots and the mean particle location with a white ring.

Faces Classification

In this project, I implemented face recognition using Principal Component Analysis (PCA), (Ada)Boosting, and the Viola-Jones algorithm.

Activity Classification

This project takes in a video of a human walking, jogging, running, boxing, waving or clapping, and correctly classifies the behavior. I created Motion History Images (MHIs), and used these images to perform activity classification as features in machine learning to train and classify the activity.

DEMO

Activity Classification

Computational Photography

Image Capture

The course explored how computation impacts the entire workflow of photography, aimed at capturing light from a (3D) scene to form a (2D) image. The traditional camera pipeline can be transformed by changing the sensor, illumination, lens, and other factors.

Camera Obscura Epsilon Photography

Camera Obscura

This photo depicts an outdoor scene.

This photo represents the same outdoor scene created with a camera obscura (pinhole camera).

I blacked out a machine shed with cardboard and painter’s tape and experimented with different pinhole sizes to create a camera obscura. I captured an image of the Iowa farmscape onto a white fiberglass panel.

This diagram shows the setup and measurements between the projection screen, camera, and pinhole.

Epsilon Photography

This image is a composite of 5 different photos that were captured with varying planes of focus.

55mm, 1s, f/5.6, ISO 160

55mm, 1s, f/5.6, ISO 160

55mm, 1s, f/5.6, ISO 160

55mm, 1s, f/5.6, ISO 160

55mm, 1s, f/5.6, ISO 160

Image Rendering

Images can be represented as multi-dimensional arrays on which mathematical operations can be performed to generate interesting artifacts.

Projects explored operations such as alpha blending, feature detection, and planar homography projections.

Panoramas

Panoramas

I wrote an algorithm to create panoramas by finding matching features between input images and then computing the homography to warp and blend the images together.

These input images were taken near near Bayanzag (Flaming Cliffs), Ömnögovi Province, Mongolia..

These input images were taken near near Bayanzag (Flaming Cliffs), Ömnögovi Province, Mongolia.

These input images were taken near near Bayanzag (Flaming Cliffs), Ömnögovi Province, Mongolia.

Seam Carving

Resizing of images is most effective when it considers the image content and preserves important features. The algorithm called seam carving supports content-aware image resizing for reducing and expanding image dimensions, as well as changing the aspect ratio.

A seam is an optimal horizontal or vertical path of pixels in an image. The path is selected by an image energy function, which defines the importance of a pixel. We select the low-energy pixels for removal or insertion. By repeatedly removing or adding seams we change the dimensions of image.

DEMO

Computer Vision

Seam Carving: Content-Aware Image Resizing

Photo to Cross-Stitch

This project takes a photo as input, and creates a cross-stitch needlework pattern as output with the specified number of colors and canvas background color.

I computationally found the best set of colors using K-means clustering, and ensured that detail is not lost by applying well-defined edges, vivid colors, high detail in the foreground, and subtle backgrounds.

GitHub