Table Game Tracker

An Exercise in Image Processing

We built on the success of the Enhanced Water Pong project by stripping the product of all of its wires and building on its functionality. We attached a small cell phone camera to the ceiling that could communicate wirelessly with our FPGA board. By comparing frames, our system could detect all of the aspects of gameplay.

Once initialized, the camera would spew data continuously. Obtaining, interpreting, and displaying this data was a huge challenge. The camera sent data in YCbCr format, meaning that it would alternate between sending grayscale and color values. Since we only needed grayscale images, we read the grayscale values and ignored the color information, allowing us to store one byte of data for each pixel.

Detecting a Made Shot

We used a technique called Template Matching, which essentially compares a template image to every possible location in a larger reference image. If there was a high correlation between our template image and a point in our reference image, our algorithm knew it had found a picture of the template image inside of the reference image. Our system used an empty (unmade) cup as the template image. This allowed the system to count the number of cups in play and recognize when there was a decrease in correlation for a particular cup, indicating that the cup had been made.