Archive for July, 2011

Visualizing TV Dialog Using Closed Caption Data

Difficulty Level = 8 [What’s this?]

One of the coolest things you can do with the nootropic design Video Experimenter shield for Arduino is decode the closed caption data embedded in NTSC (North American) television broadcasts. I figured out how to do this and documented it in a another project, so if you want to understand all the details of how to capture and decode closed captions, refer to that project. With this project, I take it a step further and show how the spoken dialog embedded in a television show can be visualized on a computer in a “cloud” of words. This is the same type of cloud (often called a “tag cloud”) that you see on blogs, where the frequency of a particular word is reflected in the size of the word. More frequent == larger word.

Hardware Setup

First, here’s how the hardware is set up. It’s really simple. The Video Experimenter needs a composite video feed from a TV tuner like a DVR (e.g. Tivo) or VCR. You can also use a DVD player because DVDs usually have closed captioning data. The USB cable connects to your computer where you run a Processing sketch (program) to visualize the words as they are decoded by the Arduino. The Processing sketch dynamically builds the TV cloud as the words are extracted from the closed caption stream!

Hardware setup


Demo Video

Here’s a video where I show a TV cloud of spoken dialog being created dynamically. I superimposed a video of the television broadcast so you can correlate the broadcast with the Processing application, but note that the Processing application doesn’t acutally display the video. Words spoken with higher frequency are larger.

Example TV Clouds

I always have noticed that whenever I happen to see a US national news broadcast, all the commercials are for drugs. I guess only old people watch the news on TV anymore. Here’s a TV cloud of the commercials shown during NBC Nightly News. Can you guess which drugs are being advertised? Can you guess which maladies they claim to cure? Look at all those nasty side effects!

TV cloud made from drug commercials. Click to enlarge.


Here is a TV cloud made while watching a baseball game. For US readers familiar with baseball, can you guess which teams were playing? Answer is at the end of this post.

TV cloud built from part of a baseball game broadcast. Click to enlarge.


The Software

The Arduino sketch is fairly simple, and for details on how it works, please see the in-depth article about decoding closed captions.
Download the Arduino sketch

The Processing sketch reads words from the serial line and filters out any word less than 3 letters and some very common words like “the”, “and”, “for”, etc. This application relies on the very nice OpenCloud Java library, so you’ll need to download that and use it in your Processing environment. Create this structure in your Processing sketchbook libraries directory: opencloud/library/opencloud.jar
Download the Processing sketch


Answer to the baseball broadcast question: Kansas City Royals vs. Minnesota Twins (go Twins!)

Published by Michael, on July 18th, 2011 at 7:17 pm. Filed under: Arduino,Level 8,Processing,Video. | 13 Comments |

Video Experimenter on the Seeeduino Mega

The nootropic design Video Experimenter shield uses some pretty advanced features of the Arduino’s ATmega328 microcontroller. One downside of this is that you can’t use the Video Experimenter shield on an Arduino Mega. Why? Well, the designers of the Arduino Mega didn’t connect a lot of the ATmega1280/ATmega2560 pins to headers on the board so that you could use them! And, as it turns out, the pins with key features utilized by the Video Experimenter are not connected to anything!

To perform video overlay, the Video Experimenter relies on an input capture pin (to capture the exact time that the pin has changed state). Even though the ATmega1280/ATmega2560 has 4 input capture pins, none of them are connected!

Important pins not even connected!

And to capture video images in the Arduino’s memory, Video Experimenter uses the analog comparator in the chip. But the AIN0 pin for the analog comparator is not connected! What were the Arduino Mega designers thinking?

Fortunately, there is the Seeeduino Mega. The guys at Seeed Studio broke out nearly all the pins on the ATmega1280 so that you can use them. I love the Seeeduino Mega because it provides so many pins on a rather small board.

Simply make 5 connections with jumper wires and you can use the Video Experimenter on the Seeeduino Mega. No code changes necessary!

By connecting 5 jumper wires, you can use the Seeeduino Mega

Here are the connections to make:
11 to 9 (white wire in picture above)
7 to 29 on the Seeeduino Mega (yellow wire)
INPUT pin on the Video Experimenter to PE2 on the Seeeduino Mega (green wire)
SYNCOUT pin on the Video Experimenter to PD4 on the Seeeduino Mega (gray wire)
VSYNC pin on the Video Experimenter to 21 on the Seeeduino Mega (brown wire)

Now you can use the Video Experimenter with an Arduino that has a more powerful processor. It really helps to have 8K of SRAM instead of 2K. Now you can do text and graphics overlay with higher resolutions, like 192×128. Have fun!

Published by Michael, on July 13th, 2011 at 2:02 pm. Filed under: Arduino,Level 1,Video. | 14 Comments |

Digit Shield Limited Edition: Blue

Blue Digit Shields are now available as a limited edition. I have only a very small stock of these fully assembled and tested Digit Shields for Arduino. Blue LEDs and LED displays are much more expensive than red and green, but I am able to offer these at the same price as assembled red and green Digit Shields. They are extremely bright and look very cool! Get one while you can!

Blue is beautiful. You may need sunglasses.

Published by Michael, on July 4th, 2011 at 8:10 am. Filed under: Arduino. | No Comments |