Arduino Computer Vision

Difficulty Level = 5 [What’s this?]

The Video Experimenter shield can give your Arduino the gift of sight. In the Video Frame Capture project, I showed how to capture images from a composite video source and display them on a TV. We can take this concept further by processing the contents of the captured image to implement object tracking and edge detection.

The setup is the same as when capturing video frames: a video source like a camera is connected to the video input. The output select switch is set to “overlay”, and sync select jumper set to “video input”. Set the analog threshold potentiometer to the lowest setting.

Object Tracking

Here is an Arduino sketch that captures a video frame and then computes the bounding box of the brightest region in the image.

This project is the example “ObjectTracking” in the TVout library for Video Experimenter. The code first calls tv.capture() to capture a frame. Then it computes a simple bounding box for the brightest spot in the image. After computing the location of the brightest area, a box is drawn and the coordinates of the box are printed to the TVout frame buffer. Finally, tv.resume() is called to resume the output and display the box and coordinates on the screen.

Keep in mind that there is no need to display any output at all — we just do this so we can see what’s going on. If you have a robot with a camera on it, you can detect/track objects with Arduino code, and the output of the Video Experimenter doesn’t need to be connected to anything (although the analog threshold potentiometer would probably need some adjustment).

If you use a television with the PAL standard (that is, you are not in North America), change tv.begin(NTSC, W, H) to tv.begin(PAL, W, H).

#include <TVout.h>
#include <fontALL.h>
#define W 128
#define H 96

TVout tv;
unsigned char x, y;
unsigned char c;
unsigned char minX, minY, maxX, maxY;
char s[32];

void setup()  {
  tv.begin(NTSC, W, H);


void initOverlay() {
  TCCR1A = 0;
  // Enable timer1.  ICES0 is set to 0 for falling edge detection on input capture pin.
  TCCR1B = _BV(CS10);

  // Enable input capture interrupt
  TIMSK1 |= _BV(ICIE1);

  // Enable external interrupt INT0 on pin 2 with falling edge.
  EIMSK = _BV(INT0);
  EICRA = _BV(ISC01);

void initInputProcessing() {
  // Analog Comparator setup
  ADCSRA &= ~_BV(ADEN); // disable ADC
  ADCSRB |= _BV(ACME); // enable ADC multiplexer
  ADMUX &= ~_BV(MUX0);  // select A2 for use as AIN1 (negative voltage of comparator)
  ADMUX |= _BV(MUX1);
  ADMUX &= ~_BV(MUX2);
  ACSR &= ~_BV(ACIE);  // disable analog comparator interrupts
  ACSR &= ~_BV(ACIC);  // disable analog comparator input capture

// Required
ISR(INT0_vect) {
  display.scanLine = 0;

void loop() {

  // uncomment if tracking dark objects

  // compute bounding box
  minX = W;
  minY = H;
  maxX = 0;
  maxY = 0;
  boolean found = 0;
  for (int y = 0; y < H; y++) {
    for (int x = 0; x < W; x++) {
      c = tv.get_pixel(x, y);
      if (c == 1) {
        found = true;
        if (x < minX) {
          minX = x;
        if (x > maxX) {
          maxX = x;
        if (y < minY) {
          minY = y;
        if (y > maxY) {
          maxY = y;

  // draw bounding box
  if (found) {
    tv.draw_line(minX, minY, maxX, minY, 1);
    tv.draw_line(minX, minY, minX, maxY, 1);
    tv.draw_line(maxX, minY, maxX, maxY, 1);
    tv.draw_line(minX, maxY, maxX, maxY, 1);
    sprintf(s, "%d, %d", ((maxX + minX) / 2), ((maxY + minY) / 2));
    tv.print(0, 0, s);
  } else {
    tv.print(0, 0, "not found");


What if you want to find the darkest area in an image instead of the brightest? That’s easy — just invert the captured image before processing it. Simply call tv.fill(INVERT).

Edge Detection

The Arduino is powerful enough to do more sophisticated image processing. The following sketch captures a frame then performs an edge detection algorithm on the image. The result is the outline of the brightest (or darkest) parts of the image. This could be useful in object recognition applications or
robotics. The algorithm is quite simple, especially with a monochrome image, and is described in this survey of edge detection algorithms as “Local Threshold and Boolean Function Based Edge Detection”.

This project is the example “EdgeDetection” in the TVout library for Video Experimenter.

Published by Michael, on March 20th, 2011 at 3:02 pm. Filed under: Arduino,Level 5,Video. | 78 Comments |

78 Responses to “Arduino Computer Vision”

  1. @Sam, That would be pretty hard for an Arduino to do…
    Finding a single bright spot is a simple algorithm. Matching an object to a template is much harder.

    Comment by Michael on September 30, 2013 at 10:32 AM

  2. Sir actually I need to know much about how to use image proceesing in ardunio. Is we shold write the code in matable or in any other language please help me

    Comment by rakshitha on March 17, 2014 at 11:43 AM

  3. hi,
    I want to track the number of lasers, I wont to know if the arduino strong enough for this (and if so do you have code for something similar), and if not, how can I send the frames from the arduino to computer where I will perform the processing.

    Comment by mei on December 13, 2014 at 6:36 AM

  4. Use the TVout pollserial functionality to write serial data to the computer. Capture the frame, then use pollserial to write the whole framebuffer. See all the examples for the product.

    Comment by Michael on December 13, 2014 at 7:26 AM

  5. Where can I find the position of the components (resistors, capacitors and diodes) on Video Experimenter shield??

    Comment by rachel on January 12, 2015 at 10:49 AM

  6. rachel:

    Comment by Michael on January 12, 2015 at 11:35 AM

  7. Where I could get an explanation of each component, purpose, role, voltage and current levels, etc. – to the level of the resistor / diode lonely.
    I just want to understand how the component is enabled and working,I ordered the component for Engineering College work, and I could not understand how it works, in addition’d love to know what the rate of processing of the component when I print on the screen, because once I’ve written code of tracking the number of lasers always component does not print all the data on the screen, In addition’d love to know if you think there is a solution to this situation …
    Thank you very much for your help

    Comment by rachel on January 24, 2015 at 7:00 AM

  8. Michael,

    Can this thin track, let’s say, red object? Because I looked at your .c files and get_pixel() function seems to only return 0,1, or 2 (black, white, or invert). Please let me know.

    Comment by Rey on August 4, 2015 at 9:10 PM

  9. No, it cannot detect color.

    Comment by Michael on August 4, 2015 at 9:16 PM

  10. That’s pretty awesome!
    Do you think the similar result could be achieved with ArduCAM module?

    Comment by Anton on November 8, 2017 at 5:25 PM

  11. Maybe. I have an ArduCam, and I think it provides images as JPG. So, you’d have to be decoding the JPG data as you read it from the camera and determine if pixels are on or off. It would be hard because there’s not enough memory to read in the whole image and then process it. You have to do it on the fly as you read data from the camera. I don’t know how….

    Comment by Michael on November 10, 2017 at 7:32 AM

  12. Does this code work if use an IP camera and connect it to arduino via network shield?

    Comment by Anonymous on December 6, 2018 at 2:54 PM

  13. No, this requires a composite video input to process the video.

    Comment by Michael on December 7, 2018 at 8:58 AM

Leave a Reply