«

»

Nov
16

Touchlib Gradient Filter

Background

Touchlib is a open-source “library for creating multi-touch interaction surfaces” based on computer vision. It uses a filter chain to process the incoming camera image and detect blobs corresponding to fingers on the surface.

Motivation

Currently, Touchlib applies a filter to every single pixel of an image with the same intensity. For example, if you use a threshold filter the threshold will be the same for the whole input image. However, we encountered the problem that in our diffused illumination setup certain areas of the surface where brighter (or darker) than others-which resulted in a non-uniform touch sensitivity.

Solution

We adopted some of the filters in a way so that they take a grayscale bitmap (of the same resolution as the camera image) as additional input. Each pixel of the map (i.e. its grayscale value) determines how strong the filter is going to be applied for the corresponding source pixel. In order to create such a map, we implemented a calibration software which allows the easy adjustment of these maps. By putting the finger on different surface areas one after another the calibration software determines optimal filter values and interpolates pixels inbetween calibration points.

Implementation

Filter

We found that scaler and rectify filter had the most influence on the recognition performance for differently lit areas. Therefore, adopted a combined scaler/rectify filter, using a separate gradient map for each step. The filter is added to config.xml and takes the following arguments: rectifyFilename and scalerFilename.

<maskedscaler label="maskedscaler3">
 <rectifyFilename value="rectifyMap.png"/>
 <scalerFilename value="scalerMap.png"/>
</maskedscaler>

The following images are used during filtering:

  • ptrSrc – pointer to source image data
  • ptrDes – pointer to destination image data
  • ptrMas – pointer to grayscale map for scaler
  • ptrRec – pointer to grayscale map for rectify

The actual filtering takes place as usual in kernel():

/* for each pixel in kernel() */
 
// scaler
 
value = (int) (ptrSrc[x] * ptrSrc[x]  * 
 
 ((float) ptrMas[x] / 128.0f));                        
 
// rectify
 
if(value <= ptrRec[x])
 
 ptrDes[x] = 0;
 
else {                
 
 if(value > 255)
 
   value = 255;
 
 ptrDes[x] = value;                
 
}

Calibration

Calibration is currently realized as a separate filter, which has to be inserted during the calibration step into config.xml instead of the actual filter, taking the following arguments: scalerFilename, rectifyFilename, and stepSize (for distance between calibration points).

<maskedscalercalib label="maskedscalercalib">
 <rectifyFilename value="rectifyMap.png"/>
 <scalerFilename value="scalerMap.png"/>
 <stepSize value="40"/>
</maskedscalercalib>

A grid of calibration points is displayed. Place the finger into the highlighted calibration point and hold down the right mouse button until the resulting touch looks ok (fine adjusting is possible using sliders). Press the left mouse button to proceed to the next calibration point, ALT right mouse button to go one point back. Pressing CTRL left mouse button cycles through display modes: calibration, resulting grayscale maps, and preview.

The auto-calibration is based on the detected blob size. If no blob is detected in the calibration point’s vicinity, the filter values are adjusted to be more sensitive; vice versa for too many detected blobs. Then, the filter values are adjusted for the detected blob to match a predefined size. Resulting maps are automatically saved (as PNG and data file).

Downloads

2 comments

  1. Edgars says:

    Hi!
    The link to the source is broken, can You fix it or it’s not available any more?

    e.s.

  2. Dominik says:

    Fixed!

Comments have been disabled.