Release 1.0.8 with GPGPU and MacOS X support available

Version 1.0.8 of the Scene application binaries and source code are available for download. This release features GPU accelerated post processing of the object blobs using morphological dilation and erosion filters. The Scene GUI incorporates two new sliders that set the number of dilations and erosions performed during post processing (dilations are always done before erosions). Morphological filtering can improve the tracking robustness in many common scenarios.

Scene 1.0.8 offers GPU implementations of all the background subtraction methods using OpenCL. When an OpenCL compatible device is detected the GPU versions are used by default, otherwise Scene falls back on the CPU versions (although see the XML configuration below). The release includes 32 bit and 64 bit binaries for Windows, 32 bit and 64 bit binaries for Linux, tested on Ubuntu 14.04.3 with NVIDIA graphics cards, and a Mac version of the Scene application for MacOS X 10.9 or higher (Cocoa). Also provided are Visual Studio 2010 and XCode projects.

Depending on the computer hardware resources, the image resolution, and the background model employed, the GPU implementations can show a very significant increase in performance compared to the CPU versions, in both the 32 bit and 64 bit binaries.

Scene 1.0.8 also features simple UDP streaming of the source image, the background model image, or the binary image with the tracked object blobs. The Scene Processing Library 3.0 includes a demo streaming client that allows viewing the video frames sent from the main GUI application (based on VideoReceiverThread).

In addition, Scene allows sending messages to multiple TUIO clients. Please see the XML Configuration File section below.

Thanks for the feedback from the people who are using Scene!

Description

Scene is an open source multiplatform computer vision framework that performs background subtraction and object tracking, using two traditional algorithms and three more recent algorithms based on neural networks and fuzzy classification rules. It was mainly designed as a toolkit for the rapid development of interactive art projects that explore dynamics of complex environments (for example public spaces).

This framework is developed by Laurence Bender and Ignacio Guerra at the Laboratorio de Arte Electrónico e Inteligencia Artificial of the Universidad Nacional de Tres de Febrero (UNTREF) in Caseros, province of Buenos Aires, Argentina, as part of the Electronic Arts research program.

The Scene GUI is a standalone application with a simple user interface which implements the widespread TUIO protocol, originally designed by the reacTIVision project for transmitting the state of tangible objects and multi-touch events on a table surface.

For each detected foreground object, Scene sends a TUIO cursor message and a TUIO blob message to the client application. The cursor message carries the center of mass position, velocity and acceleration values. The blob message carries the bounding box position, size, velocity and acceleration values, and the area of the blob. Each blob is assigned a unique session ID, which allows the simultaneous identification and tracking of several objects.

The framework includes a set of client projects for several programming languages, which serve as a starting point for the development of interactive applications. They are basically extensions of the TUIO 1.1 clients that also handle TUIO blobs, and include a SceneBlob class that simplifies object tracking and analysis. Example clients are available for the languages Java and Processing, see the downloads section. Existing TUIO clients may also be used with Scene.

The Scene application presently runs and compiles under the Windows, Linux, and MacOS X operating systems. Under Windows it supports video files and any capture source with a proper WDM driver, such as USB, USB2, FireWire and DV cameras. In Linux, only video4linux2 compatible cameras are supported. The Scene 1.0.8 binaries for Windows also include a video file for testing the program (thanks Rui Penha).

Application Handling

Before starting the Scene application make sure you have a supported camera connected to your system, or an existing video defined in the XML Configuration file (in the Windows version). When running, the application GUI shows three video windows: the source image, the background model image, and a binary image with the tracked object blobs.

A group of option buttons below the source video window determine whether to draw the center of mass positions, the blob bounding boxes, and the blob session IDs in the source and tracking video windows. Below the tracking video window there are two sliders that adjust the minimum and maximum areas of the extracted blobs, relative to the video image size. Two other sliders set the number of dilations and erosions performed on the blobs during post processing.

Under Windows, the Source menu allows the user to select a connected camera or a video file as the image input for Scene. When a camera is selected in the Windows version, the item 'Camera Options' is enabled. Choosing this option, or pressing 'Ctrl-O', shows a system dialog that allows the adjustment of the available camera parameters. Under Linux and MacOS X only camera capture is supported, and the overall camera settings are configured within the Scene.xml configuration file (see below).

The Model menu allows the user to select one of the five available background subtraction algorithms implemented in Scene. Pressing 'Ctrl-T' selects the training option, which sets the background model to the current source image. When a background subtraction method is selected, a group of sliders that allow the configuration of its parameters appear in the lower right quadrant of the GUI. A more detailed description of the methods and their parameters can be found in the BGS Methods section.

XML Configuration File

Common settings can be edited within the file "Scene.xml".

The <camera width="320" height="240" /> XML tag sets the image size of the captured video when using a camera as the input source.

Since Scene 1.0.2, the application allows video files as input sources under Windows. The XML tag <video file="myvideo.avi" /> defines the path to the file. In order to correctly reproduce videos, the necessary codecs must be previously installed in the system.

In the Linux and MacOS X versions of Scene the <video /> tag is not used, since only camera capture is supported. An additional <camsettings /> XML tag allows the configuration of the overall camera settings.

The Scene application usually sends the TUIO messages to port 3333 on localhost (127.0.0.1). You can change this setting by editing the XML tag <tuio host="127.0.0.1" port="3333" />.

The application also supports multiple TUIO clients. For example, inserting the tags

<tuio host="127.0.0.1" port="3333" />
<tuio host="192.168.1.10" port="3333" />
<tuio host="192.168.1.12" port="3333 /">

sends the TUIO messages to three different clients on separate hosts.

The <stream /> XML tag contains the image streaming parameters. The image attribute can be set to "source", "tracking" or "background". The width and height attributes define the size of the streamed images, while the quality attribute sets the jpeg compression quality of the individual frames (0-100). For example, the following enables streaming of the source image to localhost on port 9100:

<stream enable="1" image="source" host="127.0.0.1" port="9100" width="320" height="240" quality="60" />

New to Scene 1.0.5 or greater is the <gpu /> XML tag, that enables GPU processing by default. When the "enabled" attribute is set to 0, only the CPU implementations of the background subtraction methods and morphological filters are used. Note: GPU processing in MacOS X is disabled on older Intel HD Graphics 4000 devices.

The remaining XML tags store program parameters that are set in the Scene GUI. These tags are usually not edited by the user and are automatically updated when closing the application or changing the background model.

Compilation

The source distribution includes projects for Windows, Linux and MacOS X platforms, and their respective build systems.

Windows
Visual Studio 2010 projects are included, as well as all the necessary libraries and headers (wxWidgets, DirectShow, libjpeg, OpenCL). The projects should build right away without any additional configuration. Since release 1.0.4 of Scene only Visual Studio 2010 projects are available. Previous releases also included Visual Studio 2005 projects.

Linux
A Makefile is included in order to build Scene. The application requires that the following libraries and headers be installed: libwxbase3.0, libwxgtk3.0, libjpeg8, and libOpenCL.

MacOS X
A project for Xcode is included (tested with version 7.2), complete with the necessary frameworks and additional required libraries.

Citation

 @inproceedings{Scene,
 author    = {Bender, Laurence and Guerra, Ignacio},
 title     = {Scene: visión por computadora para las artes electrónicas utilizando
              métodos de sustracción de fondo acelerados mediante técnicas GPGPU},
 booktitle = {Tercer Congreso Argentino de la Interacción-Persona Computadora,
              Telecomunicaciones,Informática e Información Científica (IPCTIIC 2014)},
 address   = {Huerta Grande, Córdoba, Argentina},
 year      = {2014},
 month     = {11},
 url       = {http://scene.sourceforge.net}
 }
				  

Bender, Laurence; Guerra, Ignacio. Scene: visión por computadora para las artes electrónicas utilizando métodos de sustracción de fondo acelerados mediante técnicas GPGPU. Tercer Congreso Argentino de la Interacción-Persona Computadora, Telecomunicaciones, Informática e Información Científica (IPCTIIC 2014), Huerta Grande, Córdoba, Argentina, 2014. (PDF in spanish)

Thanks

To the reacTIVision project for inspiration, great code, and help with the speech.

To the wxWidgets project for making multiplatform GUI development a breeze.

To the CCV project for their graphic interface design.

To TinyXML and oscpack for dealing with the details.

To the Independent JPEG Group for their solid image compression library.

License

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software Foundation Inc.,
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.

Laurence Bender