SentiSight, object recognition technology

Content of this page

• Why SentiSight?
• Algorithm
• Reliability Tests and Technical Specifications
• System Requirements
• Algorithm's Demo
• Related products

Why SentiSight?

Neurotechnology's SentiSight is intended for developers who want to use computer vision-based object recognition in their applications. SentiSight enables the learning of objects and searching for learned objects in images from almost any camera, webcam, still picture or live video.

• Universal. The SentiSight algorithm is designed to be as universal as possible. It can support web cameras, surveillance cameras and can input images from the picture. It is tolerant to object scale, rotation, pose etc. Some of the potential applications for SentiSight technology include:
  • Search engines that recognize objects in picture files (either local or on the Web);
  • Security systems;
  • Parts recognition in production lines;
  • Robot vision;
  • Road signs recognition;
  • Machine vision.
• Fast. SentiSight can process video streams in real time, so it can be used for real-time applications.
• Webcam capable. Though high quality cameras will provide better recognition quality, a simple webcam is enough for SentiSight operation.
• Flexible licensing model and competitive price.

Algorithm

The SentiSight 1.1 object recognition algorithm implements advanced visual-based object learning and recognition.

1. Object learning

In order to recognize an object in an image, the appearance of an object must first be memorized. In the learning phase, SentiSight algorithms extract specific object features from a video stream or single image and save them into what is known as a model template.

In many cases there is more information in a video or single image than just the object you want SentiSight to learn, like a background, other objects in the room or a hand holding the object. For this reason, certain steps should be taken during the "learning" process to provide information about the exact location of the desired object in the image. This should be done with a mask of the object in the image. A mask explicitly specifies which pixels of the image belong to the object and which ones are part of the background. Thus, only object-specific information will be included into the model template.

If there is no way to provide a mask for the image, SentiSight can still learn the object. However the other background elements would be learned together with the object. This can affect the ability of the algorithm to recognize the unique qualities of the object and may result in the object being misclassified with other objects that have the same background.

However, for a lightweight movable object SentiSight does provide a fully automatic learning procedure. To learn a lightweight movable object in the SentiSight 1.1 SDK, the user should do following steps:

1. Choose a static background (preferably smooth) and direct the camera to it.
2. Choose a holder – an object that will be used to hold and move the learned object. A user's hand can be the "holder".
3. This "holder" should be presented it to the camera first, in various poses and configurations (if it is not rigid object) so that it can be learned by SentiSight.
4. After the holder has been learned, SentiSight is ready to learn the object itself, by having the holder rotate and move the object closer and further from the camera.
5. Input the learned object name (ID) into the system.

The process is somewhat different for objects that cannot be moved or if only images of the objects are available. In these cases, one should set the mask of the object manually or do not provide a mask at all.

2. Recognition

For the recognition of the object, the camera is directed to the scene where the learned object is presented or may appear. No other action is required. When the object appears in the vision field, it is recognized by SentiSight, which outputs the object's name (ID) and coordinates.

The SentiSight algorithm is tolerant to large variations of object scale, object rotation and translation. When an object is learned, the algorithm creates a model with possible views from different sides, in different 3D poses and in different lighting conditions.

SentiSight's object recognition is comparably fast – around 10 frames per second for a single object model (320 X 240 resolution). However for tasks when an even faster response is needed, the SentiSight 1.1 library has a tracking mode that enables tracking speeds up to 20 frames per second. Tracking is initialized if an object is recognized and located, then tracks the object until it changes somewhat in appearance, at which point tracking is reinitialized by recognition. The tracking feature is sensitive to complex backgrounds, and tracking is more difficult with homogenous objects

All performance evaluations were performed using a PC with 2.4 GHz Intel Core2 Duo CPU

Reliability Tests and Technical Specifications

SentiSight 1.1 was tested with object images from many cameras. At 0.1% False Acceptance Rate (FAR), the recognition rate is from 70% to more than 99% depending on object structural appearance, transparency, etc. For objects with well defined intenal structure, the recognition rate is 98% - 99% at 0.1% FAR.

System Requirements

• PC with 1GHz processor supporting SSE2 technology;
• 256 Mb of RAM.

Algorithm Demo

There are 2 SentiSight 1.1 demo applications for Microsoft Windows 2000/XP/2003/Vista that can be downloaded for evaluation of the SentiSight 1.1 vision based object recognition algorithm.

• SentiSight 1.1 full demo application learns and recognizes objects from almost any camera or webcam, image and video files. You may need to view SentiSight Demo Usage Guide for more convenient evaluation of SentiSight technology.
• SentiSight 1.1 simple demo application is intended for fast and simple evaluation of SentiSight 1.1 technology. The application has wizard-based user interface and uses only webcam for object learning and recognition.

Internet connection is not required to run both demo applications.

SentiSight 1.1 SDK trial is also available for downloading.

Related Products

SentiSight 1.1 SDK is based on SentiSight 1.1 technology.