Persons or vehicles recognition and tracking for video surveillance systems
SentiVeillance SDK is designed for developing software that performs biometric face identification, detects moving pedestrians or vehicles or other objects and performs automatic license plate recognition using live video streams from digital surveillance cameras.
The SDK is used for passive identification – when passers-by do not make any efforts to be recognized. List of possible uses includes law enforcement, security, attendance control, visitor counting, traffic monitoring and other commercial applications.
Available as a software development kit that allows solution development for Microsoft Windows and Linux platforms.
Features and Capabilities
- Real time pedestrians and vehicles tracking and classification.
- Biometric facial identification and matching against watchlist database.
- Automated license plate recognition (ALPR) for moving vehicles.
- Color, size and movement direction estimation for vehicles and other objects.
- Gender classification, age evaluation, facial expression and attributes detection.
- Automatic operation logs and reports events, updates watchlist from video stream.
- Large surveillance systems support with multiple cameras.
- Ready-to-use server for integration into video management systems (VMS) optionally available.
- Reasonable prices, flexible licensing and free customer support.
The SentiVeillance 8.0 technology has these specific capabilities:
- Real time performance. SentiVeillance technology performs face recognition, pedestrian or vehicle classification and tracking in real time. The technology is designed to run on multi-core processors to achieve fast performance.
Three modalities for surveillance systems.
Depending on the surveillance system design, these modalities may be used separately or together, combining the corresponding data for tracked persons or vehicles:
Biometric face recognition – based on deep neural networks and provides these capabilities for surveillance systems:
- Multiple face detection, features extraction and template matching with the internal database in real time.
- Facial identification reliability enables using large watchlist databases. Persons wearing face masks or respirators can be recognized without separate enrollment.
- Face tracking is performed in all successive frames from the video source until they disappear from camera field of view. The face tracking algorithm uses dynamic face and motion prediction models that make it robust to occlusions like other objects or even other faces. The algorithm is able to continue tracking a face even when it re-appears after being fully covered by occlusions (like walls, furniture, posters etc). Also, if the pedestrian tracking algorithm tracks a person without their face visible, and later their face is recognized, the whole tracking history will be associated with the person's identity, as well as the further non-face tracking.
- Gender classification (optional) for each person in the frame.
- Age determination (optional) for each person in the frame.
- Face mask, smile, open-mouth, closed-eyes, glasses, dark-glasses, beard and mustache attributes detection (configurable).
Vehicle or human detection, classification and movement tracking – performs object detection of moving and static objects in the scene, their classification and tracking until they disappear.
These features are available for surveillance systems:
- Object classification. SentiVeillance allows to perform object classification, locations and tracking based on its type. Over 20 classes are recognized: pedestrian, bus, truck, sedan, SUV etc.
- Color estimation. The algorithm returns most likely color estimation for cars and pedestrians. The estimated color values are: red, orange, yellow, green, blue, silver, white, black, brown, grey.
- Make and model estimation for vehicles. A neural networks-based performs the estimation of the tracked vehicle's make (brand) and model.
- Clothing analysis. When a pedestrian is detected, their appearance can be evaluated for the presence of different clothes types. Over 20 clothes and headwear classes are recognized, like shirts, trousers, skirts or coats, as well as certain national clothings. Also, this algorithm can estimate if the person is partially or totally naked.
- Movement direction estimation. The algorithm estimates vehicle or pedestrian movement direction as angle relative to camera orientation.
- Tolerance to object visibility. The detection algorithm works with partially visible objects and from great distance.
- Combination with facial recognition and ALPR. If a pedestrian was tracked without their face visible, and later the person is recognized by the facial recognition algorithm, the whole tracking history will be associated with the person's identity, as well as the further non-face tracking. Vehicle tracking can be updated in the similar way with the help of the ALPR algorithm.
Automated license plate recognition (ALPR) – once a vehicle has been detected, SentiVeillance ALPR algorithm detects and reads the license plate:
- Traffic data processing. SentiVeillance algorithms can simultaneously read vehicle license plates from multiple moving vehicles.
- Tolerance to camera position. Depending on camera resolution, the ALPR algorithm can read license plates from longer distance and higher angle.
- Preventing cheating with replaced license plates. Integrators can use vehicle recognition and ALPR modalities together for making software logic which checks if recognized license plate corresponds other registration data, like vehicle color or type, and not being spoofed or moved from another vehicle.
- Biometric face recognition – based on deep neural networks and provides these capabilities for surveillance systems:
- Automatic operation. A system based on SentiVeillance 8.0 SDK is able to log on the fly all events. It can be configured to automatically report events like match with a watch list, or perform automatic enroll from video.
- Large surveillance systems support. SentiVeillance 8.0 SDK allows to integrate its technology into surveillance systems with multiple cameras. A common PC with a GPU can process multiple video streams simultaneously.
- Thermal imaging. Besides optical and near-infrared cameras, SentiVeillance SDK includes a specific programming sample for processing video streams from thermal imaging cameras.
- Video files processing. SentiVeillance also accepts data from video files. The video files can be processed in real time as coming from a virtual camera or can be processed at maximum speed depending on hardware resources available.