SAFR : Solution Guide - Anonymous Traffic Monitoring

Table of Contents

Summary

This use case describes how to use SAFR to perform Anonymous Demographics collection and reporting using face detection.

Introduction

This document describes how to setup anonymous traffic monitoring for purposes of understanding the age, gender and sentiment make up of people in your facility.  It uses video cameras in your facility to analyze and record age, gender and sentiment of people.  The information provided is useful to know the percentage distribution of people in your facility by age, gender and sentiment.  Because the data is collected anonymously, people may be counted more than once if they appear in front of different cameras as they pass through your facility or pass through the same location multiple times.  

In this tutorial we'll learn how to setup SAFR to perform the following useful functions:

  • Train the system to detect and measure age, gender and sentiment for people passing in front of the camera
  • Select and set up cameras to detect demographics
  • Optimize the system to maximize the number of subjects for which information can be captured while balancing against the accuracy of that information

Background

About SAFR

SAFR from RealNetworks is a highly accurate facial recognition platform powered by artificial intelligence that works with IP cameras and readily available hardware to match faces in real time.  SAFR provides industry-leading performance to offer the following capabilities:

  • Scalable: can reliably match against millions of faces in under a second
  • Designed specifically to recognize faces in real-world conditions, including people in motion, in dim lighting, and at occluded angles
  • Non-literal matching: handles multiple variations, including masks, makeup, spectacles, hair styles, and aging
  • High tolerance to yaw, tilt, and rotation
  • Allows error rate tuning to minimize false rejections and false acceptances, or to optimize between the two

RealNetworks computer vision topped the list for “labeled faces in the wild” (LFW) achieving a 99.8% accuracy in a test conducted by the University of Massachusetts and 0.048 for Wild Faces FNMR according to the National Institute of Standards and Technology (NIST).

SAFR runs on Windows, Linux and Apple Mac computers as well as iOS and Android devices. SAFR software integrates cameras, door locks and alert systems with face recognition technology to enhance access control and security.  SAFR software is comprised of three primary components:  

  • SAFR Desktop and SAFR Mobile (Desktop and Mobile variants)  (Note: SAFR Edge is used for the installer name which includes SAFR Desktop and Actions. )
  • SAFR Actions (Windows, Linux or Mac)
  • SAFR Cloud Service or SAFR Server on Windows or Linux (Note: SAFR Platform is a set of products including SAFR Server, Desktop and Actions.)
    • On-Premises installs are important in cases where policy does not allow data to leave the site.
    • Cloud deployments are simpler and require less hardware but running costs are higher.

About Anonymous Traffic Monitoring

About Privacy

Demographic Data (age, gender, emotion) is extracted directly from the face image.  At no time is a face signature generated in order to obtain this information.  Demographic data collected can be optionally associated with a biometric signature but is not required.  Demographic data is stored as part of an event which records the time and metadata associated with an action such as the camera where the recognition occurred.

To increase privacy, storage of face images should be turned off for this use case.  When performing age, gender and sentiment analysis, the image is sent to the SAFR Platform (which can be local or in the cloud) for analysis.  The image is is analyzed and the desired metrics extracted and the image discarded.  Unless configured to learn new faces, nothing is stored. The information from the analysis is then returned to the client.  If configured to report events, the client will generate an event call to the SAFR Platform with just the age, gender and sentiment.  This event does not need to include the event image for this use case and thus should be turned off.

For anonymous demographics collection, the image nor biometric signature is stored.  The information stored is not associated with an individual nor could be used to create an association with an individual.  Thus, it is entirely free of Personally Identifiable Data (PII).

About Cameras

The goal in anonymous traffic monitoring is to determine the age, gender and/or sentiment of individuals at one or more locations.  SAFR works with any IP-based but to achieve good results the quality must meet certain criteria.  To perform this, cameras must be placed so that faces are of sufficient and face is oriented sufficiently towards the camera and not occluded by hats or other people.  Further, face image must be at least 60 pixels high. 

About SAFR Desktop App

The SAFR Desktop allow you to connect to a video feed and process video to perform demographics collection.  

About SAFR Actions

SAFR Action is an application designed to allow one or more actions to be taken when a recognition event occurs.  For the purpose of demographics collection, SAFR Actions is useful to log desired demographics to a CSV file for integration into a reporting system.  Optionally SAFR Events APIs exist to extract events for a desired time range in JSON format.  Use of SAFR Event APIs are not covered in this document.  For more information about SAFR Event APIs, refer to your SAFR Download Portal.

Planning your System

Before you get started, think about how you want to install SAFR.  The following components are needed for demographics collection:

  • SAFR Platform - You can use the cloud or install the platform on-site.  In this tutorial we'll assume SAFR Cloud is used.
  • SAFR Desktop for processing video feeds from cameras
  • SAFR Actions for processing events and logging demographics data to CSV file.
  • One or more Cameras

Below provides an example of how the components are laid out.

The SAFR Edge and SAFR Actions could be run on a single computer but here they are shown separately.  If you have many cameras, you may end up needing to install more than one machine to process the video feeds.  Typically one machine can handle up to 8 cameras.

Ingress and Egress cameras will provide information on traffic in and out of the facility.  Other cameras can be used to measure traffic within the space and allow for heat map reporting to identify popular areas or traffic flows.

(lightbulb) See Connect your cameras for information on camera placement.

Installation and Configuration

Installing your cameras

Before you start, you should have your IP cameras installed and connected to your Network.  Please see Appendix A for information on camera selection. Later we'll describe how to get the information for how to connect SAFR to your camera.

Install and Configure SAFR Desktop

 If you have not already done so, download (from SAFR Download Portal) and download "SAFR Edge" for your platform.

  • For Windows machines, the Full version is intended for machines with nvidia video cards.  With an nvidia card installed, significantly improved performance can be realized.
  • The "SAFR Edge" installer includes both SAFR Desktop and SAFR Actions Apps
  • After installation proceed to Configuring SAFR App below.

Configuring SAFR App

  1. Start the SAFR Desktop application from the Windows Start Menu or Mac Applications Folder
  2. Choose "Secure Access" from the dropdown in the in the top right of the main window
  3. Open SAFR Preferences Menu (SAFR > Preferences on Mac or Tools > Preferences on Windows) and configure as follows:
    1. Account Tab:
      1. Change User Site and User Source to identify the building or campus where the camera is located.  This attribute is information but can be for filtering or grouping in reports.
    2. Cameras Tab
      1. Set the Source attribute for each camera to identify each camera separately. This attribute is information but can be for filtering or grouping in reports.

 

Connect your cameras

Camera 1

 Open the SAFR Desktop App

  1. Start the SAFR App through the Windows Start Menu or Mac Applications folder
  2. On Mac or PC, select a camera from the dropdown in the top left of the main window if necessary

Make sure the camera resolution is sufficiently large so that faces are at least 60 pixels tall (160 pixels is preferable).  Following table provides face size for various camera resolutions.  Face size can be adjusted by either larger cameras or placing camera / using camera zoom.  If using camera zoom, only use optical zoom.  Using digital zoom will negatively impact recognition by degrading quality of image.

Camera Resolution
 
Face size at 5% height
(1/20th of vertical height)
Face size at 10% height
(1/10th of vertical height) 
4k / 8 MP (4096 x 2160 pixels)110 pixels220 pixels

2k / 2 MP (2048x1152)

55 pixels110 pixels
Full HD / 1.3 MP (1280x1080)50 pixels100 pixels
HD (1920x720)35 pixels70 pixels
  • Color Key: Good Results - Ok Results - Poor Results

Camera 2+

  1. In SAFR Desktop, choose File > New
  2. Perform above step for 2nd and subsequent cameras
  3. After adding each camera, check System CPU and GPU load.  Stop before reaching 100% load on either.

(lightbulb) When you quit the SAFR Desktop application, SAFR stores its state and will restore the state upon start up. When restarted, SAFR will open up with the same number of windows and each window will restore its previous settings, including the camera it was connected to.

(warning) IMPORTANT: When closing SAFR use "Quit SAFR" from the File Menu. Closing SAFR via the Window Close button will only close the window and you will loose state (settings and connected camera) for that window.

 

Configure SAFR Desktop for Anonymous Traffic Monitoring

Demographics Collection Configuration

  1. Optimize camera location/orientation and adjust optical zoom to get at least 80 pixels high face (from chin to forehead) as noted above
  2. Adjust focus of the camera to the closes position where all faces are within the camera's field of view (i.e. ensure no one will be to the left or right of the camera view)
  3. Adjust SAFR Settings to be accepting of the image quality achievable.
  4. Ensure SAFR Desktop is in Stranger Monitoring mode (thus only recognizing)
  5. To assist in optimization, 

In Preferences > Detection:

  • Minimum searched face size: 60
  • Minimum required face size: 55 (filter out any background faces)

In Preferences > Recognition:

  • Minimum required center pose quality:
    • To allow recognition: 0
  • Minimum required face sharpness quality:
    • To allow recognition: 0
  • Minimum required face contrast quality:
    • To allow recognition: 0
  • Proximity Threshold Allowance: 0.13 (used to further relax the strictness of the matching)
    • Increase to 0.2 or 0.3 if matching % is too low

In Preferences > Events enable (check) following:

  • Report events
  • Include Unrecognizable Events from Camera (try to determine demographics even if insufficient quality to recognize identity)
  • Include Stranger Events (not enrolled / learned)

Turn following on only during testing to allow for checking results in realtime thru Events Viewer window

  • Preserve Event Face Image (useful to compare with values given for age, gender and sentiment)
  • Preserve Event Scene Thumbnail Image (Useful to see which camera - can also use value of "Source")

 

Configure SAFR Actions

Use following procedure to configure SAFR Actions to write log file.

Add Log Script

Add the logging script to SAFR Actions script directory. This script logs demographics each time a person is detected.

Scripts directory is in following location:

  • Windows: C:\Program Files\RealNetworks\SAFR\ares\scripts
  • Linux: /Library/RealNetworks/SAFR/ares/scripts

Log to File Sample Script

Write Log Batch file
@echo off
REM Replace ? with null string to prevent for loop from interpreting ? as wildcard and skipping (because no matching file).
set args=%*
set args=%args:?=%
REM Set delayed expansion so vars expanded in each iteration of for loop using !var!
SETLOCAL EnableDelayedExpansion
REM Create filename in format of YY_MM_DD for log file rolling
	REM GET localtime from windows (LocalDateTime=YYYYMMDDHHMMSS.ms)
	for /f "tokens=2 delims==" %%G in ('wmic os get localdatetime /value') do set DT=%%G
	set FileName=!DT:~0,4!_!DT:~4,2!_!DT:~6,2!
	set FmtDate=!DT:~0,4!-!DT:~4,2!-!DT:~6,2!
	set FmtTime=!DT:~8,2!:!DT:~10,2!:!DT:~12,2!
Set LogFile=C:\Users\Public\Documents\safr_events_%FileName%.csv
set log_string="%FmtDate%","%FmtTime%"
REM Loop and process each argument (not currently used)
REM for %%v in (%%args%%) do (
REM    CALL set log_string=%%log_string%%,"%%~v"
REM )
REM Append arguments list from SAFR Actions already in Quoted CSV format
set log_string=log_string,%*
REM Print log line
if not exist %LogFile% (
	REM The following list should match tags added to SAFRActions.config.  Use SAFRActionsVars.xls to generate list
	echo date,time,person_id,person_ext_id,name,first_name,last_name,person_type,company,idClass,source,site,age,sentiment,gender,similarityScore,event_type,event_id,event_epoch_start_time,validation_phonea,validation_email,home_location,personTags > %LogFile%
)
echo %log_string% >> %LogFile%

 

Log File Location and Rotation

The log file is rotated dail.  The script writes its output to SAFR Logs directory by default to a file called "safr_events_YYYY_MM_DD.csv

  1. Windows: C:\ProgramData\RealNetworks\SAFR\ares\logs\
  2. Linux: /Library/RealNetworks/SAFR/ares/

 

Test Script

Test the script to ensure it is working.  Open a DOS prompt, change to above directory and type following:

write_log.bat "one","two","three"

Check that the log file is in SAFR Actions Logs directory and that it contain a header line and the first row should included the 3 fields from above.

 

SAFR Actions Configuration File

These instructions assume you are creating a brand new SAFR Actions configuration file.  If you already have modified SAFR Actions configuration file to perform other actions, you will need to add the "rules" section to that file.  To simplify, these instructions also assume you are editing the configuration file instead of editing thru the application GUI. 

(warning) Make sure the SAFR Actions Application is not running before editing the configuration file.

On the machine where SAFR Actions is installed:

  1. Open SAFRActions.config which can be found in following locations:
    1. Windows: C:\ProgramData\RealNetworks\SAFR\ares\config\
    2. Linux: /Library/RealNetworks/SAFR/ares/
  2. Replace the contents with the sample config below
  3. Save and close the file
  4. Start SAFR Actions app
    1. Make sure new config is loaded.  If not, close SAFR and copy config again.
  5. Set the following in SAFR Actions Application (so the application performs encoding of the password for you)
  6. Change environment to your environment
    1. If your download portal is safr.int2.real.com, set to INT2 (Partner Cloud
    2. If your download portal is safr.real.com, set to Production
  7. Set userId, userPwd to your SAFR account
  8. Choose File > Save to load and save changes
    1. Changes should take effect immediately

SAFR Actions Sample Config File

SAFRActions.config File
{
  "directory": "main",
  "environment": "INT2",
  "rules": [
    {
      "event": { },
      "triggers": [
        {
          "actions": [
            "scripts\\write_log.bat \"#D\",\"#E\",\"#N\",\"#F\",\"#U\",\"#T\",\"#O\",\"#a\",\"#S\",\"#I\",\"#A\",\"#M\",\"#G\",\"#l\",\"#V\",\"#v\",\"#s\",\"#p\",\"#e\",\"#H\",\"#t\""
          ]
        }
      ]
    }
  ],
  "userId": "USERNAME",
  "userPwd": "PASSWORD"
}

 

(warning) Notes:

  • Make sure the system clock on SAFR Desktop and SAFR Actions is within a few seconds of each other or events may not trigger.  If needed adjust the date/time.
  • SAFR Action Service
    • On Windows, the SAFR Actions app runs as a service and you can exit the application and service will continue to listen for and react to events
      • Because it runs as a Windows Service, you can log out the user but do not let the computer sleep
    • On Mac, the SAFR Actions App must be kept running.  You can close the window but do not quit the application or log out the user
      • Alternatively, you can run SAFR Actions on Mac via command line.  See App Documentation for more details.

 

Customizing Log Output

SAFR Actions has many tokens that can be included in the log output.  

The are listed in SAFRActionsTokens.xlsx. SAFRActionsTokens.xlsx can be used to customize the columns generated by SAFR Actions.  Use this file to generate the header and action command as follows:

  1. Open SAFRActionsTokens.xlsx
  2. Set "Yes" on all fields you want included in the command output
  3. Edit Batch/Shell Script:
    1. Open write_log.bat or write_log.sh
    2. In the Excel file, copy the contents of the cell to the right of "Headers" and paste into write_log.bat file.
    3. Save and close the file.
  4. Open SAFR Actions Application:
    1. Find the "actions" row.
    2. Locate the next row that starts with the logging script name
    3. Replace the arguments that follow the script name as follows:
      1. In the Excel file, copy the contents of the cell to right of of "Tokens
      2. Replace existing tokens with new tokens (leave the script filename)
    4. Choose File > Save in SAFR Actions Application

Note: If editing the SAFRActions.config file directly, use the alternate tokens list in the Excel file that includes "\" escapes.

 

Test Log Output

The easiest way to test your SAFR Alerts is to use the mobile app to trigger an event.  You can install the application from your SAFR Download Portal.  Put the mobile application into "Stranger Monitoring" which generates events for unregistered people.

Test the alert as follows:

  1. Open SAFR Mobile App
  2. Put the mobile application into "Stranger Monitoring" mode (see mode at top)
  3. Hiding your face from the camera and then unhide and wait for face to be recognized
  4. Upon recognition
    1. A purple oval should be drawn around your face 
    2. A row should be added to the CSV File.

If you no row is added, check the SAFR Actions log to see if there are any clues there.   You can find the log here:

  • Windows: C:\ProgramData\RealNetworks\SAFR\ares\logs
  • Mac: /Library/RealNetworks/SAFR/ares

Possible issues and resolutions:

  • If you see errors with login correct account info
  • If you see "Too Late", machine running SAFR App and machine running SAFR Actions may not be set to same time.  Correct this and try again.
  • If you see other error, check message and attempt to correct or contact your RealNetworks.

 

Test Demographics Results and Optimize

The easiest way to test the demographics results is to use the events window.  Open the event window thru SAFR Desktop Tools / App menu and configure it to show the last 5 minutes of data.  Then have test subjects walk in front of the camera and check the reported age, gender and sentiment from that which you know from the event image.

Optimizaing

  • If age is under estimated, then recognition should be configured to be more strict - in particular for sharpness and contrast settings noted above.
  • if too few people are being recognized, try relaxing the center prose quality, sharpness or contrast settings noted above.

Get image quality and size for detected faces by choosing "Recognition Candidates" from the View menu.  When Recognition Candidates is enabled, following information is available:

The values reported above can be directly compared to the values in the Recognition preferences (Tools > Preferences > Recognition) to understand what settings are required for SAFR to attempt demographics analysis.  Only the recognition settings (circled in red) are relevant to demographics collection.  Merging and identification are relevant to learning faces and thus not applicable to this use case.  Bear in mind that if the quality or size values are too small, then accuracy will suffer.  In some cases its better to improve image quality by adjusting or upgrading camera than to lower quality bar for recognition.

Tuning for best results takes time.  Modify settings methodically and note results with each change.  If results go far off track, don't be afraid to reset to defaults and start over.

 

Reporting

There are 2 options for reporting demographics:

  1. Reports tab on the SAFR Web Console contains two reports:
    1. Dashboard - A snapshot of the current people count by age, gender and sentiment
    2. PDF Report - A PDF with number of people and number of appearances of people by age, gender and sentiment
  2. CSV File - A CSV File that contains one line per appearance of a person with attributes that allows grouping by person, age, gender and sentiment.

 

Appendixes

Appendix A - Camera Selection


A 1080p USB camera such as Logitech C920 HD Webcam 1080p Webcam is good enough if you can place SAFR Desktop near the camera.
A table top IP camera such as Hikvision DS-2CD2442FWD-IW also works fine.

 

Attachments:

Demographics.jpg (image/jpeg)
RecognitionCandidates.jpg (image/jpeg)
RecognitionSettings.jpg (image/jpeg)
Demographics.jpg (image/jpeg)