SAFR : Solution Guide - Time Clock

Introduction

This document describes how to use SAFR to track work time on a job site.  It tracks time when people enter a job site and leave the site and provides examples of basic reports to report on time on the job site.  The excel version of the report tracks only total time between very first appearance of a person to last appearance each day.  The SQL version tracks the same plus decrements time when user was off site during the day for more than a pre-defined period of time.  In all cases 

  • Measure time in and time out for each worker by day
    • Measured by very first and very last recognition each day.  If either is missing (but one exists), assume 8 hours)
  • All workers will be registered (if not then guard will register them before entering)
  • Ignore multiple recognitions within 1 min (use only the 1st or last recognition) to allow for lingering in front of camera
  • Ignore recognitions if reentry within 10 min (to allow for accepting deliveries)
  • Decrement time if exit and reentry are within 2 hours
  • Log results as CSV

This solution uses SAFR to record any tine a person leaves or entered a facility.  To do this, each door is equipped with two cameras - an entry and exit camera.  These cameras will be used with SAFR Desktop to record entry and exit times of each individual and those times will be logged using SARF Actions.

Planning

The following diagram demonstrates the layout for this solution.

On each entry point, 2 cameras are required.  One camera should face people entering the facility and a 2nd should face people existing the facility.  SAFR Desktop is run on a computer to process the feed for each camera.  The cameras should be named so that its function is clear.  The camera capturing people entering should have "Entrance" in its name while the camera capturing people existing should have "Exit" in its name.  Other parts of the name should be representative of the door location.  So for example, above cameras would be labeled as shown.

If equipped with electrified locks, each door can be optionally controlled to open/close upon successful detection of a recognized person.  A registration kiosk (optional) can be used to add new people to the database.  Alternatively if all staff photos are already maintained, users can be added by loaded their photos into the SAFR Desktop application.

Optional Actions

  • If desired, a light can also be triggered upon entry.  This could be useful to ensure users are recognized as the enter and exit.  Please see tutorial on triggering lights with SAFR for more information.
  • An email or SMS can also be sent upon certain events.  For example, if an unrecognized person or a VIP enters.  See tutorial on sending notifications with SAFR for more information.

System Requirements

Hardware

  • Windows or Mac meeting minimum system requirements to run SAFR Desktop and SAFR Actions Apps

Software

  • SAFR Desktop for Windows or Mac
  • SAFR Actions for Windows or Mac
  • Excel or SQL Database (example provided here was done out of AWS Redshift running Postgress.  Other DB options may require modification to the scripts)

Installation

Installing your cameras

Before you start, you should have your IP cameras installed and connected to your Network.  Have your IP Address for the camera available and the RTSP URL for connecting to your camera.  The URL can be found in the administration interface for the camera or try googling for your make and model.  

Cameras should be placed so that faces are at least 220 pixels high for this application to get the best possible recognition results.  Placement should be such that face is not easily obscured but not too high to prevent occlusion by hats.  You may want to ask users to look directly into the camera as they enter.

Install and Configure SAFR Desktop

 If you have not already done so, download (from SAFR Download Portal) and download "SAFR Edge" for your platform.

  • For Windows machines, the Full version is intended for machines with Nvidia video cards.  With an Nvidia cad installed, significantly improved performance can be realized.
  • The "SAFR Edge" installer includes both SAFR Desktop and SAFR Actions Apps
  • After installation proceed to Configuring SAFR App below.

Configuring SAFR App

  1. Start the SAFR Desktop application from the Windows Start Menu or Mac Applications Folder
  2. Choose "Enrolled and Stranger Monitoring" from the dropdown in the in the top right of the main window (these modes have default actions triggered when a use is recognized)
  3. Open SAFR Preferences Menu (SAFR > Preferences on Mac or Tools > Preferences on Windows) and configure as follows:
    1. Account Tab:
      1. Change User Site and User Source to a suitable value
        1. For Entrance, include "Entrance" in the name such as "FrontEntrance"
        2. For Exit, include "Exit"
    2. Events Tab:
      1. If desired, check which "Listen for event replies" to see a message on the SAFR Desktop screen when an event is triggered
    3. Recognition Tab: If you have having problems being recognized, try adjusting following in the Recognition Tab:
        Minimum required face size / To allow identification:  set to 120 (this allows recognition when face image is <220 default)
        Minimum required center pose quality / To allow recognition:  Increase to allow non-frontal recognition
        Minimum required face contrast quality / To allow recognition:  Increase if you have a very bright background 
        Identify recognition threshold / Camera:  Increase value to allow more lenient recognition

Connect your cameras

 Open the SAFR Desktop App

  1. Start the SAFR App through the Windows Start Menu, Mac Applications folder or iOS/Android Desktop
  2. If your cameras support ONVIF, you can simply select the camera from the dropdown list in SAFR Desktop
  3. If not, obtain the RTSP URL as noted above and add cameras via the "Camera" tab in Preferences
    1. Close the preferences and select the newly added camera from the dropdown list
  4. Connect remaining cameras by choosing "File > New" from SAFR Desktop and connecting for each camera you have
(warning) When closing SAFR use "Quit SAFR" from the File Menu. Closing SAFR via the Window Close button will only close the window and you will loose state (settings and connected camera) for that window.

Learning/Register a Person

For best recognition results its important to start out with a good quality photo to train the system on each identity.   Below describes using different methods for adding user identities.  In add cases its critical that you start out with high quality photos at least 220 pixels high from chin to forehead.

From Photos

To learn a person from a photo of that person, do the following:

  1. Open SAFR Desktop
  2. Choose File > Open
  3. Open one or more photos
    1. Any photo that has sufficient quality for recognition will show a purple oval around the face with option to click and add a name.
  4. In each learned photo, type a name if desired.

From SAFR Mobile App

For testing, you can use the SAFR Mobile App to register a person who will act as a test subject.  

  1. Go to the SAFR Download Portal and click on the link for SAFR Mobile App to install
  2. After installing the SAFR Mobile app, launch the application.  The app will show the front facing camera by default.
  3. Hold the camera so your face fills the screen and wait for the screen to display "Tap face to register"
  4. Tap the face and enter your name.  You are now registered.

 

Configure SAFR Desktop for Optimal Results

 

Recognition Camera Configuration

  1. Optimize camera location/orientation and adjust optical zoom to get at least 80 pixels high face (from chin to forehead) as noted above.  Larger is better.
  2. Adjust focus of the camera to the closes position where all faces are within the camera's field of view (i.e. ensure no one will be to the left or right of the camera view)
  3. Adjust SAFR Settings to be accepting of the image quality achievable.
  4. Ensure SAFR Desktop is in Stranger Monitoring mode (thus only recognizing)
  5. To assist in optimization, 

In Preferences > Detection:

  • Minimum searched face size: 60
  • Minimum required face size: 55 (filter out any background faces)

In Preferences > Recognition:

  • Minimum required center pose quality:
    • To allow recognition: 0
  • Minimum required face sharpness quality:
    • To allow recognition: 0
  • Minimum required face contrast quality:
    • To allow recognition: 0
  • Proximity Threshold Allowance: 0.13 (used to further relax the strictness of the matching)
    • Increase to 0.2 or 0.3 if matching % is too low

In Preferences > Events enable (check) following:

  • Report events
  • Include Unrecognizable Events from Camera (try to determine demographics even if insufficient quality to recognize identity)
  • Include Stranger Events (not enrolled / learned)

Turn following on only during allow for checking results in realtime thru Events Viewer window

  • Preserve Event Face Image (useful to compare with values given for age, gender and sentiment)
  • Preserve Event Scene Thumbnail Image (Useful to see which camera - can also use value of "Source")

 

Configure SAFR Actions

Use following procedure to configure SAFR Actions to write log file.

Add Log Script

Add the logging script to SAFR Actions script directory. This script logs demographics each time a person is detected.

Scripts directory is in following location:

  • Windows: C:\Program Files\RealNetworks\SAFR\ares\scripts
  • Linux: /Library/RealNetworks/SAFR/ares/scripts

Log to File Sample Script

Write Log Batch file
@echo off
chcp 65001 2>nul >nul
REM Replace ? with null string to prevent for loop from interpreting ? as wildcard and skipping (because no matching file).
set args=%*
set args=%args:?=%
REM Set delayed expansion so vars expanded in each iteration of for loop using !var!
SETLOCAL EnableDelayedExpansion
REM Create filename in format of YY_MM_DD for log file rolling
    REM GET localtime from windows (LocalDateTime=YYYYMMDDHHMMSS.ms)
    for /f "tokens=2 delims==" %%G in ('wmic os get localdatetime /value') do set DT=%%G
    set FileName=!DT:~0,4!_!DT:~4,2!_!DT:~6,2!
    set FmtDate=!DT:~0,4!-!DT:~4,2!-!DT:~6,2!
    set FmtTime=!DT:~8,2!:!DT:~10,2!:!DT:~12,2!
Set LogFile="C:\Users\Public\Documents\safr_events_%FileName%.csv"
set log_string="%FmtDate%","%FmtTime%"
REM Loop and process each argument (not currently used)
REM for %%v in (%%args%%) do (
REM    CALL set log_string=%%log_string%%,"%%~v"
REM )
REM Append arguments list from SAFR Actions already in Quoted CSV format
set log_string=%log_string%,%*
REM Print log line
if not exist %LogFile% (
    REM The following list should match tags added to SAFRActions.config.  Use SAFRActionsVars.xls to generate list
    echo date,time,person_id,person_ext_id,name,first_name,last_name,person_type,company,idClass,source,site,age,sentiment,gender,similarityScore,event_type,event_id,event_epoch_start_time,validation_phonea,validation_email,home_location,personTags > %LogFile%
)
echo %log_string% >> %LogFile%

 

Log File Location and Rotation

The log file is rotated daily.  The script writes its output to SAFR Logs directory by default to a file called "safr_events_YYYY_MM_DD.csv

  1. Windows: C:\ProgramData\RealNetworks\SAFR\ares\logs\
  2. Linux: /Library/RealNetworks/SAFR/ares/

 

Test Script

Test the script to ensure it is working.  Open a DOS prompt, change to above directory and type following:

write_log.bat "one","two","three"

Check that the log file is in SAFR Actions Logs directory and that it contain a header line and the first row should included the 3 fields from above.

 

SAFR Actions Configuration File

These instructions assume you are creating a brand new SAFR Actions configuration file.  If you already have modified SAFR Actions configuration file to perform other actions, you will need to add the "rules" section to that file.  To simplify, these instructions also assume you are editing the configuration file instead of editing thru the application GUI. 

(warning) Make sure the SAFR Actions Application is not running before editing the configuration file.

On the machine where SAFR Actions is installed:

  1. Open SAFRActions.config which can be found in following locations:
    1. Windows: C:\ProgramData\RealNetworks\SAFR\ares\config\
    2. Linux: /Library/RealNetworks/SAFR/ares/
  2. Replace the contents with the sample config below
  3. Save and close the file
  4. Start SAFR Actions app
    1. Make sure new config is loaded.  If not, close SAFR and copy config again.
  5. Set the following in SAFR Actions Application (so the application performs encoding of the password for you)
  6. Change environment to your environment
    1. If your download portal is safr.int2.real.com, set to INT2 (Partner Cloud
    2. If your download portal is safr.real.com, set to Production
  7. Set userId, userPwd to your SAFR account
  8. Choose File > Save to load and save changes
    1. Changes should take effect immediately

SAFR Actions Sample Config File

SAFRActions.config File
{
  "directory": "main",
  "environment": "INT2",
  "rules": [
    {
      "event": { },
      "triggers": [
        {
          "actions": [
            "scripts\\write_log.bat \"#D\",\"#E\",\"#N\",\"#F\",\"#U\",\"#T\",\"#O\",\"#a\",\"#S\",\"#I\",\"#A\",\"#M\",\"#G\",\"#l\",\"#V\",\"#v\",\"#s\",\"#p\",\"#e\",\"#H\",\"#t\""
          ]
        }
      ]
    }
  ],
  "userId": "USERNAME",
  "userPwd": "PASSWORD"
}

 

(warning) Notes:

  • Make sure the system clock on SAFR Desktop and SAFR Actions is within a few seconds of each other or events may not trigger.  If needed adjust the date/time.
  • SAFR Action Service
    • On Windows, the SAFR Actions app runs as a service and you can exit the application and service will continue to listen for and react to events
      • Because it runs as a Windows Service, you can log out the user but do not let the computer sleep
    • On Mac, the SAFR Actions App must be kept running.  You can close the window but do not quit the application or log out the user
      • Alternatively, you can run SAFR Actions on Mac via command line.  See App Documentation for more details.

 

Customizing Log Output

SAFR Actions has many tokens that can be included in the log output.  

The are listed in SAFRActionsTokens.xlsx. SAFRActionsTokens.xlsx can be used to customize the columns generated by SAFR Actions.  Use this file to generate the header and action command as follows:

  1. Open SAFRActionsTokens.xlsx
  2. Set "Yes" on all fields you want included in the command output
  3. Edit Batch/Shell Script:
    1. Open write_log.bat or write_log.sh
    2. In the Excel file, copy the contents of the cell to the right of "Headers" and paste into write_log.bat file.
    3. Save and close the file.
  4. Open SAFR Actions Application:
    1. Find the "actions" row.
    2. Locate the next row that starts with the logging script name
    3. Replace the arguments that follow the script name as follows:
      1. In the Excel file, copy the contents of the cell to right of of "Tokens
      2. Replace existing tokens with new tokens (leave the script filename)
    4. Choose File > Save in SAFR Actions Application

Note: If editing the SAFRActions.config file directly, use the alternate tokens list in the Excel file that includes "\" escapes.

 

Test Log Output

The easiest way to test your SAFR Alerts is to use the mobile app to trigger an event.  You can install the application from your SAFR Download Portal.  Put the mobile application into "Stranger Monitoring" which generates events for unregistered people.

Test the alert as follows:

  1. Open SAFR Mobile App
  2. Put the mobile application into "Stranger Monitoring" mode (see mode at top)
  3. Hiding your face from the camera and then unhide and wait for face to be recognized
  4. Upon recognition
    1. A purple oval should be drawn around your face 
    2. A row should be added to the CSV File.

If you no row is added, check the SAFR Actions log to see if there are any clues there.   You can find the log here:

  • Windows: C:\ProgramData\RealNetworks\SAFR\ares\logs
  • Mac: /Library/RealNetworks/SAFR/ares

Possible issues and resolutions:

  • If you see errors with login correct account info
  • If you see "Too Late", machine running SAFR App and machine running SAFR Actions may not be set to same time.  Correct this and try again.
  • If you see other error, check message and attempt to correct or contact your RealNetworks.

 

Test Recognition Results and Optimize

The easiest way to test the recognition results is to use the events window.  Open the event window thru SAFR Desktop Tools / App menu and configure it to show the last 5 minutes of data.  Then have test subjects walk in front of the camera and check the results.

Optimizaing

  • if too few people are being recognized, try relaxing the center prose quality, sharpness or contrast settings noted above.

Get image quality and size for detected faces by choosing "Recognition Candidates" from the View menu.  When Recognition Candidates is enabled, following information is available:

The values reported above can be directly compared to the values in the Recognition preferences (Tools > Preferences > Recognition) to understand what settings are required for SAFR to attempt demographics analysis.  Only the recognition settings (circled in red) are relevant to demographics collection.  Merging and identification are relevant to learning faces and thus not applicable to this use case.  Bear in mind that if the quality or size values are too small, then accuracy will suffer.  In some cases its better to improve image quality by adjusting or upgrading camera than to lower quality bar for recognition.

Tuning for best results takes time.  Modify settings methodically and note results with each change.  If results go far off track, don't be afraid to reset to defaults and start over.

 

Reporting

There are 2 options for reporting time:

  1. Excel - Use Sample Excel Pivot table generated from the output CSV File.  The CSV File that contains one line per appearance of a person with attributes that allows grouping by person time.  From Excel its only possible to report overall time based on min time observed at entry per day and max time observed at exit per day.
  2. SQL - Use Sample SQL designed for AWS PostGress DB (using RDS Service) to build a report from the CSV file.  The CSV file is ingested and use provided sample SQL to generate example reports as follows:
    1. Overall time on site - The duration represented between the 1st entry for the day and the last entry for the day.  Before computing duplicate detections occurring when person lingers in front of the camera are combined into a single event.  This time is reported as a duration in a separate table / report by person and by day
    2. Time on Lunch break - The duration represented between a configurable time out and return time in greater than 30 minutes (configurable).  This time is reported as a duration in a separate table / report by person and by day and can be subtracted from overall time reported above.
    3. Ignore brief exit / entry -  Any occurrences of exit and subsequent entry are ignored if less than 20 minute (configurable).  This time is reported as a duration in a separate table / report by person and by day.

(warning) It should be noted that the provided reports are provided as-is and not intended as working supported code.  These are examples provided in hopes of reducing effort for integrating a new system but are not supported by RealNetworks.  Support and maintenance of the provided samples are entirely the responsibility of the recipient.

Excel Report Sample

The Excel report sample consists of 2 tabs:

  • timelog - The "timelog" data tab is where the raw log data from SAFR log file is added.  Data should be cleared before adding to the tab.  New data should be pasted in as text to avoid overwriting the data range.
  • report - The 'report' tab includes a pivot table and 2 calculated columns that computes the hours per person per day.

The report tab uses an Excel Pivot Table that references the timelog tab for its data.  The reference intentionally includes extra rows so that larger time logs can be pasted in and the Pivot Table will not need to be re-generated.  This this process requires the manual step of pasting new data from CSV but has the advantage of simplicity if a reporting infrastructure is not already available.

Example CSV Source and Sample Excel to generate source manually:

events_log_source.xlsx
events_log.csv

Process

The process for updating the report with new data is as follows:

  1. Copy new CSV file from SAFR Actions logs directory to local machine
    (warning) Do not open the CSV file from the server because Excel will block new writes toe the file while it is open. 
  2. Open new CSV file, click anywhere in the table of data and press Control-A (this selects all rows) then copy the selection
  3. Open events_log_report_examlpe XLS 
  4. Select cell A1 of the "timelog" tab
  5. Paste the data copied from the CSV file in Step 2
  6. Go to the report tab.  Right click the pivot table in Columns A-C and choose "Refresh".
  7. If needed, extend the formulas in columns E and F to produce results for all pivot table rows.

 

About the Report

The Excel Pivot Table in 'report' tab computes the max and min time each day by person.  It groups by month and day.  This does not take into account lunch or any other entry/exit during the day.  Only the smallest entry observed and largest entry observed is computed.

The Hours column reports the total hours worked by a single person.  The Dur column should be ignored as it is used to compute intermedia durations. This is needed to handle conditions where no entry or no exit was reported (in which case the Hours column defaults to 8 hours)

If anyone was absent the entire day, they are not included in the report.  If anyone is not recorded on entry or exit, then the report defaults those users to 8 hours (instead of reporting 0 hours).  Any other default can be used.

 

Sample XLS:  events_log_report_example.xlsx

SQL Report Sample

The SQL report sample demonstrates how, with good reporting tools, more information can be determined from the data reported by SAFR. 

To use the SQL report sample, an SQL database is required.  In this example, AWS RDS (using Postgress) is used.  Other databases can be used but some modification may be required.  Any updates to the script are the responsibility of the recipient and RealNetworks makes no obligation to provide update.

Process

Again, this process is described assuming use of AWS RDS (Postgress) and S3 to load data to the database.  

  1. Move the CSV file to your S3 bucket
  2. Modify the script to reflect your AWS S3 credentials and the path of the file uploaded to S3
  3. Run the two functions to load them into memory:
    1. real_mongo_timestamp - Convenience function to convert epoch timestamp reported by SAFR Actions to date object
    2. to_hhmmss - Convenience function to format time interval as string
  4. Create schema (rename as appropriate)
  5. Run Query 1 to import data from S3.  If an error results during Query 1, run Query 2 to check for errors.
  6. Run Query 3 to convert epoch timestamp to date object
  7. Query 4 is provided as informational to demonstrate exporting data back to S3
  8. Run Query 5 to remove successive dups (i.e. multiple detections if they linger in front of the camera)
  9. Run Query 6 Get first and last records by day and compute total worktime
  10. Run Query 7 Removes the 1st and last entry for each user each day
  11. Run Query 8 Create additional columns duration and unique grouping id
  12. Run Query 9 to compute total break time

 

About the Report

  1. Query 5 Remove successive dups - (i.e. multiple detections if they linger in front of the camera)
  2. Query 6 First/last and total worktime by day
  3. Query 8 Additional columns duration and unique grouping id
  4. Query 9 - Total Break time
  5. Query 10 (not available yet) - Total worktime - total breaktime

 

Sample SQL: events_reports.sql

 

Attachments:

Time_Tracking_Architecture.png (image/png)
Time_Tracking_Architecture.png (image/png)
Time_Tracking_Architecture.png (image/png)
events_reports.sql (application/x-upload-data)
events_log_report_example.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
events_log_source.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
events_log.csv (text/csv)
5_20_19__4_30_PM.jpg (image/jpeg)
Text_Import_Wizard_-_Step_3_of_3.jpg (image/jpeg)
Import_Data_Submit.jpg (image/jpeg)
Text_Import_Wizard_-_Step_3_of_3.jpg (image/jpeg)
Text_Import_Wizard_-_Step_2_of_3.jpg (image/jpeg)
Text_Import_Wizard_-_Step_1_of_3.jpg (image/jpeg)
Import_Data_Submit.jpg (image/jpeg)
External_Data_Range_Properties.jpg (image/jpeg)