SAFR : Solution Guide - Visitor Announcement

Table of Contents

Summary

This use case describes how to use SAFR to perform Visitor Announcement using face detection and send notification emails.

Introduction

This document describes how to setup Visitor Announcement system such as sending a notification message to a receptionist or sales assistant when a registered person arrives at the entrance of your office.

In this tutorial we'll learn how to setup SAFR to perform the following useful functions:

  • Train the system to recognize a specific person
  • Set up cameras to detect registered persons
  • Configure the system to send a notification email based on known lists of persons

Background

About SAFR

SAFR from RealNetworks is a highly accurate facial recognition platform powered by artificial intelligence that works with IP cameras and readily available hardware to match faces in real time.  SAFR provides industry-leading performance to offer the following capabilities:

  • Scalable: can reliably match against millions of faces in under a second
  • Designed specifically to recognize faces in real-world conditions, including people in motion, in dim lighting, and at occluded angles
  • Non-literal matching: handles multiple variations, including masks, makeup, spectacles, hair styles, and aging
  • High tolerance to yaw, tilt, and rotation
  • Allows error rate tuning to minimize false rejections and false acceptances, or to optimize between the two

RealNetworks computer vision topped the list for “labeled faces in the wild” (LFW) achieving a 99.8% accuracy in a test conducted by the University of Massachusetts and 0.048 for Wild Faces FNMR according to the National Institute of Standards and Technology (NIST).

SAFR runs on Windows, Linux and Apple Mac computers as well as iOS and Android devices. SAFR software integrates cameras, door locks and alert systems with face recognition technology to enhance access control and security.  SAFR software is comprised of three primary components:  

  • SAFR Desktop and SAFR Mobile (Desktop and Mobile variants)  (Note: SAFR Edge is used for the installer name which includes SAFR Desktop and Actions. )
  • SAFR Actions (Windows, Linux or Mac)
  • SAFR Cloud Service or SAFR Server on Windows or Linux (Note: SAFR Platform is a set of products including SAFR Server, Desktop and Actions.)
    • On-Premises installs are important in cases where policy does not allow data to leave the site.
    • Cloud deployments are simpler and require less hardware but running costs are higher.

About Visitor Announcement

About Cameras

The goal in visitor announcement is to recognize an individual at the entrance or reception of a company or event venue.  The person may use moniker instead of face recognition.  Thus, cameras will need to handle face recognition or badge detection.  The system, however, works with decent IP-based or USB Cameras as the badge detection doesn't require any special condition.

About SAFR Desktop or Mobile App

The SAFR Desktop and Mobile Apps allow you to connect to a video feed and process video to perform face detection.  

About SAFR Actions

SAFR Action is an application designed to allow one or more actions to be taken when a recognition event occurs. For example SAFR actions could be used to send an email if a VIP person is observed at a particular camera.  SAFR Actions can be installed on any computer and connects to the SAFR platform to monitor for recognition events. Actions can be triggered based upon a wide range of attributes associated with each recognition event such as person type, id Class (threat, concern or stranger), site (building), source (camera) and more.  Actions can include things like sending email or messages.

This document describes how to use SAFR Actions to send a message upon detection of someone's face.

Planning your System

Before you get started, think about how you want to install SAFR.  The following components are needed for visitor announcement:

  • SAFR Platform - You can use the cloud or install the platform on-site.  In this tutorial we'll assume SAFR Cloud is used.
  • SAFR Desktop for processing video feeds from cameras
  • SAFR Actions for processing events and send notification emails via external SMTP server
  • One or more Cameras

Below provides an example of how the components are laid out.

The SAFR Edge and SAFR Actions could be run on a single computer but here they are shown separately.  If you have many cameras, you may end up needing to install more than one machine to process the video feeds.  Typically one machine can handle up to 8 cameras.

In this use case we assume one might want to lock doors and prevent access if a threat is detected.  That's entirely optional and not covered here.

 

Installation and Configuration

 

Installing your cameras

Before you start, you should have connected your USB camera to the PC running SAFR Desktop or have your IP cameras installed and connected to your Network.  Please see Appendix A for information on camera selection. Later we'll describe how to get the information for how to connect SAFR to your camera.

Install and Configure SAFR Desktop

 If you have not already done so, download (from SAFR Download Portal) and download "SAFR Edge" for your platform.

  • For Windows machines, the Full version is intended for machines with Nvidia video cards.  With an Nvidia card installed, significantly improved performance can be realized.
  • The "SAFR Edge" installer includes both SAFR Desktop and SAFR Actions Apps
  • After installation proceed to Configuring SAFR App below.

Configuring SAFR App

  1. Start the SAFR Desktop application from the Windows Start Menu or Mac Applications Folder
  2. Choose "Secure Access" from the dropdown in the in the top right of the main window
  3. Open SAFR Preferences Menu (SAFR > Preferences on Mac or Tools > Preferences on Windows) and configure as follows:
    1. Account Tab:
      1. Change User Site and User Source to a suitable value
    2. Event Tab:
      1. Check "Display Reply Message" of Listen for Event Replies section

Connect your cameras

 Open the SAFR Desktop App

  1. Start the SAFR App through the Windows Start Menu or Mac Applications folder
  2. On Mac or PC, select a camera from the dropdown in the top left of the main window if necessary

Install and Configure SAFR Actions

Run the SAFR Edge installer as described above in Install and Configure SAFR Desktop.

 

Importing faces

Importing faces from photos

At some point you may want to import faces from existing photos and add them to your SAFR people directory. SAFR can also detect multiple faces from a single photo (such as a large group shot, for example) as long as all detected faces meet the quality criteria you have specified in your SAFR recognition preferences. Imported faces are stored in the directory specified in the User Directory setting of your SAFR account preferences

To import a person from a photo, do the following:

  1. Open SAFR Desktop
  2. Choose File > Import Faces…
  3. Open one or more .jpg or .png files
    1. Any photo that has sufficient quality for recognition will show a blue oval around the face with option to click and add a name.
  4. Add person attributes as follows if desired
    1. Choose Tools > People (or SAFR > People on Mac)
    2. Sort by "Enrollment Data" and set Sort order to "Descending" in order to see the newly added entries
    3. Modify person attributes for any of the newly imported faces

Processing recorded Video files

Settings for each mode are the same as when processing live video feed with the exception of the following settings which are specifically used only when processing recorded video files

  • Recognition > Identity Recognition Threshold > Video
  • Events > Include Unidentified Events from Video

To recognize and extract faces from an MP4 or MOV file, perform the following steps:

  1. Open SAFR Desktop
  2. Choose File > Open
  3. Browse to any saved MP4 or MOV file.
    1. Any face within the video that has sufficient quality for recognition be added to the database.
  4. Add names as follows if desired
    1. Choose Tools > People (or SAFR > People on Mac)
    2. Sort by "Enrollment Data" and set Sort order to "Descending" in order to see the newly added entries
    3. Add name or modify person attributes for any of the newly imported faces

From SAFR Mobile App

For testing, you can use the SAFR Mobile App to register a person who will act as a test subject.  

  1. Go to the SAFR Download Portal and click on the link for SAFR Mobile App to install
  2. After installing the SAFR Mobile app, launch the application.  The app will show the front facing camera by default.
  3. Hold the camera so your face fills the screen and wait for the screen to display "Tap face to register"
  4. Tap the face and enter your name.  You are now registered.

 

 

Configure SAFR Desktop for Optimal Results

 

Recognition Camera Configuration

  1. Optimize camera location/orientation and adjust optical zoom to get at least 80 pixels high face (from chin to forehead) as noted above.  Larger is better.
  2. Adjust focus of the camera to the closes position where all faces are within the camera's field of view (i.e. ensure no one will be to the left or right of the camera view)
  3. Adjust SAFR Settings to be accepting of the image quality achievable.
  4. Ensure SAFR Desktop is in Stranger Monitoring mode (thus only recognizing)
  5. To assist in optimization, 

In Preferences > Detection:

  • Minimum searched face size: 60
  • Minimum required face size: 55 (filter out any background faces)

In Preferences > Recognition:

  • Minimum required center pose quality:
    • To allow recognition: 0
  • Minimum required face sharpness quality:
    • To allow recognition: 0
  • Minimum required face contrast quality:
    • To allow recognition: 0
  • Proximity Threshold Allowance: 0.13 (used to further relax the strictness of the matching)
    • Increase to 0.2 or 0.3 if matching % is too low

In Preferences > Events enable (check) following:

  • Report events
  • Include Unrecognizable Events from Camera (try to determine demographics even if insufficient quality to recognize identity)
  • Include Stranger Events (not enrolled / learned)

Turn following on only during allow for checking results in realtime thru Events Viewer window

  • Preserve Event Face Image (useful to compare with values given for age, gender and sentiment)
  • Preserve Event Scene Thumbnail Image (Useful to see which camera - can also use value of "Source")

Test Recognition Results and Optimize

The easiest way to test the recognition results is to use the events window.  Open the event window thru SAFR Desktop Tools / App menu and configure it to show the last 5 minutes of data.  Then have test subjects walk in front of the camera and check the results.

Optimizaing

  • if too few people are being recognized, try relaxing the center prose quality, sharpness or contrast settings noted above.

Get image quality and size for detected faces by choosing "Recognition Candidates" from the View menu.  When Recognition Candidates is enabled, following information is available:

The values reported above can be directly compared to the values in the Recognition preferences (Tools > Preferences > Recognition) to understand what settings are required for SAFR to attempt demographics analysis.  Only the recognition settings (circled in red) are relevant to demographics collection.  Merging and identification are relevant to learning faces and thus not applicable to this use case.  Bear in mind that if the quality or size values are too small, then accuracy will suffer.  In some cases its better to improve image quality by adjusting or upgrading camera than to lower quality bar for recognition.

Tuning for best results takes time.  Modify settings methodically and note results with each change.  If results go far off track, don't be afraid to reset to defaults and start over.

 

SAFR Actions Config File

These instructions assume you are creating a brand new SAFR Actions configuration file.  If you already have modified SAFR Actions configuration file to perform other actions, you will need to add the "rules" section to that file.  To simplify, these instructions also assume you are editing the configuration file instead of editing thru the application GUI. 

(warning) Make sure the SAFR Actions Application is not running before editing the configuration file.

On the machine where SAFR Actions is installed:

  1. Open SAFRActions.config which can be found in following locations:
    1. Windows: C:\ProgramData\RealNetworks\SAFR\ares\config
    2. Linux: /Library/RealNetworks/SAFR/ares/
  2. Replace the contents with the sample config below
  3. Save and close the file
  4. Start SAFR Actions app
    1. Make sure new configure file is loaded.  If not, close SAFR and copy configure file again.
  5. Set the following in SAFR Actions Application (so the application performs encoding of the password for you)
  6. Change environment to your environment
    1. If your download portal is safr.real.com, set to Production
  7. Set userId, userPwd to your SAFR account
  8. Choose File > Save to load and save changes
    1. Changes should take effect immediately

 

SAFR Actions Screen Config File

 

 

SAFR Actions Sample Config File

 

SAFRActions.config File
{
  "directory": "main",
   "emailDef": [
     {
       "attachments": [
         "cvos:\/\/obj\/#x\/face|event_photo.jpg"
       ],
       "label": "strangerDetected",
       "message": "<h1>Stranger Detected</h1><img src=\"cid:event_photo.jpg\" \/><br\/>Classification: #a<br\/>Site: #I<br\/>Camera: #S<br\/>",
       "recipients": [
         "xxxx@xxxx.com"
       ],
       "subject": "Stranger Detected"
     },
     {
       "attachments": [
         "cvos:\/\/obj\/#x\/face|event_photo.jpg"
       ],
       "label": "registeredDetected",
       "message": "<h1>Person Detected<\/h1><img src=\"cid:event_photo.jpg\" \/><br\/>Name: #N<br\/>Person Type: #T<br\/>Classification: #a<br\/>Site: #I<br\/>Camera: #S<br\/>",
       "recipients": [
         "xxxx@xxxx.com"
       ],
       "subject": "Person Detected"
     }
   ],
   "environment": "INT2",
   "rules": [
     {
       "event": {
         "hasPersonId": false,
         "idClass": [
             "Concern",
             "Threat"
         ],
         "site": "Realnetworks Australia",
         "source": "albert-home-windows"
       },
       "triggers": [
         {
           "actions": [
             "@emailSend strangerDetected"
           ],
           "daysOfWeek": [
             "Mon",
             "Tue",
             "Wed",
             "Thu",
             "Fri"
           ],
           "reply": {
             "message": "Stranger Detected."
           },
           "timesOfDay": [
             {
               "end": "24:00",
               "start": "00:00"
             }
           ]
         }
       ]
     },
     {
       "event": {
         "hasPersonId": true,
         "personType": [
            "staff",
            "admin",
            "customer",
            "guest"
         ],
       },
       "triggers": [
         {
           "actions": [
             "@emailSend registeredDetected"
           ],
           "daysOfWeek": [
             "Mon",
             "Tue",
             "Wed",
             "Thu",
             "Fri",
             "Sat",
             "Sun"
           ],
           "reply": {
             "message": "#a Detected!"
           },
           "timesOfDay": [
             {
               "end": "24:00",
               "start": "00:00"
             }
           ]
         }
       ]
     }
   ],
   "userId": "XXXXXX",
   "userPwd": "XXXXXX"
 }

(warning) Notes:

  • Make sure the system clock on SAFR Desktop and SAFR Actions is within a few seconds of each other or events may not trigger.  If needed adjust the date/time.
  • SAFR Action Service
    • On Windows, the SAFR Actions app runs as a service and you can exit the application and service will continue to listen for and react to events
      • Because it runs as a Windows Service, you can log out the user but do not let the computer sleep
    • On Mac, the SAFR Actions App must be kept running.  You can close the window but do not quit the application or log out the user
      • Alternatively, you can run SAFR Actions on Mac via command line.  See App Documentation for more details.

 

Configure Outbound Email Server

Configure credentials to allow SAFR to send email through your email server.

  1. Get an SMTP Server account you can use for sending emails.  If using Gmail you'll need to set that account 
  2. Open Tools > Email Server Config - You'll enter the SMTP
    1. You will be prompted to log into SAFR Cloud if you have not already
      1. Choose "SAFR Partner Cloud"
      2. Enter SAFR cloud account user/pwd
      3. Click Connect
    2. In the Configure Email Server dialog, enter all requested info (read from your SAFR cloud account if previously filled out elsewhere)
      1. Sender Email - Email username for SMTP Account (e.g. me@gmail.com)
      2. Email Password - Password for above account
      3. Sender Name: Display name for the From line
      4. From Email Address - Email for the From line (will not work with all email servers  - if not supported then the Sender Email is populated)
      5. Email Server Address (e.g. smtp.gmail.com)
      6. Email Server Port: (587 for default for SMTP)
    3. Note:  If using Gmail, you may need to change Gmail to use "Less secure app access".  If you receive an error in the SAFRActions log "EMAIL SEND ERROR: <date> 500 Report server internal error" this may be the reason.
      1. To confirm, log into your gmail account you have configured to send SMTP.  Look for an email with title "Critical security alert".  
        1. Message body will state "Someone just used your password to try to sign in to your account from a non-Google app."
        2. Clicking "Check activity" and choosing Yes to "Were you recently prevented from signing in to your Google Account?" will show a message saying "Less secure app was blocked".
      2. To resolve this
        1. Log into your Google Account (Click on your avatar in upper right and choose "Google Account")
        2. Go to Security > Less Secure App Access and turn "Allow less secure apps" to "ON".

 

Appendixes

Appendix A - Camera Selection


A 1080p USB camera such as Logitech C920 HD Webcam 1080p Webcam is good enough if you can place SAFR Desktop near the camera.
A table top IP camera such as Hikvision DS-2CD2442FWD-IW also works fine.

 

Attachments:

moniker.config (application/octet-stream)
moniker.config (application/octet-stream)
SAFRActions.config (application/octet-stream)
SAFRActions.config (application/octet-stream)
SAFRActions.config (application/octet-stream)
VisitorNotificationArchitecture.jpg (image/jpeg)
email_def.PNG (image/png)
rule1.PNG (image/png)
rule2.PNG (image/png)