Panoptes Actions

Analyze Action

The Analyze action instructs panoptes to run a face recognition action on an image or a set of images for the purpose of finding out what kind of center pose quality, contrast, and sharpness values can be expected for faces. Notethat panoptes does not learn those faces. Neither does it attempt to match those faces against face information stored in an Identity Database. It tries to find as many faces as it can in the images, and it prints information about the found faces to the console.

Use this action on a set of images you have never processed before to get an idea about what center pose quality (cpq), contrast (cq), sharpness (sq) and face size values you can expect. You can then use those values as a basis for choosing the min-cpq, min-cq, min-sq, and min-size values for the Learn and Recognize actions.

If the fd-enable-liveness parameter is set to true, RGB liveness detection is enabled and the face detector will produce two additional values for the detected face: a liveness score and a liveness status. RGB liveness detection can be used only with high sensitivity face detector. (i.e. with -Cfd-type=high_sensitivity)

Panoptes uses the settings below in analyze mode which can be overridden via the -C command line options. (e.g. -Cmin-search-size, -Cfd-enable-liveness, etc.)

Setting Default Range Description
fd-type normal normal, high_sensitivity Face detector to use. high_sensitivity is slower than normal, but has better accuracy, especially on masked faces.
fd-enable-liveness false true, false When enabled, RGB liveness detection will be used. (Available only with high_sensitivity face detector.)
fd-liveness-mode min none, last, avg, min, max Liveness detection mode used to calculate liveness response in multi-model setup.
min-search-size 40 15 - 720 Minimum face width/height, in pixels, used for face search with the normal face detector.
fd-high-sensitivity-resolution 640x480 1280x720, 720x1280, 640x480, 640x640, 480x640, 480x480, 480x320, 320x480, 320x240, 320x320, 240x320, 240x240, 128x128 Resolution used for face search with the high_sensitivity face detector.

Below is an invocation of panoptes that instructs it to analyze an image and print recognition information about every person found in the image:

> panoptes analyze image_with_person.jpeg

Note: The path to the image file/image directory must immediately follow the action word.

Invocation of panoptes that will analyze and image and additionally calculate liveness score for every person found in the image. Liveness mode is changed to the average of the Texture and Context models:

> panoptes analyze image_with_person.jpeg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true -Cfd-liveness-mode=avg

Examples

The directory sample_images contains sample images that can be used to test RGB liveness detection.

RGB liveness detection achieves best results if a high resolution image is provided for liveness detection. Invocation of panoptes that will provide low resolution image for face detection and high resolution image for RGB liveness detection:

> panoptes analyze-highres ../sample_images/Live_LowRes_240x320.jpg ../sample_images/Live_HighRes_1920x2560.jpg  -Cfd-type=high_sensitivity -Cfd-enable-liveness=true

If only Live_LowRes_240x320.jpg was provided to the Analyze action, RGB liveness would not be calculated because the face size wouldn't satisfy the default requirements. For all of the following examples, RGB liveness detection and face detection are done on a single image, but if possible RGB liveness detection should be provided with a higher resolution image.

Live person images are expected to return:

> panoptes analyze ../sample_images/Live_1.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Live_2.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true

Live person images that don't satisfy face image quality requirements:

> panoptes analyze ../sample_images/Live_Blur.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Live_MinFaceSize.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Live_FaceContext.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Live_CPQ.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true

Fake person images:

> panoptes analyze ../sample_images/Fake_Paper_1.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Fake_Paper_2.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true
> panoptes analyze ../sample_images/Fake_Device_1.jpg -Cfd-type=high_sensitivity -Cfd-enable-liveness=true

Learn Action

This action instructs panoptes to learn a face/person/identity from a JPEG image or a set of persons from a directory with JPEG images. Panoptes focuses on the person with the largest projected face in the image and learns only this person. You may specify metadata for the person. This metadata is stored together with the person's face signature in the person database. Panoptes assigns a unique identifier (GUID) to the person. This GUID can be used with other actions to refer to this person.

Panoptes uses a default set of face recognizer settings, but you can override the settings via -C command line argument described previously. The following table lists the supported settings and their default values:

Setting Default Range Description
min-cpq 0.59 0 - 1 Required minimum center pose quality.
min-cq 0.45 0 - 1 Required minimum contrast quality.
min-sq 0.45 0 - 1 Required minimum sharpness quality.
min-confidence 1 0 - 1 Required minimum confidence.
min-search-size 40 15 - 720 Minimum face width/height in pixels used for face search during detection.
min-size 146 1 - 720 Minimum face width/height in pixels for recognition acceptance.
select largest any, largest, at-location Face selection policy.
loc-x 0.5 0 - 1 X-coordinate of the location if selection policy is at-location.
loc-y 0.5 0 - 1 Y-coordinate of the location if selection policy is at-location.
fd-type normal normal, high_sensitivity Face detector to use. high_sensitivity is slower than normal, but has better accuracy, especially on masked faces.
fd-enable-liveness false true, false When enabled, RGB liveness detection is used. (Only available with high_sensitivity face detector.)
fd-high-sensitivity-resolution 640x480 1280x720, 720x1280, 640x480, 640x640, 480x640, 480x480, 480x320, 320x480, 320x240, 320x320, 240x320, 240x240, 128x128 Resolution used for face search with the high_sensitivity face detector.
fr-model regular regular, masked Face recognition model to use. The masked model is optimized for recognition on masked faces.
fr-optimization speed speed, accuracy Face recognition model optimized for either speed or accuracy.

Below is an invocation of panoptes, instructing it to learn the person visible in the image and to add the metadata name with the value Foo to the person.

> panoptes learn image_with_person.jpeg -Dname=Foo

Note: As mentioned previously, the path to the image file/image directory must immediately follow the action word.

Recognize Action

This action instructs panoptes to recognize a person or a set of persons. You may pass a path to a single JPEG image or a path to a directory containing one or more JPEG images. Panoptes runs a face recognition operation on every image to extract person information, and matches up this person information with the person information stored in the person database. It then lists the unique person ID (GUID) and metadata for every recognized person.

Panoptes uses a default set of face recognizer settings, but you can override the settings. The following table lists the supported settings and their default values:

Setting Default Range Description
min-cpq 0.0 0 - 1 Required minimum center pose quality.
min-cq 0.0 0 - 1 Required minimum contrast quality.
min-sq 0.0 0 - 1 Required minimum sharpness quality.
min-confidence 0.93 0 - 1 Required minimum confidence.
min-search-size 40 15 - 720 Minimum face width/height in pixels used for face search during detection.
min-size 0 1 - 720 Minimum face width/height in pixels for recognition acceptance.
select largest any, largest, at-location Face selection policy.
loc-x 0.5 0.5 X-coordinate of the location if selection policy is at-location.
loc-y 0.5 0.5 Y-coordinate of the location if selection policy is at-location.
fd-type normal normal, high_sensitivity Face detector to use. high_sensitivity is slower than normal, but has better accuracy, especially on masked faces.
fd-enable-liveness false true, false When enabled, RGB liveness detection is used. (Only available with high_sensitivity face detector.)
fd-high-sensitivity-resolution 640x480 1280x720, 720x1280, 640x480, 640x640, 480x640, 480x480, 480x320, 320x480, 320x240, 320x320, 240x320, 240x240, 128x128 Resolution used for face search with the high_sensitivity face detector.
fr-optimization speed speed, accuracy Face recognition model optimized for either speed or accuracy.
fr-model regular regular, masked Face recognition model to use. The masked model is optimized for recognition on masked faces.

Below is an invocation of panoptes, instructing it to print out all recognized persons visible in the given JPEG image. This invocation also shows how to override one of the recognizer settings with a custom value:

> panoptes recognize image_with_person.jpeg -Cmin-confidence=0.86

Below is an invocation of panoptes which instructs it to print out all recognized persons visible in the given JPEG image. This invocation also shows how to override face detector and face recognition default settings:

> panoptes recognize image_with_person.jpeg -Cfd-type=high_sensitivity -Cfd-high-sensitivity-resolution=128x128 -Cfr-model=masked -Cfr-optimization=accuracy -Cfd-enable-liveness=true

If a person was learned using the regular/masked face recognition model, the same model should be used when attempting to recognize the face. Different face recognition models produce different facial signatures for the same face and should not be compared. If a different model is used when attempting to recognize a face, no error will be returned but the person is unlikely to be recognized.

When using the high_sensitivity face detector, this message can occur:

ERROR: OpenCL library not loaded - dlopen failed: library "libOpenCL-pixel.so" not found
ERROR: Falling back to OpenGL

This error is actually only a warning and can be ignored.

Benchmark Action

This action instructs panoptes to instantiate all eSDK capabilities (i.e. face detection, face recognition, emotion estimation, age estimation, gender detection, and object/person detection), measure the duration of each process, and generate a benchmark output before ending execution. You may pass a path to a single JPEG image or a path to a directory which contains one or more JPEG images.

Face recognition is configurable in the same way as it is in the Recognize action, described above. In addition, the object/person detector can be configured by specifying the model type (object detector or person detector), model size (normal or tiny), input size (normal, large, or small) and precision.

When running panoptes benchmark for the first time, it may take a while (depending on the device, up to 10 minutes) for benchmark to start as all the models have to be optimized before the first run.

Setting Default Options Description
model-size Normal Normal, Tiny Model size. The tiny model is smaller and faster but has lower accuracy.
input-size Normal Small, Normal, Large Size to which the input image is resized to.
Normal = 416px
Small = 320px
Large = 608px
model-precision Half Float, Half, Int8 Precision of the model. The "Float" value represents "float32", while the "Half" value represents "float16".

Below is an invocation of panoptes which will benchmark the eSDK on a single image with a person detector using fp16 precision:

> panoptes benchmark image_with_person.jpeg -Omodel-type=p -Omodel-precision=fp16

Interpreting Confidence Values

Confidence Value Interpretation
>= 1.0 Certain match. (Values > 1.00 are possible indicating even greater certainty.)
[0.93 1.0> Close match.
[0.86 0.93> Possible match with low confidence.
[0.82 0.86> Similar face with no confidence of match.
[0.00 0.82> Different face.

Note: As mentioned previously, the path to the image file/image directory must immediately follow the action word.

List Action

The List action instructs panoptes to display the person ID and metadata of all persons in the person database. You can optionally specify a known person ID to limit the display to just this person.

Remove Action

The Remove action takes a person ID and instructs panoptes to delete all information about this person from the person database.

Edit Action

The Edit action instructs panoptes to edit the metadata of a person, specified by their person ID. Metadata key-value pairs can be added or updated with the -D switch. A metadata key-value pair may be deleted with the -R switch.

Encrypt Action

The Encrypt action instructs panoptes to encrypt the person store database file with the provided password. This action also allows you to change the password of an existing database file. Note that the PANOPTES_DB_PWD environment variable must be set to the old password so that panoptes is able to decrypt the existing database file before encrypting it with the new password.

Important: Don't forget to update the PANOPTES_DB_PWD environment variable with the new password after the Encrypt action has completed and before you try to execute a Learn or Recognition action.

See Also