Luxand FaceSDK – Face Matching

Luxand FaceSDK provides the API to extract face templates and match them. A template extracted from a face can be stored in a database and can then be used to match faces using the FSDK_MatchFaces function.

Using the FSDK_GetFaceTemplate or FSDK_GetFaceTemplateInRegion function, the created template will have higher accuracy, thus allowing for better overall recognition rates (lower false rejection level at the same false acceptance level). If your priority is higher speed, first you need to detect eyes with FSDK_DetectEyes or FSDK_DetectEyesInRegion, and then pass the detected coordinates to FSDK_GetFaceTemplateUsingEyes. However, the accuracy of the template will be lower in this case. The FSDK_MatchFaces function returns the facial similarity level. You may consider similarity to be equal to the probability that templates belong to one and same person.

More precisely: if the access control system provides access to a person when similarity is higher of threshold x, the possibility of providing erroneous access to another person is 1-x. For example, if the decision to provide access to a person is based on the code

if (similarity > 0.99)
   AllowAccess();

the possibility of erroneous access to another person is 0.01, or 1%.

A facial template is non-reversible, i.e. there is no way to create the original face image using a template. To determine if the matched templates belong to the same person (with a specified error possibility), you can compare the facial similarity value with a threshold calculated by the FSDK_GetMatchingThresholdAtFAR or FSDK_GetMatchingThresholdAtFRR functions.

Please note: it is also recommended to retain both the original face images and their templates in the database. This is because future versions of Luxand FaceSDK may offer an improved template extraction algorithm, together with changes to the template format.

Note that it is not recommended to extract facial templates if the face is only partially visible in the image (that is, if some facial features are outside the image), as this may increase the false acceptance rate. Check to make sure that the region containing the face is completely included inside the image boundaries. Note that Tracker API always performs this check, and is recommended to use when recognizing subjects on video.

A face template is stored in the FSDK_FaceTemplate data structure.

In .NET, there is no specific data type for a template. Instead, it is stored in an array of bytes of FSDK.TemplateSize length. Below is an example of retrieving facial template in C#.

C# Example:

templateData = new byte[FSDK.TemplateSize];
FSDK.GetFaceTemplate(imageHandle, out templateData);

C++ Declaration:

typedef struct {
    char ftemplate [13324];
} FSDK_FaceTemplate;

Delphi Declaration:

FSDK_FaceTemplate = record
    Template: array[0.. 13324-1] of byte;
end;
PFSDK_FaceTemplate = ^FSDK_FaceTemplate;

Java and Android Declaration:

The class FSDK_FaceTemplate has the following property:

FSDK_FaceTemplate = record
    public byte template[];

VB Declaration:

Public Type FSDK_FaceTemplate
    FaceTemplate(13342) as Byte
End Type

  FSDK_GetFaceTemplate Function

This function is used to extract a template from a facial image. The function first detects a face, then detects its facial features and extracts the template.

If there is more than one face in the image, the template is extracted for the face with the most clearly visible details. If there is no clearly visible face, the function returns an error code. To set the threshold determining the accepted quality for faces, use the FSDK_SetFaceDetectionThreshold function.

If the face position or its features or eye centers are known, it is more efficient to use the FSDK_GetFaceTemplateInRegion or FSDK_GetFaceTemplateUsingEyes functions. To extract the template for a specific face, use the FSDK_GetFaceTemplateInRegion function.

C++ Syntax:

int FSDK_GetFaceTemplate(HImage Image, FSDK_FaceTemplate* FaceTemplate);

Delphi Syntax:

function FSDK_GetFaceTemplate(Image: HImage; FaceTemplate: PFSDK_FaceTemplate): integer;

C# Syntax:

int FSDK.GetFaceTemplate(UInt32 Image, byte[] FaceTemplate);

VB Syntax:

Function FSDKVB_GetFaceTemplate(ByVal Image As Long, ByRef FaceTemplate As Byte) As Long

Java Syntax:

int FSDK.GetFaceTemplate(HImage Image, FSDK_FaceTemplate.ByReference FaceTemplate);

Android Syntax:

int FSDK.GetFaceTemplate(HImage Image, FSDK_FaceTemplate FaceTemplate);

CImage Syntax:

byte[] FSDK.CImage.GetFaceTemplate();

Parameters:

Image – handle of the image from which to extract the face template.

FaceTemplate – pointer to the FSDK_FaceTemplate structure, used to receive the face template.

Return Value:

Returns FSDKE_OK if successful. If no faces are found, or the quality of the image is not sufficient, the function returns the FSDKE_FACE_NOT_FOUND code.

  FSDK_GetFaceTemplateInRegion Function

Extracts a template for a face located in a specific region returned by FSDK_DetectFace or FSDK_DetectMultipleFaces.

The function detects facial features in a specific region and extracts a template. The face detection stage is not performed. This function can be useful if an approximate face size and position is known, or to process a specific face returned by FSDK_DetectMultipleFaces. The function produces no error if the face is not clearly visible. This is because it assumes that if face detection functions return a detected face position, the face is of sufficient quality.

If facial features or eye centers are known, it is more efficient to use the FSDK_GetFaceTemplateUsingFeatures or FSDK_GetFaceTemplateUsingEyes function.

C++ Syntax:

int FSDK_GetFaceTemplateInRegion(HImage Image, TFacePosition* FacePosition, FSDK_FaceTemplate* FaceTemplate);

Delphi Syntax:

function FSDK_DetectFacialFeatures(Image: HImage; FacePosition: PFacePosition; FaceTemplate: PFSDK_FaceTemplate): integer;

C# Syntax:

int FSDK.GetFaceTemplateInRegion(int Image, ref FSDK.TFacePosition FacePosition, out byte[] FaceTemplate);

VB Syntax:

Function FSDKVB_GetFaceTemplateInRegion(ByVal Image As Long, ByRef FacePosition As TFacePosition, ByRef FaceTemplate As Byte) As Long

Java Syntax:

int FSDK.GetFaceTemplateInRegion(HImage Image, TFacePosition FacePosition, FSDK_FaceTemplate.ByReference FaceTemplate);

Android Syntax:

int FSDK.GetFaceTemplateInRegion(HImage Image, TFacePosition FacePosition, FSDK_FaceTemplate FaceTemplate);

CImage Syntax:

byte[] FSDK.CImage.GetFaceTemplateInRegion(ref FSDK.TFacePosition FacePosition);

Parameters:

Image – handle of the image from which to extract the face template.

FacePosition – pointer to the face position structure.

FaceTemplate – pointer to the FSDK_FaceTemplate structure, used to receive the face template.

Return Value:

Returns FSDKE_OK if successful.

  FSDK_GetFaceTemplateUsingEyes Function

Extracts a face template using the detected eye centers.

The function receives eye centers coordinates detected by the FSDK_DetectFacialFeatures, FSDK_DetectFacialFeaturesInRegion, FSDK_DetectEyes or FSDK_DetectEyesInRegion functions and extracts a face template. Face detection, facial feature detection, and eye centers detection are not performed. This function can be useful when facial features or eye centers for a specific face are already detected. The function produces no error if the face is not clearly visible, since it assumes that if the face and its facial features or eye centers are already detected, the face is of sufficient quality.Note that the FSDK_GetFaceTemplate, FSDK_GetFaceTemplateInRegion and FSDK_GetFaceTemplateUsingFeatures functions return templates that could be matched with higher accuracy.

C++ Syntax:

int FSDK_GetFaceTemplateUsingEyes(HImage Image, FSDK_Features* eyeCoords, FSDK_FaceTemplate* FaceTemplate);

Delphi Syntax:

function FSDK_ GetFaceTemplateUsingEyes(Image: HImage; eyeCoords: PFSDK_Features; FaceTemplate: PFSDK_FaceTemplate): integer;

C# Syntax:

int FSDK.GetFaceTemplateUsingEyes(int Image, FSDK.TPoint[]eyeCoords, out byte[] FaceTemplate);

VB Syntax:

Function FSDKVB_GetFaceTemplateUsingEyes(ByVal Image As Long, ByRef eyeCoords As TPoint, ByRef FaceTemplate As Byte) As Long

Java Syntax:

int FSDK.GetFaceTemplateUsingEyes(HImage Image, FSDK_Features eyeCoords, FSDK_FaceTemplate.ByReference FaceTemplate);

Android Syntax:

int FSDK.GetFaceTemplateUsingEyes(HImage Image, FSDK_Features eyeCoords, FSDK_FaceTemplate FaceTemplate);

CImage Syntax:

byte[] FSDK.CImage.GetFaceTemplateUsingEyes(ref FSDK.TPoint[] eyeCoords);

Parameters:

Image – handle of the image to extract the face template from.

eyeCoords – pointer to the FSDK_Features array containing eye centers coordinates.

FaceTemplate – pointer to the FSDK_FaceTemplate structure for receiving the face template.

Return Value:

Returns FSDKE_OK if successful.

  FSDK_GetFaceTemplateUsingFeatures Function

Extracts a face template using the detected facial feature coordinates.

The function receives facial feature coordinates detected by the FSDK_DetectFacialFeatures or FSDK_DetectFacialFeaturesInRegion functions and extracts a face template. Face detection, facial feature detection, and eye centers detection are not performed. This function can be useful when facial features for a specific face are already detected. The function produces no error if the face is not clearly visible, since it assumes that if the face and its facial features are already detected, the face is of sufficient quality.

The function determines if facial features, starting with the 2nd, are equal to zero or uninitialized. In this case, the functions calls FSDK_GetFaceTemplateUsingEyes instead.

C++ Syntax:

int FSDK_GetFaceTemplateUsingFeatures(HImage Image, FSDK_Features* FacialFeatures, FSDK_FaceTemplate* FaceTemplate);

Delphi Syntax:

function FSDK_ GetFaceTemplateUsingFeatures(Image: HImage; FacialFeatures: PFSDK_Features; FaceTemplate: PFSDK_FaceTemplate): integer;

C# Syntax:

int FSDK.GetFaceTemplateUsingFeatures(int Image, ref FSDK.TPoint[]FacialFeatures, out byte[] FaceTemplate);

VB Syntax:

Function FSDKVB_GetFaceTemplateUsingFeatures(ByVal Image As Long, ByRef FacialFeatures As TPoint, ByRef FaceTemplate As Byte) As Long

Java Syntax:

int FSDK.GetFaceTemplateUsingFeatures(HImage Image, FSDK_Features FacialFeatures, FSDK_FaceTemplate.ByReference FaceTemplate);

Android Syntax:

int FSDK.GetFaceTemplateUsingFeatures(HImage Image, FSDK_Features FacialFeatures, FSDK_FaceTemplate FaceTemplate);

CImage Syntax:

byte[] FSDK.CImage.GetFaceTemplateUsingFeatures(ref FSDK.TPoint[]FacialFeatures);

Parameters:

Image – handle of the image to extract the face template from.

FacialFeatures – pointer to the FSDK_Features array containing facial feature coordinates.

FaceTemplate – pointer to the FSDK_FaceTemplate structure for receiving the face template.

Return Value:

Returns FSDKE_OK if successful.

  FSDK_MatchFaces

Match two face templates. The returned value determines the similarity of the faces.

C++ Syntax:

int FSDK_MatchFaces(FSDK_FaceTemplate* FaceTemplate1, FSDK_FaceTemplate* FaceTemplate2, float* Similarity);

Delphi Syntax:

function FSDK_MatchFaces(FaceTemplate1, FaceTemplate2: PFSDK_FaceTemplate; Similarity: PSingle): integer;

C# Syntax:

int FSDK.MatchFaces(byte[] FaceTemplate1, byte[] FaceTemplate2, ref float Similarity);

VB Syntax:

Function FSDKVB_MatchFaces(ByRef FaceTemplate1 As Byte, ByRef FaceTemplate2 As Byte, ByRef Similarity As Single) As Long

Java Syntax:

int FSDK.MatchFaces(FSDK_FaceTemplate.ByReference FaceTemplate1, FSDK_FaceTemplate.ByReference FaceTemplate2, float Similarity[]);

Android Syntax:

int FSDK.MatchFaces(FSDK_FaceTemplate FaceTemplate1, FSDK_FaceTemplate FaceTemplate2, float Similarity[]);

Parameters:

FaceTemplate1 – pointer to the FSDK_FaceTemplate structure, using the first template for comparison.

FaceTemplate2 – pointer to the FSDK_FaceTemplate structure, using the second template for comparison.

Similarity – pointer to an integer value, used to receive the similarity of the face templates.

Return Value:

Returns FSDKE_OK if successful. Returns FSDKE_INVALID_TEMPLATE if any of the face templates is in an invalid format.

Returns FSDKE_UNSUPPORTED_TEMPLATE_VERSION if any of the templates is created with an unsupported version of FaceSDK.

  FSDK_GetMatchingThresholdAtFAR Function

This function returns the threshold value for similarity to determine if two matched templates belong to the same person at a given FAR (False Acceptance Rate) value. The FAR determines the acceptable error rate when two different people’s templates are mistakenly recognized as the same person. Decreasing FAR leads to an increase in FRR – i.e. with low FAR it becomes more probable that two templates from the same person will be determined as belonging to different people.

C++ Syntax:

>int FSDK_GetMatchingThresholdAtFAR(float FARValue, float* Threshold);

Delphi Syntax:

function FSDK_GetMatchingThresholdAtFAR(FARValue: single; var Threshold: single): integer;

C# Syntax:

int FSDK.GetMatchingThresholdAtFAR(float FARValue, ref float Threshold);

VB Syntax:

Function FSDKVB_GetMatchingThresholdAtFAR(ByVal FARValue As Single, ByRef Threshold As Single) As Long

Java and Android Syntax:

int FSDK.GetMatchingThresholdAtFAR(float FARValue, float Threshold[]);

Parameters:

FARValue – the desired FAR value. Varies from 0.0 (means 0%) to 1.0 (means 100%).

Threshold – pointer to a float variable to store the calculated Threshold value.

Return Value:

Returns FSDKE_OK if successful.

Example

FSDK_FaceTemplate template1, template2;

float MatchingThreshold, Smilarity;
FSDK_GetMatchingThresholdAtFAR(0.02, &MatchingThreshold);

FSDK_GetFaceTemplate(img1, &template1);
FSDK_GetFaceTemplate(img2, &template2);
FSDK_MatchFaces(&template1, &template2, &Similarity);
if (Similarity > MatchingThreshold)
   printf("Same Person\n");
else
   printf("Different Person\n");

  FSDK_GetMatchingThresholdAtFRR Function

This function returns the threshold value for similarity to determine if two matched templates belong to the same person at a given FRR (False Rejection Rate) value. The FRR determines the acceptable error rate when two templates of the same person are identified as belonging to different people. Decreasing FRR leads to an increase in FAR – i.e. with low FRR it becomes more probable that two different people’s templates will be recognized as the same person.

C++ Syntax:

int FSDK_GetMatchingThresholdAtFRR(float FRRValue, float* Threshold);

Delphi Syntax:

function FSDK_GetMatchingThresholdAtFRR(FRRValue: single; var Threshold: single): integer;

C# Syntax:

int FSDK.GetMatchingThresholdAtFRR(float FRRValue, ref float Threshold);

VB Syntax:

Function FSDKVB_GetMatchingThresholdAtFRR(ByVal FRRValue As Single, ByRef Threshold As Single) As Long

Java and Android Syntax:

int FSDK.GetMatchingThresholdAtFRR(float FRRValue, float Threshold[]);

Parameters:

FRRValue – the desired FRR value. Varies from 0.0 (means 0%) to 1.0 (means 100%).

Threshold – pointer to a float variable, used to store the calculated Threshold value.

Return Value:

Returns FSDKE_OK if successful.

 

Next chapterGender and Facial Expression Recognition

Contents