Skip to main content

Selphi Widget Android

1. Introduction​

This manual presents configuration and functioning of the FacePhi Selphi Android Widget for applications developed in Android operating system as well as its integration in Android projects. The following points are explained:

  • Properties, methods and communication of this widget
  • Creation of an Android project which integrates the widget for any Android application

1.1. Widget version​

Retrieve version via static method 'getWidgetVersion()'

    public static String getWidgetVersion()

Returns the widget's actual version in string format. This method is static so it doesn´t require launching the widget to perform this operation.

1.2. Minimum requirements​

  • Android API level: 15
  • RAM: 2 GB

If Injection Attack Detection (IAD) feature is used then:

  • API level 24
  • RAM: 3 GB

2. API (Application programming interface)​

2.1. Properties​

The Android widget provided is an Activity class which allows you to extract facial characteristics as well as configure the widget with appropriate values for extraction.

Passing boot arguments between the main class of the project and the widget is done through Parcelables classes. The class used for this communication is WidgetConfiguration.

When a new widget is created in an Android application, you can set up some properties which determine its functioning. This widget configuration is handled by WidgetConfiguration class. It contains several methods that allow you to set the following properties:

2.1.1. ResourcesPath​

It sets the route of the resources file that the widget will use for its graphical configuration. The route of this file must be specified related to the assets folder of the main application.

2.2. Methods​

You have the following methods to configure the widget, at WidgetConfiguration class:

2.2.0. setLicense​

    public void setLicense(String license)

Sets the contents of the license that will be needed for some widget features.

2.2.1. setLivenessMode​

	public void setLivenessMode(WidgetLivenessMode liveness)

It sets the widget liveness mode. The permitted values are:

  • LIVENESS_NONE: Indicate that must not be activated the photo detection mode in the authentication processes.
  • LIVENESS_MOVE: Indicates that active liveness movement mode must be activated in the authentication processes.
  • LIVENESS_PASSIVE: Indicate that the liveness detection is implemented at server side, therefore the BestImage will be sent to the server, or the correspondent TemplateRaw.

2.2.2. setStabilizationMode​

	public void setStabilizationMode(boolean sm)

Sets a stabilization mode prior to any process in the widget. With this mode, the widget is forced not to start any process if the user is not facing forward and without moving their head.

2.2.3. setQRFlag​

	public void setQRFlag(boolean qr)

Indicates if the widget needs to read a QR before the authentication process.

2.2.4. setUserTags​

	public void setUserTags(byte uTags)

It sets 4 bytes with data that may be configured by the main application and will be incorporated to the templates generated by the extractor.

2.2.5. setResourcesPath​

	public void setResourcesPath(String path)

It sets the route where the file of the widget resources is found. This file contains both the graphical resources and the location resources. This route should be specified related to the assets folder of the main application.

2.2.6. setLocale​

	public void setLocale(String locale)

It forces the widget to use the language configuration indicated by locale parameter This parameter accepts a language code and local identification code. If the file of the widget resources doesn´t have a location for the locale selecting its configuration would use the language by default.

2.2.7. setFullScreen​

	public void setFullscreen(boolean fs)

It sets if appropriate the widget starts in full screen mode, hiding the bar status of Android.

2.2.8. setCameraPreviewSize​

	public void setCameraPreviewSize(Point cameraPreviewSize)

It sets the camera resolution moving an object that contains the width and the height in the image pixels.

2.2.9. setFrontFacingCameraAsPreferred​

	public void setFrontFacingCameraAsPreferred()

It sets as a preferred camera for the authentication process the front camera of the device.

2.2.10. setBackFacingCameraAsPreferred​

	public void setBackFacingCameraAsPreferred()

It sets as a preferred camera for the authentication process the back camera of the device.

2.2.11. logImages​

public void logImages(boolean logImages)

Activate or not the return of the images list which have been captured during the execution of the extraction process. If the input parameter is true, at the time to execute the getImages() method will return the list of processed images. In another case, it will get back an empty list.

In case of not using images, it is recommended to keep this deactivated mode to improve the storage performance in the devices.

2.2.12 setTutorialFlag​

    public void setTutorialFlag (boolean debug)

Sets the option to show a tutorial of the process it's about to perform. Once the tutorial finishes, the widget will continue with the process.

2.2.13. setDebug​

	public void setDebug (boolean debug)

It sets the debugging mode of the widget.

2.2.14. setVideoFilename​

	public void setVideoFilename(String filename)

Sets the absolute path of the file name where a video of the process will be recorded. The application is responsible for requesting the necessary permissions to the phone in case that path requires additional permissions. By default, the widget will not perform any recording process unless a file path is specified using this method.

2.2.15. generateTemplateRawFromBitmap​

	public static byte[] generateTemplateRawFromBitmap(Bitmap img)

Generate a templateRaw from a native Android image. This method is static so it doesn´t require launching the widget to perform this operation.

2.2.16. generateTemplateRawFromByteArray​

	public static byte[] generateTemplateRawFromByteArray(byte[] img)

Generate a templateRaw from a byte array. This array must contain the representation of the image in jpg or png format. This method is static so it doesn´t require launching the widget to perform this operation.

2.2.17 getWidgetVersion​

    public static String getWidgetVersion()

Returns the widget's actual version in string format. This method is static so it doesn´t require launching the widget to perform this operation.

2.2.18 setShowAfterCapture​

    public void setShowAfterCapture(boolean value)

Enables a preview of the captured selfie prompting the user to accept it or repeat it.

2.2.19 setExtractionDuration​

    public void setExtractionDuration(FPhiWidgetExtractionDuration extractionDuration)

Sets the amount of time the widget will keep extracting user's facial features. The allowed values are:

  • Short: 1 second extraction duration. (Default value)
  • Medium: 2 seconds extraction duration.
  • Long: 3 seconds extraction duration.

2.2.20 setPreferredOrientation​

    public void setPreferredOrientation(WidgetOrientation value)

Sets the allowed orientations the widget will permit.

The allowed values are:

  • FULL_SENSOR: All orientations allowed.
  • FULL_SENSOR_NO_REVERSE: All orientations but reverse portrait allowed.
  • PORTRAIT: Portrait allowed. (Default)
  • LANDSCAPE_LEFT: Landscape left allowed.
  • LANDSCAPE_RIGHT: Landscape right allowed.
  • PORTRAIT_REVERSE: Reverse portrait allowed.
  • PORTRAIT_SENSOR: Portrait and reverse portrait allowed.
  • LANDSCAPE_SENSOR: Landscape left and landscape right allowed.
  • LOCKED: All orientations allowed but the widget won't rotate dynamically.

2.2.21 setCameraFlash​

    public void setCameraFlash(bool value)

Enable/disable camera flash if available.

2.2.22 setJPGQuality​

    void setJPGQuality(float value)

Sets jpeg compression quality. Default value = 0.92f.

2.3. Android widget integration​

The following sample code shows how to integrate a widget in an Android application:

	/**
	* Creates an intent that invokes the widget.
	*/
	@Override
	public void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);

		setContentView(R.layout.main);

		Intent intent = new Intent(this, Widget.class);
		startActivityForResult(intent, 1);
	}

In the code above, Intent object is created, which allows us to switch between activities. This object is generated by calling the following class of Android widget:

	Intent intent = new Intent(this, Widget.class);

In order to run an Activity from our main application, we have to declare our activities in the AndroidManifest.xml:

	<activity
	android:name="com.facephi.selphi.Widget"
	android:label="Your app name" >
	</activity>

It is necessary to grant permits to the camera for using it.

	<!-- Camera permissions -->
	<uses-permission android:name="android.permission.CAMERA" />
	<uses-feature android:name="android.hardware.camera" />

It is necessary to establish the file in which the graphical customization is found.

	Intent intent = new Intent(this, Widget.class);
	
	WidgetConfiguration conf = new WidgetConfiguration("fphi-widget-resources-SelphiPhiLive-1.2.zip");
	intent.putExtra("configuration", conf);
	
	startActivityForResult(intent, 1);

This WidgetConfiguration object is generated and it will be used to communicate with the widget and specify the extractor configuration. Command to send the configuration to the extractor is as follows:

	intent.putExtra("configuration", conf);

Where conf is our WidgetConfiguration object.

It is mandatory that the first parameter of putExtra(…, …) method to be a string whose content is configuration as it is indicated in the example, since the widget is waiting for this string at the call. Otherwise, communication will not be performed.

The following step is to initialize Activity by using startActivityForResult() method. This method will launch the widget Activity and will wait for any response. The call will be as follows:

	startActivityForResult(intent, 1);

First parameter is the Intent in which we have inserted extractor settings. The second parameter is a constant used to know from where the call has been done, each call must use a different constant in order to differentiate the flow of the application.

2.4. Widget response​

When an activity that has been called closes and returns to another, an event called onActivityResult(…) is launched. When the extraction process has been completed, this event will be launched, and it will return all the information needed for its communication. Parameters received are:

  • requestCode: Integer code which matches with the constant that you have passed to the main activity as second parameter at startActivityForResult()”.
  • resultCode: The widget will return RESULT_CANCELLED or RESULT_OK depending on the whole process if any mistake has occurred during the extraction, if the user cancelled the process manually or if it will return the second one.
  • data: When the widget finishes its process, it returns an Intent as final result.

onActivityResult:

	protected void onActivityResult(int requestCode, int resultCode, Intent data) {
		
		if (data == null) return;
			
		WidgetResult result = data.getParcelableExtra("result");
		if (result == null){
			Toast.makeText(getBaseContext(), getResources().getString(R.string.message_no_results),Toast.LENGTH_LONG).show();
			return;
		}

		// 1. The extraction ends with errors
		if (resultCode == RESULT_CANCELED) {
			if (result.getException() != null) {
				if (result.getException().getExceptionType() != null) {
					String exc_message = "";
					switch(result.getException().getExceptionType()) {
					case StoppedManually:
						exc_message = getResources().getString (R.string.message_user_stopped_manually);
						break;
					case Timeout:
						exc_message = getResources().getString(R.string.message_timeout_error);
						break;
					case CameraPermissionDenied:
						exc_message = getResources().getString(R.string.message_camera_permission_error);
						break;
					case SettingsPermissionDenied:
						exc_message = getResources().getString(R.string.message_settings_permission_error);
						break;
					default:
						exc_message = getResources().getString(R.string.message_unexpected_error);
						break;
					}
					
					Toast.makeText(getBaseContext(),exc_message, Toast.LENGTH_LONG).show();
				}
			}
		}

		// 2. The extraction has been successful 
		// The application can recover the information of the template.
		byte[] result = result.getTemplateRaw();
	}

Using any mode (Authenticate or Register), you may obtain the extraction result WidgetResult In order to obtain the extraction result template, use the following call:

	WidgetResult result = data.getParcelableExtra("result");
	byte[] result = ucResult.getTemplateRaw();

Each WidgetResult contains the following information:

  • getTemplateRaw(): It retrieves the generated raw template after the extraction process.
  • getImages(): If logImages() flag is set, it retrieves the images obtained during the extraction process.The images are retrieved from highest to lowest by its "facial score" so the best of the image of the extraction process is found in the 0 position of array.
  • getBestImage(): Returns the best image extracted from the authentication process. This image is the original size image taken from the camera.
  • getBestImageCropped(): Returns a cropped image centered on the user's face. This image is obtained from the "bestImage". This is the image that should be used as a characteristic image of the user who carried out the process as an ‘avatar’.
  • getLivenessDiagnostic(): It retrieves the final diagnostic of the liveness process.
  • getQRData(): It retrieves the data of QR codes captured.
  • getIADBundle: It retrieves the encrypted data from the injection attack detection analysis.
  • getException(): It retrieves an object of type WidgetException, which have information about the message getMessage() and the type of the exception getExceptionType().

Available values for the exception type are:

  • WidgetExceptionType.StoppedManually: Exception which is thrown when the user stops the process manually.
  • WidgetExceptionType.Timeout: Exception which is thrown when a specific amount of time is reached without a valid extraction.
  • WidgetExceptionType.CameraPermissionDenied: Exception which is thrown when the widget can’t access to the camera.
  • WidgetExceptionType.SettingsPermissionDenied: Exception which is thrown when the widget can’t modify settings.
  • Other values: Other exceptions which are thrown by the widget.

2.4.1. IFPhiWidgetEventListener interface​

This interface allows our widget to send information to the main application about important events that occur during the execution.

To achieve this, you need to create a class that implements this interface, which only has one method:

	public void onEvent(long time, @NonNull String type, @NonNull String info)

This method receives as parameters the time of the event, encoded as UnixTime in milliseconds, the type of event and the information of the event associated with this particular event.

There are mainly 3 types of events:

  • Events about view changes or widget status changes.
  • User events such as button clicks or swipe movements.
  • Events about the authentication that is currently in progress. These events can be about errors not detecting a face, about wrong movements or even not following the indications about the current process.

With these events we can comunicate important data to analyze user behavior while using our technology.

After creating the class that implements this interface, the application will call the configuration object method:

	conf.setIFPhiWidgetEventListener_classname("name.class.interface")

Passing the name of the class that implements this interface.

2.5. Data recovery example​

The following code shows the user how to recover the information that the widget provides to the user:

	// Retrieves the widget result
	WidgetResult result = data.getParcelableExtra("result");

The WidgetResult class contains information about the extraction result, any exception that may happen and a list of images for each step:

	// Retrieves an object with the exception and its type
	result.getException();

	// Retrieves the image list
	result.getImages();

	// Retrieves the user template
	result.getTemplateRaw();

The getImages() method returns a list of FPhiImage stored in the native heap of the device.

In order to transform this object to Bitmap use the widget's static method FPhiWidgetAndroid.GetBitmap(width, height, config)

IMPORTANT NOTE: In order to return the user images in the widget configuration, you must set the logImages property to true.

Due to the important memory load produced by the image processing, it is required to activate the android property largeHeap in the AndroidManifest.xml:

	<application android:largeHeap="true">
	 ...
	</application>

3. Customize the widget​

The widget allows you to customize texts, images, font and colours. The customization is made by the .zip file provided with the widget. This zip is composed of a file titled widget.xml that contains the definition of all widget screens. Each with a serie of elements which allow to make the customization. The zip file also contains a folder with graphical resources and another folder with the translation of the texts.

3.1. Basic description​

3.1.1. Text customization​

The customization of texts is done editing the texts of the translation files inside the resources folder.

	/strings/strings.es.xml
	/strings/strings.xml

3.1.2. Images customization​

The customization of images which the widget use it must be the images in the .zip of resources. In the zip there are 3 folders:

	/resources/163dpi
	/resources/326dpi
	/resources/489dpi

These folders represent at different screen densities, it may be generated as many density folders as desired. In these folders are the versions of the images for each resolution.

It is necessary to add the images in all folders because, once determined the optimal resolution for the device, the widget only loads the images of the folder with the selected resolution. The images are referenced from the file widget.xml.

3.1.3. Colour customization​

The customization of the colour of the buttons is done from the file widget.xml. You can customize any colour of any graphical element that appears in the widget. It is simply enough to modify the colour of the property desired.

3.1.4. Font customization​

The customization of the font must be placed in the folder /resources/163dpi and it can be referenced from the file widget.xml. To change the font of text elements is enough modifying the font property and put the name of the file.

In the following section, there is more information about the contents of the resources bundle and the way they can be modified.

3.2. Advanced description​

3.2.1. Widget.xml​

This file contains the definition of the properties that can be configured in the authentication processes. It is divided by navigation screens and inside of each screen tag are found all the properties that can be modified.

It is possible, via code, to select the location by the local property. This parameter accepts a string with the language code desired (for example,”es” or ”es_ES”).

3.2.2. String folder​

This folder contains string.xml file for each translation that may support. The name must be formed as follows:

	strings.(language).xml

Being (language) the language code. For example, strings.es.xml would be the translation in Spanish, strings.en.xml the translation in English, strings.es_ES.xml the Spanish from Spain or strings.es_AR.xml the Spanish from Argentina.

The language can be forced or let the widget select based on the device configuration. When deciding the language to apply the following sequence of steps is performed:

  • Searching by location code (for example, "es_AR").
  • If there isn´t any coincidence, it is possible to search for the generical language ("es").
  • If there isn´t any result, it is possible to use the language by default.

3.2.3. Resources folder​

It contains the folders with all the necessary resources to be modified, divided in densities. It is mandatory to generate the images of all densities, since the widget is expecting to find them in the corresponding folder to the density of the device. It also can create new folders with the density expected.

3.2.4. BACKGROUND element​

The background element is composed of 4 segments which it can give colour independently:

  • top: defines the background colour of the segment or upper panel.
  • middle_top: defines the background colour of the segment or panel where the image of the is located.
  • middle_bottom: defines the background colour of the segment or panel where text is located.
  • bottom: defines the background colour of the segment or the lower panel.

It also can be configured certain properties which is used only in specific screens. Below, we list them making reference to the screens which are used:

  • pagination_separator (RegistrationTips,FaceMovementTips): defines the colour separation between the lower panel and the panel under the camera.
  • mirror_border_color (RegistrationTips, FaceMovementTips): defines the colour of the border of the circle which surrounds the image of the camera or of the video of the registration tips. To this element also is called mirror.
  • mirror_border_width (RegistrationTips, FaceMovementTips): defines the width of the edge of the circle which surrounds the image of the camera or the video of the registration tips. If we don´t want to show the edge, we have to assign a value of 0.0 to this property.
  • mirror_mist_color (StartExtractor): Defines the colour of the centre circle in the previous screen to the extraction. This colour must have always a transparency value, we should let show the image of the camera for the user can place properly before to start with the extraction. The colour format when it is included a transparency value is RGBA (the alpha value will be indicated with the last byte).
  • mirror_color (Results): defines the background colour of the circle that show the results of the registration process.

3.2.5. BUTTON element​

  • background: defines the background colour of the button.
  • decorator: defines the colour of the shadow of the button.
  • foreground: defines the colour of the font of the button in case the content is a text.
  • content_type: defines the type of the content of the button. There are 2 different types:
    • RESOURCE_ID: Content must contain the name of a file in the resources bundle.
    • TEXT_ID: Content must contain the identificator of a literal of the translations file in the resources bundle.
  • content: defines the content of the button, image or text.
  • font: defines the type of font used if the content of the button is text.
  • font_size: defines the size of the font if the content of the button is text.

3.2.6. TEXT element​

The text elements are used to define the graphical aspect of the texts of each widget screens. These are the properties which can be modified:

  • color: defines the text colour.
  • font: defines the type of the font used to show the text.
  • font_size: defines the size of the font.

The results screen of the registration the two texts that define the registration quality has forced their colour to the colour of the bar that indicates the punctuation.

3.2.7. IMAGE element​

  • value: defines the name of the file that contains the image to show.

The image elements only have the property that defines the file where the physical image is located in the resources bundle. The images are obtained of the bundle searching in the appropriate folder according the density of the device.

3.2.8. VIDEO element​

  • value: defines the name of the file that contains the video to show.

The video elements only have the property that defines the file where the physical video is located in the resources bundle.

4. Integration in Android applications​

4.1. Minimum requirements​

For a correct integration of the face recognition widget in an Android application, you will need to have the following Android operating system version:

  • API level 15 (Android 4.0.3)
  • RAM: 2 GB

If Injection Attack Detection (IAD) feature is used then:

  • API level 24 (Android 7.0)
  • RAM: 3 GB

Regarding mobile device architectures:

  • armeabi-v7, x86, arm64 y x64

4.2. Content provided​

SDK components provided for integration of the widget are the following:

  • Android Studio: Libraries Android Archive Library (Android Archive Library) called fphi-core-widget-android.aar and fphi-selphi-widget-android.aar.
  • IAD version: Add additional IAD libraries license_manager-X.X.X.aar and iad-X.X.X.-release.aar.

This distribution contains the necessary libraries and resources already packed and ready for its use. Additionally, a .zip file is delivered that contains the graphical configuration and location of the widget, by which it can be personalized the physical appareance of the widget for a better integration with the final application.

Also, together with the SDK is provided an example of programming for Android Studio that helps to the integrator to consume the widget in an Android application.

4.3. Android widget configuration​

The methods to configure the widget are shown below:

	// Create the configuration object with all previous values
	WidgetConfiguration conf = new WidgetConfiguration();

	// Set the file path of graphical resources and location
	conf.setResourcesPath(path);

	// Set the liveness mode extraction (LIVENESS_NONE, LIVENESS_PASSIVE)
	conf.setLivenessMode(mode);

	// Set if its required or not the widget force the QR code before the authentication process
	conf.setQRFlag(QRflag);

	// Set the camera resolution 
	conf.setCameraPreviewSize(cameraPreviewSize);

The Android Widget provided by FacePhi is responsible for performing the extraction of the facial features of the user, and thus will generate a facial template representative of the user. The facial template should be sent to a web server for the registration of the users by face, authentication and identification of the user. Therefore, the Android application that integrates the widget should consult a web service that will be posted on a server accessible via Internet.

On the server side, the web application uses the FacePhi facial recognition libraries for processes involving registration and improving users and comparison of facial templates.