@nativescript/mlkit-core

A plugin that provides a UI component to access the different functionalities of Google's ML Kit SDK.

Installation
Use @nativescript/mlkit-core
API
License

Installation

cli

npm install @nativescript/mlkit-core

Use @nativescript/mlkit-core

The usage of @nativescript/mlkit-core has the following flow:

Registering and adding MLKitView to your markup.
Setting the detectionType and listening to the detection event.

To access all the vision APIs at once, set the detectionType property to 'all' and identify them in the detection event's handler.

To access a specific API, Barcode scanning for example, set the detectionType property to the API name ('barcode' for Barcode scanning), AND import that API's NativeScript plugin(@nativescript/mlkit-barcode-scanning).

Check if ML Kit is supported To verify if ML Kit is supported on the device, call the static isAvailable() method on MLKitView class.

if (MLKitView.isAvailable()) {
}

Request for permission to access the device camera by calling requestCameraPermission():

mlKitView.requestCameraPermission().then(() => {})

The following are examples of registering and using MLKitView in the different JS flavors.

Core

Register MLKitView by adding xmlns:ui="@nativescript/mlkit-core" to the Page element.
Use the ui prefix to access MLKitView from the plugin.

xml

<ui:MLKitView
cameraPosition="back"
detectionType="all"
detection="onDetection"
/>

Angular

In Angular, register the MLKitView by adding MLKitModule to the NgModule of the component where you want to use MLKitView.

import { MLKitModule } from'@nativescript/mlkit-core/angular';

@NgModule({
 imports: [
 MLKitModule
 ],
 declarations: [
 AppComponent
 ],
 bootstrap: [AppComponent]
})

Use MLKitView in markup.

html

<MLKitView
cameraPosition="back"
detectionType="all"
(detection)="onDetection($event)"
></MLKitView>

Vue

To use MLKitView, register it in the app.ts by passing it to the use method of the app instance.

import { createApp } from'nativescript-vue'

import MLKit from'@nativescript/mlkit-core/vue'
import Home from'./components/Home.vue'

constapp=createApp(Home)

app.use(MLKit)

Use MLKitView in markup.

html

<MLKitViewcameraPosition="back"detectionType="all"@detection="onDetection" />

Vision APIs optional modules

Important

Detection works only for optional modules installed

Barcode Scanning

cli

npm i @nativescript/mlkit-barcode-scanning

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { BarcodeResult } from'@nativescript/mlkit-barcode-scanning';
onDetection(event: DetectionEvent){
if(event.type === DetectionType.Barcode){
constbarcode:BarcodeResult[] = event.data;
 }
}

For more details, see @nativescript/mlkit-barcode-scanning

Face Detection

cli

npm install @nativescript/mlkit-face-detection

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { FaceResult } from'@nativescript/mlkit-face-detection';

onDetection(event: DetectionEvent){
if(event.type === DetectionType.Face){
constfaces:FaceResult[] = event.data;
 }
}

For more details, see @nativescript/mlkit-face-detection

Image Labeling

cli

npm install @nativescript/mlkit-image-labeling

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { ImageLabelingResult } from'@nativescript/mlkit-image-labeling';
onDetection(event: DetectionEvent){
if(event.type === DetectionType.Image){
constlabels:ImageLabelingResult[] = event.data;
 }
}

For more details, see nativescript/mlkit-image-labeling

Object Detection

cli

npm install @nativescript/mlkit-object-detection

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { ObjectResult } from'@nativescript/mlkit-object-detection'
onDetection(event: DetectionEvent){
if(event.type === DetectionType.Object){
constobjects:ObjectResult[] = event.data;
 }
}

For more details, see @nativescript/mlkit-object-detection

Pose Detection

cli

npm install @nativescript/mlkit-pose-detection

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { PoseResult } from'@nativescript/mlkit-pose-detection';
onDetection(event: DetectionEvent){
if(event.type === DetectionType.Pose){
constposes:PoseResult= event.data;
 }
}

For more details, see @nativescript/mlkit-pose-detection

Text Recognition

cli

npm install @nativescript/mlkit-text-recognition

import { DetectionType, DetectionEvent } from'@nativescript/mlkit-core';
import { TextResult } from'@nativescript/mlkit-text-recognition';
onDetection(event: DetectionEvent){
if(event.type === DetectionType.Text){
consttext:TextResult= event.data;
 }
}

For more details, see @nativescript/mlkit-text-recognition

API

detectWithStillImage()

import { DetectionType, detectWithStillImage } from"@nativescript/mlkit-core";

async processStill(args) {
try {

result: { [key: string]: any } =awaitdetectWithStillImage(image: ImageSource, options)
 } catch (e) {
 console.log(e);
 }
 }

Detects barcode, pose, etc from a still image instead of using the camera.

image: The image to detect the object from
options: An optional StillImageDetectionOptions object parameter specifying the detection characteristics.

MLKitView class

The MLKitView class provides the camera view for detection.

It has the following members.

Properties

Property	Type
`detectionEvent`	`string`
`cameraPosition`	CameraPosition
`detectionType`	DetectionType
`barcodeFormats`	BarcodeFormats
`faceDetectionPerformanceMode`	FaceDetectionPerformanceMode
`faceDetectionTrackingEnabled`	`boolean`
`faceDetectionMinFaceSize`	`number`
`imageLabelerConfidenceThreshold`	`number`
`objectDetectionMultiple`	`boolean`
`objectDetectionClassify`	`boolean`
`torchOn`	`boolean`
`pause`	`boolean`
`processEveryNthFrame`	`number`
`readonly latestImage?`	ImageSource
`retrieveLatestImage`	`boolean`

Methods

Method	Returns	Description
`isAvailable()`	`boolean`	A static method to check if the device supports ML Kit.
`stopPreview()`	`void`
`startPreview()`	`void`
`toggleCamera()`	`void`
`requestCameraPermission()`	`Promise<void>`
`hasCameraPermission()`	`boolean`
`on()`	`void`

StillImageDetectionOptions interface

interfaceStillImageDetectionOptions {
detectorType:DetectionType

barcodeScanning?: {
barcodeFormat?: [BarcodeFormats]
 }
faceDetection?: {
faceTracking?:boolean
minimumFaceSize:?number
detectionMode?:'fast'|'accurate'
landmarkMode?:'all'|'none'
contourMode?:'all'|'none'
classificationMode?:'all'|'none'
 }
imageLabeling?: {
confidenceThreshold?:number
 }
objectDetection?: {
multiple:boolean
classification:boolean
 }
selfieSegmentation?: {
enableRawSizeMask?:boolean
smoothingRatio?:number
 }
}

Enums

DetectionType

exportenumDetectionType {
Barcode='barcode',
DigitalInk='digitalInk',
Face='face',
Image='image',
Object='object',
Pose='pose',
Text='text',
All='all',
Selfie='selfie',
None='none',
}

CameraPosition

exportenumCameraPosition {
FRONT='front',
BACK='back',
}

BarcodeFormats

exportenumBarcodeFormats {
ALL='all',
CODE_128='code_128',
CODE_39='code_39',
CODE_93='code_93',
CODABAR='codabar',
DATA_MATRIX='data_matrix',
EAN_13='ean_13',
EAN_8='ean_8',
ITF='itf',
QR_CODE='qr_code',
UPC_A='upc_a',
UPC_E='upc_e',
PDF417='pdf417',
AZTEC='aztec',
UNKOWN='unknown',
}

FaceDetectionPerformanceMode

exportenumFaceDetectionPerformanceMode {
Fast='fast',
Accurate='accurate',
}

License

Apache License Version 2.0

Previous: Payments: IAP & Subscriptions
Next: Barcode Scanning

On this page

Contents
Installation
Use @nativescript/mlkit-core
API
License

Contributors

NativeScript

Edit this page

Last updated:

@nativescript/mlkit-core ​

Contents ​

Installation ​

Use @nativescript/mlkit-core ​

Core ​

Angular ​

Vue ​

Vision APIs optional modules ​

Barcode Scanning ​

Face Detection ​

Image Labeling ​

Object Detection ​

Pose Detection ​

Text Recognition ​

API ​

detectWithStillImage() ​

MLKitView class ​

Properties ​

Methods ​

StillImageDetectionOptions interface ​

Enums ​

DetectionType ​

CameraPosition ​

BarcodeFormats ​

FaceDetectionPerformanceMode ​

License ​