- Homepage
- Feature Generation
Feature Generation
When it comes to the data collected using the Moonsense SDK, the raw data by itself does not provide the necessary characterization that is representative of a particular user behavior or action we want to derive. There is a process of refinement that needs to be done in order for the data to be actionable. This process of generating actionable features from the raw sensor data collected by the Moonsense SDK is called Feature Generation.
Out of the box, the Moonsense SDK provides useful Feature Generators that can be used by various machine learning classifiers to derive a variety of signals for a number of use cases around fraud and risk.
Users may prefer to use the Feature Generators over the raw sensor data for one or more of the following reasons:
Provide actionable data to ML models for useful signals.
Reduce the dimensionality of the data before transmission over the network thereby lowering the amount of data that is uploaded and subsequently stored.
Transform the data in a way to preserve user privacy.
Eliminates the noise in the raw sensor data, thereby making the data more actionable.
Possibility of using on device machine learning models that prevents the data ever leaving the device.
The following document details the steps involved integrating the Moonsense Features SDK and enabling Feature Generation in your existing product.
Integration
The Moonsense Features SDK is an extension of the Moonsense SDK and therefore relies on the base SDK being installed and configured correctly. Refer to the Getting Started guide to set up the base SDK correctly:
Before integrating the Features SDK make sure you have the correct entitlements set up that grants access to the SDK using the repo access token. This should allow you to correctly download and access the Features SDK.
ℹ️ The Features SDK requires a license that is separate from the main SDK license. If you are interested in the Features SDK, please reach out to support@moonsense.io.
With the entitlements in place configure the Feature SDK dependency using the following lines
// Add the following to your project's gradle file
implementation("io.moonsense:android-features-sdk:<latest_version>")
# The Moonsense iOS Features SDK is available as a Swift Package. Simply use the the link to the public repo, https://github.com/moonsense/moonsense-ios-features-sdk, as the Package URL when adding the Swift Package to your project.
npm install --save @moonsense/moonsense-web-features-sdk
With the packages installed we now have access to the Feature Generators available via the SDK.
Feature Generation
It is important to understand how the feature generators work before we talk about usage. Feature Generators in a nutshell take the data captured by the Moonsense SDK via the Bundle object and convert them into features for use. The way it does is via Transforms and Operators. The role of a Transform is to map an entry in a Bundle into an intermediate output that can be summarized using an Operator. The Operator acts as a reducer operating on the mentioned transformed stream to generate statistical outputs like mean, median, min, max and so on.
Below is an example of a Feature Generator:
CompositeFeatureGenerator(
listOf(
PointerTransform(
"tool_type_touch" to { pointer ->
if (pointer.type == Pointer.Type.TOUCH) {
1.0
} else {
null
}
}
)
),
Operator.COUNT
)
CompositeFeatureGenerator(transforms: [
PointerTransform(transforms: [
"tool_type_touch": { $0.type == .touch ? 1.0 : nil }
])],
operators: [.count]
)
new CompositeFeatureGenerator(
[
new PointerTransform({
tool_type_touch: (d) => d.type === bundle.Pointer.Type.TOUCH ? 1 : null,
}),
],
[
Operator.COUNT
]
),
All feature generators in the SDK are based off of the CompositeFeatureGenerator. The CompositeFeatureGenerator allows you to describe features in an expressive manner that makes it easy to generate features that precisely fit your needs. As shown above the CompositeFeatureGenerator accepts a PointerTransform that flags every Pointer event that is of type TOUCH. The Count Operator then returns the total count of touch type events included in the Bundle. The feature that is generated is available in the Bundle via the features field in the following format:
{ pointer-tool_type_touch : { count : 10.0 } }
In this manner we can extend the CompositeFeatureGenerator to express a wide variety of features that that fit our use case. Below is the comprehensive list of Transforms and Operators available to the user
| Transform |
|---|
| Accelerometer |
| FocusChange |
| Gesture |
| Gyroscope |
| InputChange |
| KeyPress |
| LinearAccelerometer |
| Magnetometer |
| Orientation |
| Pointer |
| Click |
| FormSubmit |
| MouseWheel |
| ViewScroll |
| Operator |
|---|
| Min |
| Max |
| Mean |
| Median |
| Standard Deviation |
| Sum |
| Kurtosis |
| Percentile |
| Root Mean Square |
| Skewness |
| Zero Crossing |
| Count |
* Gesture is defined as a collection of Pointer events with the same device id. The Click, FormSubmit, MouseWheel and ViewScroll transforms are only available on web. Percentiles report the 0th, 25th, 50th 75th and 100th values.
The Features SDK comes with two stock ready to use generators that makes the integration and usage fairly straightforward.
Device Motion
The DeviceMotionFeatureGenerator included in the SDK provides the count, mean, standard deviation, root mean square, kurtosis, skewness, zero crossing and percentile values for each individual axis reported by the accelerometer, linear accelerometer, gyroscope, magnetometer and orientation sensor types.
Pointer
The PointerFeatureGenerator included in the SDK provides useful insights around user interaction patterns included in the Bundle. The list of features included in this generator are:
- Count of touch, mouse, stylus, inverted stylus and unknown tool types reported by all the Pointer events in the Bundle.
- The count, max, min, median, mean and standard deviation for size and pressure values recorded by each unique Pointer event.
- The count, max, min, median, mean and standard deviation for distance, displacement and curviness(displacement/distance) recorded by each unique
Gesture. - The count, max, min, median, mean and standard deviation of the bounding box covering all the touch points contained in a unique
Gesture.
Usage
Users can choose to specify which Feature Generator they plan to use by providing the generator as an argument to the SessionConfig when starting a session. For example:
// specify the feature generators to use
val sessionConfig = SessionConfig(
featureGenerators = listOf(
DeviceMotionFeatureGenerator(),
PointerFeatureGenerator(),
// report a mean of the magnitude vectors reported by the accelerometer
CompositeFeatureGenerator(
listOf(
AccelerometerTransform(
"magnitude" to {
sqrt((it.x * it.x) + (it.y * it.y) + (it.z * it.z))
}
)
),
Operator.MEAN
)
)
)
// provide the created session config
val session = Moonsense.startSession(
duration = TimeUnit.SECONDS.toMillis(15),
sessionConfig = sessionConfig
)
let sessionConfig = SessionConfig(featureGenerators: [
DeviceMotionFeatureGenerator(),
PointerFeatureGenerator(),
CompositeFeatureGenerator(transforms: [
AccelerometerTransform(transforms: [
"magnitude": { sqrt(($0.x * $0.x) + ($0.y * $0.y) + ($0.z * $0.z))}
])],
operators: [.mean]
)
])
// provide the created session config
let session = try? Moonsense.startSession(duration: 15.0,
sessionConfig: sessionConfig)
// specify the feature generators to use
const featureGenerators = [
new DeviceMotionFeatureGenerator(),
new PointerFeatureGenerator(),
// report a mean of the magnitude vectors reported by the accelerometer
new CompositeFeatureGenerator(
[
AccelerometerTransform({
magnitude: (d) => Math.sqrt((d.x * d.x) + (d.y * d.y) + (d.z * d.z)),
})
],
[
Operator.MEAN
]
),
];
// create a 15 second session using the feature generators
const session = Moonsense.startSession({
duration: 15000, // 15 seconds
featureGenerators,
});
As shown above, the Feature Generators when specified via the SessionConfig instruct the SDK to generate the corresponding feature and add them to the Bundle created by the SDK.
Note that the Feature Generators work on a per Bundle basis and only summarizes values based on what it reads off the provided Bundle. The generated features are subsequently available in the features field in the Bundle.
Refer to the documentation for additional information around usage:
The features generated can be found using the appropriate key(accelerometer-x for eg.). The summarized feature should be available in the associated Feature struct as either a BytesList, DoubleList, DoubleMap, Int64List or StringList.
The feature generators can be specified along side raw sensors to be captured when starting a session. The following table describes the outcome when using a combination of the two when used in the SessionConfig.
| sensorTypes | featureGenerators | Comment |
|---|---|---|
| Not Provided | Not Provided | All available sensors are captured and no features are generated |
| Provided | Not Provided | The specified sensors are captured and no features are generated |
| Not Provided | Provided | The specified features are generated and no sensors are captured |
| Provided | Provided | The specified sensors are captured and specified features are generated |