Data Model

The core data structure used by the SDK and the Moonsense Cloud is the Bundle. The Moonsense SDK transports data to serialized as a Bundle to the Moonsense Cloud and all of the consumption mechanisms use that (either Webhooks or the Python SDK)

Bundle

A Bundle is a collection of data points collected from multiple sensors on a mobile devices. All fields are optional and it's up to the client SDK to decide what data to collect and send and at what frequency.

FieldTypeDescription
accelerometer_dataArray[Accelerometer]Device acceleration including the gravity component.
client_timeClockDevice time when the bundle was constructed.
batteryBatteryDevice battery level and status data.
pointer_dataArray[Pointer]Pointer movement data as reported by the touchscreen or mouse.
linear_accelerometer_dataArray[Accelerometer]Device acceleration without the gravity component.
indexint32Continuous monotonic increasing index generated by the client.
text_change_dataArray[TextChange]Text change events triggered for a target input element.
key_press_dataArray[KeyPress]Key press events captured when available.
focus_change_dataArray[FocusChange]Loss or gain of focus for a target input element.

SealedBundle

A SealedBundle is a server-side envelope for a Bundle generated by the client SDK. It is used to append additional server-side state extracted from the request to each batch of events captured from the client.

FieldTypeDescription
bundleBundleThe bundle as received in the client request.
app_idstringApplication ID that was the source of the data.
credential_idstringCredential ID associated with the App.
session_idstringSession ID for a specific recording.
user_idstringPlaintext User ID extracted from the authentication token.
server_time_millisint64The server time in millis when the bundle envelope was constructed.
install_idstringUnique app install ID generated by the OS.
client_user_idstringThe client side user ID (e.g after customer authenticates an user).

Accelerometer

An accelerometer sensor reports the acceleration of the device along the three sensor axes. The acceleration values are in SI units (m/s^2). A typical device has two types of accelerometers: one that measures the acceleration of the device including the force of gravity, and one that doesn't include gravity.

FieldTypeDescription
determined_atint64Time of event dispatch in milliseconds, relative to an arbitrary timeline.
xdoubleX-axis acceleration in SI units (m/s^2).
ydoubleY-axis acceleration in SI units (m/s^2).
zdoubleZ-axis acceleration in SI units (m/s^2).

Battery

FieldTypeDescription
capacityint32Remaining battery capacity as an integer percentage of total capacity (with no fractional part).
stateBattery.StateCurrent battery state (discharging, charging or full)

Battery.State

Values
UNKNOWN
CHARGING
DISCHARGING
FULL

Clock

Neither Android, nor iOS provide a trusted time source. The best we can do is record monotonic and non-monotonic time coordinates and rely primarily on relative time measurements when computing features to be used to train models.

FieldTypeDescription
wall_time_millisint64Standard wall clock (time and date) expressed in milliseconds since the epoch. This is controlled by the user or the phone network and it may jump backwards or forwards unpredictably. This clock should only be used when correspondence with real-world dates and times is important.
timer_millisint64This clock MUST be guaranteed to be monotonic, and is suitable for interval timing when the interval does not span device sleep. This clock stops when the system enters deep sleep (CPU off, display dark, device waiting for external input), but is not affected by clock scaling, idle, or other power saving mechanisms.
timer_realtime_millisint64This clock MUST be monotonic, and needs to continue to tick even when the CPU is in power saving modes, so is the recommend basis for general purpose interval timing.

ClosedRange

A range (or interval) defines the boundaries around a span of values; ; for example, "integers from 1 to 100 inclusive."

FieldTypeDescription
lower_bounddoubleThe smallest value in the range (inclusive)
upper_bounddoubleThe largest value in the range (inclusive)

FocusChange

Represents a loss or gain of Focus for a target element.

FieldTypeDescription
determined_atint64Time of event dispatch in milliseconds, relative to an arbitrary timeline.
typeFocusChange.TypeThe type of focus event that was detected.
targetTargetElementRepresents the target field that detected the key event.

FocusChange.Type

Values
UNKNOWN
FOCUS_GAINED
FOCUS_LOST

KeyPress

Represents a keyboard event or key event. This data is collected whenever available.

FieldTypeDescription
determined_atint64Time of event dispatch in milliseconds, relative to an arbitrary timeline.
typeKeyPress.TypeThe type of key event that was detected.
special_keystringProvides a string value for a special, logical or modifier key that was pressed, for eg. 'Home', 'Left Shift' etc. The value reflects the actual key that was pressed and is therefore not masked. There is a possibility that the value returned by this field differs across platforms.
masked_keyint32The unicode value for the key pressed. This field is masked for privacy reasons and does not correspond to the exact key that was pressed. The pressed key is guaranteed to be a unicode character. Defaults to '0' if unavailable.
targetTargetElementRepresents the target field that detected the key event.
alt_keyboolFlag to specify the event as including an 'Alt Key'
control_keyboolFlag to specify the event as including an 'Control (Ctrl) Key'
meta_keyboolFlag to specify the event as including an 'Meta Key'. This includes the Windows Key (⊞) on windows and the Command Key (⌘) on MacOS
shift_keyboolFlag to specify the event was captured while the 'Shift' Key was active

KeyPress.Type

Values
UNKNOWN
KEY_UP
KEY_DOWN

Offset2D

A 2D floating-point offset that represents a distance in a Cartesian space from a separately-maintained origin point.

FieldTypeDescription
dxdoubleThe X component of the offset.
dydoubleThe Y component of the offset.

Pointer

Generic data model for touch, stylus, or mouse events. Pointer events operate in the coordinate space of the screen, scaled to logical pixels. Logical pixels approximate a grid with about 38 pixels per centimeter, or 96 pixels per inch. This allows analysis be performed independent of the precise hardware characteristics of the device. In particular, features such as touch slop can be defined in terms of roughly physical lengths so that the user can shift their finger by the same distance on a high-density display as on a low-resolution device.

FieldTypeDescription
determined_atint64Time of event dispatch in milliseconds, relative to an arbitrary timeline.
typePointer.TypeThe type of the pointer event that was used.
buttonsint64Bit field calculated using a bitwise AND operation over constants that represent individual buttons on a device.
deltaOffset2DDistance in logical pixels that the pointer moved since the last data point. Note that the delta is reported per gesture. The delta values reset when a new gesture is started.
deviceint64Unique identifier for the pointing device. Note that all platforms iOS, Android and Web report a new device id for each new gesture. The device can be used as a logical grouping of Pointer events that belong to a particular gesture.
distancedoubleThe distance of the detected object from the input surface.
distance_rangeClosedRangeThe range of possible values for distance.
obscuredboolSet if an application from a different security domain is in any way obscuring this application's window. Note that this field is not implemented as there is no means of deriving this information on all platforms.
orientationdoubleThe orientation angle of the detected object, in radians.
positionOffset2DCoordinate of the position of the pointer, in logical pixels in the global coordinate space.
pressuredoubleThe pressure of the touch.
pressure_rangeClosedRangeThe range of possible values for pressure.
radius_majordoubleThe radius of the contact ellipse along the major axis, in logical pixels.
radius_minordoubleThe radius of the contact ellipse along the minor axis, in logical pixels.
radius_rangeClosedRangeThe range of possible values of radius;
sizedoubleThe area of the screen being pressed.
synthesizedboolSet if the event was generated by some automated mechanism.
tiltdoubleThe tilt angle of the detected object, in radians.
is_software_keyboardboolSet if the touch was detected to have come from a software keyboard.

Pointer.Type

Values
UNKNOWN
STYLUS
INVERTED_STYLUS
TOUCH
MOUSE

TargetElement

Corresponds to a target element that is interacted with when collecting events using the SDK.

FieldTypeDescription
target_idstringA unique identifier that represents an element that is being interacted with. The id is guaranteed to be different for each element in the application. The SDK makes the best effort to ensure the consistency of this id. The user can override this value by providing their own id.
target_typestringThis type data accompanies the target_id. The SDK makes a best attempt at determining the type of the target based on the properties of the element. The user can override this value by providing their own type.

TextChange

Represents a message that is fired everytime a text change event happens on a specified input target.

FieldTypeDescription
determined_atint64Time of event dispatch in milliseconds, relative to an arbitrary timeline.
targetTargetElementRepresents the target input field that detected the text change event.
focusboolThis value represents whether the input field represented by target(see above) is in focus or not.
masked_textstringProvides a snapshot of the text contained in the input field represented by the target at the determined time. Note that the caller can infer the characters typed based on how this text changes over time. For privacy purposes this string is masked and does not correspond to the actual characters typed.
truncated_textboolReturns true if the captured text exceeds the maximum allowable limit. In cases like these the text is truncated and this flag returns a true.
actionTextChange.ActionDetermines if the text change was a result of a specific action.

TextChange.Action

ValuesDescription
UNKNOWNEncompasses all the other types of text change events
CUTincluding typing, swipe, autofill, autocomplete. Essentially anything that is not cut or paste.
PASTE