Introduction to Touch Events

The Moonsense SDK provides a seamless mechanism to observe touch/pointer events when integrated. The overall goal of capturing touch events with the SDK is to model the reality of users interacting with the user interface of the application.

This document provides context around the information contained within the Pointer events. It also describes the nuances of how these events are populated per platform (i.e. iOS, Android & Web).

Pointer

When a touch, stylus or mouse event is detected by the SDK, a Pointer event is generated. Pointer events operate in the coordinate space of the screen, scaled to logical pixels. Logical pixels approximate a grid with about 38 pixels per centimeter or 96 pixels per inch. This allows analysis be performed independent of the precise hardware characteristics of the device. In particular, features such as touch slop can be defined in terms of roughly physical lengths so that the user can shift their finger by the same distance on a high-density display as on a low-resolution device.

The Pointer event contains the determined_at field which captures the time the event occurred. This time is reported as milliseconds and is relative to an arbitrary timeline. The type field reports the type of input was used to generate the Pointer event: TOUCH, STYLUS, INVERTED_STYLUS or MOUSE.

All Pointer events contain the coordinate position of the pointer in logical pixels. When a Pointer event is part of a gesture, the delta field is computed on subsequent events. This field reports the distance in logical pixels that the pointer has moved since the last data point. The delta field will always be empty in the first Pointer event of a gesture. The device field is a unique identifier per gesture and can be used as a logical grouping of Pointer events that belong to that particular gesture. Note that all platforms (iOS, Android and Web) will report a new device for each new gesture.

The Pointer event contains information about the radius of the contact ellipse along the major (radius_major) and minor (radius_minor) axis, reported in logical pixels. The radius_range field will contain the upper and lower bounds for the reported radius. The size field will return the area of the screen being pressed. See the Platform Considerations section below for more information on how these fields are reported on different platforms.

In some instances the Pointer event will contain the pressure and pressure_range fields. The pressure field will contain the pressure, or force, associated with the event as well as the upper and lower bounds of the pressure range.

When a stylus or an Apple Pencil is used, the Pointer event may contain one or more of the following fields: distance, distance_range, orientation and tilt. The distance field reports the distance of the input device from the input surface and the distance_range field reports the possible range of distance values. The orientation field reports the orientation angle, in radians, of the input device. The tilt field reports the tilt angle, in radians, of the input device.

The buttons field of the Pointer event is a bit field calculated using a bitwise AND operation over constants that represent individual buttons on a device. The possible values for this field are device dependent. See the Platform Considerations section below for more information.

Lastly, when a Pointer event is captured on an iOS device, the is_software_keyboard field will be true if the event was detected on the Window that contains the software keyboard.

At the present time the Pointer event contains two other fields, synthesized and obscured. The obscured field is deprecated and will be removed in a future version. The intent of the synthesized field was designed to represent whether the event was generated by some automated mechanism. At the present time this field returns false on all platforms.

Platform Considerations

Android

type

Android reports type as TOUCH for events originating from the finger, STYLUS for an actual stylus, MOUSE for a connected mouse device and INVERTED_STYLUS when using the back end of a stylus i.e. the eraser.

device

device is reported as a unique identifier that corresponds to a single gesture. A gesture is defined as a series of Pointer events that start with PointerChange.DOWN followed by zero or more PointerChange.MOVE events and finally PointerChange.UP. Starting a new gesture will report a different id for device. When using multi-touch, each finger will report a unique device id. The device id will never be the same across two gestures.

distance and distance_range

The distance and distance_range fields are only available when the type is STYLUS. In most cases the distance_range is reported from 0.0 to 1.0.

pressure and pressure_range

pressure and pressure_range are only supported on a subset of Android devices. Keep an eye on the data reported by the device to determine if the pressure readings are recorded. Devices that report pressure also report an associated pressure_range.

orientation and tilt

On Android, orientation is only reported for TOUCH and STYLUS types when available. tilt is only reported for the STYLUS type.

radius_minor, radius_major, radius_range and size

The fields radius_minor, radius_major, radius_range and size are only reported when available.

buttons

Android only reports button clicks when the type is either MOUSE or STYLUS. For a MOUSE, the button field will return a mask that can be used to determine if the primary or secondary button was clicked.

is_software_keyboard

Android does not report touch events for the on-screen keyboard. This field will always return a false.

iOS

type

All touch events from iPhone devices will always return the type of TOUCH. When an Apple Pencil is used on iPad, the type will be STYLUS. When an iPad keyboard with a trackpad is connected AND the UIApplicationSupportsIndirectInputEvents setting is set in your app's Info.plist, movements via the iPad's trackpad will return the type of MOUSE. The type INVERTED_STYLUS is not available on iOS.

device

On iOS, the device field is an opaque, unique identifier that corresponds directly to the UITouch event responsible for the Pointer event. All phases of the UITouch will have this same device value.

radius_minor and size

iOS does not support the radius_minor or size field.

pressure and pressure_range

iOS only reports pressure on iOS devices that support 3D Touch and iPad devices using an Apple Pencil.

distance and distance_range

The distance and distance_range fields are not available on iOS.

orientation and tilt

The orientation and tilt properties are only available when an Apple Pencil is used as the input device.

buttons

The buttons attribute is only available on iOS 13.4 or later and when the UIApplicationSupportsIndirectInputEvents setting is set in your app's Info.plist. The buttons field will contain the raw value of the UIEvent's buttonMask optionSet. With this you can determine whether the primary or secondary button was pressed during the event.

Web

type

Web reports type as TOUCH for events originating from the finger, STYLUS for an actual stylus, and MOUSE for a connected mouse device. INVERTED_STYLUS is not supported by browsers.

device

device is reported as a unique identifier that corresponds to a single gesture. A gesture is defined as a series of Pointer events that start with pointerdown followed by zero or more pointermove events and finally pointerup. Starting a new gesture will report a different id for device. When using multi-touch, each finger will report a unique device id. The device id will never be the same across two gestures.

distance and distance_range

The distance and distance_range fields are not available as part of the Web SDK.

pressure and pressure_range

pressure and pressure_range are only supported on a subset of devices. Keep an eye on the data reported by the device to determine if the pressure readings are recorded. Devices that report pressure also report an associated pressure_range.

orientation and tilt

On Web, orientation is only reported when the rotation of the pointer can be determined.

radius_minor, radius_major, radius_range and size

The fields radius_minor, radius_major, and size correspond to the PointerEvent.width and PointerEvent.height. The larger of the two will be reported as the radius_major with the smaller of the two being reported as the radius_minor. The size is the area of the square produced by these measurements. radius_range is not supported on Web.

buttons

Web reports buttons as hardware supports. The number represents one or more buttons being pressed. For more than one button pressed simultaneously, the values are combined. For more information on values, see the MDN entry.

is_software_keyboard

Web does not support the is_software_keyboard flag.