Thinking beyond touch

Designing multimodal applications to support platforms of the future.

Note: This is more thought on how we would design mobile applications of the future, not an answer.

Mobile applications are not just limited to mobiles with upcoming AR glasses tethered to smartphones or the announcement of Windows 11 supporting the android application. Your apps might find being used in other platforms and mediums. This also calls for rethinking how we design and develop interactions for mobile applications. The future is more multi-modal.

Nreal

Windows 11

Chromebook

Samsung DeX

When we design for mobile applications we focus on touch input, but a multimodal approach will need to focus on states like hover and focus.

Different types of interactions that we need to consider for developing future applications.

Keyboard, Mouse, and Controllers

Gesture

Gaze

Voice

Fluid Interfaces

Apart from supporting multimodal interactions, we need to focus on designing interfaces and taking advantage of different screen sizes, and taking advantage of extra size or landscape mode in large screens.

https://material.io/design/layout/responsive-layout-grid.html#breakpoints