Moving towards setting the input standard for the Metaverse.
We started Wearable Devices Ltd. with a vision to transform interaction with technology to be as natural and intuitive as real-life experiences. The future of computing is tilting toward wearable computers such as Smartwatches and AR/VR devices, which require instant and relevant output interaction based on the user’s movement, aim and perspective – translating the user’s intentions and inputs into actions.
The pace of technology advancements has always been dictated by user interfaces. We asked ourselves – how can we extend our hands to reach and touch the digital world? It was obvious that the solution must provide a natural, intuitive, and simple user experience, which doesn’t require elaborate hardware, heavy compute power, or forced user interactions limited to line of sight and field of view.
We envisioned an interface that will provide a Handsfree, Eyes-Free and Touchless interactive experience. Such a human machine interface must provide an extension of our body. Such an extension which one can move and manipulate without paying full attention - in a similar manner to a hand movement. Our hand is the best user interface ever invented, so utilizing finger movements and hand gestures seemed obvious. The challenge was to create an interface which has the right form factor together with an exceptional user experience.
The human wrist is a highly valuable “real-estate” for sensing the human body. A wristband device seemed like the perfect location and form factor. It can be attached to a watch’ case, providing a socially acceptable, discreet and fashionable device. It leaves the inner side of the wrist area free for sensing the nerve bundles and arteries which pass directly beneath the skin, enabling effective bio-potential sensing capabilities.
This is how Mudra - “Gesture” in Sanskrit – started: a neural input wristband to control digital devices using subtle finger movements. Sensors on the wrist detect electrical impulses sent from the brain to the fingers through motor neurons. Using advanced electronics and AI-machine learning algorithm, it deciphers each unique neural pattern with its associated finger movement. We then bind each movement, or gesture, with a control function, to achieve natural and seamless interaction.
The Mudra platform detects discrete gestures such as individual finger or tap movements, continuous interactions which involve applying fingertip pressure between two fingers, and mid-air gestures which combine arm movements. These gestures can function as swipe, scroll, select, drag & drop, and more.
There are numerous challenges on the path to delivering a neural interface wristband product. It must be accurate yet comfortable, stylish yet robust. It must provide a great user experience. All of which require the development of cutting-edge electronics and software algorithms, packed into an ergonomic and stylish band design. Bringing such a product to market brings its own challenges: adopting a new interface requires a change in long formed user habits and perceptions.
One of the first public steps in realizing our vision, was launching the Mudra Band for Apple Watch Indiegogo campaign. Thanks to an amazing backers support, the product gained mass media attention. The Mudra Band won the CES 2021 Innovation Awards Honoree award and the Best Wearable Award of CES 2021.
The notion of touchless Apple Watch control was recently adopted by Apple with AssistiveTouch. While it uses coarse gestures, it is fully integrated with the watch’s operating system and provides users with the functionality that the Mudra Band originally sought to deliver - managing calls, controlling music, taking selfies, and dismissing notifications.
Since controlling the watch itself can now be performed using AssistiveTouch, we decided to shift our focus on to the next step in the realization of our vision - touchless control of the iPhone.
Mudra-Air Touch
The new functionality adds touchless control of the iPhone. It allows users to swipe screens, launch applications, browse between apps, adjust controls, and interact with all phone applications with an air-mouse and simple finger gestures. Intuitive scrolling, tapping to take a selfie and switching back to the home screen is as easy as a flick of the wrist.
You will be able to operate and control your iPhone from a distance or, when your hands are occupied, when you wear gloves, and while multi-tasking. We plan to integrate this touchless functionality into the iPad, Mac and AppleTV very soon. We work parallely on releasing the above-mentioned watch functions later this year.
We believe that the mixture of air-touch interactions and finger movement gestures is the perfect realization of our goal – setting the input standard for the Metaverse.