When you take a photo with your phone, the AI camera can identify what's in the frame (for example, the blue sky or spread of food), and apply intelligent enhancements that correspond to the scene, generating detailed and vibrant images. You can remain productive, even when your hands are occupied, thanks to voice assistants like HiVoice or Siri on your phone. Make payments with AI-powered face recognition, in which your phone stores and analyzes your facial profile, even recognizing your behavioral patterns, to determine when it is you making a purchase, or logging in to an account.
In 2017, the release of Huawei's Kirin 970 captured the industry's imagination. As the first-ever mobile chipset to integrate an independent neural processing unit (NPU), the Kirin 970 represented a milestone for mobile AI. The following year, Huawei unveiled its HiAI platform, opening up Huawei's chipset, device, and cloud capabilities to global developers, and providing invaluable assistance by bolstering a wide range of apps with newfound intelligence.
Since its advent, HUAWEI HiAI has focused on applying innovative new technology from the bottom layer of the system on up. Distributed intelligence forms the key to transforming device software and hardware from isolated capabilities into a collaborative, mutually-reinforcing ecosystem. In this way, HUAWEI HiAI enables software and hardware makers to facilitate innovation in their respective areas of expertise, and contribute to the seamless user experience of tomorrow.
Technology seems like magic, in a sense. It helps unlock unforeseen potential, enriching the lives of countless individuals, in particular, those with disabilities. New apps are constantly being introduced, offering life-altering capabilities.
Huawei has teamed up with IIS Aragon and DIVE Medical to jointly launch the TrackAI project, which is dedicated to helping ophthalmologists run visual tests for children, using Huawei smart devices equipped with HiAI. Numerous medical institutes around the world, in China, Spain, Vietnam, Mexico, and Russia, among other countries, have begun amassing the plethora of data required to train the AI algorithm, through their work with over 2,000 visually-impaired children.
TrackAI's complete detection system consists of the Device for an Integral Visual Examination (DIVE), a Huawei P30 mobile phone, and a Huawei MateBook E tablet. The system displays visual stimuli on a screen, and uses an eye track to detect the child's focus. The system can also learn the differences between children with and without an eye disease. Lastly, the Huawei P30 smartphone runs a pre-trained machine learning model, powered by HiAI, to detect whether the child has a visual impairment.
The HiAI-powered TrackAI system can help stop an eye disease in its tracks
Compared with traditional models, in which data is uploaded to the cloud for analysis, with the results transferred back to devices, HUAWEI HiAI's on-device analysis is remarkably efficient. In leveraging the local Kirin chipset for AI processing, users have access to real-time analysis, with less latency. Even in rural areas where Internet access is spotty, doctors can still use the TrackAI system for testing. This provides for tremendous benefits for children's healthcare in developing countries.
On-device processing also provides for enhanced privacy safeguards. Users can rest assured that data is stored only on their device, avoiding the risks associated with cloud storage, such as data leak.
Huawei has drawn on AI to improve the lives of those living with disabilities, in a myriad of other ways.
In collaboration with the Polish Blind Association, Huawei developed Facing Emotions, an app designed to assist the blind and visually-impaired perceive emotions through the power of sound. The app uses the rear camera and AI on Huawei phones to translate human emotions into unique sounds. For the millions of people who are unable to see faces and read emotional cues, this offers a truly life-altering capability, bringing them closer to friends and loved ones. Imagine the joy of "hearing" a smile for the first time!
There's also StorySign, an app that helps deaf children read by translating the text from selected books into sign language. Huawei partnered with the European Union of the Deaf, Penguin Books and Aardman Animations, as well as other organizations, in developing StorySign. When a child opens a selected reading book in front of them, then opens the app and holds the phone over the page, an avatar signs the story, while the app highlights each word that has been signed. Thanks to multi-lingual Optical Character Recognition (OCR) and document adjustment technology, StorySign now supports more than 10 European sign languages. StorySign enriches of lives of deaf children and their parents, opening up a wondrous world of storytelling and literature, in which they learn to read and sign together.
HUAWEI HiAI powered StorySign supports more than 10 European sign languages
Huawei has also provided crucial assistance to the Chinese-developed Qimingtong app, which is designed to help the visually-impaired better interact with the world around them. The app reads text out loud, such as that from newspaper articles, letters, and product user guides. It also leverages HUAWEI HiAI capabilities, including face detection and facial feature detection, and enables the visually-impaired to take pictures following simple voice instructions.
All of the above apps are powered by AI, and it is HUAWEI HiAI's primary mission is to make app development effortless.
HUAWEI HiAI provides developers with access to the truly boundless potential from Huawei's chipset, device, and cloud technology. On the chipset side, developers benefit from cutting-edge NPU acceleration. Device capabilities bolster face, image, text, and speech recognition, while cloud technology enables apps to provide timely scenario-based services.
Developers are dreamers, and HUAWEI HiAI is the platform that helps them fulfill their dreams. Since its debut in 2018, HUAWEI HiAI has connected more than one million developers, and 4000 partners.
SketchAR, a tool for teaching drawing using augmented reality (AR) and AI, offers a prime exactly of how HiAI has revolutionized user experience. It enables users to transmit an image from their device onto any surface, such as a sheet of paper or white wall. The image can then be used on their device as a template for manually drawing on the surface of choice. SketchAR utilizes HUAWEI HiAI's NPU acceleration, to boost image recognition speeds by up by 40%, for improved accuracy and greater responsiveness.
The Chinese-developed app Lvmuxia (Green Screen Compositor) helps users composite a captured video with a background video. Typically, green screen compositing requires powerful computing capabilities, and could previously only be accomplished on the cloud. But as on-cloud computing poses a number of challenges for developers, including high costs, high latency, and privacy risks, it was not practical. By working with HUAWEI HiAI, the Lvmuxia app has overcome those challenges, providing on-device AI capabilities, and shortening the app's time to market.
While attending a conference or lecture, you may want to take photos of the PowerPoint slides for future reference, but the quality of the images can be poor, particularly if you were seated in a corner, or there is some sort of visual obstruction. HiAI provides an elegant solution, with its document adjustment feature, in which photos are straightened, clarifying text and removing unwanted corners. With the OCR feature, you can even add notes or correct mistakes in the text.
From single-device AI to distributed AI
In November 2019, Huawei introduced HiAI 3.0, an open AI capability platform that allows smart devices to share AI computing power between them, representing the tremendous leap from single-device AI to distributed AI.
In its infancy, HUAWEI HiAI 1.0 only supported single type devices. 2.0 expanded to support such devices as phones, tablets, and smart screens. HUAWEI HiAI 3.0 goes even further, pooling hardware resources to form a super device. Powered by distributed AI, devices mutually reinforce each other, providing users with the best possible experience, given the resources at its disposal.
Smart devices are designed to fulfill specific needs, but each category of device comes with drawbacks. For example, smart TVs, watches, and earphones excel in collecting images, videos, and sensor data, but fall short in terms of sheer AI computing power. Smartphones are equipped with increasingly powerful photography features, but still don't compare to dedicated televisions and surveillance cameras, in many regards. Their sound collection capabilities also pale in comparison to a speaker's microphone array.
Huawei developed HiAI 3.0 in response to the ever-present need for enhanced capabilities on smart devices. It works by pooling the hardware resources from different devices to form a super intelligent system. When enriched with shared AI capabilities, all of the participating devices are equipped to provide seamless, cross-device intelligence that is responsive to any and every user whim.
By drawing from such basic distributed technologies as distributed virtual bus and device virtualization, HUAWEI HiAI 3.0 facilitates high-speed connectivity between devices, allowing them to share capabilities and reinforce each other. This has seemingly countless applications in real life.
For example, fitness apps have often been regarded as less effective than professional personal trainers. But HiAI 3.0 helps turn such apps into viable personal trainers in their own right. HiAI 3.0 connects the user's smart TV, phone, and speaker to form a wholly-integrated, super intelligent system. The system can use the TV's camera to capture the user's posture, the phone's AI computing power to analyze the images, and determine whether the user's posture is standard based on their skeleton information, before finally having the speaker remind the user to correct their posture by voice.
When the user gets into their car, their phone can automatically connect to the car and utilize the in-car microphone to pick up sounds, while the in-car camera uses the phone chipset's AI computing for driver monitoring. If any driver fatigue or distractions are detected, an alert will be played.
On-device, distributed AI allows devices to "see", "hear", "sense", and "calculate" with greater precision and sensitivity. Fragmented experiences are merged into one consistent, cross-device experience, and device silos are connected to form a super device.
This basic understanding underpins the new paradigm that is Huawei's Seamless AI Life strategy — unbounded intelligence in all scenarios. Powered by AI, diverse hardware resources, including those from smartphones, are pooled, and mutually reinforcing, providing the seamless flow of information across all usage scenarios, and the connected intelligence that will power innovation.