Invention: HoloLens mixed-reality smart glasses
Consumers may soon interact with the real world – and each other – through a new generation of data glasses. Developed by Brazilian software engineer and hardware inventor Alex Kipman and marketed by Microsoft, the HoloLens mixes reality with hologram-like overlays. Business people and hospital patients alike stand to benefit as applications like holographic phone conferencing and computer-assisted surgery adopt the technology.
Consumers today have access to a wealth of information through
their tablets and smartphones. But, on the flip side, they also spend close to
ten hours a day looking at computer screens. That's why computer expert Alex
Kipman at Microsoft wants to direct the human gaze back to the real world,
without losing access to computing power.
The inventor's mixed-reality data glasses, called the Microsoft
HoloLens, blend views of the real world with digitally overlaid "holograms".
Wearers can view and manipulate these lifelike images - for instance the three-dimensional
image of another person - while still seeing the real world around them.
Loaded with computing power, the HoloLens reads the
environment through the inventor's patented sensors. Users control the device
without keyboards, game pads or touchscreens. Instead, the HoloLens can interpret
hand gestures and voice commands thanks to built-in machine-learning algorithms.
The resulting mixed-reality experience integrates computers seamlessly into daily
activities - both at work and play.
Societal benefit
Some industry insiders believe that mixed-reality devices,
like the HoloLens smart glasses, could replace smartphones. Experts at Gartner predict
that, by 2020, 30% of web browsing sessions will be done without a screen.
Major drivers for the technology include consumer demand for assisted
navigation, gaming and live translations. In fact, by 2020, 100 million
consumers will rely on augmented reality - as mixed reality is sometimes called - for personal shopping assistance (comScore).
Developers are currently creating a host of applications for
Kipman's data glasses in the Windows Mixed Reality software environment of
Windows 10. For example, the thyssenkrupp industrial group includes the glasses
in their metering and quality control operations, while the famous American
medical institution Cleveland Clinic offers an interactive digital human
anatomy curriculum. In 2017, while operating on a patient to remove a malignant
muscular tumour, Spanish surgeons used the HoloLens to simultaneously view live
diagnostic pictures from magnetic resonance imaging and radiography data feeds.
Economic benefit
Perfected over the course of seven years, the HoloLens was
first introduced by Microsoft as a prototype in 2015. It features Kipman's
patented sensors for the company's Kinect video game controller. The HoloLens
is currently available as a development edition aimed at software programmers
for EUR 2 556. An enterprise version costs EUR 4 260.
Microsoft has sold around 50 000 HoloLens units, relatively few
compared with sales of over one million virtual-reality headsets in the third
quarter of 2017 alone. But the corporation, which earned EUR 73 billion
during the fiscal year 2017, is strategically "seeding" the market for
mixed-reality products through strategic development partnerships with
companies such as Lowe's, Boeing, Saab and Volvo.
Consulting firm Digi-Capital estimated the world market for augmented-
and virtual-reality devices at EUR 3.3 billion in 2016. The firm
projects that, driven by next-generation mobile augmented-reality hardware, the
market will reach EUR 85 billion by 2021. Once Microsoft has built an
attractive mixed-reality library, products such as the three-dimensional teleconferencing
app Holoportation could well join bestsellers like the Office 365 commercial suite
with over 100 million users.
How it works
Current-generation mixed-reality glasses include
"smart" ski goggles that project data such as altitude, temperature
and pulse into the wearer's field of vision. But these devices display only two-dimensional
text and graphics, while requiring a connection to a smartphone for computing
power.
The HoloLens is a stand-alone device, independent from
external PCs, tablets or smartphones. It contains all necessary computing power
in its custom-built processors. These include a CPU (central processing unit),
GPU (graphics processing unit) and custom HPU (holographic processing unit), an
industry first.
These processors collect several terabytes of data every
second as the HoloLens reads its environment. Data arrives through an inertial
measurement unit (IMU), a depth-sensing camera, a 2.4-megapixel video camera,
four microphones and an ambient light sensor.
As the name suggests, the HoloLens owes its innovative edge
to the waveguide lenses that "trick" the human brain into
interpreting photons as solid, three-dimensional things - hence holograms. They are computed on the GPU and displayed through
the device's "light engines", two tiny liquid-crystal-on-silicon projectors
which beam light onto the two lenses.
The image is adjusted between the two lenses for clarity and
depth, then blended with the real world. As a result, holographic images appear
to exist in front of the wearer - from a few centimetres to many metres away -
with true depth of field.
The inventor
Born in Curitiba, Brazil, Alex Kipman fell in love with
computer programming by playing video games at the age of six. "In
software, you are only bound by the limits of your imagination," says Kipman.
He joined a global company commensurate with his imagination when he joined
Microsoft in 2001 after earning his Bachelor of Science degree in Computer Software
Engineering from Rochester Institute of Technology.
The inventor landed his first smash hit - and became a star
of the tech community - when his Kinect motion sensor broke all records as the fastest-selling
consumer electronics device in 2008. So far, he is the primary inventor on more
than 150 US and Asian patents as well as three granted European patents,
including the technology behind the HoloLens.
Today, as Technical Fellow of the Windows and Devices Group
at Microsoft, Kipman remains closely involved in developing new technology for
the HoloLens and its related Windows Mixed Reality software platform. At just
40, his honours already include a Popular Mechanics Breakthrough Award (2009)
and being named the IPO Education Foundation's Inventor of the Year (2012).
In his private life, the outspoken futurist likes to retreat
to the simplicity of his family's off-the-grid farm in the Brazilian
rainforest. Unplugged from digital devices, he seeks out real-life outdoor
experiences with his wife and daughter.
Did you know?
The HoloLens takes a time-honoured technological principle
into the age of wearable computers. As early as the 16th century, theatres
employed a technique to project images onto angled sheets of transparent glass
for visual effects. In the 1940s, projecting text onto angled glass became the
principle behind heads-up displays used in fighter jets. Important data such as
altitude and speed could be projected into a pilot's field of view, allowing
full focus on the action without distraction. The same principle has been
applied for decades in autocues to display text to a speaker, unseen by the
audience.
Fast-forward to the 21st century and Kipman's HoloLens can
make people's holographic likeness appear in thin air with new levels of
sophistication - for instance during teleconferencing calls. And because the
mixed-reality headset requires no computer screen or keyboard, it could be part
of a revolution that would replace smartphones as the main communications
devices.
Looking into a not-so-distant
future, Kipman has predicted the "death" of the smartphone. Current
industry trends appear to support his statement - such as the shift towards voice search with digital assistants
like Cortana from Microsoft, Siri from Apple and Alexa from Amazon. Consumers
may one day be talking and gesturing to holographic "ghosts", while
the smartphone becomes a spectre of the past.
