OPENXR: UNIFYING REALITY - UNIFYING REALITY - BRENT E. INSKO, PHD - KHRONOS GROUP
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Unifying Reality
OpenXR: Unifying Reality
Brent E. Insko, PhD
Lead XR Architect at Intel &
OpenXR Working Group Chair
© Khronos® Group Inc. 2020 - Page 1
What is OpenXR?
OpenXR is a royalty-free, open standard that provides
high-performance access to Augmented Reality (AR) and
Virtual Reality (VR)—collectively known as XR—platforms
and devices.
© Khronos® Group Inc. 2020 - Page 2
OpenXR handles communication to and from an
application and an XR platform
Application XR device
© Khronos® Group Inc. 2020 - Page 3
OpenXR handles communication to and from an
application and an XR platform
Application XR device
• Head & hand pose information
• Controller/input state
• Display configuration / form factor
© Khronos® Group Inc. 2020 - Page 4
OpenXR handles communication to and from an
application and an XR platform
• Image(s) to display
• Audio
• Haptic responses
Application XR device
• Head & hand pose information
• Controller/input state
• Display configuration / form factor
© Khronos® Group Inc. 2020 - Page 5
What Problems are we trying to Solve with OpenXR?
© Khronos® Group Inc. 2020 - Page 6
Consider the variety of XR Platforms available…
© Khronos® Group Inc. 2020 - Page 7
Cross Platform XR Application Development
Before OpenXR, program to
each platform’s proprietary API
© Khronos® Group Inc. 2020 - Page 8
Cross Platform XR Application Development
With OpenXR, program to a single,
common, high-performance API
© Khronos® Group Inc. 2020 - Page 9
OpenXR is used with a 3D API
Application or Engine
Display,
High-performance, low-latency composition and
Cross-platform access to XR
3D rendering and composition* optical correction HMDs and sensors
Multiview parameters XR application lifecycle
Context priority Input device discovery and events
Front buffer rendering Sensor tracking and pose calculation
Tiled rendering (beam racing) Frame timing and display composition
Variable rate rendering Haptics Control
OpenXR is strongly influenced by Vulkan with a
* OpenXR can be used with other 3D APIs shared spec toolchain and support for API layers.
OpenXR is a “lower-frequency” API than Vulkan
such as Direct3D, OpenGL and OpenGL ES and is a much smaller spec
© Khronos® Group Inc. 2020 - Page 10OpenXR Flexibility
VR
Oculus Quest / Quest 2 AR VR
Valve Index and other
SteamVR compatibile HMDs Oculus Rift S
Microsoft
HoloLens 2
HTC Vive Cosmos Windows Mixed Reality
All In One Tethered
© Khronos® Group Inc. 2020 - Page 11Structure of an OpenXR App
• Get started
- Create an Instance
- Choose extensions, layers, bind to graphics API
• Find out where/how to run
- Get HMD characteristics - mono/stereo, form factor etc.. /user/hand/left/input/a/click
(/interaction_profile/ControllerCorp/fancy_controller/input/a/click)
• Set up interaction/input handles
- Bind physical inputs to actions OpenXR Runtime
grab_object, teleport etc. .../input/trigger/click
.../input/a/click
Explode
Teleport
• Prepare your immersive experience
- Create Session
- Create Swapchain to drive the display XrAction: “Teleport”
• Participate in the frame loop
- Update positions
- Handle input and haptics
- Manage swapchain to drive
imagery to the display
- Poll for events
© Khronos® Group Inc. 2020 - Page 12Applications, Game Engines, Rendering Engines,
Development Browsers using OpenXR! - Fall 2020 - 2021
Conformant AR and VR platforms released - Summer & Fall 2020
Conformance suite developed and Adopters program released - July 2020
OpenXR 1.0 Specification Released - SIGGRAPH, July 2019
0.90 Provisional Specification Released - GDC, March 2019
OpenXR Working Group Formed - 2017 © Khronos® Group Inc. 2020 - Page 13Latest Developments with OpenXR
© Khronos® Group Inc. 2020 - Page 14OpenXR Widespread Industry Adoption
Conformant Implementations
Microsoft HoloLens 2 Oculus Rift S
HTC Vive Cosmos
Windows Mixed Reality Headsets Quest and Quest 2
Valve SteamVR Varjo Collabora
Transitioning from OpenVR to OpenXR Preliminary support shipping now Open-source Implementation
Engine Support
Unreal Engine Unity Engine Chromium 81 for Chrome and Edge Browsers
Support in 4.24. Optimizations in 4.25 OpenXR Plugin Preview in 2020.2+ OpenXR default backend for WebXR
© Khronos® Group Inc. 2020 - Page 15OpenXR Applications
Blender 2.83 integrates OpenXR to deliver native VR
scene inspection capabilities
Minecraft’s RenderDragon rendering engine uses
OpenXR for desktop VR support
Google Chrome
Google Chromium
Microsoft Edge
uses OpenXR as its default backend for WebXR,
Microsoft Flight Simulator Sim Update 2 enabling Google Chrome and Microsoft Edge
uses OpenXR for VR functionality browsers to use any OpenXR-compatible hardware
© Khronos® Group Inc. 2020 - Page 16What’s Next for OpenXR?
© Khronos® Group Inc. 2020 - Page 17OpenXR Overlays Extension
• Simultaneous in-world applications in OpenXR
• Can overlay one app’s 2D content “on top” of an
existing application’s content
• Overlay applications add a rich content into other XR
Applications: e.g. Desktop OS windows in-world, In-
game HUD, Virtual keyboard, Chat (e.g. Pluto VR's
Pluto)
© Khronos® Group Inc. 2020 - Page 18Hand Tracking Extension • Standardize the definition of a common set of hand joints • Five fingers are named as: thumb, index, middle, ring and little. • Each finger has 5 joints: tip, distal, intermediate, proximal, metacarpal, except thumb lacks intermediate joint. • Wrist joint - the pivot point of hand motion & Palm joint - the center of hand and easy access to palm orientation. • This extension fits both VR and AR experiences. © Khronos® Group Inc. 2020 - Page 19
Advanced UI Cross-Vendor OpenXR Extensions
Developers can build cross-platform applications that use
advanced UI solutions from different technology vendors
OpenXR API layers can be used implement extensions
Hand Tracking Eye Tracking
26 unique joints per hand for fully Eye gaze interaction for intuitive interfaces
articulated hands visible to the user 2-Step Interaction
Shipping on HoloLens 2 Hand-eye coordination
and Microsoft Hand Mesh Extension Natural aiming
for HoloLens 2 layers over it
Shipping on HoloLens 2
Ultraleap developer preview available
© Khronos® Group Inc. 2020 - Page 20What are we looking at for the future of OpenXR?
• Fixing a few things from OpenXR 1.0
• Expanding support for AR and MR
- Scene understanding
• Additional User input/output capabilities
- Expanded haptics support
- Expanded body tracking support
- Expanded controller and device support
© Khronos® Group Inc. 2020 - Page 21OpenXR and Edge Server Applications with 5G
OpenXR will enable AR Significant industry
applications to run portably effort in developing
on edge server architectures Low latency this use case
Sensor Data
Runtime using 5G implemented
across device and server and
accessed through OpenXR API
Sensor handling
MEC (Multi-access Edge
Display composition Computing) Server
1. Processes sensor data, can include
Wireless mobile device machine learning for environmental
with display and sensors lighting, occlusion, scene semantics,
object reconstruction and UI
Generated 2. Generates imagery from 3D
models, can include stereo, foveal
Augmentations & rendering, ray-tracing, optics pre-
Scenes distortion, varifocal processing
© Khronos® Group Inc. 2020 - Page 22Now is the time to get involved with OpenXR!
© Khronos® Group Inc. 2020 - Page 23Now is the time to get involved with OpenXR!
Khronos open sourced conformance tests and launched Adopters Program
Officially conformant runtimes from:
Microsoft, Oculus, HTC, Valve, Varjo and Collabora
Hand and eye tracking cross-vendor extensions for advanced UI
OpenXR being used by real software:
Unreal! Blender! Chrome! Edge! Unity! Flight Simulator! Minecraft!
Virtual Reality
Now is the time for application V
developers to leverage OpenXR
for widespread deployment!
A
Augmented Reality
© Khronos® Group Inc. 2020 - Page 24How to get started…
https://openxr.org
• OpenXR Forum and Slack Channel
- Forum: https://khr.io/openxrfeedback
- Discussion: https://khr.io/slack
© Khronos® Group Inc. 2020 - Page 25What Resources Are Available?
200+ page specification Reference Pages Overview Guide
© Khronos® Group Inc. 2020 - Page 26Github resources
• https://github.com/KhronosGroup/OpenXR-Docs
- Contains the source for generating the
specification document and reference pages
-
- Contains the openxr header files
• https://github.com/KhronosGroup/OpenXR-
Registry
- Contains the specification, reference pages,
and overview guide
• https://github.com/KhronosGroup/OpenXR-SDK
- Contains the source for: the loader, some
basic API layers, and a test sample
- For the current best example code, see:
src/tests/hello_xr
© Khronos® Group Inc. 2020 - Page 27Additional Resources from OpenXR-Supporting Companies
• Collabora: OpenXR Masterclass from Laval-Virtual 2020
• Collabora: OpenXR easy-to-read example for OpenGL/Linux
• Holochip: OpenXR Samples
• Microsoft: Getting started with OpenXR
• Microsoft: OpenXR app best practices
• Microsoft: OpenXR performance
• Microsoft: OpenXR troubleshooting
• Microsoft: Getting started with OpenXR
• Microsoft: OpenXR Samples for Mixed Reality Developers
• Oculus: OpenXR for Oculus
• Ultraleap: OpenXR Hand Tracking in Unity
• Unity: OpenXR Plugin
• Unreal: Developing with OpenXR
© Khronos® Group Inc. 2020 - Page 28Summary
• What is OpenXR?
• What are the Problems we are trying to Solve?
• What are the latest developments and future of OpenXR?
• How to get started…
© Khronos® Group Inc. 2020 - Page 29Widespread Industry Support
OpenXR is a collaborative design that integrates many lessons from proprietary
‘first-generation’ XR APIs to create a new generation API with cutting-edge
capabilities and a flexible, extensible, future-proof architecture
© Khronos® Group Inc. 2020 - Page 30OpenXR Win-Win-Win
XR End-Users
Can run the apps they want on
their system – reducing market
confusion and increasing
consumer confidence
XR Vendors
Can bring more
applications onto their
platform by leveraging
the OpenXR content
ecosystem
www.khronos.org/openxr/
XR ISVs
Can easily ship on more
platforms for increased
market reach
© Khronos® Group Inc. 2020 - Page 31https://www.khronos.org/openxr/
© Khronos® Group Inc. 2020 - Page 32You can also read
