Where will Apple, Google and Meta lead us?
In 2023, there were many exciting tech announcements, especially for XR enthusiasts, leading to many new and exciting applications. However, they also gave us a glimpse into the future plans of the tech giants. Those are actively driving the shift from mobile computing to spatial computing, further locking us into their platforms. Nevertheless, we, the developers in particular, should not only praise the strategies of Big Tech, but also critically question them. This shift is also giving us a rare opportunity to create a new open spatial web, where we can disrupt the monopolies’ platform economies of the web 2.0 era.
XR applications soon have the potential to leave their niche purposes behind. The following five areas of consideration are therefore intended to support my assertion that developers are overcoming some previous hurdles to develop applications with a high degree of usability and utility and that we are heading for the home stretch of mass adoption.
The growing technology ecosystem
It is apparent that Big Tech considers XR or the Metaverse as the future vision of computer-human interaction and is pushing the development of spatial computing with immense investments. This is being driven primarily by GAMAM (formerly GAFAM or Google, Apple, Facebook / Meta, Amazon and Microsoft), but also by players such as: NVIDIA, Disney, Unity, Epic Games, Niantic, Snap, HTC, Nokia, Tesla, ByteDance, Qualcomm, Samsung, and many more. They organize themselves in alliances and (standardization) consortia like the Kronos Group, OpenARCloud, OpenUSD Alliance or the Open Metaverse Standards Forum to enforce standards for communication in virtual worlds and interoperability of spatial content. Many well-known tech companies are active members here.
Google and Apple have chosen to bring XR applications to the mass market through small incremental steps, developing a full version of “spatial computing” behind closed doors for now. They understood that the smartphone and the operating system is a gateway. Users will be further and more deeply intertwined into their eco-systems before the big roll out of Spatial Computers begins.
Google is pushing more on the application level, giving developers tools to increase the utility of their applications. Most notably through Maps, Cloud (GCP) and the release of the GeoSpatial API, the company is providing the basic architecture necessary to enable location-based and persisted content in XR applications. They have been working with Adobe to reach a broader base of content creators.
Apple, on the other hand, is increasingly trying to create usability by perfecting the device genre through novel mature features of their wearables (Apple Watch, EarPods and iPhone) and with the combination of AI assistants and device networking. This culminated in arguably the most memorable event of the year, WWDC, which featured the announcement of the Apple Vision Pro.
Meta’s response quickly followed. Meta used to be heavily dependent on Apple and Google, as its services mainly run on the two platforms’ operating systems. In order to create its own ecosystem and become a gatekeeper itself, it has been taking a very profound and progressive approach for a few years now and is trying to gain market share for XR applications early on. This is to be achieved on the one hand by placing and linking the brand with the “metaverse” (social virtual worlds) and on the other hand by early availability of consumer oriented devices (Meta Quest Platform). Like the other two, Meta has recognized how significant the intersection of the two technologies AI and XR will be in the future, since this year has seen significant progress in the diffusion of the use of Large Language Models. It is apparent that the development of the technology enjoys relevance on a global level and is being driven forward at the highest level. This was most recently outlined by Mark Zuckerberg in his keynote at Meta Connect ’23
Whether and how the respective strategies can work out, I have illuminated in parts in my article “10 reasons why Meta will have it tough against Apple”.
The shift in content creation and software development
Artificial Intelligence: In order to sell Spatial Computer, manufacturers themselves must ensure that applications are developed with appropriate utilities. The year 2023 was precisely the moment when the latest breakthroughs in AI development came to fruition. In the area of content creation, we were shown throughout the year the capabilities that Large Language Models (LLMs) are already having. Since content creation for XR applications is significantly more complex than for linear 2D media, AI can simplify processes on both the user and developer sides. In particular, the field of photogrammetry is worth mentioning. Until recently, 3D replications of the real world involved significant effort. Here, technologies such as Neural Radiance Fields or Gaussian Splatting (NeRFs and GaSps) are revolutionizing the creation and visualization of digital twins. Meta has made its own LLM “LLAMA 2” openly available, which can create program code from text input, for example. Other example apps for process optimization and development of XR applications are Polycam, LumaAI, ChatGPT, Midjourney or SkyboxAI.
Interfaces: Computer interfaces of the future will also be trained using machine learning. Apple showed impressive AI-assisted gesture control at Vision Pro, and Meta is demonstrating the best hand-tracking model on Quest. Also for all other input and output possibilities like, eye-tracking, body-tracking, spatial audio, displays etc. the methods are becoming more robust and accessible for developers.
Tools: The development environment “Unity” takes a central role for the creation of interactive worlds, since it offers a very low-threshold access for developers. The creative development of individual ideas now comes into focus, because complex content creation pipelines and the handling of devices are simplified in such a way that work can be done much faster and thus more interactively. Almost every basic pipeline for content and app generation involves Unity. OpenUSD also establishes a standard that enables the exchange of 3D formats in a way that was previously unthinkable. At the application development level, significant API advances include OpenXR for providing APIs to hardware platforms, ARKit and ARCore as APIs for computer vision and tracking, and WebXR for creating web-based XR applications.
Also since this year, developers get easy developer tools to design the new interfaces of the future thanks to RealityKit, Qualcomm Snapdragon Spaces or the new version of MixedRealityToolkit. At WWDC or Google IO, development toolkits for XR applications were the core topics.
XR funding, research, ethics and investments
Funding: Not only have many countries recognized the value of XR and are investing in national funding programs, but XR is also recognized as relevant at the EU level. The European Commission has a funding program, Horizon Europe, in which XR is an integral part. One example is the XRECO program. These grants aim to fund innovative projects that have the potential to drive the XR industry and support startups, companies and educational institutions.
Research: Universities and private research institutes are working at the forefront of XR development. They are researching everything from improved user interfaces and more realistic virtual environments to new learning methods and safer and more efficient ways to use XR in everyday life.
Ethics: As XR evolves, so does the need to create ethical guidelines and regulation. It is important to ensure that XR technologies are used in a way that respects privacy, promotes inclusion, and does not have harmful effects on users. This is another area where the European Commission is taking action with the “EU Initiative for Web 4.0 and Virtual Worlds” presented on July 11, 2023, to establish frameworks for the development of a metaverse strategy. It is not only important for the safety of users but also helps developers to keep their moral compass intact.
Investments: In 2023, no less than two of the major tech companies, Apple and Meta, invested in the Munich location. In addition to such direct investments, Bavaria also invests in specialized XR hubs or the expansion of business networks. These hubs provide companies and researchers with the necessary infrastructure and resources to work collaboratively on the next generations of XR solutions. The Bavarian Business Association (VBW) has been organizing network meetings of business enterprises with local development companies specializing in XR applications since last year.
Expanding areas of XR use
Still, the question of utility of XR applications is the biggest driver of adoption. We already see niche applications in many sectors that are on the verge of scalable market maturity. These applications are diverse and naming them is beyond the scope of this article. Some of the most prominent application areas have grown tremendously this year:
Education: Schools and colleges are buying large amounts of headsets to explore the possibilities of a more lively and interactive education. Virtual field trips, simulations and interactive learning sessions make learning more effective and engaging. The XR HUB Bavaria has launched Adriane’s blog on its website, which is a wonderfully descriptive summary of the uses of new learning worlds.
Healthcare: From rehabilitation to surgery, XR is finding application in medicine to treat patients and give doctors a more accurate and detailed view of medical procedures.
Entertainment: Video games, movies and other media use XR to create an immersive experience. It allows users to immerse themselves in stories like never before.
Industry: From engineering to architecture, companies are using XR for prototyping, training and to improve workflows.
Tourism: location-based tours or experiences allow people to perceive guided tours in various culturally-historically relevant places that convey architecture, knowledge and history in a much more memorable way thanks to the use of XR applications.
A major influence on all areas of use was created by new capable and stable authoring tools that make the distribution of applications scalable and lower the barrier to entry for less tech-savvy people.
New business models and monetization of XR applications
The most important indication for a relevant diffusion of XR is if it enables new business models or boosts the incentive value creation. Both are already in full swing. Amazon has launched “Amazon Anywhere” in Niantic’s augmented reality game “Peridot,” which allows physical products to be sold within the application. Shopify is adding an augmented reality feature to preview products, allowing customers to interact with the product in 3D. Snap, too, has long pursued a strategy of introducing users to augmented reality technology and encouraging content developers to create filters which can be monetized. The most hands-on utility of XR applications in the mainstream is therefore likely to be seen in the e-commerce space. Other mass-oriented businesses that are already successful include Holoride with their in-car entertainment platform and Hologate with their Virtual Reality Arcades licencing.
B2B business models are still the main source of revenue for XR applications. They often align with software-as-a-service models, giving companies new ways to train, recruit employees, or impart knowledge. Local examples are Holo-Light, AR-Experts with “GiRi” or Blickwinkel Tour with the Spatial Presentstion Suite.
All five of the above considered areas clearly show the existing relevance of XR applications and the inevitable shift from mobile to spatial computing. Through further development and expansion, XR technology will soon feel like “normal”. In my article Let’s stop confusing people about “XR”, I talked about how we as developers should use the term to bring the concept to the masses. If we now focus on creating utility of applications and forget about talking about the benefits of the technology, it becomes apparent that we are looking at an extremely exciting future. We are counting on the tech giants to not only lead us to the dark side of technology where the current negative effects of closed platforms are exacerbated, but also to promote interoperable open standards so that we developers can create and monetize inclusive and useful applications of high social value.
