I have been experimenting with augmented reality on various platforms for a while, and this article describes how to get started with Google’s AR Core platform. First published on gooroo.io.
In the past couple of years, if not more, the technology of virtual, augmented and, lately, mixed realities have taken a huge leap forward both for consumers as well as enterprise. Initially augmented reality (AR) almost entirely consisted of overlaying data on the real world, triggered by a shape, a logo, a QR code, or something else that could be easily recognised. Most of all it was very similar to a heads up display. Not really interacting with the real world, not really being part of your reality.
In January 2015 Microsoft announced the HoloLens Mixed Reality headset, which was released a year later. The HoloLens took augmented reality to an entirely new level, and Microsoft called this “mixed reality” (MR). The groundbreaking aspect of it is the environmental scanning and spatial mapping capabilities, which means the device knows the physical layout of your surroundings to an amazing degree of details.
3D model of my living room captured with HoloLens
This detailed awareness of the environment means that digital assets, or holograms, knows where their surroundings are and can act accordingly. You can have people sitting next to you on a chair or couch, place digital flowers on top of a real table, have characters navigate your actual living room and much much more. This spatial mapping and tracking capability essentially left other augmented reality implementations for dead.
Apple announced ARKit as part of iOS version 11, which, using only the camera on the device, can also track and environment, and not long after Google announced their ARCore. Both of these technologies are groundbreaking in being able to track the physical environment using existing hardware, and relying primarily on the camera to do the tracking and mapping. This means that you can use any new-ish iOS or Android device to view highly convincing augmented reality experiences.
ARCore and Unity 3D
To start building ARCore apps is not difficult. However, because the framework is still in preview release only a small number of devices are supported. If you don’t have one of these devices, it isn’t as much fun going through the tutorials.
To set up your development environment and start developing ARCore experiences with Unity is pretty straight forward and you can follow this guide. It will give you a simple app that shows the spatial grid and you can then place little Android robots on the real surfaces of the room.
Once you have your development environment set up, there are a number of fun things you can try out, if you are up for reading the documentation. This will improve over time, but for now it is very limited and examples to try out are hard to come by.
ARCore and Xamarin
If you aren’t a Java developer (I certainly aren’t) and still want to work with ARCore, there is now also a project to port over the API for using in C# projects with Xamarin. This is important, because the Xamarin community already has over 1 million developers signed up to use their products. That is 1 million more developers to use the ARCore technology. On top of that you can use Visual Studio to build your ARCore app, however, it should be said it is also very early days for Xamarin for ARCore, so bugs and “shenanigans” are to be expected.
To get started with Xamarin and ARCore you’ll need the community edition of Visual Studio with the Xamarin tooling installed. Then follow the instructions at the GitHub page for the ARCore Xamarin project.
ARCore and Unreal Engine
A popular tool for creating games is Unreal Engine, which has been used in creating great games and experiences. The unreal engine offers proven tooling to start building 3D experiences, which is why it fits so well in with ARCore and AR in general. Follow this guide to set up you development environment. Compared to the two approaches above, Unity and Xamarin, using the Unreal Engine does require quite a bit more low level setup and preparation. You will have to build you own version of the Unreal Engine source code and then set it up with the Android and ARCore tooling. The result can be as convincing as the other platforms.
Three.js and ARCore
There is also kind of a web approach to using AR Core. This is one of the most experimental technologies using ARCore still and requires a special build of Google Chromium in the form of an app for Android or iOS. The setup is not too difficult, but it does install very experimental software on your phone. The great thing is that you can use the very popular three.js JavaScript library to build AR experiences, and in particular the helper library three.ar.js provides a lot of the basic building blocks for it.
THREE.ARUtils.getARDisplay().then(init);
function init(display) {
vrDisplay = display;
// Set up three.js scene
renderer = new THREE.WebGLRenderer({ alpha: true });
scene = new THREE.Scene()
// Set up our ARView with ARPerspectiveCamera arView = new THREE.ARView(vrDisplay, renderer); camera = new THREE.ARPerspectiveCamera(vrDisplay, 60, window.innerWidth / window.innerHeight, vrDisplay.depthNear, vrDisplay.depthFar); vrControls = new THREE.VRControls(camera); update(); } function update() { // Update our controls/camera, the ARView rendering, // and our three.js scene vrControls.update(); arView.render(); renderer.clearDepth(); renderer.render(scene, camera); vrDisplay.requestAnimationFrame(update); }
Still Early Days
Although there are at least 4 different ways to get started using ARCore, it is still really early days. If you do want to take the jump and build an Android AR experience, expect bugs to surface, things to change with little or no notice, and some frustration. At least you will be on the forefront of AR development, and once the platform is stable and available to the general consumer, millions of users will have access to it.
I would suggest to choose your tool for development based on any experience you might have. If you haven’t got any experience, I would recommend using Unity 3D, as it is very well documented, has a large user base and can be used to build for a range of platforms, such as Xbox, iOS, mixed reality and more.
Have you played with AR or any other related technology? Let me know if the comments below.