How to interact the Basic Sample scene

The following is a walk through for playing the Basic Sample scene on a compatible device. The purpose of this sample is to demonstrate a minimal AR setup with persistence. The other samples are more advanced; a good handle of this one is recommended before exploring the others. Don't worry, it's simple!

How to start the Sample App

Open 6DSDKSampleApp. The app will have the 6D.ai logo.

On the first launch, a pop-up will ask you to authorize usage of the phone's camera; another pop-up will also ask you to authorize usage of location data. Every application build on top of the 6D.ai SDK needs user consent to these two things, or the 6D SDK functions will not work properly.

warningPlease make sure that Location Settings aren't disabled under Settings > Privacy.

Open the Basic Sample scene

The scene presents itself as a camera viewport with two cloud buttons, a mesh button, and three rotating 6D logos floating in AR.

The top button cloud_upload performs an API call to SaveToARCloud().

The bottom button cloud_download performs a call to LoadFromARCloud().

In addition, a colored tint starts spreading over objects: that's the real time world mesh.

As you use the scene and explore the real world with your phone's camera, the 6D SDK scans and maps the area. You can control when to save and load those scans with those two buttons.

When you load, a phase called relocalization allows the 6D SDK to find where it is, relative to a previously saved map of the same location. This allows the coordinate system of the AR experience to be identical between sessions and devices, allowing the persistence of objects (since 3D coordinates of virtual objects map to the same physical location) and multiplayer (since the location map can be used by multiple devices simultaneously).

Walk around your room, move your camera around. Everything that enters your field of view is getting mapped and can be pointed at later for relocalization. You can visually see the map by the colored tint of the mesh.

After a few seconds of scanning, hit the cloud_upload Save to AR Cloud button.

You can now completely close the Sample App.
Open the Sample App again and select the Basic Sample scene again.

You'll find that the logos are now floating at a different fixed arbitrary location. Immediately hit the Load from AR Cloud button cloud_download and the scene will download your previously saved map to relocalize your coordinate set.

When the prompt says "Relocalizing..." make sure that you point your camera at one of the areas you had previously scanned.

After a successful relocalization, you'll notice that the logos teleported back to their original location. This is Persistence. You will also note that the previous saved mesh will be drawn on screen. Please see notes at the end of this document for more details on how this system works.

The 6D logo positions also need to be saved somewhere. In this example, we are saving them to our 6D web service we allow developers to also use for rapid prototyping. Once the map relocalizes, we download the file which contains the positions of the logos, parse it, then move the logos to their respective positions.

In the background, mapping is now resuming, allowing you to save again and make subsequent relocalizations faster and more accurate.

You can also drag the 6D logos around and reposition them.

You'll notice that the logos' rotating yellow star always rests on top of the mesh as you drag them, no matter how irregular the surface is. This is real world interaction.

Use the toggle mesh button to make the mesh rendering disappear and see how the virtual objects still interact and get occluded by, or intersected with, real objects like furniture.

This is occlusion.

Relocalization with saved mesh data

Starting with SDK version 0.17.0, relocalization and loading of previously saved meshes is now possible. A few notes about how this system currently works so that developers understand how to use it. We still have many improvements to come around mesh merging, and will update this page and release notes as functionality is updated.

  1. It is important to always start an app session by downloading any currently saved meshes via the download button cloud_download (SixDegreesSDK_LoadFromARCloud()). The app will automatically attempt to relocalize across all saved maps that exist for your current location. This process will timeout after 30 secs, but you can stop it early with SixDegreesSDK_CancelLoad() if needed.
    1. After a successful relocalization, you will notice that any mesh you had been building locally is cleared from memory and the downloaded mesh corresponding to your current location is drawn. The SDK will immediately start to add to this mesh as you move around.
    2. Skip to step 3 if relocalization doesn’t succeed, map data is not available at your location, or if you just want to start from scratch.
  2. After you have built up some additional mesh data, use the upload button cloud_upload (SixDegreesSDK_SaveToARCloud()) to add your newly collected mesh to the stored maps in the 6D Reality Platform. Note that map data size increases linearly with usage, making file transfer more significant when calling SixDegreesSDK_LoadFromARCloud() / SixDegreesSDK_SaveToARCloud() - essentially each map you upload is saved, and the database of all saved maps is transferred back and forth with each upload/download action.
  3. warning During an app session, if you upload before you have downloaded or relocalized, your upload will be used to replace any existing stored maps for your location. This functionality is added to enable developers to easily clean out old saved maps during testing and will not function this way once we get to a general release candidate.