Meshing API Guide

Please review to the 6D.ai SDK Function Definitions for additional info on functions mentioned in this guide.

Meshing is at the moment only generated on-the-fly, in the current AR session. In later SDK releases we'll include cloud-based meshing in order to allow sessions to use meshes generated from other devices.

Mesh information comes in three different buffers:

Those buffers must be allocated on the app side, using the sizes suggested by SixDegreesSDK_GetMeshBlockInfo(), and passed as pointers to SixDegreesSDK_GetMeshBlocks() to be populated.

A block is a cube of 22.4cm by 22.4cm by 22.4cm (8.82in); each triangle of the mesh is fully contained inside a block. The size of that cube is returned by SixDegreesSDK_GetMeshBlockSize(), expressed in meters (consistent with the coordinate system).

Each block is described in the block buffer by 6 consecutive integers: [ x, y, z, vertex_count, face_count, version ].

(x, y, z) represents the coordinates in space where the cube is, with (0, 0, 0) being the cube containing geometry between 0 and 0.224 on all 3 axes.

The vertex count and face count indicate how many vertices and triangles are contained in each block. The version value increments every time triangles in the block are updated.

Each vertex is described by 6 consecutive floats: [ x, y, z, normal_x, normal_y, normal_z ]. (x, y, z) represents the vertex's coordinates in space, (normal_x, normal_y, normal_z) is its normal vector used for lighting and physics.

Each face is made of 3 consecutive indices [ a, b, c ], referring to 3 vertices, starting with 0 for the first one.

The diagram above details a simple example of block, vertex, and face buffers returned by SixDegreesSDK_GetMeshBlocks(). In this example, the mesh is a polygon spanning 2 blocks (= 12 int values), described by 7 vertices (= 42 floats) and 3 faces (= 9 int indices).

The first block (yellow), at coordinates (0, 0, 0), has a square made of 2 faces (= 6 indices) and 4 vertices (= 24 floats). The second block (pink), at coordinates (1, 0, 0), has a triangle made of 1 face (= 3 indices) and 3 vertices (= 18 floats). You can notice that some vertices are duplicated (vertex 1 and 4, for instance), because they are part of a different block.

Blocks are 22.4cm a side because they are made of 8x8x8 voxels, a number that facilitates parallel processing at a voxel size that gives the best tradeoff of precision and accuracy with our current depth estimation.

The sample apps split meshes into chunks made of blocks so each Mesh object (in Unity) or geometry node (in SceneKit) is roughly 1 meter a side. This allows calls to SixDegreesSDK_GetMeshBlocks() to only update a subset of the total extent of the mesh, which isn't an issue for rendering but causes significant slowdowns for physics colliders.

Blocks also allow very efficient iterations through the mesh, for instance to find the Y value of the ground plane, without reading every single vertex.