3D scanning and 3D model with texture mapping

I am new here and would appreciate help in coding or an explanation what to use in swift for an app which will be able to capture LiDAR scanning and RGB data from taken pictures, generate a 3D mesh, and create .OBJ, .MTL, and .JPEG file set for further manipulation of 3D model. I am able to create from LiDAR scanning 3D mesh and .OBJ file but can not generate .MTL and .JPEG for a texture of 3D model.

Answered by DTS Engineer in 917517022

To build on the earlier pointer to Object Capture: PhotogrammetrySession can write an OBJ file with its MTL file and texture maps. The documentation for modelFile(url:detail:geometry:) says: "If url refers to a directory, the request saves an OBJ object and every texture map there." The request is available in iOS 17 and later.

I tried this on an iPhone running iOS 27.2.1 with the Scanning objects using Object Capture sample. The sample requires a device with a LiDAR Scanner. I added a second request that points at a folder:

try FileManager.default.createDirectory(at: objFolderURL, withIntermediateDirectories: true)
try session.process(requests: [
    .modelFile(url: usdzURL),
    .modelFile(url: objFolderURL)
])

After I scanned a pencil holder, the folder held an .obj file, its .mtl file, and a .usda file. It also held three texture maps: diffuse, normal, and ambient occlusion. The textures are 2048 x 2048 PNG files. If your pipeline needs JPEG, convert them and update the file names in the .mtl file to match.

On iOS, PhotogrammetrySession.Request.Detail lists .reduced as the only supported detail level, and my scan had 25,000 triangles.

Your LiDAR scanning already gives you the mesh and the .obj file. Object Capture is a different path, to a textured model. ARMeshGeometry provides vertices, normals, faces, and classifications, but no texture coordinates, so an .obj built from that mesh has no texture mapping.

I’m certainly not an expert in this technology, but it sounds like you’re looking for Object Capture.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

To build on the earlier pointer to Object Capture: PhotogrammetrySession can write an OBJ file with its MTL file and texture maps. The documentation for modelFile(url:detail:geometry:) says: "If url refers to a directory, the request saves an OBJ object and every texture map there." The request is available in iOS 17 and later.

I tried this on an iPhone running iOS 27.2.1 with the Scanning objects using Object Capture sample. The sample requires a device with a LiDAR Scanner. I added a second request that points at a folder:

try FileManager.default.createDirectory(at: objFolderURL, withIntermediateDirectories: true)
try session.process(requests: [
    .modelFile(url: usdzURL),
    .modelFile(url: objFolderURL)
])

After I scanned a pencil holder, the folder held an .obj file, its .mtl file, and a .usda file. It also held three texture maps: diffuse, normal, and ambient occlusion. The textures are 2048 x 2048 PNG files. If your pipeline needs JPEG, convert them and update the file names in the .mtl file to match.

On iOS, PhotogrammetrySession.Request.Detail lists .reduced as the only supported detail level, and my scan had 25,000 triangles.

Your LiDAR scanning already gives you the mesh and the .obj file. Object Capture is a different path, to a textured model. ARMeshGeometry provides vertices, normals, faces, and classifications, but no texture coordinates, so an .obj built from that mesh has no texture mapping.