Choosing the software#
New programs for Gaussian Splatting are constantly being released. This offers enormous freedom in the field, but it doesn’t make the choice any easier.
Since I want to learn Blender anyway, I created all of my first splats using the Blender plugin Skysplat. Once you’ve installed Blender and Skysplat (instructions), you’re ready to go.
The training process#
The basic training process usually follows these steps:
- A video is selected as the starting point.
- Frames are extracted from the video.
- The frames are translated into a camera path and the base points of the 3D space are set.
- The actual training of the splats
Depending on the software, some steps happen automatically or minor adjustments are required.
Number of frames#
Depending on scene complexity and the desired quality, more or fewer frames are needed for training. The more frames you select, the more VRAM-intensive and slower the training process becomes. The upside is a better end result.
To get a feel for the process, 150 frames are usually completely sufficient at the start.
The training#
Usually the best image is reached after about 30,000 training iterations. Some programs take a snapshot every 5,000 training iterations so that you can review the intermediate results at your leisure. This becomes necessary when you let the training process run for a very long time and notice that so-called overfitting has occurred. In machine learning, overfitting means that the model has been adapted too closely to the training dataset. This leads to disproportionately good results on the training dataset, but worse results on unknown data.
Since Gaussian Splats allow a view in 3D space, they enable new perspectives that were not part of the training dataset. For this reason, overfitting should be avoided.