In a typical simulation step there is a call to find the distance to the next boundary. So the computation cost of this method is important in the overall CPU time of all simulations.
Pre-processing to reduce the time taken by Navigation
In complex setups there can be tens, hundreds or sometimes thousands or more physical
volume inside one mother volume. The Geant4 Geometry Modeller (the Navigator) computes
information of each of those volumes, and stores it in a ‘optimisation’ data structure, in
order to reduce the number of candidate volumes intersected for each simulation step.
The method use is similar to methods used for ray tracing in graphics and visualisation,
but customised for HEP applications and other
domains in which a large number of volumes can co-exist at the same level.
The same structure can be duplicated in several parts of the geometry hierarchy tree. But
there can also be multiple different logical volumes which contain
a large number of daughter volumes.
The optimisation operation in Geant4 is called ‘voxelisation’, as the information is used
to create ‘smart voxels’: virtual slices or divisions of along one or more axes.
First each volume is asked its extent along a particular axis. Then the mother
volume is ‘sliced’ along this axis into a number of virtual divisions (slices),
each of which contains a set of integer that identify the daughter volumes it
contains. Neighbouring slices which contain the same sets are then merged into an
‘extended slice’.
Then a similar division is done along a second (and possibly the remaining third
axis) in those extended slice that contain at least a threshold
number of daughter volumes (currently 2, 3 and 4 for the 1st, 2nd and 3rd axis
respectively). When considering the daughter volumes which are contained inside
the extended slices, the boundary of the extended slice are passed to Geant4
solid, and can be used to calculate a fully-optimised extent on the subsequent axis/
axes. All possible orders of the cartesian axes are tried, keeping the one which
scored best in our metric.
This method is further described in the note which proposed it: Pure tracking and
Geometry in GEANT 4 CERN-RD44
(Geant4) development note, 1995.
The calculations and creation of the ‘smart voxels’ are done during the
initialisation of the geometry. For complex setups the computations can require
substantial time, and the resulting data structures can be large. To obtain a
report of the volumes which required the most time and memory, you need to set the
verbose level of a run:
The output of this looks as follows:
G4GeometryManager::ReportVoxelStats -- Voxel Statistics
Total memory consumed for geometry optimisation: 975520 kByte
Total CPU time elapsed for geometry optimisation: 59.2 seconds
Voxelisation: top CPU users:
Percent Total CPU System CPU Memory Volume
------- ---------- ---------- ------- ----------
7.25 4.29 0.00 56584k VolumeNum02
5.77 3.41 0.00 54619k VolumeNum20
5.13 3.04 0.00 47491k VolumeNum04
5.02 2.97 0.00 47165k VolumeNum03
4.84 2.87 0.00 44924k VolumeNum09
4.44 2.63 0.00 44599k VolumeNum18
4.36 2.58 0.00 44474k VolumeNum19
0.90 0.53 0.00 17809k VolumeNum01
Voxelisation: top memory users:
Percent Memory Heads Nodes Pointers Total CPU Volume
------- ------ ------ ------ -------- --------- ----------
5.80 56584k 153350 727309 1498811 4.29 VolumeNum02
5.60 54618k 146544 704707 1444035 3.41 VolumeNum20
4.87 47491k 81390 688903 1212961 3.04 VolumeNum04
4.83 47165k 80893 684272 1203466 2.97 VolumeNum03
4.64 45273k 76920 656525 1163615 2.83 VolumeNum05
4.63 45172k 76750 655021 1161179 2.82 VolumeNum06
You can then use this information to decide whether to revise the voxel density
parameter ‘smartlessness’, to reduce memory or CPU time, or seeking to improve
navigation performance in a particular volume.
For information on this please look in the section Logical Volumes.
Using parallelism to reduce the time for geometry initialisation
The computation work to create the ‘smart voxel’ structure for all logical volumes (which
have daughter volumes) is normally carried out by the initial (main) thread, before tasks
or worker threads are created.
In Geant4 11.3 we introduce the ability to share the work of initialising the creation
of the ‘smart voxel’ structures between tasks or threads. Currently this is an option. To
turn this on, a user must configure the G4GeometryManager by request it:
auto geomMgr= G4GeometryManager::GetInstance();
G4bool verbose= true;
geomMgr->RequestParallelOptimisation(true, verbose);
Using ’true’ for verbose ensures that the report of the memory and CPU time consumed by volumes is produced (in standard output.)