Geant4 Testing and Quality Assurance and User Support
Report to Workshop and TSB - September 1999

John Allison, 13th September 1999
A separate document reviews current status and plans at July 1999.
A thorough survey of system testing tools and recommendations for development was submitted to the Collaboration and presented at the Geant4 Workshop by Fred Jones in September 1999.

The Geant4.0.1 Release

  • Some new functionality (improvements to photon evaporation, low energy electromagnetic processes)
  • Automated build with granular libraries (recommended as default)
  • RogueWave emulation with the "STL Interface"
  • Extensive bug fixing (some thrown up by the STL Interface, some left over from Geant4.0, some new ones)
  • All tests run successfully except:
  • reading STEP file on DEC (see comments on DEC below)
  • exampleN04 loop on DEC in optimised mode (see comments on DEC below)
  • intermittent bus error on HP when using integral-approach electromagnetic processes
  • Some remaining platform problems:
  • AIX compiler very slow
  • AIX compiler produces very big object files
  • DEC miscompiles min and max in some cases (a new compiler is available which seems to fix it but not yet tested)
  • SG neglected partly because of hardware problems on SI group machine and partly because no manifest user demand
  • Successfully implemented on HP, Linux, NT (under Cygwin) and SUN
  • Optimised binary libraries released for HP, Linux and SUN for both RogueWave and STL versions.
    • BUT...
  • No new system tests since Geant4.0 (except for low energy electromagnetic processes)
  • Existing system tests not very demanding
  • It was 6 weeks late

    • Release Post Mortem

    A special meeting on 29th July identified some issues:
  • New code (low energy) not brought too late into system testing
  • Resources not fully deployed
  • Insufficient training of new members of the T & QA team
  • Over-use of talent from other working groups
  • System testing of visualization, user interfaces and environments undertaken but not reported
  • Inadequate or not-followed procedures (it is already the best documented of all our activities)
  • Software process compromised (sometimes you have to)
    • and recommended:
  • Clearly identifying what needs to go into the release and design corresponding tests
  • Improving the tests by:
  • increasing their scope
  • using "statistical tests", i.e., calculating quantities such as mean energy deposition which can be compared with expectation using statistical significance tests (obviates insignificant platform dependencies and code changes which change only the detail)
  • Training new members
  • Review and document procedures
  • Include the use of tools such as Insure++, CodeWizard, SUN Workshop

    • Plans

  • All the above recommendations
  • Development Plan proposed by Fred Jones (following talk):
  • Adoption of "browsers" - Bonsai and LXR
  • Adoption of automatic testing system based on Tinderbox: with customisation for Geant4 (tagged code)

    • Issues

  • Collaboration with category coordinators on test design and development
  • QA