An Alternate Larsoft Build Process Patrick Gartung FNAL - - PowerPoint PPT Presentation

An Alternate Larsoft Build Process

Patrick Gartung FNAL gartung@fnal.gov

Larsoft librarians biweekly meeting Nov 18, 2014

Some Context

• Ben Morgan (University of Warwick) created geant4 cmake files
• Created cmake files for Art (fnal-art) and FNALcore (cpp0x, cetlib, fhicl-cpp and

MessageFacility)

– LBNE/FNALCore git repo – LBNE/fnal-art git repo

• Exports FNALCore and Art library interface using only cmake native macros
• Can be used in any cmake file by adding FNALCore and Art install directories by

adding to cmake prefix path and calling find_package(FNALCore 1.0.0) and find_package(art 1.11.03)

– FNALCore talk – Art build talk – Art build update talk

• Motivated by desire to build on OSX with system install of compiler and externals.
• Makes it possible to build using lbne-build which is based on worch.

Objectives

• Rewrite the cmake files for the larsoft package

to only use cmake macros available directly from the cmake package.

• Use the changes that Ben Morgan made to the

art package as a guide.

• Record the options used to invoke cmake in a

worch config

Getting larsoft into github

• Checkout all of the larsoft repos from FNAL redmine ( aka

upstream )

• Create an empty repo for each in LBNE account on github

– https://github.com/LBNE

• Push upstream develop and master branches to github
• Github rejected because of 120MB file that was added in

initial svn commit to larsoft and then removed.

• Followed directions that github gave for removing large files

– https://help.github.com/articles/working-with-large-files/

Setting up build environment

• Started a build from scratch using lbne-build package ( lbne

specific config for worch )

– https://github.com/LBNE/lbne-build

• Spent some time debugging why “make test” was failing for

FNALCore and fnalart packages

• Updated config for the latest version of FNALCore and fnalart
• Updated sqlite3 version and attempted to use the

FindSQLite3.cmake macros which should work without passing extra -D options to cmake.

• Figured out that the cmake macros work for a system install of

sqlite3 but not in worch environment. Fell back to passing -D

• ptions to cmake.
• Updated clhep and root versions needed by fnalart.

Build first larsoft package – larcore

• Started with larcore because of dependency chain

Larcore<-Lardata<-Larevt<-Larsim<-Larreco<-Larana

• Quickly added rules to build regular libraries and

add header files in install area

• Needed to create cmake modules directory and

copy over Find*.cmake from fnalart.

• Found a bug in the way root was configured while

generating dictionary library

• DGCCXML_EXECUTABLE={gccxml_install_dir}/bin/gccxml

Comparing it to upstream

• Library sizes were much smaller than upstream
• Location of lib and bin directories different

– OK because lib location exported in cmake files

• All header files in same location
• Needed to add .gdml and .pl and .C files in

gdml dir, .fcl files in job dir, and test scripts in bin

Making it equivalent to upstream

• After adding cxx flag -g sizes were comparable
• Dug through cetbuildtools macros and ups config

for larcore to reverse engineer install_gdml(),install_fhicl() and install_scripts() macros.

• Cmake globbing patterns, ups set variables, etc.
• Didn't implement install_source() to install the

library source files following the install pattern of fnalart.

Side effect of resizing larcore repo

• While comparing gdml dir found file was missing

larcore/Geometry/gdml/icarus.gdml

• In the repo I found icarus.gdml.REMOVED.git-id
• I had passed the wrong flag to the “BFG” utility recommended by github

– BFG removed top N files by size not files over github 100MB limit

• Both “BFG” and “git filter-branch” removed the one file over 100MB but all

commits after that are rewritten changing their commit id.

• Asked github for an exception to the 100MB file size limit for the larsoft
• repos. They graciously raised the size limit to 200MB.
• All larsoft repos now cloned to github with the upstream commit ids.
• Cloned the lbnecode repo on github just to be sure there were no size limit

problems.

Building lardata

• Added lines to larcore cmake files to export library interfaces following

fnalart example

• Added larcoreConfig.cmake.in to be used by other package to import

larcore interfaces

• Added GENIE and LHAPDF to worch config and built them.
• Figured out where the get the sources for nutools package.
• Modified nutools cmake files to not use cetbuildtools macros and stored

changes as a patch to upstream in worch. Only built libSimulationBase.so and copied associated header files. All other cmake files commented out for now.

• Turns out nutools was not needed for lardata because source code

using it had been moved to uboonecode repo.

Building larevt and larsim

• Nutools header needed to build larevt

–

Filters/FinalStateParticleFilter_module.cc:28:39: fatal error: SimulationBase/MCNeutrino.h: No such file or directory

• More nutools libraries need to be built for larsim

– EventGeneratorBaseCRY – EventGeneratorBaseGENIE

• Need to modify nutools cmake files so they work

without cetbuildtools and ups. Then they can be built with worch.