A Brief Introduction to Chroma S. Collins, University of Regensburg - - PowerPoint PPT Presentation

A Brief Introduction to Chroma

• S. Collins, University of Regensburg

Outline

◮ What is chroma? ◮ What can it do? ◮ Design. ◮ XML input and running chroma. ◮ How to get it. ◮ How to compile it.

2

Other chroma lectures and tutorials

Several chroma lectures and tutorials have been given before (see chroma web site):

◮ HackLatt’09/08/07/06, Balint Jo´

• , Dave Richards.

◮ Numerical Methods Lecture, Seattle Summer School, Balint

Jo´

• ,

◮ Lattice Practices 2008, Carsten Urbach and Andreas Jüttner.

Some have a different emphasis - e.g. hmc.

3

What is chroma?

Chroma is a C++ software package for performing Lattice QCD calculations. Chroma is based on QDP++.

◮ QDP++ provides an interface where operations can be

applied on lattice wide objects.

• e.g. LatticeColorMatrix, LatticeFermion, LatticePropagator

e.g. Gamma(G5) ∗ quark_propagator ∗ Gamma(G5)

◮ The level of abstraction is such that user code written using

the interface can be run unchanged on a single processor or multiprocessors with parallel communications.

◮ Architectural dependencies are hidden below the interface.

For chroma: storage, IO, manipulation and parallelization delegated to QDP++.

4

What is chroma?

QDP++ itself uses other packages

• QIO for data parallel IO + XpathReader (included in QDP++

download), libxml2

• QMT for multi-threading,
• QMP for parallel communications.

Q-packages developed as part of the SciDAC initiative. Additional target architecture specific optimized packages, Chroma - SSE Wilson Dslash (included in chroma download)

• CG-DWF (included in chroma download)
• BAGEL Wilson Dslash
• BAGEL clover

QDP++ - BAGEL QDP Also need BAGEL generator for BAGEL optimizations.

5

What is chroma?

There is a hierachy of software packages needed for chroma

• Scalar build very simple: chroma and qdp++

(QIO/XpathReader included, need libxml2)

• Parallel build more complicated depending on target.

Chroma compiles on any target machine,

• Optimization packages mean good performance on some

targets, SSE2 based machines, BlueGene/L(P).

6

What can chroma do?

The chroma software package can generate configurations using the executables

◮ purgaug - for pure gauge configurations using heat bath and

• verrelaxation algorithms,

◮ hmc - for hybrid Monte Carlo for dynamical

configurations (including “sea quarks”). The executable chroma can be used to calculate many

• bservables, using the configurations generated.

The configurations and observables can be calculated in many different ways (different actions, gauges, using different algorithms etc),

◮ naturally, orientated towards the programme of the JLab

lattice group + collaborators (who wrote the code),

◮ constantly expanding,

• some measurements will not be refined or working properly.

7

What observables can be calculated?

An incomplete list: Eigenvalue/-vector measurements

◮ Eigenvalue/-vector calculation via Kalkreuter-Simma

algorithm Gluonic observables

◮ Plaquette, Wilson loops

Hadronic observables

◮ Meson and baryon 2pt + 3pt functions ◮ Meson-meson 4pt functions ◮ Static-light correlators ◮ Hybrid mesons ◮ Nucleon-nucleon 4pt functions

8

Some of the available actions

Gauge actions

◮ Wilson gauge action ◮ Lüscher-Weisz gauge action

(tree level and 1-loop) Fermion actions

◮ Staggered fermions (naïve and Asqtad) ◮ Wilson fermions ◮ Clover improved Wilson fermions

(Standard, SLiNC)

◮ Domain wall fermions ◮ Overlap fermions (many approx. schemes in 4D and 5D and

deflation)

9

Some of the available BCs, inverters, sources and sinks

Boundary conditions

◮ Periodic, antiperiodic BCs ◮ Dirichlet BCs ◮ Twisted BCs

Inverters

◮ Conjugate gradient ◮ BiCGStab, BiCGStab with “reliable” updates for mixed

precision

◮ Minimal Residual ◮ EigCG

Quark sources and sinks

◮ Point, Shell (gauge invariant Gaussian) ◮ Derivative (covariant derivative and displacements) ◮ Momentum wall ◮ Stochastic

10

Gauge fixing and link smearings

Gauge fixing

◮ Axial gauge ◮ Coulomb gauge ◮ Landau gauge ◮ and Random gauge transformation

Link smearings

◮ APE ◮ HYP ◮ Stout

11

Design

Here, only mention basics and what is relevant for understanding input files.

◮ Chroma is an object-orientated code. ◮ Makes use of design patterns - general solutions to commonly

• ccuring problems (there are books on this).
• e.g. handle, factory function, factory, singleton.

◮ Implementation of design patterns comes from LOKI library

(A. Alexandrescu). How to achieve a code which

◮ has a high degree of flexibility at run time in terms of what is

calculated and how,

◮ where the calculations themselves may be complicated

involving several steps,

◮ (easily) extendable?

12

Design

Split the calculation into modules or tasks,

◮ a task may appear repeatedly in different calculations, e.g.

make a source for the propagator, invert on the source to

• btain the propagator.

◮ natural in object-oriented programming.

When calculating observables the tasks are referred to as inline measurements in chroma. At run time need to specify what tasks need to be performed,

◮ many tasks possible, ◮ many ways to perform the task (e.g. what inverter to use for

generating the propagator).

13

Solution: use the Factory pattern

Digression: object-oriented programs are designed in terms of

• bjects,

◮ a class is used to define the type of object,

e.g. class InlinePropagator{}, class InlineMakeSource{},

◮ classes for objects with similar properties can be derived from

a base class (inheritance),

e.g. class InlineMeas{}, class InlinePropagator: public InlineMeas {}, class InlineMakeSource: public InlineMeas {},

◮ One can create instances of the object using a creation

function, (c.f. new InlinePropagator())

e.g. InlinePropagator* createInlineProp(), InlineMakeSource* createInlineMakeS(),

Can do InlineMeas *my_inline; my_inline = createInlineProp(); my_inline = createInlineMakeS();

14

The Factory pattern

This is essentially a map from an id (or key) to a creation function for the object. Map:

“PROPAGATOR” −

→ InlinePropagator* createInlineProp() Can now create different inline measurements by querying the map.

std::map<std::string, InlineMeas* (*)(void)> factory_map; factory_map.insert(make_pair("PROPAGATOR",createInlineProp())); factory_map.insert(make_pair("MAKE_SOURCE",createInlineMakeS()));

with

InlineMeas *my_line = (factory_map["PROPAGATOR"])();

This pattern is used throughout chroma,

• e.g. smearing, inverters, boundary conditions, . . .,
• in the input file factory keys are capitalized.

More complicated: chroma uses handles, maps are singletons etc.

15

IO

Output of observables calculated within an inline measurement is dealt with within the inline measurement. Lattice objects in chroma (defined in QDP++),

• e.g. LatticeColorMatrix, LatticeFermion,

LatticePropagator, are needed as input for inline measurements or created by the measurement. These objects are assigned an ID (string) and stored in memory as “NamedObjects”,

• using a map, TheNamedObjMap (singleton).
• Input/Output dealt with using inline measurements (separated

from the calculation of observables).

16

Documentation

Chroma is a very large package - difficult to get an overview (QDP++ better documented). Useful:

◮ doxygen documentation - generated from the code, accessible

from chroma website or generate from source code yourself,

◮ cd chroma/docs; make

◮ lectures/tutorials, ◮ notes - Huey-Wen Lin’s compilation of notes at

http://www.jlab.org/~hwlin/Chroma_notes/chroma_note.html,

also available as latex source files on chroma website or in chroma source code (chroma/docs/notes),

◮ chroma is changing all the time, look at comments in git

repository. Find out how to invoke different gauge actions/boundary conditions/inline measurements by looking in the test files.

17

Test files

These are written for regression tests and can be found in sub-directories of source code: chroma/tests

◮ chroma/tests/chroma (chroma inline measurement tests) ◮ chroma/tests/hmc (HMC tests) ◮ chroma/tests/purgaug (pure gauge configuration generation

tests)

◮ . . .

For each test there is

◮ an input file for the test, ◮ the corresponding output file.

No guarantees - there can be/have been bugs in chroma. Less likely in older (well used) measurements.

18

Citations

If you use chroma in your simulations you must cite the appropriate people. Always R. G. Edwards, B. Jo´

• ,

arXiv:hep-lat/0409003, Nucl. Phys B140(Proc. Suppl) 882, 2005. SSE Optimised Dslash code C. McClendon, Jlab preprint JLAB-THY-01-29. BAGEL assembly generator P. A. Boyle, http://www.ph.ed.ac.uk/~paboyle/bagel/Bagel.html, 2005. QUDA GPU Library M. A. Clark, R. Babich, K. Barros, R. C. Brower, C. Rebbi, arXiv:0911.3191v1 [hep-lat] Look at the chroma website for the most up-to-date references.

19

Running

For generating pure gauge configurations: purgaug -i config.ini.xml -o config.out.xml Input file: XML,

• what gauge action to use, lattice size, how many updates, any
• bservables to calculate . . ..

Output:

• standard output (screen): can include status, performance,

timings etc.

• XML file, config.out.xml: log of calc., can contain physics e.g.

the plaquette or the values of any observables that were calculated. XML output may be lengthy.

20

Advantages and disadvantages of using chroma

Plus

◮ Very powerful, general software available for free!

• Many observables can be calculated with a large choice of

actions, inverters etc.

• All parallelization done behind the scenes.

◮ Many features chosen at runtime, lattice size, inverter, actions

etc.

• No need to recompile.

◮ Runs on basically every architecture, with target specific

improvements

• good performance on many machines.

◮ How long would it take to write the code that you want?

21

Advantages and disadvantages of using chroma

Minus

◮ Lack of documentation means it is painful to

• compile - what options on what platforms, many packages to

compile,

• compose the input XML file, what calculations are possible,

what input parameters are needed?

• understand and extract the observables calculated from output

files.

◮ Compilation of chroma takes a long time. ◮ XML makes input and output lengthy. ◮ Need a reasonable level of C++ to make additions (even to

cut and paste).

22

Overview of the input information

The program purgaug needs the following information in order to generate configurations.

◮ The starting configuration, whether starting from scratch,

before thermalisation, or continuuing from already thermalised configurations.

◮ The seed for the random number generator and the number of

thermalisation sweeps and the number of sweeps in total.

◮ The gauge action information, include lattice size, boundary

conditions etc.

◮ The Monte Carlo algorithm, how many heat bath sweeps and

• verrelaxation sweeps per update.

◮ Inline measurements, what observables to measure on the

configurations as they are generated.

23

Input XML: general structure

General structure of an input XML file: <?xml version="1.0"?> <!-- XML statement --> <purgaug> <Cfg> <cfg_type>WEAK_FIELD</cfg_type> <!-- config. format --> <cfg_file>dummy</cfg_file> <!-- file name --> </Cfg> <MCControl>..</MCControl> <!-- Seed and no. updates --> <InlineMeasurements>..</InlineMeasurements> <!-- observables to measure --> <HBItr>..</HBItr> <!-- gauge action and No. of HB and Ovr --> </purgaug>

24

Input XML

XML:

◮ content is wrapped up in tags <purgaug></purgaug>, ◮ tags can be nested, ◮ comments with <!-- -->. ◮ best viewed in mozilla family of browsers

(firefox/iceape/iceweasel/icecat) or using an editor that can handle XML. Tags are mostly self-explanatory (more details are given in the tutorial)

◮ <CFG>< /CFG> - specifies the configuration to be read in

at start-up. Has object id default_gauge_field.

◮ <cfg_type>< /cfg_type> - the type or format of the

configuration,

• UNIT (free-field), WEAK_FIELD (small fluct.), DISORDERED

(random), CLASSICAL_SF. All generated within chroma.

• CPPACS, KYU, MILC, NERSC, SCIDAC (also reads ILDG

format), SZIN, SZINQIO. Formats allowed for external files.

25

Additional information

For this tutorial chroma has already been installed, so you do not need to download and install the software yourself. However, the following slides give you some information of the necessary steps involved.

26

Getting the software

Chroma website: http://usqcd.jlab.org/usqcd-docs/chroma/ & links The latest stable versions of chroma and it’s dependencies (qdp++/qmp/bagel/. . .) are available as tarballs.

◮ e.g. chroma-3.38.0.tar.gz, qdp++-1.36.1.tar.gz,

qmp-2.1.6.tar.gz Chroma source management system is Git,

◮ list of projects on gitweb, chroma.git, qdp++.git etc.

To clone chroma repository with submodules, (git version 1.6.5.2

• r better):

◮ git clone git://git.jlab.org/pub/lattice/usqcd/chroma.git ◮ cd chroma ◮ git submodule update --init --recursive

Older cvs repository.

27

Compiling

Chroma and dependencies need to be compiled, “built”, in sequence, - dependencies first then chroma. What you need to compile depends on scalar/parallel and the platform:

◮ scalar: qdp++, chroma ◮ multi-threading: qmt, qdp++, chroma ◮ parallel: qmp, qdp++, chroma (gmp, libxml already installed) ◮ parallel (BlueGeneL/P): bagel, bagel_qdp, qmp, qdp++,

bagel_wilson_dslash, bagel_clover, chroma Chroma takes the longest to compile > 30 minutes. Each package built using: configure; make; make install,

◮ but configure has many options.

28

Compiling

Example of a few of the qdp++ configure options:

./configure

• -prefix=QDP++_INST_DIR

QDP++ installation dir

• -enable-Nd=Nd

# of spacetime dim., default=4

• -enable-Nc=Nc

# of colours, default=3

• -enable-Ns=Ns

# of spin comp., default=4

• -enable-parallel-arch=arch

arch = scalar, parscalar

• -enable-precision=prec

prec = single, double

• -enable-sse
• ptimize code with intel sse
• -with-qmp=QMP_INST_DIR

QMP installation directory

Don’t repeat the above flags for chroma.

◮ additional flags: --enable-sse-wilson-dslash

For what configure options are available use configure --help. Examples can be found in jlab-standard-chroma-standard build package in the git repository.

29

jlab-standard-chroma-standard-build

Used by JLab to automate, download, building and installing of chroma & dependencies. Scripts for different platforms & builds (scalar/parallel)

◮ scalar.sh ◮ bgp_build.sh ◮ ranger_build.sh

Look in these scripts to see what packages are needed. Look in package/machine/build-type/build/configure.sh for useful flags

• e.g. chroma/bgp/parscalar-bgp-double-sloppy/build/configure.sh

With a small amount of work, these scripts can also (almost) work for you.

30

Compiling

Once chroma is installed then find executables in

◮ CHROMA_INST_DIR/bin/chroma ◮ in same directory, hmc, purgaug + others ◮ also chroma-config, usage: chroma-config --[options]

>chroma-config --prefix /psi_devel/chroma_3.38.0_scalar-single64 >chroma-config --version 3.38.0 >chroma-config --Nd 4 >chroma-config --parallel-arch scalar

31