Other Moore Programs#

In addition to the XDIGI2CSV program, the (X)DIGI2CSV repository also includes two other Moore programs, XDIGI2MDF and XDIGI2ROOT, which can be used to convert (X)DIGI files into MDF or ROOT formats, Like the The XDIGI2CSV Program, these programs can be set up and configured using the configuration file.

To set up the input: use the moore_input section of the configuration file, as described in the Set Up Moore Input section
To set up the output directory: use the output section of the configuration file, as described in the Setting Up Output Directory section
To configure the program: use {program} section of the configuration file, where {program} is either xdigi2mdf or xdigi2root, depending on which program you want to run. The options for these programs are described in the default configuration file setup/config_default.yaml.

It’s worth noting that the algorithms behind XDIGI2MDF and XDIGI2ROOT are available in the master branches of the LHCb stack. This means that you can execute these programs using the lb-run Moore/latest or the lb-run Moore/{specific version} command instead of relying on your local build of the LHCb stack. To do this, simply modify the build/moore option in your configuration file to:

build:
  moore: "lb-run:Moore/latest"

XDIGI2MDF Program#

The XDIGI2MDF program can be used to convert (X)DIGI files into MDF format.

An example YAML configuration file for running the XDIGI2MDF program can be found in jobs/examples/xdigi2mdf/xdigi2mdf.yaml:

build:
  moore: "lb-run:Moore/latest"
moore_input:
  paths: "{XDIGI2CSV_REPO}/data/SMOG2Arv56_127-Extended.digi"
  dddb_tag: dddb-20210617
  conddb_tag: sim-20210617-vc-md100
output:
  outdir: "xdigi2mdf-smog2"
computing:
  program: "xdigi2mdf"

In this example, the .digi file is located at data/SMOG2Arv56_127-Extended.digi The latest version of the LHCb stack available on CVMFS is used, rather than a local build.

To run the XDIGI2MDF program, use the following command:

./run/run.py -c jobs/examples/xdigi2mdf/xdigi2mdf.yaml

Alternatively, you can use the following command:

./run/moore/run.py xdigi2mdf -c jobs/examples/xdigi2mdf/xdigi2mdf.yaml

You can also customize the XDIGI2MDF program by changing the options in the xdigi2mdf section of the configuration file:

xdigi2mdf:
  # Whether to use Retina clusters
  retina_clusters: true
  # Name of the dumped MDF file
  mdf_filename: "dumped_mdf"
  # name of the directory where the binary geometry files are saved
  geo_dirname: "geometry"

XDIGI2ROOT Program#

The XDIGI2MDF program can be used to convert (X)DIGI files into ROOT format.

An example of YAML file to configure the XDIGI2ROOT program is shown in jobs/examples/xdigi2root/xdigi2root.yaml:

build:
  moore: "lb-run:Moore/latest"
moore_input:
  paths: "{XDIGI2CSV_REPO}/data/SMOG2Arv56_127-Extended.digi"
  dddb_tag: dddb-20210617
  conddb_tag: sim-20210617-vc-md100
output:
  outdir: "xdigi2root-smog2"
computing:
  program: "xdigi2root"

To run the XDIGI2ROOT program, use the following command:

./run/run.py -c jobs/examples/xdigi2root/xdigi2root.yaml

or equivalently

./run/moore/run.py xdigi2root -c jobs/examples/xdigi2root/xdigi2root.yaml

The retina_clusters option in the xdigi2root section can be used to specify whether to use Retina clusters or not:

xdigi2root:
  # Whether to use Retina clusters
  retina_clusters: true

How to Add a Moore Program#

To add a Moore program called MYMOOREPROGRAM, follow these steps:

Create a folder called run/mymooreprogram and a python file called run/mymooreprogram/moore_program.py. This file should contain the code that configures the Moore algorithm to run. It may read the YAML file that configures the algorithm, whose path is given by the environment variable MYMOOREPROGRAM_PROGRAM_CONFIG:

from tools import envvar
yaml_path = envvar.get_environment_variable("MYMOOREPROGRAM_PROGRAM_CONFIG")
with open(yaml_path, mode='r') as config_file:
    config = yaml.load(config_file, Loader=yaml.SafeLoader)

# get the parameters
param1 = config["param1"]
param2 = config["param2"]
# the output directory is always given by the `outdir` key
# and is determined by the `output` section of the configuration
outdir = config["outdir"]
...

In setup/default_config.yaml, add a new section with the default values for the parameters of your program:
```
mymooreprogram:
    param1: defaultvalue1
    param2: defaultvalue2
    ...
```

In definitions/dconfig.py, make the following changes:

Add your program to the list of moore_programs:

moore_programs = [
    "xdigi2csv",
    "xdigi2mdf",
    "xdigi2root",
    "test",
    "mymooreprogram"
]

If you want to use the eos output mode, you need to provide the datatype, which should be csv for the XDIGI2CSV program, root for the XDIGI2ROOT program, and mdf for the XDIGI2MDF program.

program_to_datatype = {
    "xdigi2csv": "csv",
    "xdigi2mdf": "mdf",
    "mdf2csv": "csv",
    "xdigi2root": "root",
    "mymooreprogram": "mydatatype"
}

f any of the parameters of your algorithms correspond to paths, you need to append them to the path_list_location list, in order for them to be properly handled. For example, if the parameter param2 corresponds to a path or a list of paths:
```
path_list_location = [
    {"moore_input": ["paths"]},
    {"allen_input": ["indir", "paths", "geodir"]},
    {"output": ["outdir"]},
    {"build": ["moore", "allen_standalone"]},
    "include",
    {"mymooreprogram": ["param2"]}
]
```
This way, the program will be able to properly resolve the paths, taking into account relative paths and any environment variables that may be used in the paths.

Add a description of your program to the config_sections dictionnary so that the parser can display this description if queried with --help:

config_sections = {
    ...
    "mdf2csv": "Conversion from MDF to CSV",
    "test": "Test program for Moore",
    "mymooreprogram": "Here is what mymooreprogram does"
}

Finally, add a description of the options in config_description so that they can be properly handled by the Argument Parser:

config_description["mymooreprogram"] = {
    "param1": dict(
        type=bool,
        help="A flag that does this"
    ),
    "param2": dict(
        nargs="+",
        help="This is one or more paths that correspond to this"
    ),
}

Even if you do not have parameters, define an empty dictionnary config_description["mymooreprogram"] = {}

Once you have added your custom Moore program, you can run it just like any other Moore program. To do so, define a configuration file, for example mymooreprogram_config.yaml, and include the input, output, and program configuration for your custom program. Here is an example configuration file:

moore_input:
  paths: "{XDIGI2CSV_REPO}/data/SMOG2Arv56_127-Extended.digi"
  dddb_tag: dddb-20210617
  conddb_tag: sim-20210617-vc-md100
mymooreprogram:
    param2:
    - path1
    - path2
output:
  outdir: "outdir"
computing:
  program: "mymooreprogram"

Once you have created the configuration file, you can run your program using the following command:

./run/run.py -c mymooreprogram.yaml --param1

Here, the --param1 flag is an example of modifying a parameter of your custom program at runtime. You can modify any of the parameters of the configuration in this way.

Alternatively, you can also run your program using the following equivalent command

./run/moore/run.py mymooreprogram -c mymooreprogram.yaml --param1

Other Moore Programs

Contents

Other Moore Programs#

XDIGI2MDF Program#

XDIGI2ROOT Program#

How to Add a Moore Program#