run folder#

This folder contains the helper scripts that allow to run Moore and Allen programs.

Scripts#

run/run.py main script#

Helper python script to run jobs with different backends.

The available backends are - local: to run a job locally - condor: to run many jobs in parallel on LXPLUS using HTCondor

The program to run is specified in computing/program in the configuration file, or using the –program option.

To execute this script, run the following command:

./run/run.py -c configuration_file.yaml [OPTIONS]

run/mdf2csv/run.py script#

run/moore/run.py script#

An helper script that can be used to run one of the moore program (XDGI2CSV, XDIGI2ROOT or XDIGI2MDF). The input is defined in the moore_input section of the YAML file, and the output directory is defined in the output section

The helper script has 3 features

1. It can run a function before the main program, which is configured in moore_config in the moore_config.program_preruns dictionary. 2. It runs the main Moore program, whose Moore option file is defined in run/{program}/moore_program.py. 3. It can run a function after the main program (e.g., to convert a CSV file into a PARQUET file!), which is configured in moore_config in the moore_config.program_postruns dictionary.

To run the helper script, use the following command:

./run/moore/run.py {program} -c configuration_file.yaml [other arguments]

Replace {program} with xdigi2csv, xdigi2mdf, or xdigi2root, depending on which program you want to run.

Modules#

run/moore/mooreconfig.py module#

Module that contains helper functions and variables to configure algorithms that are run from Moore.

run.moore.mooreconfig.check_no_conversion(format, compression=None)[source]#

Check whether a conversion is needed (that is, the end products are not an uncompressed CSV file(s))

Parameters:

  • format (str) – file format (csv, arrow or parquet)

  • compression (Optional[str]) – conversion algorithm to use (None if no compression)

Return type:

bool

Returns:

Whether a conversion is needed

run.moore.mooreconfig.program_postruns = {'xdigi2csv': <function xdigi2csv_conversion>}#

Associates a Moore program with a function to run after running the program. It takes as input the keyword arguments given by the pre-run function.

run.moore.mooreconfig.program_preruns = {'xdigi2csv': <function xdigi2csv_prerun>}#

Associates a Moore program with a function to run before running the program It takes as input the configuration that it may alter in-place, and return the arguments to pass to the post-run function, or None if no post-run functions shall be run

run.moore.mooreconfig.xdigi2csv_conversion(intermediate_outdir, outdir, format, compression=None, keep_original=False)[source]#

Implements the conversion from uncompressed CSV to the desired format for the given configuration.

Parameters:

  • intermediate_outdir (str) – Output directory of the Moore algorithm

  • outdir (str) – where to save the converted files

  • format (str) – file format (csv, arrow or parquet)

  • compression (Optional[str]) – conversion algorithm to use (None if no compression)

  • keep_original (bool) – whether to keep the original files. If set to False, the files are removed

run.moore.mooreconfig.xdigi2csv_prerun(config)[source]#

Modify the configuration in place to prepare the conversion from uncompressed CSV to the chosen format.

Parameters:

config (dict) – configuration that is modified

Return type:

dict | None

Returns:

The keyword arguments to pass to xdigi2csv_conversion(), or None if there is no conversion