Setup Guide#

This page explains how to set up this repository, for the first time as well as for every new session.

For the First Time#

  1. To clone the repository for the first time, use either ssh or https

    • Using ssh

      git clone ssh://git@gitlab.cern.ch:7999/gdl4hep/xdigi2csv.git

    • Using https

      git clone https://gitlab.cern.ch/gdl4hep/xdigi2csv.git

  2. If you are on an LXPLUS machine, the LHCb environment is already sourced. If not, you need to source the LHCb environment by running the following command:

    source /cvmfs/lhcb.cern.ch/lib/LbEnv

    If you do not have a mounted CVMFS, please refer to this section for instructions on how to mount CVMFS on an Ubuntu machine.

  3. Install the xdigi2csv_env python environment with pyarrow, which is required to compress or convert the CSV files obtained by the XDIGI2CSV algorithm. To do so, run:

    ./setup/install.sh

    This command will install the environment in setup/xdigi2csv_env according to the requirements specified in setup/requirements.txt.

  4. If you wish to use the XDIGI2CSV program, you need to setup the LHCb stack where the repositories are set to the anthonyc-persistence_csv branches. If you are on an LXPLUS machine, the necessary stack can be found at /afs/cern.ch/work/a/anthonyc/public/tracking/stack. The path to Moore within this directory is already set up by default, so no additional steps are required. If you are not on an LXPLUS machine, please refer to the second to last section for instructions on how to set up the LHCb stack.

  5. To find LFNs from a bookkeeping path, PFNs from LFNs, and download LFNs on a grid node outside of CERN using HTCondor, you must explicitly set the location where the Dirac proxy is written whenlhcb-proxy-init is invoked. You can do this by setting the X509_USER_PROXY environment variable in your .bashrc file (or by editing the setup/setup.sh file) to your desired location, for example:

    export X509_USER_PROXY=$HOME/.grid.proxy

    When a job submitted, the environment variable X509_USER_PROXY is explicitely passed to the HTCondor instance so it can access the LHCb proxy.

  6. If you plan to use the the MDF2CSV program in Allen, you need to build the Allen repository in the anthonyc-persistence_csv_mdf branch. If you are not on an LXPLUS machine, the simplest way to do this is to set up Allen standalone, following the instructions in the last section.

Every Time#

lhcb-proxy-init#

If you need to access Dirac-related resources, you should initialize your proxy by executing the command:

lhcb-proxy-init

You do not need to run this command locally, as ganga will automatically run it for you if needed. However, you must run it if you want to run a job on HTCondor that requires the proxy.

source setup/setup.sh#

Do not forget to source setup/setup.sh to set up the environment variables appropriately, using the command:

source setup/setup.sh

This will:

  • Create the XDIGI2CSV_REPO environment variable, which is the path to the root of the repository

  • Source the xdigi2csv_env python environment if it was installed

  • Set up ganga so that it works with Dirac. This also requires CVMFS.

If you also need to source the LHCb environment on CVMFS, you can run the following command instead:

source setup/setup_remote.sh

Setting Up the LHCb Stack Outside of LXPLUS#

In order to set up the LHCb stack in the anthonyc-persistence_csv branches in your own space or machine, you can follow a modified version of the official documentation.

  1. Install the configuration files of the LHCb stack by executing the following command:

    curl https://gitlab.cern.ch/rmatev/lb-stack-setup | python3 - stack

    This will create a folder named “stack” that contains the LHCb stack.

  2. Add the following lines to the stack/utils/config.json configuration file:

        "binaryTag": "x86_64_v2-centos7-gcc12+detdesc-opt",
    "gitBranch": {
        "Moore": "anthonyc-persistence_csv",
        "Allen": "anthonyc-persistence_csv"
    }

    This will set anthonyc-persistence_csv as the default branch in Allen and Moore.

  3. Compile Moore and its dependencies within the stack by executing the following command:

    cd stack
make Moore

    Note

    When building on your local machine, you can use the make Moore BUILDFLAGS='-j 15' command to limit the number of cores to 15, for example. By default, 75 cores are used if no limit is specified.

  4. To enable the XDIGI2CSV repository to locate your stack, you need to specify the path to the build directory of Moore. To do this, create or modify setup/config.yaml with the following content:

    build: # path to the build directories of Moore and Allen standalone
    moore: /path/to/stack/Moore/build.x86_64_v2-centos7-gcc11-opt

    For example, if the repository and stack directories are at the same level, the content of config.yaml should be:

    build: # path to the build directories of Moore and Allen standalone
    moore: ../../stack/Moore/build.x86_64_v2-centos7-gcc11-opt

    where the path is relative to the config.yaml file.

Setting Up Allen Standalone Outside of LXPLUS#

To set up Allen standalone outside of LXPLUS, follow these steps:

  1. Clone the Allen repository

    # SSH
git clone ssh://git@gitlab.cern.ch:7999/lhcb/Allen.git
# HTTPS
git clone https://gitlab.cern.ch/lhcb/Allen.git

  2. Check out the appropriate branch anthonyc-persistence_csv_mdf

    git checkout anthonyc-persistence_csv_mdf

  3. Refer to the Allen Documentation](https://allen-doc.docs.cern.ch), specifically the section “BuildAllen” to build Allen.

  4. In order for the XDIGI2CSV repository to use your local Allen standalone installation, you need to provide the path to the build directory of Allen. To do this, create or modify setup/config.yaml with the following content

    build:
    allen_standalone: "/path/to/Allen/build"

    Replace /path/to/Allen/build with the actual path to the build directory of Allen.