vasca.resource_manager

Resource manager for VASCA

Module Contents

Classes

ResourceManager

Manages access to external resources.

Data

CLASS_DIR
PACKAGE_DIR

API

vasca.resource_manager.CLASS_DIR = 'dirname(...)'

vasca.resource_manager.PACKAGE_DIR = None

class vasca.resource_manager.ResourceManager(verbose=False)

Manages access to external resources.

Resources like catalog files or large metadata files may be stored on remote storage systems such as DESY Sync & Share (SAS) or LUSTRE. Users may synchronize these resources onto their local systems to perform software analysis on the data. To aid collaborative development, the ResourceManager provides an interface to find the locations of resources on the user’s local system.

verbosebool, optional

Enable verbose printout to stdout (default is False for no printout).

flagsdict

Dictionary holding all flags.

metadatadict

Dictionary holding the parsed contents of the YAML-config files located at CLASSDIR/resource_metadata. Additional info my be added e.g. the set status of the env vars and the path they are pointing to.

The implementation requires a set of environment variables. The full list of variables is defined in CLASSDIR/resource_metadata/resource_envs.yml. Users need to make sure these variables are known to the system. This can be achieved in two ways. Either the user manually sets each variable via the shell config file (.bashrc, zshrc, etc.) or the instance of ResourceManager automatically sets the variables if the config file at PACKAGEDIR/.env is found.

Initialization

__enter__()

__exit__(exc_type, exc_value, traceback)

_load_metadata()

_load_env(config_path=PACKAGE_DIR + '/.env', overwrite=False)

Verify and set the required environment variables (env vars).

…

_log_env_status()

Verify and log if env vars are set.

…

_check_resource_catalog()

Dummy function atm.

Function to check the resource_catalog.yml file for consistency. Potential problems to look out for are:

• duplicate resource IDs/names for a given storage

• multiple resources with the same name at different storages

Additionally check if cataloged files really exist

_check_resoruce_env_info()

Verify if

• not multiple env vars exist referring to one project and storage system

get_path(resource, storage)

Returns the local path to a given resource at the corresponding storage system.

The path is constructed from the local path to the synced remote directory and the resource path relative to that directory.

resourcestr

Name of the resource for which the path is requested.

storagestr

Name of the storage system where resource is saved at.

pathstr

Full local path to resource

KeyError

When either one of resource or storage names is not listed in resource_catalog.yml

ValueError

When the inference of the needed environment variable fails or the resource could not be found at the inferred path.

This method requires specific environment variables to be declared on the users system during runtime. The variable names are listed in CLASSDIR/resource_metadata/resource_envs.yml.

>>> rm = ResourceManager()
>>> rm.get_path(resource="gal_visits_list", storage="sas_cloud")
str(/local/path/to/resource.file)