In this section we cover the following topics:
We are developing and testing the RBC software under Linux using Conda packaging system as it conveniently provides all the necessary dependencies for the RBC package as well as for the OmniSciDB software.
In the following, we explain how to setup Conda environments for developing RBC as well as OmniSciDB software, how to get the software, and how to run and test the software.
To create the needed development environments, follow the instructions below.
Install and setup conda (unless it is already installed):
Create rbc-dev
environment:
Create omniscidb-dev
environment:
It is recommended to keep the rbc and omniscidb development environments separate to minimize the risk for any software version conflict.
OmniSciDB can be built for CPU-only or CUDA-enabled mode.
For CUDA-enabled OmniSciDB development, create omniscidb-cuda-dev
environment:
In addition, make sure that your system has CUDA Toolkit installed and the CUDA driver functional (run nvidia-smi
to verify). The highest CUDA Toolkit version that OmniSciDB supports is currently 11.0
Here follow the instructions for installing the CUDA Toolkit version 11.0.3:
Checkout rbc
sources:
Checkout omniscidb
sources:
Although the omniscidb-dev
environment contains all the prerequisites for building and running OmniSciDB server within the conda environment, we slightly adjust the conda environment to make the management of the building process for different build targets easier. For that, use the following script for activating the omniscidb-dev
environment:
The script activate-omniscidb-internal-dev.sh
must be sourced (not run via bash
or another shell) to the existing terminal session. The activate script will show various information about how to develop the OmniSciDB software within a Conda environment.
The activate script above uses a custom script /usr/local/cuda/env.sh
for setting CUDA environment variables for conda environment. Use the following commands to install the env.sh
script:
The basic workflow for developing and testing RBC package contains the following commands:
The omniscidb related rbc tests are run only when the omniscidb server is running. The server must be started with the options --enable-runtime-udf --enable-table-functions
, see OmniSciDB development section below.
By default, it is assumed that the omniscidb server can be accessed behind the localhost port 6274. Only in the case, the server is running elsewhere, one needs to configure the rbc testing environment as follows:
Create a configuration file with the following content (update credentials as needed):
The default location for the configuration file depends on the system but one can set an environment variable OMNISCI_CLIENT_CONF
that should contain a full path to the configuration file. Default locations of the configuration file are shown below:
The basic workflow for developing and testing OmniSciDB software in the context of RBC development is as follows:
that will build CUDA-enabled omniscidb server. See the instruction from the activate script about how to build CPU-only omniscidb server, for instance.
To execute the omniscidb test-suite, make sure that the current working directory is the build directory and run:
To start omniscidb server with runtime UDF/UDTFs support enabled, run: