RedisAI Development ¶
The following sections discuss topics relevant to the development of the RedisAI module itself. We'll start by referring to the general design, followed by the source code layout, and how to prepare your local test and development environment.
General Design ¶
RedisAI bundles together best-of-breed technologies for delivering stable and fast model serving. To do so, we need to abstract from what each specific DL/ML framework offers and provide common data structures and APIs to the DL/ML domain.
As a way of representing tensor data we've embraced dlpack - a community effort to define a common tensor data structure that can be shared by different frameworks, supported by cuPy, cuDF, DGL, TGL, PyTorch, and MxNet.
RedisAI provides the following data structures:
- Tensor : represents an n-dimensional array of values
- Model : represents a computation graph by one of the supported DL/ML framework backends
- Script : represents a TorchScript program
Source code layout ¶
Inside the root are the following important directories:
src: contains the RedisAI implementation, written in C.
opt: contains the helper scripts to set up the development environment.
tests: contains the unit and flow tests, implemented with python and RLTest.
deps: contains libraries RedisAI uses.
We'll focus mostly on
, where the RedisAI implementation is contained,
exploring what there is inside each file. The order in which files are
exposed is the logical one to follow in order to disclose different layers
of complexity incrementally.
This is the entry point of the RedisAI module, responsible for registering the new commands in the Redis server, and containing all command functions to be called. This file is also responsible for exporting of Tensor, Script and Model APIs to other Modules.
Contains the helper methods for both creating, populating, managing and freeing the Tensor data structure, as well as the methods for managing, parsing and replying to tensor related commands or operations.
Contains the helper methods for creating, populating, managing and freeing the Model data structure. Also contains methods for managing, parsing and replying to model related commands or operations, that take place in the context of the Redis main thread.
The helper methods that are related to async background work are available at
Contains the helper methods for creating, populating, managing and freeing the PyTorch Script data structure. Also contains methods for managing, parsing and replying to script related commands or operations, that take place in the context of the Redis main thread.
The helper methods that are related to async background work are available at
Contains the helper methods for parsing, running the command in the background, and replying to DAG structured commands.
Contains the structure and headers that create, initialize, get, reset, and free the structures that represent the context in which RedisAI blocking commands operate, namely
Contains the methods that are related to async background work that was triggered by either
commands, and called from
This file also contains the function signatures of the reply callbacks to be called in order to reply to the clients, after the background work on
Contains the structure for managing per-device queues that are used for decoupling the work from the main thread to the background worker threads. For each of the incoming
commands, the request is queued and then executed asynchronously to one the device queues.
Contains the structure and headers that create, initialize, get, reset, and free run-time statics, like call count, error count, and aggregate durations of
The statistics are ephemeral, meaning that they are not persisted to the Redis key space, and are reset when the server is restarted.
Contains the structure and headers of an API for creating, initializing, getting, resetting, and freeing errors of the different backends.
backends.h and backends directory
Contains the structure and headers methods required to register a new backend that can be loaded by the module.
To do so, the backend needs to implement and export the following methods:
model_create_with_nodes: A callback function pointer that creates a model given the
RAI_ModelOptsand input and output nodes.
model_create: A callback function pointer that creates a model given the
model_run: A callback function pointer that runs a model given the
model_serialize: A callback function pointer that serializes a model given the
script_create: A callback function pointer that creates a script.
script_free: A callback function pointer that frees a script given the
script_run: A callback function pointer that runs a model given the
folder you will find the implementations code required to support the following DL/ML identifiers and respective backend libraries:
tensorflow.cexporting the functions to to register the TensorFlow backend
tflite.cexporting the functions to to register the TensorFlow Lite backend
torch.cexporting the functions to to register the PyTorch backend
onnxruntime.cexporting the functions to to register the ONNXRuntime backend
Building and Testing ¶
You can compile and build the module from its source code - refer to the Building and Running section of the Quickstart page for instructions on how to do that.
Configring your system ¶
Building in a docker (x86_64)
The RedisAI source code can be mounted in a docker, and built there, but edited from the external operating system. This assumes that you are running a modern version of docker, and that you are making a recursive clone of this repository and all of its submodules. This assumes that you have jinja installed, as the docker files are geneated from the dockerfile.tmpl in the opt/build/docker directory.
git clone --recursive https://github.com/RedisAI/RedisAI cd RedisAI/opt/build/docker make # note, to build with GPU support instead make GPU=1
After this, you can run the create docker and mount your source code with the following command, from within the RedisAI folder.
docker run `pwd`:/build -it redisai:latest bash
Continue to edit files on your local machine, and rebuild as needed within the docker, by running the command below, from /build in the docker:
make -C opt
Building on bare metal
The instructions below apply to Ubuntu 18.04 only . RedisAI can be built on other platforms, as documented is the system-setup.py file . This assumes that you're cloning the RedisAI repository.
git clone --recursive https://github.com/RedisAI/RedisAI cd RedisAI sudo apt-get -qq update -y sudo apt-get -qq install -y \ build-essential \ ca-certificates curl wget unzip \ gawk \ libopenblas-dev libmpich-dev \ git-lfs clang-format-10
Ensure that clang-format points to clang-format-10:
sudo update-alternatives --install /usr/bin/clang-format clang-format /usr/bin/clang-format-10 10
Install cmake v3.19.5 from the cmake repository .
Building on bare metal, with build scripts
These instructions apply to Ubuntu 16.04 and 18.04 . RedisAI can be built on other platforms, but these are the supported Platforms. This assumes you're cloning the RedisAI repository.
git clone --recursive https://github.com/RedisAI/RedisAI cd RedisAI sudo ./opt/system-setup.py
To compile RedisAI, run make -C opt , from the root of the repository.
The module includes a basic set of unit tests and integration tests, split across common and backend specific files. To run them you'll need:
- lcov (for coverage analysis, on Linux)
- Python and a few packages (e.g. pytest, redis, etc.)
redis-server in your PATH, or in
To run all tests in a Python virtualenv, follow these steps:
$ mkdir -p .env $ virtualenv .env $ source .env/bin/activate $ pip install -r tests/flow/tests_setup/test_requirements.txt $ make -C opt test
Integration tests are based on RLTest, and specific setup parameters can be provided to configure tests. By default the tests will be ran for all backends and common commands, and with variation of persistency and replication.
To understand what test options are available simply run:
$ make -C opt help
For example, to run the tests strictly designed for the TensorFlow backend, follow these steps:
$ make -C opt test TEST=tests_tensorflow.py
For coverage analysis we rely on
that can be run by following these steps:
$ make -C opt build COV=1 SHOW=1 $ make -C opt test COV=1 SHOW=1