C++ library for GPU accelerated linear algebra

Jump to:

Bug Reports
Citation
Distribution License
Linking
Speed
Development
Features
Miscellaneous

Bug Reports

I found a possible bug in the code and/or documentation. How do I report it ?
Please make a small and self-contained program which exposes the bug, and then open an issue at the GitLab repo, or inform the developers via email.
The small program should have a main() function and use only functions/classes from Bandicoot, Armadillo, and the standard C++ library (no other libraries). We appreciate your time to make a bug report.

Citation

How do I cite Bandicoot in my work ?
Please cite the following papers if you use Bandicoot in your research and/or software.
Citations are useful for the continued development and maintenance of the library.
Ryan Curtin, Marcus Edel, and Conrad Sanderson.
Bandicoot: C++ Library for GPU Linear Algebra and Scientific Computing.
Technical report, 2023.

Distribution License

What is the distribution license for Bandicoot ?
Bandicoot is licensed under the Apache License 2.0.
The Apache license is a permissive license; its main conditions simply require preservation of copyright and license notices. See also a Quick Summary of the license.

Can I use Bandicoot in commercial products without releasing the source code ?
Yes. Bandicoot can be used in proprietary software, without releasing the source code. However, any software that incorporates or distributes Bandicoot in source or binary form must include a readable copy of the attributions in the NOTICE.txt file that comes with Bandicoot. The attributions can be placed in the documentation and/or other materials provided with the software.

How is the Apache License related to the GPL ?
The Apache license and the GPL are completely separate and distinct licenses. Unlike the GPL, the Apache license does not "infect" your source code. See also the comparison of open-source software licenses on Wikipedia.

How is the Apache license related to the MIT and BSD licenses ?
The Apache license is a permissive license similar to the MIT and BSD licenses, but also provides a grant of patent rights from contributors to users. As such, the Apache license can be thought of as an improved version of the MIT and BSD licenses.

Is Bandicoot a product from the Apache Software Foundation ?
No. We just use the Apache license.

Linking

I'm getting lots of unresolved symbols during compilation
Use the CMake installer to install Bandicoot as described in the README file.
Then link your programs with the bandicoot run-time library:
```
g++ prog.cpp -o prog -std=c++11 -O2 -lbandicoot
```
What does the Bandicoot run-time library do ?
The Bandicoot run-time library is generated by the CMake installer and is a wrapper for all the relevant libraries present on your system, such as OpenBLAS, LAPACK, BLAS, CUDA, cuBLAS, cuRand, cuSolver, NVRTC, OpenCL, and clBLAS. The wrapper library is meant as a convenience for the linking process.

How can I link directly with libraries without using the Bandicoot run-time library ?
Define COOT_DONT_USE_WRAPPER before including the bandicoot header. For example:
or, if you're using gcc or clang, you can declare the define directly on the command line:
- If you don't have OpenBLAS installed, on Linux, change -lopenblas to -lblas -llapack and on macOS, change -lopenblas to -framework Accelerate
- If using the OpenCL backend on macOS, also change -lOpenCL to -framework OpenCL
Can I use Bandicoot as a pure template library ?
Yes. See the answer to the previous question.
Can I use Bandicoot without LAPACK and BLAS ?
Currently LAPACK and BLAS are required for some OpenCL decompositions and so it is not possible to use Bandicoot without them.

Where do I get LAPACK and BLAS ?
- For Linux-based systems (eg. Fedora and Ubuntu) pre-built OpenBLAS, LAPACK, and BLAS packages are available. You need to explicitly install them before installing Bandicoot. Make sure you also install the related development packages (which contain header files).
- macOS comes with the Accelerate framework, which is an optimised implementation of BLAS and LAPACK. The CMake installer can make use of the Accelerate framework by default.

Can I use high-speed LAPACK and BLAS replacements (eg. OpenBLAS, MKL) ?
Yes. The CMake installer should figure out they are available on your system. Otherwise, you can directly link with such libraries -- see the answers to preceding questions.

Where do I get OpenCL, clBLAS, and CUDA libraries ?
- For Linux-based systems (e.g. Fedora and Ubuntu) pre-built OpenCL, clBLAS, and CUDA packages are available. You need to explicitly install them before installing Bandicoot. Make sure that you also install the related development packages (which contain header files).
- For macOS, OpenCL is installed as a part of Xcode.
- You can check that OpenCL is installed correctly with the clinfo utility.
- You can check that CUDA is installed correctly with the nvidia-smi utility.
How can I use Bandicoot with OpenBLAS installed in a non-standard location?
Unpack the Bandicoot tar.xz archive and DO NOT install it using the CMake installer.
Compile your programs using:
- Replace /home/xyz with whatever your home directory is called
- Replace /home/xyz/OpenBLAS-0.3.20 with the directory name containing the OpenBLAS library
- Replace <backend_link_libraries> with the appropriate libraries to link against for the backends you have enabled:
  - If using the OpenCL backend, use -lOpenCL -lclBLAS on Linux or -framework OpenCL -lclBLAS on macOS
  - If using the CUDA backend, use -lcuda -lcudart -lcublas -lcurand -lcusolver -lnvrtc
  - If both backends are enabled, use -lOpenCL -lclBLAS -lcuda -lcudart -lcublas -lcurand -lcusolver -lnvrtc
- You may also need to tell the system linker where to find the OpenBLAS library (eg. by modifying the LD_LIBRARY_PATH environment variable in Linux)

How can I use Bandicoot with OpenCL installed in a non-standard location?
The process is similar to using Bandicoot with OpenBLAS installed in a non-standard location; follow the instructions from that question, but:
- Omit -L /home/xyz/OpenBLAS-0.3.20/
- Replace -lOpenCL with -I /home/xyz/OpenCL/include/ -L/home/xyz/OpenCL/lib/ -lOpenCL
- Replace /home/xyz/OpenCL/ with the directory name containing the OpenCL headers and library
- Replace /home/xyz/OpenCL/include/ with the directory name containing the OpenCL headers (i.e. the directory containing CL/opencl.h on Linux or OpenCL/opencl.h on macOS)
- Replace /home/xyz/OpenCL/lib/ with the directory name containing the compiled OpenCL library (libopencl.so on Linux or libopencl.dylib on macOS)
Note that different distributions of OpenCL may store the headers and compiled libraries in different directories than specified above.

It is generally much easier to install the OpenCL libraries and runtime through your system's package manager and that approach should be preferred whenever possible.

See also the question on getting OpenCL libraries.

How can I use Bandicoot with clBLAS installed in a non-standard location?
The process is similar to using Bandicoot with OpenBLAS installed in a non-standard location; follow the instructions from that question, but:
- Omit -L /home/xyz/OpenBLAS-0.3.20/
- Replace -lclBLAS with -I /home/xyz/clBLAS-2.12.0-Linux-x64/include/ -L /home/xyz/clBLAS-2.12.0-Linux-x64/lib64/ -lclBLAS
- Replace /home/xyz/clBLAS-2.12.0-Linux-x64/include/ with the directory name containing the clBLAS headers (i.e. the directory containing clBLAS.h)
- Replace /hpme/xyz/clBLAS-2.12.0-Linux-x64/lib64/ with the directory name containing the clBLAS libraries (libclBLAS.so on Linux and libclBLAS.dylib on macOS)
See also the question on getting clBLAS.

How can I use Bandicoot with the CUDA toolkit installed in a non-standard location?
The process is similar to using Bandicoot with OpenBLAS installed in a non-standard location; follow the instructions from that question, but:
- Omit -L /home/xyz/OpenBLAS-0.3.20/
- Replace -lcuda -lcudart -lcublas -lcurand -lcusolver -lnvrtc with -I /usr/local/cuda-12.2/include/ -L /usr/local/cuda-12.2/lib64/ -lcuda -lcudart -lcublas -lcurand -lcusolver -lnvrtc
- Replace /usr/local/cuda-12.2/lib64/ with the directory name containing the CUDA toolkit libraries (e.g. libcuda.so on Linux and libcuda.dylib on macOS)
- Replace /usr/local/cuda-12.2/include/ with the directory name containing the CUDA headers (e.g. the directory containing cuda.h)
See also the question on getting CUDA.

Speed

How fast is Bandicoot's matrix multiplication ?
When using the OpenCL backend, Bandicoot uses clBLAS for matrix multiplication. Wehn using the CUDA backend, Bandicoot uses cuBLAS for matrix multiplication, which is tuned for NVIDIA devices. Therefore, the speed of matrix multiplication is dependent on those implementations.

How fast is Bandicoot's eigen decomposition, matrix inversion, etc ?
When using the OpenCL backend, Bandicoot uses adapted versions of the fast MAGMA and clMAGMA linear algebra libraries. When using the CUDA backend, Bandicoot uses cuSolver for decompositions, which is tuned for NVIDIA devices.

Development

Who are the developers ?
Lead development is done by Ryan Curtin, Marcus Edel, and Conrad Sanderson

Can you implement features on request ?
We are in the process of porting all relevant Armadillo functionality to Bandicoot. Hearing from our users about what is needed and useful is helpful for prioritising development efforts. If you need a feature that Armadillo implements but Bandicoot does not, please contact the developers.

Can I contribute new functionality ?
Contributions are welcome, provided they are cleanly written, tested, and have accompanying documentation. You can make a pull request at the GitLab repo.

Is there a source code repository ?
Yes. See gitlab.com/conradsnicta/bandicoot-code. Note that the repository may contain work-in-progress code that has not been tested thoroughly.

Features / Functions

Is the API stable ?
Yes, within each major version. See the documentation for more details.

I need a specific function, but can't find in the documentation.
If it's not in the documentation, it doesn't exist yet. Some functionality that is in Armadillo is inappropriate for GPUs and Bandicoot; see the Bandicoot/Armadillo conversion guide for more information. If a specific Armadillo function is not implemented in Bandicoot for your use case, plase file a bug report so that adding the function can be prioritised. You can also contribute the functionality -- see the answers to development questions.

Can I use the C++11 auto keyword with Bandicoot objects and/or expressions?
Use of C++11 auto is not recommended with Bandicoot objects and expressions. Bandicoot has a template meta-programming framework which creates lots of short lived temporaries that are not handled by auto.

Is Bandicoot a C++11 only library ?
Yes. Bandicoot requires a compiler that supports the C++11 standard or newer.

Does Bandicoot take into account possible aliasing ?
Yes. Bandicoot checks for aliasing wherever it's possible to do so. In normal usage of the library this means aliasing is always checked. However, if you're evil enough you can always construct an artificial case to defeat any alias checking mechanism; in particular, if you construct matrices using writeable auxiliary memory (externally managed memory), your code will be responsible for taking care of possible aliasing.

Is it possible to interface Bandicoot with other libraries ?
Yes. This can be done by creating matrices or vectors that use auxiliary memory (i.e. pointers to CUDA or OpenCL memory blocks), and by directly obtaining a pointer to matrix GPU memory via the .get_dev_mem() function. Transferring a Bandicoot matrix to CPU memory can be done with the conv_to() function.

Is it possible to use Bandicoot matrices with user-defined/custom element types ?
Bandicoot supports matrices with the following element types: float, double, short, int, long, and unsigned versions of short, int, and long. We are also aiming to add support for reduced-precision floating-point types (bf16/fp16/fp8/etc.) when they are supported by the GPU.

Miscellaneous

Are there code examples ?
See "examples/example1.cpp" which comes with the Bandicoot archive. See also the code snippets within the documentation, including the short example program.

How can I do ... ?
Check the documentation and/or the README file that comes with the Bandicoot archive.

Is it possible to plot Bandicoot data directly from C++ ?
Yes. Try matplotlib-cpp, gnuplot-cpp, gnuplot-iostream, scopemm. Note that converting Bandicoot matrices to Armadillo matrices via conv_to() may ease the use of the suggested plotting libraries above.

What was the motivation for Bandicoot ?
Bandicoot is being developed with the aim of being a GPU-enabled drop-in replacement for the CPU-only Armadillo linear algebra library. The intention is to allow Armadillo-based programs to be easily ported to use GPUs. Although alternative GPU-enabled linear algebra libraries do exist, it is cumbersome to rewrite large codebases that use Armadillo, and alternatives do not generally employ the same level of optimisations via template metaprogramming that Armadillo provides. Bandicoot was developed to resolve this problem, providing easy transitions to GPU-enabled code and applying the same efficient optimisation techniques used by Armadillo.