RAII Wrapper For Registering/Mapping CUDA Resources

Question

I've implemented a resource management class for CUDA interop using RAII to ensure exception safety. The goal is to handle the registration/unregistration and mapping/unmapping, of graphics resources (like buffers and textures) while managing their lifetime in a specific CUDA stream.

Key features & constraints

• C++20: Uses features like std::format and std::source_location.
• Flexible registration: Can manage any resource type by passing a registration function (e.g., cudaGraphicsGLRegisterBuffer, cudaGraphicsGLRegisterImage) and its arguments to a templated constructor.
• Stream-aware: All operations (map/unmap) use a specific CUDA stream provided at construction to avoid blocking the main thread.

Here's the code:

include/cuda_types/resource.h

#ifndef SPACE_EXPLORER_CUDA_RESOURCE_H
#define SPACE_EXPLORER_CUDA_RESOURCE_H
#include <cuda_gl_interop.h>
#include <memory>
#include "cuda_types/fwd.h"
#include "cuda_types/stream.h"
namespace raw::cuda_types {
// The definitions of exception class and macro are in separate files in the actual code 
// but i figured it would be easier to put them here
class cuda_exception : public std::exception {
private:
 std::string message;
public:
 explicit cuda_exception(std::string message) : message(std::move(message)) {}
 [[nodiscard]] const char* what() const noexcept override {
 return message.c_str();
 }
};
#define CUDA_SAFE_CALL(call) \
 do { \
 cudaError_t error = call; \
 if (error != cudaSuccess) { \
 const char* msg = cudaGetErrorName(error); \
 const char* msg_name = cudaGetErrorString(error); \
 throw cuda_exception(std::format( \
 "[Error] Function {} failed with error: {} and description: {} in file: {} on line {}", \
 #call, msg, msg_name, std::source_location::current().file_name(), \
 std::source_location::current().line())); \
 } \
 } while (0)
/**
 * @class resource
 * @brief Base class for managing external resources, takes in the constructor function to register the
 * resource and the parameters, unmaps and unregisters the stored resource in the destructor
 */
class resource {
private:
 cudaGraphicsResource_t m_resource = nullptr;
 bool mapped = false;
 std::shared_ptr<cuda_stream> stream;
protected:
 // I heard somewhere that this down here is better than directly accessing the protected member
 cudaGraphicsResource_t &get_resource();
private:
 void unmap_noexcept() noexcept;
 void cleanup() noexcept;
public:
 resource() = default;
 template<typename F, typename... Args>
 requires std::invocable<F, cudaGraphicsResource_t *, Args...>
 explicit resource(const F &&func, std::shared_ptr<cuda_stream> stream, Args &&...args)
 : stream(stream) {
 create(func, std::forward<Args &&>(args)...);
 }
 template<typename F, typename... Args>
 void create(const F &&func, Args &&...args) {
 cleanup();
 CUDA_SAFE_CALL(func(&m_resource, std::forward<Args &&>(args)...));
 }
 void unmap();
 void map();
 void set_stream(std::shared_ptr<cuda_stream> stream_);
 virtual ~resource();
 resource &operator=(const resource &) = delete;
 resource(const resource &) = delete;
 resource &operator=(resource &&rhs) noexcept;
 resource(resource &&rhs) noexcept;
};
} // namespace raw::cuda_types
#endif // SPACE_EXPLORER_CUDA_RESOURCE_H

src/cuda_types/resource.cpp

#include "cuda_types/resource.h"
#include <iostream>
namespace raw::cuda_types {
void resource::unmap_noexcept() noexcept {
 if (mapped && m_resource) {
 try {
 CUDA_SAFE_CALL(
 cudaGraphicsUnmapResources(1, &m_resource, stream ? stream->stream() : nullptr));
 mapped = false;
 } catch (const cuda_exception& e) {
 std::cerr << std::format("[CRITICAL] Failed to unmap graphics resource. \n{}",
 e.what());
 }
 }
}
void resource::cleanup() noexcept {
 unmap_noexcept();
 if (m_resource) {
 try {
 CUDA_SAFE_CALL(cudaGraphicsUnregisterResource(m_resource));
 } catch (const cuda_exception& e) {
 std::cerr << std::format("[CRITICAL] Failed to unregister resource. \n{}", e.what());
 }
 }
}
resource& resource::operator=(resource&& rhs) noexcept {
 if (this == &rhs) {
 return *this;
 }
 cleanup();
 m_resource = rhs.m_resource;
 mapped = rhs.mapped;
 stream = std::move(rhs.stream);
 rhs.m_resource = nullptr;
 rhs.mapped = false;
 return *this;
}
resource::resource(resource&& rhs) noexcept
 : m_resource(rhs.m_resource), mapped(rhs.mapped), stream(std::move(rhs.stream)) {
 rhs.m_resource = nullptr;
 rhs.mapped = false;
}
cudaGraphicsResource_t& resource::get_resource() {
 return m_resource;
}
void resource::unmap() {
 if (mapped && m_resource) {
 CUDA_SAFE_CALL(cudaGraphicsUnmapResources(1, &m_resource, stream->stream()));
 mapped = false;
 }
}
void resource::map() {
 if (!mapped && m_resource) {
 CUDA_SAFE_CALL(cudaGraphicsMapResources(1, &m_resource, stream->stream()));
 mapped = true;
 }
}
void resource::set_stream(std::shared_ptr<cuda_stream> stream_) {
 stream = std::move(stream_);
}
resource::~resource() {
 cleanup();
}
} // namespace raw::cuda_types

And it can be used like this (pseudocode)

class gl_buffer : public resource {
private:
 T* data = nullptr;
public:
 using resource::resource;
 gl_buffer(size_t* amount_of_bytes, UI buffer_object, std::shared_ptr<cuda_stream> stream)
 : resource(cudaGraphicsGLRegisterBuffer, stream, buffer_object,
 cudaGraphicsRegisterFlagsWriteDiscard) {
 map();
 CUDA_SAFE_CALL(cudaGraphicsResourceGetMappedPointer((void**)&data, amount_of_bytes, get_resource()));
 unmap();
 }
 [[nodiscard]] T* get_data() const {
 return data;
 }
 ~buffer() override = default;
};

My questions

1. Exception Safety: Is the exception handling in the destructor and move operations correct and sufficient? Should the noexcept cleanup attempts be handled differently?
2. Design Choice: The design uses a templated constructor that accepts a function pointer for resource registration. Is this a good approach for handling different types of resources (buffers vs. textures), or would a different design be better and less error-prone?

Would appreciate any help or suggestions!

Answer 1

Use std::runtime_error

You derive cuda_exception from std::exception, but now you have to implement you own what(). Why not inherit from std::runtime_error?

class cuda_exception : public std::runtime_error {
public:
 using std::runtime_error::runtime_error;
};

Avoid macros

I would always try to avoid macros. They are hard to reason about, and come with pitfalls that can even trip up experienced developers.

If you drop the requirement that you want to print the function call itself, you can just write a function that takes the function you want to call and its arguments, and then does perfect forwarding to call that. The hard part is getting a std::source_location: you can use the trick of making a struct and using a deduction guide:

template<typename F, typename... Args>
struct safe_call {
 auto safe_call(F&& function, Args&&... args) {
 cudaError_t error = std::invoke(std::forward<F>(function), std::forward<Args>(args)...);
 if (error != cudaSuccess) {
 const char* msg = cudaGetErrorName(error);
 const char* msg_name = cudaGetErrorString(error);
 throw cuda_exception(std::format(
 "[Error] Function failed with error: {} and description: {} in file: {} on line {}",
 msg, msg_name, std::source_location::current().file_name(),
 std::source_location::current().line()));
 }
 }
};
template<typename F, typename... Args>
safe_call(F&&, Args&&...) -> safe_call<F&&, Args&&...>;

And then use it like so:

safe_call(cudaGraphicsResourceGetMappedPointer, (void**)&data, amount_of_bytes, get_resource());

I still think it's worth it.

Make it safer to use

To actually use a resource, you have to make three separate function calls: map(), get_resource() and unmap(). It's easy to forget to map, or to do things in the wrong order. Ideally, you just need one function call, and it should automatically map and unmap. For example, something like:

class resource {
 ...
public:
 class mapped_resource {
 std::shared_ptr<cuda_stream> stream;
 cudaGraphicsResource_t m_resource;
 public:
 mapped_resource(std::shared_ptr<cuda_stream> stream, cudaGraphicsResource_t resource): stream(stream), m_resource(resource) {
 safe_call(cudaGraphicsMapResources, 1, &m_resource, stream->stream()));
 }
 ~mapped_resource() {
 safe_call(cudaGraphicsUnmapResources, 1, &m_resource, stream->stream()));
 }
 cudaGraphicsResource_t& operator*() {
 return m_resource;
 }
 };
 mapped_resource get_resource() {
 return mapped_resource(stream, m_resource);
 }
 ...
};

Then you can use it like so:

class gl_buffer: public resource {
 ...
 gl_buffer(size_t* amount_of_bytes, UI buffer_object, std::shared_ptr<cuda_stream> stream)
 : resource(cudaGraphicsGLRegisterBuffer, stream, buffer_object,
 cudaGraphicsRegisterFlagsWriteDiscard) {
 safe_call(cudaGraphicsResourceGetMappedPointer, (void**)&data, amount_of_bytes, *get_resource()));
 }
 ...
};

In the above, get_resource() returns a temporary mapped_resource which is valid for the duration of the call to safe_call(). If you want to call more functions on the same mapped pointer, you would assign the result of get_resource() to a variable, and it will only unmap when the variable goes out of scope.

Exception safety

Is the exception handling in the destructor and move operations correct and sufficient? Should the noexcept cleanup attempts be handled differently?

You shouldn't catch exceptions when you can't do anything useful. What you do here is wrong and potentially dangerous: you catch the error, but then just print a warning and continue on, even though the underlying issue is not dealt with.

Sure, you could argue that you can't do anything when unmapping a resource fails, but things have gone wrong behind the scenes already, and there is no guarantee what this will mean for the rest of the program's execution. At best it's just a resource leak. But it shouldn't happen anyway.

I would avoid making these functions noexcept, and instead let the exceptions propagate.

• NeKon: I want to quickly talk about each point as I see it 1) I didn't really knew you can do that, but thanks for the suggestion! 2) It's doubtful which approach will be more appropriate here, since it's used not only here but across my whole code base, and losing function name along with it's parameters really is a sacrifice. 3) I already do have this but in more "high level" manner, but overall I think I'll implement that for the safety. 4) I am not really sure about that.. But what if we still want to continue execution of the program even tho this failed? Maybe it wasn't critical
• ayaan098: If you pass the function pointer as an NTTP to safe_call, you can do some shenanigans with std::source_location to get the name of the CUDA function being called (this does however require some compiler dependent code).