Handling Handles -- Decoupling Data And State

[JohnSmoit]

Overview

In order to facilitate data oriented state management, I plan to separate vulkan state from their public API's both conceptually and literally. Rather than a single wrapper type holding some state and a public API, I'll have a public "interface" which can be bound to data via a handle/pointer/some indexer to memory managed by my context.

Reasoning

This has the benefit of me being able to align memory in the most optimal usage patterns and also handle lifetimes in a more succinct way then just relying on the user to denitialize everything manually (Of course, this being zig, objects can still be deinitialized manually).

Features

Splitting Objects into State and Data:

I plan to split each API type into a "Proxy" which contains a public API and a bound handle/set of handles, and a "Handle" which is a reference to context-managed memory.

// Data is just a normal struct, whose memory is managed by the context
// (Of course, user-provided allocators will still be passed for host memory)
const VulkanObject = struct {
    h_swapchain: vk.Swapchain,

    h_renderpass: vk.Renderpass = .null_handle,
    framebuffers: api.FramebufferSet(api.MAX_IMAGE_COUNT),
};

// Some possible ways to do handles

// Way 1: Raw pointers,
pub fn Handle(comptime T: type) type {
    return *const T;
}

// Better way: Indexer

pub const HandleIndexer = packed struct {
    value: u24,
    meta: u8,
};

pub fn Handle(comptime T: type) type {
     return struct {
          index: HandleIndexer,
          source: ObjectPool,
    };
}

const VulkanObjectHandle = Handle(VulkanObject);


const VulkanObjectProxy = struct {

    pub fn exampleAPIFunction(self: VulkanObjectProxy, arg1: int) !void {
        // ... do some stuff
        // ... possibly support maplike range-based execution
    }

    // commmon proxy functions:
    pub fn bind(self: *VulkanObjectProxy, h: VulkanObjectHandle) void {}
    
    handle: VulkanObjectHandle,
};

Proxies are the "views" into data and are the primary way operations may be perfomed on them. In order to support cache coherence, I also want to support binding multiple contiguous handles to a proxy and executing operations on them in sequence (possibly even SIMD'ing some stuff)

[JohnSmoit]

Updates:

With regards to handles, I've chosen to implement them in the manner of an indexer -- such that rather than directly referencing an object by a pointer, handles will serve as an index to a slot in the pool allocator. This allows handles to be more resilient to changes in the renderer-owned memory such as providing better tracking of use-after frees in safe builds via a generation count. Additionally, I think it'll be easier to handle defragmentation routines and stuff (if I need them) if we have the more indirect handles (which can be updated with the owned memory should it's loation change). I'll probably have the handles stored in a sort of small header/footer for the allocation available in safe mode so that use-after frees can be detected easily. Additionally, I think this footer can be used to track arbitrary resource relocations in case an existing resource gets moved during a defragmentation or smoething.

TL:DR: Wall of text