Summary

Make cuda_model_cache_limit_bytes() and cuda_model_arena_chunk_bytes() configurable through environment variables, replacing the previous hardcoded defaults. This improves flexibility for systems with constrained VRAM (e.g., Strix Halo with 128 GB unified memory).

Changes

cuda_model_cache_limit_bytes()

• Added DS4_CUDA_WEIGHT_CACHE_LIMIT_GB environment variable support.
• When set, the cache limit is value * 1 GiB.
• Falls back to UINT64_MAX (unlimited) when unset, preserving the original behavior.

cuda_model_arena_chunk_bytes(uint64_t need)

• Added DS4_CUDA_WEIGHT_ARENA_CHUNK_MB environment variable support.
• Default remains 1792 MB when unset.
• Clamped to [256 MB, 8192 MB] range to prevent pathological allocations.
• New alignment logic: when the requested need exceeds half the chunk size, the function now aligns to 64 MB boundaries instead of the previous 256 MB alignment. This reduces wasted padding for large allocation requests.
• The existing 256 MB alignment for need > bytes is preserved as a fallback.

Motivation

On systems with limited unified memory like the Strix Halo, a hardcoded 1792 MB chunk size and unlimited cache can quickly exhaust available memory. These environment variables allow users to tune allocation behavior without recompilation.

Testing

• DS4_CUDA_WEIGHT_CACHE_LIMIT_GB=40 limits total weight cache to 40 GB.
• DS4_CUDA_WEIGHT_ARENA_CHUNK_MB=512 sets arena chunk size to 512 MB.
• Values outside [256, 8192] are silently clamped.
• Unset variables retain the original defaults (UINT64_MAX and 1792 MB).