-
Notifications
You must be signed in to change notification settings - Fork 330
Implement Gaussian Splats #1748
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Someone actually tried packaging on this branch and is running into an issue, see #1749
A new issue to add to the list: when a Dynamic Pawn gets added to the level, the lighting on the splat goes haywire:
image
@david-lively reports this issue:
[UE] [2025年10月15日-16.55.01:627][ 59]LogCesium: Setting bounds: IsValid=true, Min=(X=357940124.922 Y=-120781445.651 Z=-123751071.001), Max=(X=357954552.308 Y=-120777170.055 Z=-123756954.477)
[UE] [2025年10月15日-16.55.01:627][ 59]LogCesium: Setting bounds: IsValid=true, Min=(X=357940124.922 Y=-120781445.651 Z=-123751071.001), Max=(X=357954552.308 Y=-120777170.055 Z=-123756954.477)
[UE] [2025年10月15日-16.55.01:627][ 59]LogCesium: Setting bounds: IsValid=true, Min=(X=357940124.922 Y=-120781445.651 Z=-123751071.001), Max=(X=357954552.308 Y=-120777170.055 Z=-123756954.477)
[UE] [2025年10月15日-16.55.01:627][ 59]LogCesium: Transform visible: false
validateComputeFunctionArguments:1175: failed assertion `Compute Function(Main_00005947_8a69e872): The pixel format (MTLPixelFormatR32Uint) of the texture (name:<null>) bound at index 5 is incompatible with the data type (MTLDataTypeInt) of the texture parameter (User_SplatInterface_SplatIndices [[texture(0)]]). MTLPixelFormatR32Uint is compatible with the data type(s) (
uint
).'
validateComputeFunctionArguments:1175: failed assertion `Compute Function(Main_00005947_8a69e872): The pixel format (MTLPixelFormatR32Uint) of the texture (name:<null>) bound at index 5 is incompatible with the data type (MTLDataTypeInt) of the texture parameter (User_SplatInterface_SplatIndices [[texture(0)]]). MTLPixelFormatR32Uint is compatible with the data type(s) (
uint
).'
Likely something to do with it running on the Apple GPU.
Gaussian Splats as depicted in the style of Tom Snyder:
squigglevision.mp4
Perhaps it goes without saying that there's some precision issues at play here. To some degree this problem is intractable with the requirements that a) everything happens on the GPU and b) everything must be sorted together. But there's definitely some things we can do to improve this I think. Probably if we rebased the Gaussian Splat Niagara system to be at the center of all currently loaded splats, it would help reduce precision issues instead of having it remain at (0, 0, 0) always.
I'm hoping this fixes the Android build.
This reverts commit 5e44cff.
@azrogers I've fixed the Android builds, though I'm not very satisfied with my understanding of what was going wrong. The fix was to remove the explicit reference to zlib::zlib
in CesiumGltfReader's target_link_libraries
. That was (apparently!) causing -isystem c:/android-ndk-r25b/toolchains/llvm/prebuilt/windows-x86_64/sysroot/usr/include
to be added to the clang command-line in CesiumGltfReader, which was (apparently!) causing the wrong cmath or math.h or something to be included. Like I said, not an entirely satisfying explanation.
weegeekps
commented
Oct 17, 2025
Particularly, the Hines Rowing Center tileset from the Sandcastle hovers over the terrain.
@azrogers, the Hines Rowing Center is using a WGS84 ellipsoid rather than Cesium World Terrain. Given this, it is known that it hovers over Cesium World Terrain. Here's a screenshot of it hovering in CesiumJS after the CWT terrain is selected.
imageRe: precision issues. Wow, yeah that's bad...
Probably if we rebased the Gaussian Splat Niagara system to be at the center of all currently loaded splats, it would help reduce precision issues instead of having it remain at (0, 0, 0) always.
This seems like the most reasonable solution given the situation.
EDIT: Although, this may be another instance where separate Niagara systems are warranted. That creates its own issues with sorting two splats on top of one another, but it might be an acceptable solution if precision continues to be an issue.
EDIT: Although, this may be another instance where separate Niagara systems are warranted. That creates its own issues with sorting two splats on top of one another, but it might be an acceptable solution if precision continues to be an issue.
We've been discussing this on the team. Splitting it up into multiple systems would cause some very obvious artifacts with two tilesets close by to each other, but should be fairly negligible with tilesets that are far away from each other - which is the most likely circumstance where the precision issues would manifest. The other circumstance would be if there are lots of tilesets, or a few very large tilesets, spanning a large distance - but you'd probably run into VRAM issues there before precision issues anyways.
weegeekps
commented
Oct 17, 2025
We've been discussing this on the team. Splitting it up into multiple systems would cause some very obvious artifacts with two tilesets close by to each other, but should be fairly negligible with tilesets that are far away from each other - which is the most likely circumstance where the precision issues would manifest. The other circumstance would be if there are lots of tilesets, or a few very large tilesets, spanning a large distance - but you'd probably run into VRAM issues there before precision issues anyways.
There are potentially changes that could be made on the tiler side to help with very large tilesets, allowing for the potential of better culling. It won't fix the precision issues, but it could help with any potential VRAM issues.
Let me know if you want to come to one of your stand ups and I'm willing to do so. Just need to know what day and the time.
@weegeekps
weegeekps
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we discussed verbally, this is looking good so far. My review for both this and the Cesium Native code has only been regarding implementation. On the Native side of things, the implementation is also looking good, but keep your eyes peeled for more changes regarding the rendering hints sometime soon.
The only major issue I've seen is that there appears to be some visual quality issues compared to CesiumJS. Per our discussion, it could be due to very minor differences in how we're doing the cut-off maths between the 3DGS shaders in CesiumJS and the shaders your PR.
Uh oh!
There was an error while loading. Please reload this page.
Closes #1670. This PR adds support for the
KHR_gaussian_splatting
extension in Cesium for Unreal (and, by virtue of CesiumGS/cesium-native#1262, SPZ as well). It does this using the Niagara particle system to handle rendering the splats from a custom NiagaraDataInterface. A subsystem is used to collect the splats into the singleton Niagara system.This is very very experimental! Even when this gets merged I think we'll need to call it an experimental feature. But some things we should take care of before this makes it out of a draft PR:
ensure(GroupCount.X <= GRHIMaxDispatchThreadGroupsPerDimension.X);
fails inRenderGraphUtils.h
as a result of Niagara trying to generate too many sort keys at once. This is maybe an inevitable result of trying to pack so many splats into a single Niagara system, but packing them all into a single system is necessary so they sort together (else we'd get some awful looking seams between adjacent systems). I've implemented a really dirty system to try to mitigate this (trying to avoid resetting the system more often than every five ticks), but a better solution is almost certainly needed. Ideally we can find a way to avoid this assertion that doesn't involve splitting the scene into multiple Niagara systems.(削除) I haven't yet tested it in a packaged build. It's likely the way I'm loading the Niagara system in the subsystem could break with a packaged build. (削除ここまで)✅ Should be fixed in 1476ee2.(削除) Compute shader issue on Mac laptop. (削除ここまで)✅ Fixed in 0695d2b.