Mipmaps and LOD Behavior in Skybox Shaders

Question

1. Is it beneficial to use Generate Mipmaps for a texture used in a skybox shader?

I need to use tex2Dlod to fix the edge seams issue, and I want to use the highest LOD I can reach in order to get the lowest resolution.

2. What is the highest LOD for textures with a resolution of 1024?

3. What will happen if the resolution changes to 512 but the LOD value is not changed?

4. How will tex2Dlod behave if Generate Mipmaps is disabled in the texture settings?

enter image description here

Used shader:

 Shader "Skybox/LatLong"
{
 Properties
 {
 [NoScaleOffset] _MainTexture ("Main Texture (LatLong)", 2D) = "white" {}
 }
 SubShader 
 {
 Tags { "Queue"="Background" "RenderType"="Background" "PreviewType"="Skybox" }
 Cull Off ZWrite Off
 Pass
 {
 CGPROGRAM
 #pragma vertex vert
 #pragma fragment frag
 #include "UnityCG.cginc"
 #include "Lighting.cginc"
 sampler2D _MainTexture;
 struct appdata_t
 {
 float4 vertex : POSITION;
 };
 struct v2f
 {
 float4 pos : SV_POSITION;
 float3 vertex : TEXCOORD0;
 };
 v2f vert (appdata_t v)
 {
 v2f OUT;
 OUT.pos = UnityObjectToClipPos(v.vertex);
 // Get the ray from the camera to the vertex and its length (which is the far point of the ray passing through the atmosphere)
 float3 eyeRay = normalize(mul((float3x3)unity_ObjectToWorld, v.vertex.xyz));
 OUT.vertex = -eyeRay;
 return OUT;
 }
 half2 GetLatLongCoords(float3 normalizedViewDir)
 {
 half lon = atan2(-normalizedViewDir.x, -normalizedViewDir.z);
 half lat = asin(-normalizedViewDir.y);
 half2 uv;
 uv.x = (lon / (2.0 * UNITY_PI)) + 0.5;
 uv.y = ((lat / UNITY_PI) + 0.5);
 uv.y = saturate((uv.y - 0.5) / 0.5);
 return uv;
 }
 half4 frag (v2f IN) : SV_Target
 {
 half3 viewDir = normalize(IN.vertex.xyz);
 half2 coords = GetLatLongCoords(viewDir);
 half4 col = tex2Dlod(_MainTexture, half4(coords, 2, 0));
 return col;
 }
 ENDCG
 }
 }
}

• DMGregory: Latitude-longitude isn't a very good way to store a skybox. If you need to see the whole sphere, a cubemap would be better. That saves you from needing to use a custom shader, and you won't need the tex2Dlod trick to hide the seam. It also has more even texel density, so mipmaps are only needed if the cubemap is higher-res than your field of view on your target HW. Unity can automatically convert your latitude-longitude image to a cubemap if you ask it to. Is there any reason you're not using this workflow?
• DMGregory: If you only need the upper hemisphere ( a sky dome), then a single square texture with a sphere map or paraboloid map is a common choice, which similarly eliminates the need for trig functions and tex2Dlod in the shader, since there's no seam. Pyramid maps are another option that go right up to the corners of your texture, so you don't waste any texture space.
• Ahmed Dyaa: @DMGregory, Thank you, The approach I am currently following suits me, and I do not need to change it at the moment. However, the questions I mentioned above are what I have not been able to understand so far.

Answer 1

1. Is it beneficial to use Generate Mipmaps for a texture used in a skybox shader?

Using the equirectangular / panorama / latitude-longitude texture mapping you've chosen, yes, mipmaps will be helpful for performance when looking up.

Because this projection stretches tiny areas near the poles (like the "top" of the sky) across the whole width of the texture, two pixels very close to each other on the screen will end up reading widely-separated parts of the texture, even though they want almost the same colour value in the end. This thrashes the texture cache, increasing average latency and lowering throughput / effective texture bandwidth. On mobile, that also means draining more battery. These effects get more severe as your texture resolution increases.

When mipmaps are available (and we use the right texture-fetching code), the GPU can detect that the texture is being effectively downsampled here, and swap to reading from a smaller mip for both samples. Because the mip is smaller, more of it fits in cache at a time, and there's a better chance that texels you fetched to shade one pixel can be re-used for a nearby one, improving performance.

For just one texture on a modern device, you probably won't notice the difference. But if you are using lots of these lookups for things like reflection maps, or if you're targeting lower-end mobile devices, these costs can add up.

An alternative is to use a texture projection with a more even texel distribution, like a cubemap. In that case, you can just size your cubemap to be about the right resolution for your target display size, and you won't see such wild variation in caching performance depending on where the player looks. The shader code for sampling the texture is also cheaper.

If you're using this skybox as a reflection map, then mipmaps are also useful for cheaply rendering dull / blurry reflections, in either projection format.

---

2. What is the highest LOD for textures with a resolution of 1024?

Each level of detail is one mipmap level, with the resolution halving at each step. If you had a 1024x1024 map to start with, then...

0. The original resolution (1024 x 1024)
1. The first mipmap, at half res (512 x 512)
2. The second mipmap, at quarter res (256 x 256)
3. The third mipmap, at eighth res (128 x 128)
4. The fourth mipmap, at sixteenth res (64 x 64)
5. The fifth mipmap (32 x 32)
6. The sixth mipmap (16 x 16)
7. The seventh mipmap (8 x 8)
8. The eighth mipmap (4 x 4)
9. The ninth mipmap (2 x 2)
10. The deepest mipmap (1 x 1 - a single texel)

In general, a texture whose smallest dimension is \2ドル^n\$ can have up to \$n\$ mipmaps, plus the original resolution.

Since a panorama image is usually wider than it is tall, you should use the vertical dimension when doing mipmap figuring. So a 1024x512 image can have up to 9 down-scaled mipmaps, plus the original, and a 1024x256 sky dome image can have up to 8, plus the original.

Just because a texture can have this many mipmap levels does not necessarily mean that it does. Some settings that can affect this:

• You can configure a mipmap limit to use only the smaller mip levels. This can be configured in the quality settings by platform, so for instance the same game built for desktop, mobile, and web might use different texture sizes and different mips on each destination.
• You can enable mipmap streaming, so higher-resolution versions are not loaded until needed. This means the number of mipmaps actually available might change at runtime.
• A texture created through script or imported from a format that contains mips like DDS can be configured to include only the higher resolution mips (up to a limit set by the author), skipping the lower-res mip levels.

---

3. What will happen if the resolution changes to 512 but the LOD value is not changed?

A call to tex2Dlod will sample the mip at the index specified by the w component of the texture coordinate. So this call:

tex2Dlod(_MainTexture, half4(coords, 2, 0));

...will always sample from mipmap index 0, the highest resolution available (completely losing any benefits of mipmapping as discussed in question 1 - we'll come back to this).

For a 1024x1024 image, that's the 1024x1024 version it's sampling from. If you resize the image to 512x512 (or it gets limited to that resolution by the mipmap limit, quality settings, or mipmap streaming), then it's the 512x512 version it's sampling from.

If you used 1 in the w component, then from a 1024x1024 image, it would sample the 512x512 mip, and from a 512x512 image, it would sample the 256x256 mip (in both cases: one step down from the highest available).

If the value give for the w component is greater than the deepest mip index in the texture, the deepest (lowest-resolution) mip is used instead.

As an aside, the "2" in the code above is meaningless - tex2Dlod does not use the z component of the texture coordinate, as you can see in the documentation.

---

4. How will tex2Dlod behave if Generate Mipmaps is disabled in the texture settings?

It will always sample from the highest-resolution version of the texture available, exactly what you're doing now.

---

Now, the question you didn't ask, but probably should have:

"How can I correctly fix the seam on an equirectangular panorama texture?"

I need to use tex2Dlod to fix the edge seams issue

No, that's not the correct way to fix the seam if you care about gaining the benefits of mipmapping discussed in question 1.

What we want to do instead is tell the graphics card how quickly the texture coordinate is changing between nearby pixels on the screen, so it can select/blend the best mip levels (and anisotropic filtering, if enabled) automatically.

What causes the seam is where the longitude wraps around from π (180°) to -π (-180°). We know these values reference the same part of the texture, but the texture mapping hardware sees a jump across the whole width of the texture from one pixel to the one immediately beside it, as if the texture were scaled way down, so the sampling hardware mips-down unnecessarily with the usual tex2D call.

We can use the trick I show in this answer to correct the texture gradient and ignore this spurious jump in the x coordinate.

Here's a shader that does that:

Shader "Skybox/FixedPanorama"
{
 Properties
 {
 _MainTex ("Texture", 2D) = "white" {}
 [KeywordEnum(Sky Sphere, Sky Dome)] _Mapping("Mapping", float) = 0
 }
 SubShader
 {
 Tags { "RenderType"="Background" "Queue"="Background" "PreviewType"="Skybox" }
 Cull Off
 ZWrite Off
 Pass
 {
 CGPROGRAM
 #pragma vertex vert
 #pragma fragment frag
 #pragma multi_compile_local __ _MAPPING_SKY_DOME
 #include "UnityCG.cginc"
 struct appdata
 {
 float4 vertex : POSITION;
 };
 struct v2f
 {
 float3 direction : TEXCOORD0;
 float4 vertex : SV_POSITION;
 };
 sampler2D _MainTex;
 float4 _MainTex_ST;
 v2f vert (appdata v)
 {
 v2f o;
 o.vertex = UnityObjectToClipPos(v.vertex.xyz);
 o.direction = v.vertex.xyz;
 return o;
 }
 fixed4 frag (v2f i) : SV_Target
 { 
 half2 longlat;
 longlat.x = atan2(i.direction.z, i.direction.x) * half(-0.5);
 longlat.y = asin(i.direction.y/length(i.direction));
 longlat /= half(UNITY_PI);
 
#ifdef _MAPPING_SKY_DOME
 longlat.x += half(0.5);
 longlat.y *= half(2.0);
#else
 longlat += half(0.5);
#endif
 // Compute how fast the texture coordinates are changing
 // as we move across the screen.
 half4 gradient = half4(ddx(longlat), ddy(longlat));
 // Wrap the gradient for longitude, so a jump of +/-1 becomes 0
 gradient.xz = frac(gradient.xz + half(1.5)) - half(0.5);
 // If you want to sample a smaller mip to cheaply blur the image,
 // just multiply the gradient by the size of your blur.
 // gradient *= half(4.0); // Steps down 2 mips, approximating a 4x4 blur.
 fixed4 col = tex2Dgrad(_MainTex, longlat, 
 gradient.xy, gradient.zw);
 return col;
 }
 ENDCG
 }
 }
}

Be sure to set the texture wrap mode to "Clamp" on the V axis to get correct mipmap generation (texels along the top of the sky shouldn't get blended with ones from the bottom). Here are the settings I used when writing this answer:

Clamping settings

But I'd reiterate that it's better to just use a cubemap for this - it's what they're designed for. For an equal number of texels, cubemaps have a better worst-case angular resolution than equirectangular panoramas, way less stretching and anisotropy, and no seams to fix. It also helps that the logic for sampling and filtering them correctly is built-in, so we get simpler shaders that just do the right thing out of the box. Using Unity's automatic cubemap import, you don't even have to change the textures you're using as your source files - it can transparently re-encode them as cubemaps behind the scenes.

• Ahmed Dyaa: Thank you for the clear answer. The reason I’m using a Texture 2D instead of a Cubemap is that I’m targeting mid-range phones, and a Cubemap would cost me more than a Texture 2D and require more memory. If there’s a way to use a Cubemap without a performance loss, I will certainly follow it.
• DMGregory: What leads you to believe that a cubemap would have a higher memory footprint than a texture2D? Let's say you have a 1024x512 panorama. A cubemap with sides 256x256 uses only 3/4 as much memory, and still gives better worst-case angular resolution. Have you measured the performance impact of this alternative?
• Ahmed Dyaa: How can I unwrap a cubemap the same way I do with a Texture2D in the code I’m using?
• DMGregory: Why would you need to unwrap it? Just call texCube to sample it with your direction vector, or beter yet, use one of the built-in skybox shaders that already works with cubemaps out of the box. If you mean "How can I use an image that's just the top half of a sky?" (a sky dome), then you can open that image in a photo editor, double its height, and fill the bottom half with colour or stretch the row of pixels at the horizon over the rest. Then you can import it as a normal full sphere, not a dome. This cuts into the memory advantage but you keep the sampling benefits.
• Ahmed Dyaa: Thank you, I followed your method, adjusted the image, set the cubemap resolution to 512x512, and measured the performance. I found a slight performance improvement for the cubemap compared to a 2D texture.