fix(billboards): Rasterize SVG content at billboard resolution

[donmccurdy]

Description

This PR is a draft for discussion, proposing two related changes when rendering SVG content in billboards:

1. When billboard.image is a URL for an SVG, rasterize the SVG using billboard width/height if specified.
2. Extend JSDoc types to allowing billboard.image to be an HTMLImageElement (already supported, other than types)

These changes are intended to improve visual quality for SVG content in billboards. Because SVGs are scalable, the internal viewBox and width/height are not necessarily the size the user wants to display, and if the user has added a billboard at 50px x 50px size, the billboard will appear pixelated if the (somewhat arbitrary) internal size of the SVG is different. Either of these changes independently provides a workaround for the issue. With (1) the SVG is automatically rasterized at billboard resolution, and with (2) the user can construct the HTMLImageElement themselves, resize it, and pass that to the billboard constructor.

It's not obvious to me that these changes, especially (1), are the 'right' approach ... is it OK for the billboard size to determine the resolution of the SVG in the texture atlas? What if billboard size were in meters? (fixed) But it seemed like a good place to start discussion.

Test code:

viewer.entities.add({
  position: Cartesian3.fromDegrees(-75.59777, 40.03883),
  billboard: { image: "icon.svg", width: 100, height: 100 },
});

viewer.entities.add({
  position: Cartesian3.fromDegrees(-50.59777, 40.03883),
  billboard: { image: "icon.svg" }
});

Before:

After:

Remaining work and known issues:

• Unit tests not yet added or updated
• If the same SVG URL is provided for two billboards of different size, the first determines the texture atlas resolution and the second will use a cached texture.
  • See discussion below.
• Changes to the billboard width/height probably will not change the resolution of the texture (untested).
  • See discussion below.

Issue number and link

Related:

• Poor billboard quality #4235

This PR doesn't fully fix the issue, but improves quality when the billboard and SVG sizes don't match. More fixes (probably related to pixel alignment and anti-aliasing, as discussed in the issue) would be good followups in another PR.

Testing plan

Large billboard = high-resolution SVG texture: Given a larger billboard (100px x 100px), the billboard texture should be scaled up so that the SVG appears crisp.

Billboard size in meters = default texture size: If billboard enables sizeInMeters, we cannot use width/height as indicators of billboard texture size, and the default browser-detected size will be used instead.

Multiple billboards, mixed sizes: Given multiple billboards of different sizes reusing the same SVG, each billboard must (at least) render at the expected on-screen size. Optimal texture resolution is not guaranteed - the first billboard texture for a given URL is cached — but this behavior may change in the future.

Author checklist

• I have submitted a Contributor License Agreement
• I have added my name to CONTRIBUTORS.md
• I have updated CHANGES.md with a short summary of my change
• I have added or updated unit tests to ensure consistent code coverage
• I have updated the inline documentation, and included code examples where relevant
• I have performed a self-review of my code

[github-actions]

Thank you for the pull request, @donmccurdy! Welcome to the Cesium community!

In order for us to review your PR, please complete the following steps:

• Send in a Contributor License Agreement (CLA)
• Add yourself to the contributors file

Review Pull Request Guidelines to make sure your PR gets accepted quickly.

[mzschwartz5]

My two cents:

1. This is obviously a big improvement.
2. You're right that there will be issues with multiple, different sized billboards using the same SVG. I don't think this is uncommon, either. I wouldn't let that prevent this PR from being merged, but it should be a follow up to address (mipmaps starting at the biggest billboard size?)
3. I do wonder about the drawbacks of this improvement. I think it's just that we use more memory to store these textures? How much more? (Is this going to limit the utility of cesium on platforms like mobile?)

Separately, I wonder if we have considered an SDF-approach for SVG textures. As described in this video. I think we use this technique (or something like it) for text-rendering, already. In the video, it's applied to PNGs, so I'm not actually sure if it can work with SVGs, or if it can work at runtime (they show it as a preprocess step, but it looks like it can be done dynamically)... anyway just thoughts I'm thinking. Overall, this PR is realistically going to get us most of the way there.

[ggetz]

I can confirm that @donmccurdy is now covered by a CLA.

[donmccurdy]

Thanks @ggetz and @mzschwartz5!

I wouldn't let that prevent this PR from being merged, but it should be a follow up to address (mipmaps starting at the biggest billboard size?)

Agreed, though not sure on the best followup approach yet. Another idea might be to use the existing image cache, but include image size as part of the cache key. Perhaps instead of the exact image size, that could be snapped to some nearest preset size (16, 32, 64, 128, 256) so that we're allocating "at most" 256x256x133% resolution per SVG, regardless of how many unique sizes are created in the scene.

I do wonder about the drawbacks of this improvement. I think it's just that we use more memory to store these textures? How much more?

Hm, hard to say. It could go the other way, if the browser happened to be allocating a large resolution for an SVG only displayed at 10px size. On the bright side I think there's a natural upper bound on how large the billboard's display size will be, especially if we clamp it as mentioned above.

The SVG-to-SDF idea is a good one I think! I haven't seen that done yet. Any idea how common SVG billboards are, compared to images? I'm hoping to take a closer look at the SDF-based label rendering in a future PR too, but I think the current font-to-SDF rendering dependency only supports fonts.

[donmccurdy]

I've made a small change so that sizeInMeters is considered; if width/height are not in pixels they cannot be used to set the SVG texture resolution. Remaining TODOs in the original post are still remaining – I'm hoping to make those changes today and will write an update here afterward!

[donmccurdy]

Ready for review!

Another idea might be to use the existing image cache, but include image size as part of the cache key ... snapped to some nearest preset size (16, 32, 64, 128, 256)...

I briefly implemented this, and the issue I ran into was that the ID (used as the cache key) is also returned from billboard.image here...

cesium/packages/engine/Source/Scene/Billboard.js

Line 942 in edceb42

return this._imageTexture.id;

... which makes me nervous to append anything to the user-provided URL. I'd been attempting to create an ID like ${url}@${width}x${height}.

Hoping for a safe incremental improvement, I don't think the billboard texture uses mipmaps yet (could be wrong, but didn't find the code that would generate them?), but that seems like a very promising future improvement for many use cases, not just SVGs. So perhaps let's keep an approach for this PR where 1 URL = 1 cache entry. For now N>1 SVG billboards are scaled according to the first/cached texture, but that can be improved later, hopefully with mipmaps, as @mzschwartz5 suggested from the beginning. :)

[mzschwartz5]

I think this is good to merge (+/- some minor style comments you can choose to apply. I'll wait to merge until I hear from you on those).

Based on your comments, I think a follow-up using mipmaps snapping to fixed increments is a solid approach (not too complex, good value solution).

[donmccurdy]

Thanks @mzschwartz5! I think all comments are addressed, ready to merge any time.

[mzschwartz5]

Merging! Nice work 🚀