Showcasing 3D CAD models Updates | ATOMWOLF

Rendering Outlines with a Post-processing Shader

2024-07-18T00:00:00Z

Summary: I learn about shaders to make a post-processing effect for model-viewer that adds outlines to highlight detail in 3D CAD models.

Assumed audience: curious folks on the web. You shouldn't need to know already know 3D graphics, shaders, or OpenGL.Don't lose heart! The world rewards curiosity.

Project: part of Showcasing 3D CAD Models on the Web

Tags: 3D graphics, web, computers

Created: 18 July 2024

Outlines are key to discerning details in 3D CAD models. Let’s compare the modeling view from Fusion 360 with a render from model-viewer:

Comparison of CAD modeling view (top) from Fusion 360 and a rendered view (bottom) from model-viewer

The same model looks different, even ignoring the slight camera position difference due to manual positioning. The lighting and shadows are different, the materials look different, and the colors are different. The perspective is different. Even if I adjusted the lighting, added textures, and used realistic material settings, I’d prefer looking at details in the CAD view because of the outlines. I've written about this elsewhere in this series, including Experimenting with CAD models in model-viewer.

Could I add outlines? Maybe! I didn’t know anything about how model-viewer worked. I browsed the documentation around extensions and modifications, then set the project aside to leave an opening for inspiration.

Inspired by Mœbius

A few weeks later, I ran into an in-depth article about creating a post-processing effect inspired by the French artist Mœbius.

Plate 2 of Voyage d’Hermès, by Mœbius. Image from Sci-Fi-O-Rama.

This article builds a shader, step-by-step, to add outlines, adjust shadows, and create a hand-drawn appearance to a rendered scene.

To extend model-viewer, you can build on its foundation, three.js, or use model-viewer-effects, which adds post-processing effects. The article used react-three-fiber, a library that connects React to three.js! I couldn’t copy-and-paste the code samples—I couldn’t find an example for model-viewer-effects using a custom shader—but this was exactly the inspiration I needed.

Adding a custom shader to model-viewer

I couldn’t find a demo with a custom shader and model-viewer, so I needed to create one. Per the documentation, I set up postprocessing and model-viewer-effects. model-viewer-effect’s documentation on custom effects aims folks at the Postprocessing Custom Effects wiki page. After reading and re-reading the Custom Effects wiki page, I had a small understanding of how it works.

A scene is rendered and given to the custom effects. The effect runs C-like code for each pixel of the source image, outputting a pixel used for the output image.

With much effort, I had the skeleton of a custom effect rigged up to model-viewer. I was able to use it to add a partial outline to the model. This was a promising start, but I couldn’t make any improvements. I needed a deeper understanding.

Background

WebGL is a JavaScript API that uses OpenGL ES for high-performance 2D and 3D graphics rendering in the browser.
Three.js (GitHub) is a JavaScript library that usually uses WebGL for rendering.
model-viewer (GitHub) is a JavaScript library and Web Component that uses three.js to display 3D models in the browser.
The Poimandres postprocessing library adds post-processing effects to three.js.
model-viewer-effects (GitHub) is a library and Web Component that uses the postprocessing library to add effects to model-viewer.

What is post-processing?

Post-processing involves rendering a scene to an intermediate buffer before applying effects. Some effects can be applied directly to the intermediate render. Other effects are more complex, needing multiple passes or additional information.

What is a shader?

OpenGL renders images using a rendering pipeline. The Khronos Group will remind you that an OpenGL implementation renders images, while OpenGL is only a specification and an API definition for rendering. Some pipeline steps run programs called shaders. There are different types of shaders, like vertex shaders, which run for each vertex of the geometry, and fragment shaders, which run for each fragment, which is almost the same as a pixel. Shaders vary, but the prototypical vertex shader transforms a vertex’s 3D coordinates to a 2D screen position, and the prototypical fragment shader sets the color of a pixel. OpenGL shaders are typically written in OpenGL Shading Language (GLSL), similar to C.

Creating the outline effect

There are many ways to make outlines in 3D graphics. See 5 ways to draw an outline and Generating Beautiful 3D Outlines, for instance. One method uses a fragment shader. Pixel-by-pixel, it looks for edges and draws outlines on the detected edges. It uses different versions of the scene to detect more edges.

Based on advice from some of the articles I had found, I started with a grayscale version of the rendered scene.

The grayscale version of the rendered scene's colors

I use the Sobel operator to detect edges. For each point in the image, the Sobel operator looks at the eight neighboring points to determine if the center point is an edge. Many image operations can be done by looking at a small window of pixels around each pixel. Each output pixel is created through some math done on that small window of input pixels. This is called convolution. It can be tricky to understand. The Mœbius shader article has an interactive Sobel demo, and I found a similar interactive sharpening demo. If you know of something better, please let me know! The Sobel operator isn’t perfect, but it’s easy to implement in a fragment shader. If the Sobel operator’s calculated value is higher than some threshold, I count it as an edge and color the pixel black.

Edges detected from the grayscale version of the rendered scene's colors

It does an okay job—better than I was expecting! Much of the interior geometry is lost, but the silhouette and some details are outlined.

The next aspect used to add edges is depth. The depth map is a grayscale image where the color represents the distance from the viewpoint to the object.

The depth map, where the pixel brightness represents how far away the point is from the viewpoint

Once again, the Sobel operator detects edges.

Edges detected from the depth map

Using the Sobel operator to detect edges using the depth map reveals edges around the window, the cutouts, and the silhouette. Most other edges are missed.

The next aspect used to detect edges is the normal map. The normal of a surface is the direction the surface faces, and a normal map is an image that colors each surface based on the direction the surface faces.

The normal map, where colors represent the direction the surface faces

Using the Sobel operator, again, we highlight detected edges in the normal map.

Edges detected from the normal map

Wow! For this model, at least, the detected edges create a detailed outline. Parallel surfaces without intermediate geometry seem to be the cause of most missing edges. There is at least one clever improvement to this method documented in the resources. For example, when looking at the top from an angle, the far edges of the buttons usually aren’t outlined.

Next, I combine all the detected edges.

Edges detected from the grayscale version of the scene's colors, the normal map, or the depth map

This looks pretty good!

Let’s overlay the edges on the rendered scene.

Detected edges overlaid upon the original rendered scene

Looking critically, the A and B labels are missing outlines from most viewpoints. Other edges are missing, and there are often so many outlines drawn on the standoffs that it looks like shading. There’s an issue with aliasing.

It isn’t perfect, but this looks even better than I had hoped!

Demos

While working on this article, I made a tool to take screenshots of the shader in various configurations. I kajiggered it into an interactive demo where you can play with the shader settings. As I was brainstorming ways to present an explanation, I made another demo that shows all the views of the model in synchrony.

Next Steps

Shader Improvements

Assuming this works as well on other models, it’s good enough for me to consider outlines handled. In case I change my mind (or for the inevitable tinkering), I’ve collected some articles that detail improvements and alternate approaches.

I haven’t yet reviewed the shader with care. I built this shader through experimentation and chose the various settings haphazardly. I’m certain there are real improvements to be made and bugs to fix, even before improving the actual algorithm.

Moving on

The next step is to wrap this outline effect into a Web Component that integrates with model-viewer-effects. No one should have to write JavaScript to add this effect. It’ll go inside <model-viewer> tags like the built-in effects.

Additionally, I want to create a one-page “quick and dirty” example for using a custom shader with model-viewer, and an example model-viewer-effect-alike Web Component. I don’t think custom shaders for model-viewer are commonly asked about, but I want to help document them.

Resources

WebGL: 2D and 3D graphics for the web Mozilla’s WebGL documentation. It includes reference material, tutorials, guides, articles, and talks. It seems to be a good starting point, and what I’ve read has been clear and helpful.
Khronos OpenGL wiki Khronos, the group behind OpenGL, has an in-depth wiki.
5 ways to draw an outline Overview of five ways to draw outlines in 3D graphics.
Generating Beautiful 3D Outlines Three ways to draw outlines: offset and scale, stencil and wireframe, and post-processing.
Moebius-style post-processing and other stylized shaders Quite good. Dissects “Mœbius style”, which includes outlines. Uses post-processing with depth and normals. Code is JavaScript, using react-three-fiber, but definitely doesn’t require React knowledge to understand.
Sketchy Pencil Effect with Three.js Post-Processing Uses post-processing with normals and grayscale version of the rendered scene. Code is JavaScript, using three.js.
Edge Detection Outlines
How to render outlines in WebGL
Outlines via Postprocessing
Post-processing, uses depth and normals. Code is C# for Unity.
Unity Outline Shader A detailed walkthrough creating and improving an outline shader in Unity. Code is C#. It seems to be similar but significantly improved from what I’m doing here.
Better outline rendering using surface IDs with WebGL (GitHub) Highlights issues with using normals, and assigns surface IDs instead.
Pixel-Perfect Outline Shaders for Unity Different method than our post-processing edge-detection, but good article. Uses a vertex shader to add a slightly larger mesh. Code is for Unity.
Moebius-style 3D Rendering ~8-minute video. Breaks down the “Mœbius style”, and imitates it in Unity. Helpful explanation and visuals. Beginner-friendly, but it expects 3D rendering knowledge.

Reply via email

Experimenting with CAD models in model-viewer

2024-06-10T00:00:00Z

Summary: I use Blender to export models from Fusion 360 to model-viewer, and I investigate unexpected visual problems.

Assumed audience: folks on the web

Project: part of Showcasing 3D CAD Models on the Web

Tags: web, computers, 3D graphics

Created: 10 June 2024

After exploring options for showcasing 3D CAD models on the web and compiling a list of desired features, I’m ready to experiment with Google’s model-viewer, using a real model I’ve been working on.

I’m making a small case for a T-Display S3 with a few buttons, a cutout for the screen and the USB-C port, a small prototyping PCB, and two Molex SL connectors. I exported some images from the modeling view in Fusion 360. Fusion 360, a closed-source CAD suite, is my go-to for engineering 3D models. Although Autodesk recently added some features I've been wanting for years, like Configurations, I worry about its future. Wikipedia informs me it's been recently rebranded as just "Fusion".

Showing all the components in Fusion 360’s modeling view

To show the case details, I hid the electronics and the connectors.

Showing only the printed components in Fusion 360’s modeling view

Exporting from Fusion 360

model-viewer requires models in glTF format (or GLB, glTF’s binary equivalent). Fusion 360 doesn’t export to these formats without a plugin. I found two proprietary glTF exporters, one by ProtoTech Solutions and another by SimLab, but I want to avoid a closed-source plugin. Based on the model-viewer FAQ’s recommendation, I decided to try Blender as a converter. I like Blender. Blender is open source, and it's used by professionals and hobbyists. I haven't spent more than an hour or two with it. though, which isn't even enough time to make a good donut.

I exported the model from Fusion 360 in both FBX and USDz formats for comparison.

FBX

FBX is a proprietary file format controlled by Autodesk.

When importing the FBX into Blender using defaults, the model was oddly oriented.

FBX model imported into Blender

After some tinkering, I found that setting the import options for Manual Orientation to “Y Forward” and “Z Up” in Blender’s FBX import dialog resolved the orientation issues. Exporting to glTF 2.0 with the “+Y Up” option checked produced a correctly oriented model in model-viewer.

To check scaling, I used the validation report in model-viewer’s editor. The model dimensions matched the dimensions in Fusion 360.

Fusion 360 to FBX to glTF 2.0 via Blender, viewed in model-viewer’s editor

With the Manual Orientation adjustment on importing FBX and the +Y Up option set when exporting to glTF 2.0, the glTF file has the correct orientation and scale.

USDz

USDz is an open-source format developed by Pixar, with wide support in Apple products. It’s part of Universal Scene Description (USD).

Importing the USDz file into Blender maintained the correct orientation, but I encountered scaling issues. Setting the import scale to 0.001 in Blender’s import dialog resolved this problem.

Fusion 360 to USDz to glTF 2.0 via Blender, viewed in model-viewer’s editor

With the scale adjustment on importing USDz and the +Y Up option set when exporting to glTF 2.0, the glTF file has the correct orientation and scale.

Rendering Challenges

With the model successfully imported into model-viewer, I quickly realized that, compared to Fusion 360’s modeling view, it was difficult to see details in the model in model-viewer.

model-viewer rendering with “neutral” lighting environment

model-viewer lets you control the lighting and environment around the model. The previous image showed the model using the default scene with even lighting on all sides.

model-viewer also has a “legacy” environment meant for frontward viewing.

model-viewer rendering with “legacy” lighting environment

The “legacy” environment reveals more details than the “neutral” environment. Neither option came close to the clarity of Fusion 360’s modeling view with its outline rendering.

Understanding the Rendering Differences

This isn’t a case of model-viewer or glTF being “bad”. model-viewer does a great job rendering glTF.

model-viewer rendering a damaged helmet, one of the example models in model-viewer’s editor

glTF aims for photorealistic results through “physically based rendering”. glTF's physically based rendering (PBR) uses "Bidirectional Scattering Distribution Functions" and microfacets to model light interactions with surfaces. glTF includes a consistent set of parameters, helping to standardize PBR across various rendering engines. The Khronos Group, the consortium behind glTF, has a detailed explanation of PBR in glTF.

Good results from physically based rendering need good parameters and textures for the materials. In Fusion 360, I don’t usually do much with materials, textures, or colors. For this model, I set the material to “Plastic” and set the colors to some defaults. The materials were exported through an intermediate format into glTF 2.0. Materials in physically based rendering have a base color and some surface parameters, but the rendered color of the surface may be quite different from the base color. Just like in the real world, the color of the light that bounces off a material isn't just the color of the material. It's influenced by the surface details and what the light was like before it interacted with the material. There are interactive lighting and color examples in the Lighting & Skybox model-viewer examples section. model-viewer's "neutral" lighting environment was engineered to color materials closely to the material's base colors. For more, read "Achieving Color-Accurate Presentation with glTF" and "Tone Mapping Considerations for Physically-Based Rendering". Even if I do well with the materials (and lighting), though, I should expect the render to show detail like a photo. I think the Fusion 360 modeling view does better than a photo.

Exploring solutions

Can I turn on outlines like in Fusion 360’s modeling view? model-viewer supports model-viewer-effects, an extension that changes the visuals. model-viewer-effect has an outline effect, but it only adds an outer outline around objects, not internal edges.

Don’t lose hope! All of this is open source. model-viewer-effects uses postprocessing, a library that makes effects using vertex and fragment shaders. If I can’t use postprocessing, I could plumb through model-viewer into three.js.

Next steps

Can I add outlines to model-viewer? "The Programmers’ Credo: we do these things not because they are easy, but because we thought they were going to be easy", pinboard I have no experience with WebGL, three.js, or postprocessing. I am not sure what a shader is. Let’s find out!

Reply via email

Goals for a 3D Model Viewer

2024-03-08T00:00:00Z

Summary: I list goals for a viewer meant for showcasing 3D CAD models on the web.

Assumed audience: folks on the web

Project: part of Showcasing 3D CAD Models on the Web

Tags: web, computers

Created: 08 March 2024

My hopes and dreams for a 3D CAD model viewer may be nitpicky and particular, but they’re mine. I’m planning to write about CAD models, and I want to point to details in an interactive web viewer. I started a list of goals when I decided to begin with model-viewer, and I’ve added to it.

annotations (hotspots)

I need to be able to add a caption to a model feature and to move the camera to show a specific view of an annotation.
component visibility

I need to be able to hide or show specific components of a model. For instance, I need to be able to show and hide the lid of a box.
show dimensions

I need to be able to add the overall dimensions of a model.
banana for scale

This is silly, but I’d really like the ability to add a banana next to a model. Per Know Your Meme, the earliest instance of a person posting a photo online with a banana for scale was in 2005 on the blog Rock Dog Designs. Using the Wayback Machine, I found the original post. "I don’t know how big the screen is, we’re moving and I can’t find the tape measurer. But I do have a banana. For scale. Oh wait, my husband says it’s 19inches [sic]. Oh well, I’ll leave the banana for interest. Please be interested." Other folks have made an STL viewer with a banana for scale, and you can add a banana for scale in KiCad.

Augmented Reality (AR) / Virtual Reality (VR)

It’d be nice if it were easy to use a mobile device to see what a model looks like on a desk or in a visitor’s particular environment, and it’d be nice if it were easy to use a VR headset to interact with the model.
rendering the models I want to use well

There’s a lot I don’t know about 3D rendering, and a viewer that does great rendering shoes or models from video games might not automatically do well rendering the CAD models I want to show.
open source

I wrote more about why I want to use an open-source viewer in “Choosing a 3D CAD viewer”, but one reason is that I need to be able to modify the viewer.
not reliant on centralized service or hosting

I want to be able to host my own models and the viewer itself. If the viewer changes, I don’t want to be forced to quickly update my site. I wrote more about this in “Choosing a 3D CAD viewer”.
quick to load

Adding the viewer to a page shouldn’t make the page slower in a way that people can tell. Real-world web performance is more complicated than Lighthouse scores, but the viewer should still allow perfect Lighthouse scores on mobile and desktop.
low page weight (data)

If a visitor doesn’t interact with the viewer, the browser shouldn’t need to download much extra data.
accessibility

This isn’t a complete list, but it should work well for folks without JavaScript, with vision impairments or with a screen reader, with older browsers, with slower connections, with limited or metered data, and even to folks who like printing things.
support progressive enhancement

Adding a 3D viewer to a page shouldn’t break the page for folks who can’t use the viewer. One option has the an image with good alt text and a caption, and the interactive viewer seamlessly replaces the image. If the viewer doesn’t load, the visitor still gets the image.

Next Steps

Next, I’m going to load an example CAD model into model-viewer and see how it fares.

Reply via email

Choosing a 3D CAD Viewer

2024-03-06T00:00:00Z

Summary: I investigate 3D model viewers for the web, including model-viewer, Sketchfab, View3D, and ThreePipe, to showcase CAD models on ATOMWOLF. I decide to start with model-viewer.

Project: part of Showcasing 3D CAD Models on the Web

Assumed audience: folks on the web

Tags: web, computers

Created: 06 March 2024

We’ve been putting 3D models on the web since 1994, but doing it well is tricky. The more I learn, the more complicated it seems. mitxela built their own viewer and wrote a good writeup. Among other things, it reveals a glimpse of the complexity of camera control. I want to write articles showcasing CAD models in an interactive web viewer. I looked toward existing options for inspiration.

Sketchfab

The first 3D viewer I found was Sketchfab. Sketchfab says it’s the “best 3D viewer on the web”. I don’t know about that, but I do like the annotations.

A model annotation from the Sketchfab Annotation Demo

If I were to use Sketchfab, Sketchfab would host my 3D models. I’d add an iframe with a particular Sketchfab URL to a page, and web browsers would create a page-within-a-page to show the viewer. Sketchfab has a JavaScript API that lets developers interact with the viewer in the iframe to develop some custom features.

Sketchfab’s viewer is hosted by Sketchfab, and my site would just refer to it. If Sketchfab changed their viewer, the viewers on my pages would change, whether I wanted it or not. If Sketchfab had an outage, my pages would be broken. If Sketchfab went out of business, I’d have to scramble to replace them. If the Sketchfab viewer were open source or if I could host it, I’d be less concerned. I've found that relying on this type of service tends to create emergencies for me. The more services like this I use, the more frequently my work is driven by upstream changes. Having something I host, something I update, helps reduce those emergencies. Being able to use a previous version is really helpful! Furthermore, my ability to modify the viewer is limited.

For the features I’d like, Sketchfab seems expensive. I think Sketchfab would cost me at least $129 per month.

I don’t think Sketchfab is a good fit for a 3D viewer for ATOMWOLF, but it may be a good fit for others. If I were advising a business that sold 3D models needed to get a 3D viewer out ASAP, I’d check out Sketchfab.

Autodesk Viewer

Autodesk has a web viewer. I can save my Fusion 360 projects to their storage and look at them in the browser. The viewer displays the model to look just like it does in Fusion 360. You can rotate and zoom, measure parts of the model, run saved animations, and show and hide parts of the model. I can make the project public and embed the viewer into a page.

The Autodesk viewer wouldn’t cost me anything extra, but my other concerns about Sketchfab apply here.

Goals

After Sketchfab and the Autodesk viewer, I had a better sense of what I wanted.

I don’t want to use a viewer that’s an iframe to an external site or one that only shows models hosted at an external site. I want to be able to host the models and the viewer.
I want to be able to modify the viewer. It should be open source.
I need to add text to call out model features to discuss design decisions. These are called “annotations” or “hotspots”.

After surveying more viewer options, I grouped them into categories.

Framework viewers

Framework viewers are more like libraries or examples than applications. They handle many complexities but would require development work to integrate into a site.

Three.js

Three.js is an open-source JavaScript library for 3D graphics in the browser. It uses WebGL, and it can use WebXR to do Virtual Reality (VR) things. It has good docs, a bunch of examples, and it’s been around since 2010. It’s pretty awesome. Most of the other options on this page are either built on three.js or built on things built on three.js.

I’m not sure three.js has a viewer. They have an editor and many loader examples. I’d definitely have to build a lot of the features I want.
A-Frame’s model viewer

A-Frame is a fun web framework built on top of three.js, meant for VR. My first VR development experience used A-Frame. I made a demo that played videos and used a controller for 3D space data logging. After getting through some conceptual hoops around VR development, using A-Frame was really quick. I liked it, but I don’t think I’d use A-Frame for this project.
Babylon.js’s viewer (example)

Babylon.js is similar to three.js, but the library is a bit less graphics-y and a little more “game engine-y”. The Babylon.js model viewer has a lot of features: animations, asynchronous loading of models, the Babylon.js Inspector. I didn’t see annotations, and the interface would need to be modified.

Viewer “applications”

These viewers are meant to be used as-is. They vary widely, and I gave some only a glance.

Don McCurdy’s three-gltf-viewer (GitHub)

This viewer is built on three.js. It supports animations but doesn’t appear to support annotations or VR/AR/WebXR.
kovacsv’s Online3DViewer (GitHub)

I don’t know a lot about kovacsv’s Online3DViewer. It’s open source and uses three.js. The viewer is embedded using an iframe. I don’t think it has annotations or VR/AR/WebXR support.
View3D (GitHub)
View3D is built on three.js. It’s open source. It supports annotations and animations. It supports augmented reality through WebXR, Android’s Scene Viewer, and iOS’s AR Quick Look. It appears to be maintained by a South Korean company called Naver.
ThreePipe (GitHub))

ThreePipe is built on three.js. ThreePipe says it has a “simple, intuitive API for creating 3D model viewers/configurators/editors on web pages, with many built-in presets for common workflows and use-cases.” It has plugins, including one for Gaussian splats. Gaussian splats don't seem helpful for this project, but they're interesting! It doesn’t appear to support WebXR/AR/VR yet, and it doesn’t appear to support annotations.

model-viewer (GitHub, docs)

model-viewer is a Google project. It’s open source and built on three.js. It’s got a lot of examples. It supports annotations, and AR through WebXR, AR Quick Look, and Scene Viewer. VR support seems to be in progress. It has an example showing model dimensions. There is good integration between the 3D viewer and the rest of the webpage. It feels “in the spirit of the web”.

model-viewer has a lot of polish. It can show an “interaction prompt” animation on the model, giving visitors an affordance that the model isn’t a static image. model-viewer has an editor to help with configuration. The model-viewer folks care about and track [page load performance].(https://modelviewer.dev/examples/lighthouse.html) Asynchronously loading model-viewer decreases its impact on page load performance, but model-viewer is still 60+ KB. Lite model-viewer is an easy-to-use third-party wrapper that defaults to only loading model-viewer after the visitor interacts with the model image. For a visitor that isn't going to interact with the model, using lite-model-viewer means the page size is only around 3 KB larger than just showing the image! model-viewer starts with a pre-rendered image of the scene and seamlessly replaces it with the interactive model when the model file loads. This is “in the spirit of the web” and heavily influenced my decision to start with model-viewer.

Another way model-viewer aligns with the web is that it’s a Web Component. You don’t need to write JavaScript to put a model on your page. You can add a <model-viewer> tag. Awesome!

model-viewer is open to modifications. model-viewer-effects provides postprocessing effects, like pixelation, glitching, and antialiasing. Additionally, I could hook into three.js and modify things how I like.

Conclusion

I’m going to start with model-viewer, but not without reservation.

model-viewer is maintained by folks from Google. While I’m cautious of Google’s history of discontinuing projects, the open-source nature of model-viewer gives me reassurance. When Google decides it’s done with model-viewer, the community might maintain it, but I could continue to use the last version while migrating to another option, like View3D or ThreePipe.

My next step is to discover the rest of my finicky requirements as I integrate model-viewer into ATOMWOLF.

Reply via email