Bridging
digital simulation
& physical touch.
A mixed reality temporomandibular joint simulation integrating real-time biomechanical modeling with native haptic feedback - sub-10 ms latency for surgical training, running a stable 1 kHz force-feedback loop.
- Role
- Software engineer · XR systems
- Year
- 2023
- Domain
- Medical XR · dental / oral surgery
- Platform
- Windows · Phantom / Geomagic
- Core stack
- Unity · native C++ DLLs · HLSL · P/Invoke
- Haptic loop
- 1 kHz · sub-10 ms latency
Bypassing the engine where it mattered.
TMJ disorders require extremely precise surgical interventions. The challenge was to create a simulation that not only looked photorealistic but felt physically accurate to the surgeon's hand.
Standard game engines like Unity update physics at 60–90 Hz, but realistic haptic feedback requires a loop running at 1 kHz. Any drop in frequency produces stair-stepping or vibration artifacts that break immersion and ruin training value.
The solution was a hybrid architecture - critical physics in unmanaged C++ memory space, visual updates synchronized to Unity's managed environment. No Garbage Collection spikes. No force-feedback dropouts. Indistinguishable-from-reality tool resistance.
Engineering pipeline.
Analysis of jaw kinematics and soft-tissue resistance data - the reference the simulation had to match.
Building the C++ DLLs for direct hardware communication with Phantom/Geomagic devices.
Connecting the haptic loop to Unity via P/Invoke marshalling - careful allocation patterns to prevent GC spikes.
Custom HLSL subsurface-scattering shaders for realistic flesh rendering.
Calibration with haptic devices and clinical accuracy testing against physician reference.
Under the hood.
Native haptic feedback
Bypassed Unity's standard physics to implement a direct C++ interface with Phantom/Geomagic devices, ensuring a stable 1 kHz force-feedback loop for indistinguishable-from-reality texture and resistance.
Biomechanical modeling
Mathematical models of mandibular movement constraining the virtual jaw to anatomical limits - simulating the elasticity of ligaments and muscle tissue.
Subsurface scattering
Custom HLSL shaders that simulate how light penetrates translucent oral tissue, scatters internally and exits elsewhere - giving the flesh its characteristic waxy look.
Hybrid architecture
Decoupled architecture where critical physics happen in unmanaged C++ memory space, while visuals sync to Unity's managed environment - preventing Garbage Collection spikes.