MEDICAL XR • 2023

TMJ Simulation
Haptic System

A mixed reality temporomandibular joint simulation integrating real-time biomechanical modeling with native haptic feedback, achieving sub-10ms latency for surgical training.

Role

Lead XR Engineer

Timeline

Ongoing

Platform

Windows / Mixed Reality

Tech

C++ & Unity Haptics

Loading Jaw Model...
Drag to rotate • Scroll to zoom

< 10ms

Haptic Latency

C++

Native Physics Core

1kHz

Refresh Rate

Clinical

Validations

// The Challenge

Bridging Digital Simulation & Physical Touch

Temporomandibular joint (TMJ) disorders require extremely precise surgical interventions. The challenge was to create a simulation that not only looked photorealistic but felt physically accurate to the surgeon.

Standard game engines like Unity update physics at 60-90Hz, but realistic haptic feedback requires a loop running at 1000Hz (1kHz). Any drop in frequency results in "stair-stepping" or vibration artifacts that break immersion and ruin training value.

  • Achieve stable 1000Hz haptic refresh rate within a Unity environment
  • Synchronize high-speed C++ physics with slower C# rendering
  • Simulate complex biomechanical jaw movements and tissue resistance
  • Render realistic oral tissue using Subsurface Scattering (SSS)

// Development Process

Engineering Pipeline

01

Bio-Research

Analysis of jaw kinematics and soft tissue resistance data

02

Native Core

Building the C++ DLLs for direct hardware communication

03

Integration

Connecting the haptic loop to Unity via P/Invoke marshalling

04

Shaders

Developing custom HLSL SSS shaders for realistic flesh rendering

05

Validation

Calibration with haptic devices and clinical accuracy testing

// Key Features

System Capabilities

Native Haptic Feedback

Bypassed Unity's standard physics engine to implement a direct C++ interface with Phantom/Geomagic devices, ensuring a stable 1kHz force-feedback loop for indistinguishable-from-reality texture and resistance.

Biomechanical Modeling

Implemented accurate mathematical models of the mandibular movement, constraining the virtual jaw to anatomical limits and simulating the elasticity of ligaments and muscle tissue.

Subsurface Scattering

Custom HLSL shaders written to simulate how light penetrates translucent oral tissue, scatters internally, and exits at a different point, creating the "waxy" look of real organic matter.

Hybrid Architecture

Decoupled architecture where critical physics calculations happen in unmanaged C++ memory space, while visual updates are synchronized to Unity's managed environment, preventing Garbage Collection spikes.

// Technical Stack

Under the Hood

Core & Interop

C++ 17 C# / .NET P/Invoke OpenHaptics SDK Multithreading

Rendering & Physics

Unity URP HLSL / Shader Graph Compute Shaders Soft Body Physics

Hardware

3D Systems Touch Geomagic Phantom Meta Quest 3 (MR) Varjo XR-3

Tools

Visual Studio CMake RenderDoc Git LFS 
// Example: Interfacing Unity C# with Native C++ Haptics DLL
public class HapticInterface : MonoBehaviour
{
    // Import function from unmanaged C++ DLL
    [DllImport("HapticPlugin")]
    private static extern void UpdateHapticForce(int deviceId, Vector3 position, Vector3 velocity);

    void FixedUpdate()
    {
        // Send Unity physics state to the 1kHz haptic loop
        var pos = transform.position;
        var vel = _rigidbody.velocity;
        
        // Direct memory call avoids GC overhead
        UpdateHapticForce(_deviceId, pos, vel);
    }
}