Volume Visualization Part II

-GPU Based Volume Rendering

Outline

Graphic Hardware

GPU based Marching Cubes Algorithm

GPU based Ray-casting Algorithm

Texture-Based Volume Rendering

2D-Textured Object-aligned slicing
3D-Textured View-aligned slicing

Common Volume Rendering Software

Graphic Hardware

Accelerate rendering of a virtual scene modeled

by planar polygons
Convert polygons to raster image as a fixed
sequence of processing stages
Ordering of operations is described as a graphics
pipeline

Graphics Processing Unit (GPU)

A dedicated device to manipulate and display computer

graphics

Graphic Pipeline
Scene
Description

Vertex
Geometric
Shader
Processing

Raster
Image

Programmable Pipeline

Fragment
Rasterization
Shader

Fragment
Operation

Vertex Shader

Per-vertex calculations performed here

Without knowledge about other vertices
(parallelism)

Responsibilities
Vertex and normal transformation
Blur and (Per-Vertex) Lighting
Color material application and color clamping
Texture coordinate generation

Set up data for fragment shaders

Fragment Shader

Perform per-pixel calculations

Without knowledge about other fragments
(parallelism)
Can think of a fragment as a potential pixel

Responsibilities and opportunities

Operations on interpolated values
Texture access and application
Fog and color lookup

Assign each fragment a color at the end

GPU Based Marching Cube

Background

Marching cubes (MC) and its variants

MC: Lorenson and Cline 1987
MT: Doi and Koide 1991
MC 33: Chernyaev 1995
MC *: Nielson 2003
MD: Anderson at el. 2005

GPU-based isosurface rendering

MT: Reck at el. 2004
Ray casting: Hadwiger at el. 2005

GPU-based isosurface extraction and rendering

MT: Kipfer and Westermann 2005
Klein at el. 2004

GPU Based Marching Cube

Background

Traditional pipeline
System Memory
Dataset

Acc. Data Structure

Cell
Selection

Vertex
labeling

Cell
Indexing

CPU

Vertex
Interpolation

Normal
Computation

Rendering

GPU

GPU-Friendly Marching Cubes

Parallel Viusalization of Multiple Translucent

Isosurfaces
---By Y. M. Xie, G. Y. Wang, T. T. Wong and P. A. Heng
Department of Computer Science & Engineering
The Chinese University of Hong Kong

GPU-Friendly Marching Cubes

GPU accelerated pipeline

System Memory
Dataset
Acc. Data Structure

Cell
Selection

CPU

Vertex
labeling

Cell
Indexing

Vertex
Interpolation

GPU

Normal
Computation

Rendering

GPU-Friendly Marching Cubes

GPU-Friendly Marching Cubes

Cell Selection

GPU-Friendly Marching Cubes

Vertex Labeling

GPU-Friendly Marching Cubes

Vertex Labeling

GPU-Friendly Marching Cubes

Interpolation
Start

GPU-Friendly Marching Cubes

Rendering Translucent Isosurfaces

GPU-Friendly Marching Cubes

Extract Triangles in Order

We extend the technique of object-aligned slices (OAS)

to achieve the correct order of active cells.
Solve the order of triangles in three levels:
Inter-slab order - the order of slabs within the volume to be visualized
Intra-slab order - the order of cells within each slab
Intra-cell order - the order of triangles within each active cell

We rearrange the traditional triangle table to solve the

intra-cell order.

GPU-Friendly Marching Cubes

Object-Aligned Slices

GPU-Friendly Marching Cubes

Inter- and Intra-Slab Order

Rendering depend on the

Viewing angle

GPU-Friendly Marching Cubes

Implementation and Results

GPU-Friendly Marching Cubes

Rendering

GPU-Friendly Marching Cubes

Rendering

GPU-Friendly Marching Cubes

-Some Examples

GPU-Friendly Marching Cubes

Time Statistics

Rendering speed , flame per

second

GPU-Friendly Marching Cubes

Summary

The proposed GPU-friendly MC algorithm can extract

and render isosurfaces from high-resolution 3D volume
data in real time.

With this framework, we can correctly visualize multiple

translucent isosurfaces, without sorting.

The framework can be trivially modified to implement a

wide range of MC variants.

GPU-Based Ray Casting

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger
VRVis Research Center, Vienna

Why Ray-Casting on GPUs?

Most GPU rendering is object-order (rasterization)

Image-order is more CPU-like
Recent fragment shader advances
Simpler to implement
Very flexible (e.g., adaptive sampling)
Correct perspective projection
Can be implemented in single pass!
Native 32-bit compositing

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger
VRVis Research Center, Vienna

Support for Different Grid Types

Uniform grid (voxels)

Reconstruction with
trilinear interpolation

Unstructured grid
Decomposed into
tetrahedra
Reconstruction with
linear interpolation
REAL-TIME VOLUME GRAPHICS
Daniel Weiskopf
Institute of Visualization and Interactive Systems, University of Stuttgrat, Germany

GPU-Based Ray Casting

Possible optimizations
Early ray termination
Empty space leaping
Adaptive sampling

Hardly applicable to slice-based volume rendering

Combine dynamic sampling and GPU acceleration
Support for unstructured grids

Ray casting in graphics hardware

REAL-TIME VOLUME GRAPHICS
Daniel Weiskopf
Institute of Visualization and Interactive Systems, University of Stuttgrat, Germany

GPU-Based Ray Casting

-Ray Casting in Uniform Grids

Fragment
program

Multi-pass approach
Multiple rendering passes
Constant number of steps per pass
2D textures for accumulating color and opacity
Access volume data from 3D texture map

Ray termination in separate pass

Ping-pong
scheme

REAL-TIME VOLUME GRAPHICS

Daniel Weiskopf
Institute of Visualization and Interactive Systems, University of Stuttgrat, Germany

Structure of GPU Ray Casting

Initial passes
Precompute ray
directions and lengths

Additional passes
Perform raymarching
in parallel for each
pixel
Split up full raymarch
to check for early
termination

Not terminated

Ray Setup

Ray Traversal
Integration
Check Ray
Termination

Terminated

REAL-TIME VOLUME GRAPHICS

Daniel Weiskopf
Institute of Visualization and Interactive Systems, University of Stuttgrat, Germany

Optimizations

Early Ray Termination

Compare accumulated opacity against threshold

Empty Space Skipping

Additional data structure encoding empty space in volume
Oct-tree
Encode measure of empty space within 3D texture read from
fragment shader
Ray-marching fragment shader can modulate sampling
distance based on empty space value

Travis Gorkin
GPU Programming and Architecture, June 2009

Performance and Limitations

More physically-based than

slice-based volume rendering
Guarantees equal sampling
distances

Does not incorporate

volumetric shadows
Reduced number of fragment
operations
Fragment programs made more
complex

Size base volume rendering

Optimizations work best for

non-opaque data sets
Early ray termination and empty
space skipping can be applied

Travis Gorkin
GPU Programming and Architecture, June 2009

Applications

Virtual Endoscopy

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger
VRVis Research Center, Vienna

Applications

Virtual Colonoscopy

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger
VRVis Research Center, Vienna

Applications

Virtual Colonoscopy

REAL-TIME VOLUME GRAPHICS

Markus Hadwiger
VRVis Research Center, Vienna

Texture-Based Volume rendering

steps:

Split-volume management for texture

rendering
Extraction of texture slices
Oblique plane cutting of the volume
Stereoscopic perspective rendering
Final composition of rendering image
Interactive selection of volume of interest

How does a 2D texture

work?
Texture

RGBA
For each fragment:
interpolate the
texture coordinates
(barycentric)

Texture-Lookup:
interpolate the
texture color
(bilinear)

Color
channel

REAL-TIME VOLUME GRAPHICS

Christof Rezk Salama
Computer Graphics and Multimedia Group, University of Siegen, Germany

2D-Textured Object-aligned slicing

Draw the volume as a stack of 2D textures
Bilinear Interpolation in Hardware
Decompostition into axis-aligned slices

3 copies of the data set in memory

REAL-TIME VOLUME GRAPHICS
Christof Rezk Salama
Computer Graphics and Multimedia Group, University of Siegen, Germany

2D-Textured Object-aligned slicing

Texture stack swapped based on closest to

viewpoint
Actual locations of sampling points change
abruptly when the stacks are switched

2D Textured Object-aligned slicing

2D TexturesSampling rate is fix, but can

change the sampling angle to
change the sampling rate

Sampling rate is inconsistent

d

d d

emission/absorption slightly incorrect

Super-sampling on-the-fly impossible
REAL-TIME VOLUME GRAPHICS
Christof Rezk Salama
Computer Graphics and Multimedia Group, University of Siegen, Germany

Issues with 2D-Textured ObjectAligned Slices

Fast and simple

3 times memory consumption
Data replicated along 3 principle directions

Change in viewpoint results in stack swap

Image popping artifacts
Lag while downloading new textures

Sampling distance changes with viewpoint

Intensity variations as camera moves

How does a 3D Textures work?

3 coordination system to have 3D

G
For each fragment:
interpolate the
texture coordinates
(barycentric)

Texture-Lookup:
interpolate the
RGBA
texture color
(trilinear)
REAL-TIME VOLUME GRAPHICS
Christof Rezk Salama
Computer Graphics and Multimedia Group, University of Siegen, Germany

3D-Textured View-aligned slicing

3D-Textured View-aligned slicing

Intersect slicing planes with bounding box

Sort resulting vertices in (counter)clockwise order
Construct polygon primitive from centroid as
triangle fan

Viewing angle

Resampling via 3D Textures

Sampling rate is constant
d

Supersampling by increasing the

number of slices
REAL-TIME VOLUME GRAPHICS
Christof Rezk Salama
Computer Graphics and Multimedia Group, University of Siegen, Germany

Issues with 3D-Textured View-aligned

slicing
Break

Slower, but more memory efficient

Trilinear interpolation at hardware level

Consistent sampling distance

inconsistent sampling rate for perspective
projection

Visible Human Visualization

WebGL & Kinect-based Visible Human Visualzation

3D Print Interface

Some Popular Visualization Tools

Visualization Toolkit (VTK)

What is VTK?
Open source, freely available
software for 3D computer
graphics, image processing,
and visualization
Managed by Kitware Inc.
Strictly object-oriented design
(C++)
High-level of abstraction
Use C++, Tcl/Tk, Python,
Java
Website: http://www.vtk.org/

Features of VTK

Parallel support (message passing,

multithreading)
Stereo support
Cross-platform and runs on Linux,
Windows, Mac and Unix platforms.
Event handling
3D widgets

What can VTK do?

Vector fields

Visualization techniques for visualizing

Scalar fields
Vector fields
Tensor fields
Texture
Volumetric methods

Implicit modeling and polygon reduction

Mesh smoothing, cutting, contouring, and delaunay
triangulation
Image processing
Your own algorithms

OsiriX Imaging Software

MacOS X, Apple Computers

Image processing software dedicated to DICOM
images
Also, image processing software for:
Radiology and nuclear imaging
Functional imaging
Molecular imaging.
Navigation and visualization of multimodality and
multidimensional images
Website: www.osirix-viewer.com/

OsiriX Imaging Software

Ultrafast performance
Intuitive interactive user interface
Exclusive innovative technique for 3D/4D/5D
navigation
Distributed under Open Source licensing - GPL
Open platform for development of processing tools

Features of OrisiX

DICOM File Support

DICOM Network Support
Non-DICOM Files Support
2D Viewer
3D Post-Processing
Optimization
Expansion & Scientific Research
Based on Open-Source components

Mac_OS Version

iPhone Version

Analyze Software

Developed at Mayo Clinic's Biomedical

Imaging Resource (BIR)
Software package for 3D biomedical image
visualization and analysis
Website: www.mayo.edu/

Feature of Analyze Software

Available on 5 Unix platforms, 6 windows

platforms, and PC-based Linux operating
systemes
Imaging Modalities: CT, MRI, PET, SPECT,
ECT, Ultrasound Imaging, Biomagnetic Imaging,
Radiographic Imaging

Volume Rendering in CUDA

NVIDIA CUDA SDK Code Samples
Example: Basic Volume Rendering using 3D Textures

http://developer.download.nvidia.com/compute/cuda/sdk/websi
te/samples.html#volumeRender

Travis Gorkin
GPU Programming and Architecture, June 2009

Volume Rendering in CUDA

3D Slicer www.slicer.org
Open source software for visualization and image analysis
Funded by NIH, medical imaging, MRI data
Currently integrating CUDA volume rendering into Slicer 3.2

Travis Gorkin
GPU Programming and Architecture, June 2009

Volume Rendering in CUDA

Volume Raycasting with

CUDA
Jusub Kim, Ph.D. Dissertation,
Univeristy of Maryland, College
Park, 2008
http://creator75.googlepages.co
m/cuda

Stream model for raycasting

implemented in CUDA
Efficiently balance warps of
threads and block sizes
Single instruction execution
within warp of threads
Avoid memory conflicts with
warps of threads
Travis Gorkin
GPU Programming and Architecture, June 2009

References

Chapter 39. Volume Rendering Techniques, GPU Gems

Volume 1, Ikits, Kniss, Lefohn, Hansen, 2003
http://http.developer.nvidia.com/GPUGems/gpugems_ch39.html
Interactive Visualization of Volumetric Data on Consumer PC
Hardware IEEE Visualization 2003 Tutorial

http://www.vis.uni-stuttgart.de/vis03_tutorial/
Acceleration Techniques for GPU-Based Volume Rendering
J. Krugger and R. Westermann, IEEE Visualization 2003
http://wwwcg.in.tum.de/Research/data/Publications/vis03-rc.pdf
3D Slicer: Volume Rendering with CUDA
http://www.slicer.org/slicerWiki/index.php/Slicer3:Volume_Rendering_Wi
th_Cuda

References

Volume Raycasting with Cuda, Jusub Kim, 2008

http://creator75.googlepages.com/projects
http://creator75.googlepages.com/cudachapter.pdf

Production Volume Rendering, Jerry Tessendorf, Slides

presented at University of Pennsylvania, 2008
Real-Time Volume Graphics, SIGGRAPH 2004
http://old.vrvis.at/via/resources/course-volgraphics-2004/

End!