Image synthesis and OpenGL: introduction

1. What is image synthesis?
2. Applications
3. Desirable qualities
4. Simplified pipeline
5. Sources
   

What is image synthesis?

• Creates synthetic images from numerical models
  
• Images are created to be seen by humans
• Goal: Generates plausible/legible images for the human eyes
  

Applications

Video-games

Space Invaders (1978)	Wolfeinstein 3D (1992)	Halo (2001)	Uncharted 3 (2011)

Animated movies

Gery's game	Number 9	Brave

Special effects

Abyss	Matrix	The Hobbit

Advertising

cars	Chocolate

Computer Aided Design (CAD)

General Motors (60's)	Piston	Plane

Education

Heritage conservation

Mellado et al. 2010	Larue et al. 2012

Scientific visualization

Nasa	Temperature (Conninx et al. 2011)	Airflow

Medical visualization

Haidacher, 2007

Training simulation

A380 Simulator

Interaction, virtual reality, interfaces

Cave	Collaborative environment

Desirable qualities

Realism

• What is realism?
• How to define and obtain it?
  
• Are there alternatives?
• The uncanny valley
  

See a video here

Speed

• How many frame per second (FPS) do we need? For a movie? For a video-game?
  
• How can we create images efficiently?
• Offline VS online rendering, convergence between both!
  

Simplified pipeline

• Need a "model " of the world
• 2D (images)
• 3D (world coords)
• 4D (+ time)

• A simplified version of the model involves:
• a camera
  
• some 3D objects
  
• some materials, appearance properties associated to (part of) the models
  
• some lights (ambiant / directionnal / spot / etc)
  
• some renderings (sampling strategies / antialiasing / post-processing / etc)
  

Download this simple scene

The camera

• Position
  
• Type of projection (orthographic / perspective / fish-eye / etc)
• Field of view
  
• Focus
• ...
  

Orthographic and perspective cameras	Fish-eye camera	Multi-perspective camera

The 3D models

• Points / curves / surfaces / volumes, possibly animated
  

• Infinite possibilities! (how would you represent a simple square? a smooth curve?)
• Surfaces are often reduced to quadrangular and/or triangular meshes (simple/GPUs)
• This is what we will use during this course
• Example:

• Other possible representations:
• using points (Particules / splats)
  
• using boundary curves (B-rep)
  
• using operations on simple primitives (CSG)
  
• using images (billboards)
• using implicit functions
• ...
  

• Using simulated physic models (example for soft body locomotion)
  
• Using keyframing / skeletons
• Do the animations always need to be physically correct?
  

• From scratch
• by artists / designers
• using modelisation tools: blender (free), 3DSMax, Maya, ZBrush, Modo, Meshlab (free), etc

• From real data
• using 3D scanners
• using a set of images

Scanning Michelangelo	3D model from images (GRIMAGE)

The material / appearance

• What is the material of the object? (plastic / marble / glass / etc)
  
• What are its properties? (matte / glossy / mirror / transparent / translucent / etc)
• Does its color/reflectance properties change spatially?

• Micro-effects (material): usually using a function called BRDF (Bidirectional Reflectance Distribution Function)

The BRDF defines how light is reflected on a surface	Plastic material (Phong BRDF)	Complex materials (translucency). Needs more dimensions (BSSRDF)

• Meso-effects (details): mostly with textures (texture/bump/normal mapping)

Texture mapping	normal mapping

• Macro-effects (geometry): for instance with displacement maps. See a video here: ndo2

• Hybrid representations: combines geometry, textures and/or volumes to represent natural scenes, medical data, etc
  

Clouds [Bouthors et al. 2008]	Forests [Decaudin and Neyret 2004]

Lighting effects

• No Light -> No images
  
• Image synthesis consists in modelizing light behavior:
• physical properties (intensity/color/etc)
• directory (straight lines bouncing on surfaces/projections/reflections)
• interaction with surfaces (material - BRDF/shadows/textures/reflectance)

Caustic	Global illum	shadows [Vincent et al. 2009]

The renderings

• Goal: obatining a color for each pixel of the output image
• What are the observed surfaces?
• What are the associated materials/lights/textures?
• Rasterization: fills the projected vector primitives (triangles) with colors
• Raytracing: follows the inverse light trajectory

• Sampling issue (aliasing): filtering / multisampling / supersampling (for anti-aliasing)

• Storing colors:
• Type: integer/float/etc?
• format: number of colors / compression?
• HDR images: tone mapping?

Sources

• J. Baril, F. Berthaut, N. Mellado
• Gaël Guennebaud