Relight: relightable images (RTI) (C++ and Javascript)

RTI basis supported by relight: PTM, HSH, RBF, YCC and BILINEAR (see Web3D paper for details).
Relight fitter c++ program to turn a set of photos and light directions into an RTI.
Relight.js an OpenGL Javascript library for RTI rendering on the web.
Relight Viewer.js a multiresolution viewer based on relight.js with toolbar, interface etc.
Relight web format a PCA based format

Gallery

Bottom of a bottle
Courtesy of Musee d'Histoire de Marseille

PTM, HSH, RBF comparison
Courtesy of Palazzo Blu

HSH, RBF, YCC comparison
Courtesy of Palazzo Blu

All comparisons

Download

Latest release for Linux, Windows, MacOS is available here: https://github.com/cnr-isti-vclab/relight/releases

Source code is available at: https://github.com/cnr-isti-vclab/relight

RTI supported formats

PTM: Polynomial Texture Mapping

18 planes, or 9 using the LPTM where only the luminance change

HSH: Hemispherical Harmonics

27 planes, (9 per channel)

DMD: Discrete modal decomposition

Described in Discrete Modal Decomposition for surface appearance modelling and rendering, it is not supported.

PCA + RBF

Variable number of planes using a sampled basis and radial basis function interpolation.

Named as RBF<nplanes> such as RBF18.

PCA + RBF + YCC

Same algorithm as before, but using the Jpeg color space separately for each channel, resulting in a better (adjustable) chroma fidelity. YCC<nluma><nchroma><nchroma> such as YCC1044 for 10 luminance planes and 4+4 for the chroma.

BILINEAR

Bilinear interpolation replace radial basis function in the previous algorithms. Especially good for uneven sampling of the light directions.

Relight fitter

relight is a Qt command-line program to process a stack of photos into an RTI.

	Usage: relight [-frqpsScCey] <input folder> [output folder]

	input folder containing a .lp with number of photos and light directions
	optional output folder (default is ./)

	-b : rbf(default), ptm, lptm, hsh, yrbf, bilinear
	-p   : number of planes (default: 9)
	-y   : number of Y planes in YCC
	-r   : side of the basis bilinear grid (default 8)
	-q   : jpeg quality (default: 90)
	-s   : sampling rate for pca (default 4)
	-S   : sigma in rgf gaussian interpolation default 0.125 (~100 img)
	-C        : apply chroma subsampling 
	-e        : evaluate reconstruction error (default: false)

relight can also be used to convert .ptm files into relight format:

	relight [-q] <file.ptm> [output folder]
	-q   : jpeg quality (default: 90)

libvips can be used to [generate deepzoom, zoomify and google pyramidal formats](http://libvips.blogspot.com/2013/03/making-deepzoom-zoomify-and-google-maps.html), scripts can be found in the directory test.

relight.js

relight.js is a small Javascript library to render the RTI on a WebGL canvas. It support a variety of tiled formats.

To inizialize the library:

var relight = new Relight(canvas, options);

Canvas is either a css selector or the canvas object

Options is an object with the following keys:

url: path to a directory containing the web format
layout: deepzoom, zoomify, google, iip, iiif or image for the web format
path: used for iip layout, absolute path of the dir.
light: initial light, array x, y, z.
pos: initial view object { x: y: z:}
border: for tiled formats amount of prefetching tiles arount the view (default 1)
fit: scale and center the view on load

Members:

pos: the position of the view as {x: y: z: t} where:
x and y are the coords of the center of the screen in full scale image.
z is the zoom level with 0 being the original image.
t is for interpolation in ms
light: array of x, y, z light direction
onload: function to be called when rendering is ready

Methods:

setUrl(url): change url
resize(width, height): resize canvas
zoom(amount, dt):
pan(dx, dy, dt): change the center of the view
setPosition(x, y, z, dt):
center(dt): center view (but does not change zoom
centerAndScale(dt): fit the view to the canvas
setLight(x, y, z): change light direction
draw(time): draw the canvas, use time for interpolation
redraw(): schedule an animaterequest

relight-viewer.js

It's an RTI (and images) viewer based on the relight.js library, and requires a .css file and a set of icons in the skin directory.

<!doctype html>
<html>
  <head>
    <link  rel="stylesheet" href="relight.css">
    <style>
      html, body, .relight { width:100%; height:100%; padding:0px; margin:0px; }
    </style>
    <script src="js/hammer.min.js"></script>
    <script src="relight-viewer.min.js"></script>
  </head>
<body>
  <div class="relight"></div>
  <script>
var Relight = new RelightViewer('.relight', { url: "rti-folder", layout: "deepzoom" });
  </script>
</body>
</html>

It takes the relight.js options and additionally:

pandelay: in ms the inertia of the movement (default 50)
zoomdelay: in ms the inertia of the zooming process (default 200)
zoomstep: amount of zoom for buttons and wheel (default: 0.25, meaning to 4 clicks to double the size)
lightsize: percentage of the viewer area used for light direction change (default 0.8)
scale: the size in mm of a pixel, to show the scale of the object (default 0, meaning disabled)

Relight format

Relight format is a directory containing an info.json file and a few images.

The JSON structure contains the following fields:

width:
height:
format: image extension: .jpg, .png etc.
type: one of lptm, hsh, rbf, bilinear ycc
nplanes: numer of coefficient planes
resolution: for bilinear types the side of the bilinear grid
colorspace: one of lrgb, rgb, ycc, mrgb, mycc
sigma: rbf interpolation parameter
lights: mandatory for rbf interpolation
materials: an array of materials, each material specify scale, bias and range
scale: an array of floats
bias: an array of floats
range: an array of floats
basis: an array of unsigned chars containing the basis for rbf, bilinear and ycc basis.

Each image contains 3 coefficient planes

PTM: r, g, b, 1, u, v, u^2, uv, v^2
HSH: see source code :)
RBF: PCA basis coefficients
BILINEAR: PCA basis coefficients
YCC: PCA basis coefficients as Y₀C₀C₀, Y₁C₁C₁... Y_kC_kC_k, Y_k+1Y_k+2Y_k+3

Scale and bias will be applied to the texture coefficients (from [0, 1]) as: (c - bias)*scale