A Survey of Specularity Removal Methods

Abstract

In the real world, materials often show both diffuse and specular reflections. To separate these two reflection components may help applications which need consistent object surface appearance. In fact, many algorithms used in numerous tasks of computer vision, computer graphics and image processing, work under the assumption of perfect Lambertian surfaces (or perfect diffuse reflection). They consider specular pixels (or highlights) as outliers or noise. Moreover, the two reflection components can be processed separately and afterwards recombined to produce particular visual effects [MZK*06] (dichromatic editing Figure.1).

Figure 1: Example of different visual effects produced by dichromatic editing [MZK*06]: (a) is the input image, (b) wetness effect, (c) skin colour change and (d) effect of make-up. Input image courtesy of S.P. Mallick.

Acquired textures with the presence of highlights are a classical example of how details may be lost. In this case, the simulation of the changes of the light source position will be affected by the highlights and shadows generated by the light source used during the acquisition process. Figure.2 shows an image where details and colours are completely washed out in the highlights region. To recompose the details and the colour information, highlights removal techniques are required. These techniques, starting from one or more input images, extract two intrinsic properties; the diffuse and the specular images. This is a classical ill-posed problem [Wei01], because the number of unknown variables is larger than the number of equations. In fact, we have only one equation that defines the total radiance as the sum of different terms.

Figure 2: Example of an image where the presence of highlights generates the loss of details and colour information. Details and colours are completely washed out in the highlights region.

Several specularity removal techniques are available in literature; they differ in the information they use and in how it is used. Table 1 summarises how these techniques may be classified and in which paragraph of this survey they are explained. The notation used in Table 1 for the categories is the following: CSA, colour space analysis; NA, neighbourhood analysis; POL, polarization; IS, image sequences; MFI, multiple-flash images. The notation used in Table 1 for the techniques is the following: CRM, using colour reflection model [KSK87, KSK88]; TDA, 2D diagram approach [ST95b, SK00]; BM, Bajcsy et al. method [BLL96]; USFI, use of specular-free images [TI05b, YCK06, SC09]; PDE, PDE approach [MZK*06]; IT, in-panting technique [TLQ06]; CC, use of colour information and classifier [TFA05]; TS, separation of highlight reflections on textured surfaces [TLQ06]; FR, Fresnel methods [Ang07]; HM, histogram methods [CGS06]; HFLF, high-low frequency separation [LPD07, NGR06]; MBS, multi-baseline-stereo [LB92, LYK*03, LS01]; MII, deriving intrinsic images from image sequences with illumination changes [Wei01]; CP, colour and polarisation methods [NFB97, KLHS02, MPC*07, USGG04]; MF, multi-flash methods [FRTT04, ARNL05].

Table 1: The classification of highlights removal methods. These methods are classified based on the number of image used as input an the type of information used. L and G respectively stand for global and local approach. The number, in the first column, indicates the paragraph in the paper where the technique appears in the survey.

		Single Image		Multiple Images
Tech. - Sec.	Type	CSA	NA	IS	MFI	POL
CRM - 3.1.1	G	x	-	-	-	-
TDA - 3.1.2	L	x	-	-	-	-
BM - 3.1.3	G	x	-	-	-	-
USFI - 3.2.1	L	-	x	-	-	-
PDE - 3.2.2	L	-	x	-	-	-
IT - 3.2.3	L	-	x	-	-	-
TS - 3.2.4	L	-	x	-	-	-
CC - 3.2.5	L	-	x	-	-	-
FR - 3.2.6	G	-	x	-	-	-
HM - 4	G	-	-	x	-	-
HFLF - 4	G	-	-	x	-	-
MBS - 4.1	G	-	-	x	-	-
MII - 4.2	L	-	-	x	-	-
CP - 4.3	G	-	-	-	-	x
MF - 4.4	L	-	-	-	x	-

Discussion

Several factors may influence which approach is superior to another one, such as the number of images to be captured, automatic operation vs. manual help, light constraints and the reflectance model used, merits of quality, and the hardware used during the acquisition phase. Table 2 provides a comparative summary of the specular removal methods discussed in this survey.

Table 2: Comparison of the highlight removal techniques by their characteristics. The methods are grouped into the single-image category (top) and the multiple-image category (bottom). User Interaction: MS manual segmentation, A automatic, NS no segmentation, Light Requirement: IC illuminant compensation, DM dichromatic reflectance model, and FM flash model. Hardware: S single camera, M multiple camera, * polarised filters, FS flash system

Technique	Images	User Interaction	Light Requirement	Hardware
Colour Space [KSK87, KSK88, ST95b, SK00, BLL96]	1	MS	IC-DM	S
Specular-Free Image [TI05b, YCK06, SC09]	1	A	IC-DM	S
Inpainting [TLQ06]	1	MS	IC-DM	S
PDE [MZK*06]	1	-	IC-DM	S
Textured Surfaces [TLQ06]	1	MS	IC-DM	S
Colour Classifier [TFA05]	1	-	-	S
Fresnel Coefficient [Ang07]	1	MS	No IC-DM	S
Multi-Baseline Stereo [LB92, LYK*03, LS01]	50-70	NS	DM	M
Deriving Intrinsic Images [Wei01]	40-70	NS	Lighting changes	S
Polarization [NFB97, KLHS02, MPC*07, USGG04]	6-10	NS	DM	S*
Histogram [CGS06]	200	NS	-	S
High-Low Frequency Separation [LPD07, NGR06]	4, 32	NS	-	S
Multi-flash [FRTT04, ARNL05]	4-8,2	NS	FM	FS

Finally we have performed several experiment, to compare different specularity removal techniques, on various input images starting from single-colour, multi-colour and textured surfaces, increasing the texture complexity.

References

For the references please see the paper.

@Article\{ABC11,
  author       = "Artusi, Alessandro and Banterle, Francesco and Chetverikov, Dmitry ",
  title        = "A Survey of Specularity Removal Methods",
  journal      = "Computer Graphics Forum",
  number       = "8",
  volume       = "30",
  pages        = "2208-\^a€“2230",
  month        = "December",
  year         = "2011",
  keywords     = "specularity removal, dichromatic reflection model, colour shading, shadowing, diffuse and specular reflections, highlights",
  url          = "http://vcg.isti.cnr.it/Publications/2011/ABC11"
}