Image color transfer

"Color mapping" redirects here. Not to be confused with False color.
Color mapping example
Source image
Reference image
Source image color mapped using histogram matching

Image color transfer is a function that maps (transforms) the colors of one (source) image to the colors of another (target) image. A color mapping may be referred to as the algorithm that results in the mapping function or the algorithm that transforms the image colors. The image modification process is sometimes called color transfer or, when grayscale images are involved, brightness transfer function (BTF); it may also be called photometric camera calibration or radiometric camera calibration.

The term image color transfer is a bit of a misnomer since most common algorithms transfer both color and shading. (Indeed, the example shown on this page predominantly transfers shading other than a small orange region within the image that is adjusted to yellow.)

Algorithms

There are two types of image color transfer algorithms: those that employ the statistics of the colors of two images, and those that rely on a given pixel correspondence between the images. In a wide-ranging review, Faridul and others [1] identify a third broad category of implementation, namely user-assisted methods.

An example of an algorithm that employs the statistical properties of the images is histogram matching. This is a classic algorithm for color transfer, but it can suffer from the problem that it is too precise so that it copies very particular color quirks from the target image, rather than the general color characteristics, giving rise to color artifacts. Newer statistic-based algorithms deal with this problem. An example of such algorithm is one that adjusts the mean and the standard deviation of each of the source image channels to match those of the corresponding reference image channels. This adjustment process is typically performed in the Lαβ or Lab color spaces.[2]

A common algorithm for computing the color mapping when the pixel correspondence is given is building the joint-histogram (see also co-occurrence matrix) of the two images and finding the mapping by using dynamic programming based on the joint-histogram values.[3]

When the pixel correspondence is not given and the image contents are different (due to different point of view), the statistics of the image corresponding regions can be used as an input to statistics-based algorithms, such as histogram matching. The corresponding regions can be found by detecting the corresponding features.[4]

Liu[5] provides a review of image color transfer methods. The review extends into considerations of video color transfer and deep learning methods including Neural style transfer.

Applications

Color transfer processing can serve two different purposes: one is calibrating the colors of two cameras for further processing using two or more sample images, the second is adjusting the colors of two images for perceptual visual compatibility.

Color calibration is an important pre-processing task in computer vision applications. Many applications simultaneously process two or more images and, therefore, need their colors to be calibrated. Examples of such applications are: Image differencing, registration, object recognition, multi-camera tracking, co-segmentation and stereo reconstruction.

A photograph of 21st century London recolored to match an 18th century painting by Canaletto.

Other applications of image color transfer have been suggested. These include the co-option of color palettes from recognised sources such as famous paintings and the use as a further alternative to color modification methods commonly found in commercial image processing applications such as ‘posterise’, ‘solarise’ and ‘gradient’.[6] A web application has been made available to explore these possibilities.

Nomenclature

The use of the terms source and target in this article reflects the usage in the seminal paper by Reinhard et al.[2] However, others such as Xiao and Ma[7] reverse that usage and indeed it seems more natural to consider that the colors from a source image are directed at a target image. Adobe use the term source for the color reference image in the Photoshop Match Color function. Because of confusion over this terminology some software has been released into the public domain with incorrect functionality. To minimise further confusion, it may be good practice henceforth to utilise terms such as input image or base image and color source image or color palette image respectively.

See also

References

  1. ^ Faridul, H. Sheikh; Pouli, T.; Chamaret, C.; Stauder, J.; Reinhard, E.; Kuzovkin, D.; Tremeau, A. (February 2016). "Colour Mapping: A Review of Recent Methods, Extensions and Applications: Colour Mapping". Computer Graphics Forum. 35 (1): 59–88. doi:10.1111/cgf.12671. S2CID 13038481. Retrieved 9 June 2023.
  2. ^ a b Color Transfer between Images
  3. ^ Inter-Camera Color Calibration using Cross-Correlation Model Function
  4. ^ Piecewise-consistent Color Mappings of Images Acquired Under Various Conditions Archived 2011-07-21 at the Wayback Machine
  5. ^ Liu, Shiguang (2022). "An Overview of Color Transfer and Style Transfer for Images and Videos". arXiv:2204.13339 [cs.CV].
  6. ^ Johnson, Terry (28 May 2022). "A Free-toUse Web App for Image Colour Transfer Processing". Medium.
  7. ^ Xioa, X; Ma, L (2006). "Color transfer in correlated color space". ACM: 305–309.