27 Wavelength encoding – Foundations of Image Systems Engineering

27.1 Wavelength encoding overview

When Thomas Young delivered his Bakerian Lectures (1801–1803), he introduced two propositions that challenged British orthodoxy: first, that light travels as a wave; and second, that human color vision is trichromatic because the retina possesses three distinct types of wavelength sensors. The first was a resurrection of Huygens’ idea that the establishment had rejected. The second was a novel insight about color perception that flowed naturally from the first.

Both ideas were met with fierce resistance. Henry Brougham’s scathing broadsides in the Edinburgh Review battered Young’s reputation and contributed to his withdrawal from public optical debate for more than a decade (the quotation and the story are in Note 8.1, in Section 8.4).

Young’s wave theory was vindicated a decade later. Fresnel, working independently in France, supplied the mathematics of diffraction and interference, and Arago’s observation of the bright spot at the center of a circular shadow firmly re-established the wave theory of light (Section 8.4.2).

The color idea took longer to settle. Maxwell’s experiments, described below, showed that the three-dimensionality of color is a property of the eye, not of light.

My enthusiasm for Maxwell as a color scientist is based on the fact that (a) he settled, once and for all, the question of the biological basis of trichromacy, and (b) he did so in a way that established a clear conceptual structure and experimental methodology that we still use today (Maxwell 1857a; Maxwell 1860). Maxwell then collaborated with Thomas Sutton to produce the first color photographs Figure 1.4, demonstrating that his understanding was true and useful (Maxwell 1861; Sutton 1861)¹, though the significance of that color demonstration is controversial (Klein and Elter 2023).

Maxwell did the color work just prior to taking on the challenges of unifying the equations for electricity and magnetism and discovering that light is also electromagnetic radiation. I am so pleased that color science can call Maxwell one of our own.

History of color science

The idea that human color vision is limited by the eye, rather than by the light, has a long history. The history is chronicled in a fascinating chapter by (Mollon 2003), and there is much to be learned from his account. One interesting point is that much practical knowledge was available and used by commercial firms prior to scientific analyses. The exchange between commerce and science is present throughout the field of image systems and far beyond.

I have made a few additional observations, building on Mollon’s work, in Chapter 50.

27.2 Maxwell’s color matching experiments

Thomas Young’s suggestion that three-dimensional color vision is of biological origin was just that, a suggestion. It was based on his understanding that light comprised waves and that the frequency of these waves was a continuous variable. He was aware that color could be reproduced using only three primary colors. To reconcile the continuous representation of light with the low dimensional representation of color, he wrote

Now, as it is almost impossible to conceive each sensitive point of the retina to contain an infinite number of particles, each capable of vibrating in perfect unison with every possible undulation, it becomes necessary to suppose the number limited for instance to the three principal colours … (Young 1802).

Even apart from the scurrilous attacks from Brougham (Note 8.1), serious scientists still held on to the idea that light itself was three-dimensional (Brewster 1834), or that color vision had more than three dimensions (Helmholtz 1852). It was James Clerk Maxwell who developed the experiments and conceptual framework that settled the matter.

The instrument that Maxwell used in his initial experiments, Figure 27.1, was an important choice. His illuminated spinning wheel created a tool in which the primaries were lights, not pigments. The light primaries are additive (?sec-superposition-lights), unlike the subtractive primaries of printing. The linearity of the primary stimulus combinations is an essential experimental choice that enabled him to characterize the matching experiment. He used this simple tool in several ways.

Figure 27.1: Maxwell’s color wheel, used to produce quantitative color matches. Paper discs of primary colors (red, green, and blue) were arranged in the outer ring around a smaller central comparison disc. When spun rapidly, the rotating sectors merged into uniform color appearances, allowing Maxwell to achieve a match by adjusting the proportions of the exposed sectors. Source: Cambridge Digital Library

First, Brewster had proposed that although light comprised a continuum of wavelengths, these were not independent of one another. Rather, he postulated that there were only three fundamental types of light (red, yellow, and blue). Each of these, he proposed, had energy distributed across the spectrum. To test this idea, Maxwell performed a simple experiment. He used his spinning color top to create metameric matches (for example, matching a combination of Vermilion, Emerald Green, and Ultramarine in the outer ring against a mixture of Chrome Yellow, Black, and White in the inner circle). He then viewed the metamers through a color filter. If the two matching lights were physically identical—or if light itself were physically three-dimensional such that visually indistinguishable lights had the same physical composition—any color filter placed in front of the eye would attenuate both lights identically, and the match would hold(Maxwell 1857b). He reported

The tints which before appeared identical will now be manifestly different, and will require alteration, to reduce them to equality. (Maxwell 1857b).

Because the colored glass broke the match, the two lights had to be physically distinct in their spectral composition. Therefore, their initial visual identity could not be a property of the light itself; it existed solely because the human eye collapses a continuous wavelength spectrum down to three physiological sensations. Young must have got it right. Or as Maxwell put it:

It seems almost a truism to say that colour is a sensation; and yet Young, by honestly recognising this elementary truth, established the first consistent theory of colour. So far as I know, Thomas Young was the first who, starting from the well-known fact that there are three primary colours, sought for the explanation of this fact, not in the nature of light, but in the constitution of man. (Maxwell 1872)

His experiments with the color wheel convinced Maxwell that color perception was trichromatic. But the color wheel was challenging to control and did not give him access to narrow wavelengths of light, which he understood as being fundamental. This led Maxwell to construct an apparatus in which he could measure the relationship between narrowband (spectral) lights, for both the primary and test lights (Maxwell 1860; Wandell et al. 2024).

Maxwell explained his motivation for these experiments quite clearly. He noted that a test light can be matched by adjusting the intensities of three fixed, primary lights mixed together. Associating any light with the three intensities of standard primaries quantifies that light’s appearance:

It appears therefore that the result of any mixture of colours, however complicated, may be defined- by its relation to a certain small number of well-known colours. Having selected our standard colours, and determined the relations of a given colour to these, we have defined that colour completely as to its appearance. Any colour which has the same relation to the standard colours, will be identical in appearance, though its optical constitution, as revealed by the prism, may be very different (Maxwell 1860).

27.3 The color-matching experiment

The basic design of the color-matching experiment is illustrated conceptually in Figure 27.2. The observer views a split (bipartite) field through an aperture. One half of the field is illuminated by a test light, \(T\). The other half is illuminated by the mixture of three independent primary lights, \(R\), \(G\), and \(B\). The observer adjusts the intensities of the three primaries until the two halves of the field are indistinguishable in visual appearance.

Figure 27.2: Conceptual description of the color matching experiment. The observer views a bipartite field with the test light \(T\) on one side and a mixture of three adjustable primary lights (\(R, G, B\)) on the other side. The first color matching instrument, built by Maxwell, used an ingenious but more complex optical design.

27.3.1 Linearity and superposition

Maxwell’s experiments confirmed trichromacy, but even more importantly, he tested and supported the principle of superposition in color matching (Maxwell 1860). While physical light mixtures are known to obey superposition, discovering that human perceptual matches also satisfy superposition—a set of empirical relationships often called Grassmann’s Laws—is the critical finding that allows us to characterize color vision using linear algebra.

As illustrated in Figure 27.3, superposition in color matching comprises two rules:

Proportionality (Homogeneity): Scaling the intensity of the test light by a factor \(a\) scales the required primary intensities by the same factor \(a\).
Additivity: If primary intensities \((R_1, G_1, B_1)\) match test light \(T_1\), and \((R_2, G_2, B_2)\) match test light \(T_2\), then the mixture of primaries \((R_1 + R_2, G_1 + G_2, B_1 + B_2)\) matches the combined test light \(T_1 + T_2\).

Figure 27.3: Linearity of the color-matching experiment: additivity and homogeneity.

27.4 Matrix formulation of color matching

Because color matching satisfies superposition, we can express the relationship between any test light and the primary intensities needed to match it using linear matrix algebra.

Suppose we represent a test light as a column vector, \(\mathbf{t}\), whose entries \(t[\lambda]\) specify the light’s spectral power at sampled wavelengths \(\lambda\). Because the system is linear, the primary intensities \((R, G, B)\) required for a match can be calculated by multiplying \(\mathbf{t}\) by a \(3 \times N\) matrix, \(\mathbf{C}\):

\[ \begin{pmatrix} R \\ G \\ B \end{pmatrix} = \mathbf{C} \mathbf{t} \]

Figure 27.4: Linearity of the color-matching experiment expressed in matrix tableau.

The matrix equation shows how two lights are related in appearance. Any particular test light \(\mathbf{t}_1\) matches the appearance of a second test light \(\mathbf{t}_2\) if and only if their primary matching intensities are identical:

\[ \mathbf{C} \mathbf{t}_1 = \mathbf{C} \mathbf{t}_2 \tag{27.1}\]

27.4.1 Dimensionality of the CMFs

How do we know that we should use three primary lights, and not two or four? The answer is based on experiments.

If we provide people with only one or two primary lights, there will be test lights that cannot be matched. If we provide four primary lights, observers can set satisfactory matches with many different combinations of the primaries, so the match is not unique. With three primaries, a match can be found for every test light (allowing primaries to be added to the test side, as described below), and the match is unique. Three is the right number.

27.4.2 Measuring the color-matching matrix \(\mathbf{C}\)

How do we determine the entries of the matrix \(\mathbf{C}\)? Linear systems theory provides a straightforward answer: measure the system’s response to impulse inputs. In optics, an impulse is a point of light, and the response is the point spread function (Section 36.4). In color matching, an impulse is a monochromatic (single-wavelength) test light.

Suppose the test light is a monochromatic light at wavelength \(\lambda_k\) with unit power. Its spectral vector \(\mathbf{t}\) contains zeros everywhere except for a value of 1 at entry \(k\). When we multiply \(\mathbf{C} \mathbf{t}\), this unit impulse simply extracts the \(k^{\text{th}}\) column of \(\mathbf{C}\) (Figure 27.5).

Figure 27.5: Measuring the color-matching matrix with monochromatic test lights. A unit-intensity spectral light selects a single column of the matrix \(\mathbf{C}\), directly revealing the primary intensities required to match that wavelength.

By measuring the primary intensities needed to match monochromatic test lights across all sampled wavelengths of the visible spectrum, we directly measure every column of \(\mathbf{C}\).

27.4.3 Color matching functions

The rows of the matrix \(\mathbf{C}\) are functions of wavelength, known as the color-matching functions (CMFs), conventionally written as \(r(\lambda), g(\lambda), b(\lambda)\).

An example of color-matching functions measured using three monochromatic primaries (at 645.2 nm, 526.3 nm, and 444.4 nm from the Stiles & Burch 10-degree observer) is shown in Figure 27.6.

Figure 27.6: Stiles & Burch (1959) 10-degree RGB color-matching functions measured with monochromatic primaries. In the cyan spectral region between 480 nm and 530 nm, the red primary curve is negative, indicating that the red primary light was added to the test light to achieve a match.

Reading the CMFS

When I teach, I often forget the wavelengths of the primary lights. But it is possible to deduce them from the graphs; can you see how? The principle is straightforward. When a monochromatic test light matches one of the primary lights, the primary settings will be one of these three values: \((1,0,0)\), \((0,1,0)\), or \((0,0,1)\). Read the curves to find the wavelengths at which the primaries have those values. You won’t be very precise, but you can get a close estimate.

27.4.4 Negative values in the CMF

A notable feature of these curves is that \(r(\lambda)\) takes on negative values between approximately 480 nm and 530 nm. In this spectral region, monochromatic test lights appear more saturated in color than any additive mixture of the green and blue primaries can produce. To achieve a visual match, the experimenter adds some of the red primary to the test light side of the bipartite field, desaturating it until the remaining green and blue primaries can complete the match. Because color matching obeys superposition, adding primary light to the test field is mathematically equivalent to subtracting it from the primary side, resulting in negative coefficients in the color-matching functions.

This point matters in practice when we describe the colors a display can produce. A display has three primary lights, and there is no way to add a primary to the test side. Colors that would require negative primary values cannot be produced. This limitation defines the display gamut (Section 29.1).

27.4.5 Uniqueness of the CMFs

Two groups, A and B, who use different sets of primary lights, \(\mathbf{P}_A\) and \(\mathbf{P}_B\), will measure different CMFs. But the two sets of CMFs are related.

To see why, suppose group A measures the matches to the primaries of group B. Group A can then predict the matches that group B would set for any test light.

Represent the primaries as \(N \times 3\) matrices, \(\mathbf{P}_A\) and \(\mathbf{P}_B\), whose columns are the spectral power distributions of the three primaries. Let \(\mathbf{C}_A\) and \(\mathbf{C}_B\) be the \(3 \times N\) CMF matrices measured by the two groups. By definition, \(\mathbf{P}_B (\mathbf{C}_B \mathbf{t})\) is a mixture of group B’s primaries that matches \(\mathbf{t}\). Because matched lights produce the same group A matching values, applying \(\mathbf{C}_A\) gives

\[ \mathbf{C}_A \mathbf{t} = \mathbf{C}_A \mathbf{P}_B (\mathbf{C}_B \mathbf{t}). \]

The \(3 \times 3\) matrix \(\mathbf{M}_{B,A} = \mathbf{C}_A \mathbf{P}_B\) converts group B primary values into group A primary values. Because this holds for every test light \(\mathbf{t}\),

\[ \mathbf{C}_A = \mathbf{M}_{B,A} \, \mathbf{C}_B . \tag{27.2}\]

Any two sets of CMFs are related by a \(3 \times 3\) linear transformation.

The converse also holds. Multiply both sides of Equation 27.1 by any invertible \(3 \times 3\) matrix, \(\mathbf{A}\). The predicted matches are unchanged:

\[ \mathbf{A} \mathbf{C} \mathbf{t}_1 = \mathbf{A} \mathbf{C} \mathbf{t}_2 \quad \Longleftrightarrow \quad \mathbf{C} \mathbf{t}_1 = \mathbf{C} \mathbf{t}_2 . \tag{27.3}\]

Thus the CMFs are unique only up to an invertible linear transformation. We do not need to use the measured CMFs directly. We can use any linear transformation of them that is more convenient.

27.5 CIE Standards

This freedom is what the standards bodies exploit. The Commission Internationale de l’Éclairage (CIE) defined a standard observer in 1931, based on color matches measured for small (2-degree) fields. The CIE derived the CIE 1931 \(\bar{x}(\lambda), \bar{y}(\lambda), \bar{z}(\lambda)\) functions by applying a linear transformation to measured RGB CMFs. The transformation was chosen so that the functions are non-negative, and so that \(\bar{y}(\lambda)\) equals the photopic luminosity function. A second standard was defined in 1964 for larger (10-degree) fields; the Stiles & Burch data in Figure 27.6 are from that field size.

Color matching, biology, and psychology

27.5.1 Baylor and cone response functions

Just point to Foundations of Vision. I don’t want to repeat the story in both places, cool as it is.

27.5.2 Maxwell the Psychologist

Maxwell recognized that it is possible to create an exact quantitative science based on perceptual observations, and that this science is an exploration of our conscious experience:

If the sensation which we call colour has any laws, it must be in our own nature which determines the form of these laws; and I need not tell you that the only evidence we can obtain respecting ourselves is derived from consciousness.

The science of colour must therefore be regarded as essentially a mental science. It differs from the greater part of what is called mental science in the large use which it makes of the physical sciences, and in particular of optics and anatomy. But it gives evidence that it is a mental science by the numerous illustrations which it furnishes of various operations of the mind (Maxwell 1872).

Something about the simplicity of matching compared to appearance. We have one worked out, not the other. More on this in Foundations of Vision.

27.6 Color differences

Transition comments here into the metrics chapter.