Theory of Deep Samples

This document derives the techniques for splitting and combining two non-solid samples of equal depth and thickness. These should should be used by deep image “flattening” algorithms to compute the combined colour of two samples. The formulas are defined in the document Interpreting OpenEXR Deep Pixels. This document derives those formulas, and is for information only.

Definitions

symbol

descripion

\(r\)

ratio of length of original sample to a subsample: \(0\le r \le 1\)

\(\mathbf{C_i}\)

Colour of sample \(i\)

\(\mathbf{C'_i}\)

Colour of a subsample of \(i\)

\(\alpha_i\)

alpha value of sample \(i\)

\(\alpha'_i\)

alpha value of a subsample of \(i\)

\(T_i\)

transparency/transmission of sample \(i\); \(T_a=1-\alpha_a\)

\(\mathbf{c_i}\)

colour per unit length (‘instantaneous colour’) of \(i\)

\(t_i\)

transparency per unit length (‘intanstaneous transparency’) of \(i\)

Subscripts \(_a\) and \(_b\) refer to the input samples; subscript \(_c\) refers to the combined output sample

Sample Model

Many deep compositing operations require subdividing a sample into two subsamples, or else clipping a sample at a given point. When this happens, the alpha value of the subsample(s) must be recomputed, and the colour will also change. To derive an equation for computing the new alpha, the amount of light attenuated by each subsample must be considered:

Light passing through multiple objects from left to right. Each object attenuates a fraction t of the light

Light passing through multiple objects from left to right. Each object attenuates a fraction t of the light

In this case, light passes through two objects, from left to right. Each objects attenuates the light. Light passing through any light absorbing object will by attenuated according to the transparency \(t\) of that object. Thus, if \(i\) is the amount of light entering the object, the light \(o\) leaving the object is given by

(1)\[o=t\cdot i\]

Where light passes through two objects, the light is attenuated twice (the output from the first is fed into second)

(2)\[o=t_2\cdot(t_1\cdot i)\]

where \(t_2\) and \(t_1\) are the transmission of the individual samples. In general, if the transmission \(t\) of all objects are the same, the light emerging from \(n\) adjacent samples is given by the Beer-Lambert law:

\[o=i\cdot t^n\]
(3)\[T=\frac{o}{i}=t^n\]

where \(T\) is the total transmission. If \(t\) is considered the transmission per unit length and \(n\) the length, the equation is continuous, applying also for non-integer values of \(n\) [1].

An OpenEXR deep sample can then be modelled as an object of a specified length \(n\) which absorbs light according to equation Beer’s law (3)

Alpha vs Transparency

OpenEXR images use alpha \(\alpha\) instead of transparency. \(T=1-\alpha\): if \(\alpha=1\) (so \(T=0\)) then all light is absorbed; if \(\alpha=0\) (so \(T=1\)), then all light is transmitted and the material is transparent. Substituting \(T\) into (1) and (2) combined alpha of two objects is given by the screen compositing operation:

(4)\[\alpha_c = 1-(1-\alpha_1)(1-\alpha_2)\]

where \(\alpha_c\) is the combined alpha value, and \(\alpha_1\) and \(\alpha_2\) are the alpha values of the two samples.

Throughout this document, \(T\) is preferred to \(\alpha\) where it gives simpler equations.

Computing \(t\)

OpenEXR images samples store the \(\alpha\) of the entire sample. This gives us the total transmission of the sample, not the transmission per unit length \(t\). The total transmission \(T\) and the sample length \(n\) can be used to compute the transmission per unit length:

(5)\[t=T^\frac{1}{n}\]

Sample Properties

The following are assumptions made by the formulas derived here:

  • Samples have constant optical density and colour: if a sample has length \(n\) and a subsample of length \(n'\) is extracted from it, the RGBA colour of the subsample will be the same regardless from where in the sample it is extracted. In particular, if a sample is split into \(k\) subsamples of equal length, each subsample will have the same RGBA colour.

  • Sample attenuation is non-scattering and pixel independent: light travelling through the sample is either absorbed or transmitted; it is not reflected back down the sample or scattered into neighbouring samples. Scattering causes point lights to appear blurred when passing through fog, and also tends to make fog look more optically dense than it really is, since detail is lost very quickly, even though light is being transmitted. This effect is not modelled with OpenEXR volumetric samples, and must be approximated by including the scattered light within the volume or applied as a post-process. Since the light attenuation profile throughout a pixel will not follow the Beer-Lambert equation (3), extra samples must be used to model the pixel.

  • Sample behaviour is unit and scale independent: if a sample is divided into \(k\) subsamples of equal length, then scaling the depth channels \(z_\textsf{front}\) and \(z_\textsf{back}\) of the deep image will not change the colour of the samples. When merging two deep images, prescaling the depth of each image by the same amount then merging the images is identical to merging the original images then scaling the depth of the result. This scaling property allows any unit to be used to store depth, and the unit to be changed by scaling the depth channels without modification of the RGBA channels.

  • Sample behaviour is position independent: moving a sample in depth will not change the RGBA values obtained by subdividing it. If a sample is \(10\) units long and a subsample of \(2\) units extracted from it, the subsample will have the same RGBA values regardless of the position in depth of the sample. Shifting two images by adding a constant \(c\) to the depth channels of each image then merging them is identical to merging the original images then shifting the result.

  • Sample splitting is non-destructive: using the equations derived here, the samples of a pixel can be arbitrarily subdivided. Recombining the subsamples with the over compositing operation will yield the RGBA colour of the original sample (assuming sufficient numerical precision).

Sample Splitting

Alpha of a Subsample

When a subsample of length \(n'\) is extracted from an original sample of length \(n\), the alpha value \(\alpha'\) for subsample must be computed.

The total transmission \(T'=t^{n'}\). Substituting for \(t\) computed for the entire sample length using (5) gives:

\[\begin{split}\begin{aligned} T'&=\left(T^\frac{1}{n}\right)^{n'} \nonumber\\ T'&=T^\frac{n'}{n}\end{aligned}\end{split}\]

Here, \(n\) refers to the length of the original sample, but samples are specified in OpenEXR with a front and back depth. Thus, if a sample \(T\) with front and back depths \(z_\textsf{front}\) and \(z_\textsf{back}\) is split at point \(z\) in space, the transmission \(T'\) of the front subsample is given by

\[\begin{split}\begin{aligned} T'&=T^\frac{z-z_\textrm {front}}{z_\textrm {back} - z_\textrm {front}}\\ \alpha'&=1-\left(1-\alpha\right)^\frac{z-z_\textrm{front}}{z_\textrm {back} - z_\textrm{front}}\end{aligned}\end{split}\]

Since \(({z-z_\textrm{front}})/({z_\textsf{back} - z_\textsf{front}})\) is the ratio of the original sample length to the extracted sample length, we can substitute this ratio for \(r\) for the remainder. This is equivalent to considering subdivision of a sample of unit length.

Colour of a Subsample

A sample with RGB information is modelled as a cylinder. The alpha channel(s) of the sample are represented by some non-scattering, absorbent material within it, which attenuates the light passing through the sample. The colour channel(s) of the sample is represented by infinitely many light sources, which emit coloured light (but don’t absorb anything), with identical intensities \(\mathbf{c}\), evenly spaced through the sample:

Model for a sample with colour and alpha.

Model for a sample with colour and alpha. The grey box is light absorbent material with transparency \(t\); the red dots are light sources each of which has colour \(\mathbf{c}\).

A light source at distance \(x\) into the sample will be attenuated by the amount of absorber between it and the front of the sample. According to Beer-Lambert, the total transmission of that part of the sample \(t\) is given by

\[\begin{split}\begin{aligned} t'&=T^x\\ \alpha'&=1-(1-\alpha)^x\end{aligned}\end{split}\]

The light which reaches the front of the sample from the individual light source at distance \(x\) is \(\mathbf{c} T^{x}\). If there were a finite number \(N\) sources, the total light reaching the front of the sample would be:

\[\mathbf{C} = \sum_{k=0}^{N}\mathbf{c}{T}^{k/N}\]

where \(k/N\) is the position of light \(k\), equal to \(x\). As \(N\) tends to infinity, this becomes

(6)\[\begin{split}\begin{aligned} \mathbf{C} &= \int_0^1\mathbf{c}{T}^x {\mathop{}\!\textrm{d}}x\\ &=\left.\frac{\mathbf {c}{T}^x}{\log(T)}\right|_{x=0}^{x=1} \nonumber\\ &=\frac{\mathbf{c}(T-1)}{\log(T)}\end{aligned}\end{split}\]

The value \(\mathbf{C}\) is the sample’s RGB value, as stored in the OpenEXR image. Rearranging gives us the colour of each light (or perhaps the ‘instantaneous colour’ of the sample)

(7)\[\mathbf{c} = \frac{\mathbf{C}\log(T)}{T-1}\]

If the entire sample is reduced to a subsample length \(r\), only a subsection of lights are included, and we must compute the new RGB colour \(\mathbf{C'}\) for the subsample. Hence:

\[\begin{split}\begin{aligned} \mathbf{C'} &= \int_0^{r}\mathbf{c}{T}^x {\mathop{}\!\textrm{d}}x \\ &=\frac{\mathbf{c}(T^{r}-1)}{\log(T)} \end{aligned}\end{split}\]

Substituting for \(\mathbf{c}\) from (7) gives

\[\begin{split}\begin{aligned} \mathbf{C'}&=\frac{\mathbf{C}\log(T)}{T-1}\frac{(T^{r}-1)}{\log(T)} \\ &=\mathbf{C}\frac{(T^{r}-1)}{T-1}\end{aligned}\end{split}\]

Noting that \(T-1=-\alpha\) and \(T^{r}-1=T'-1=-\alpha'\) and multiplying top and bottom by \(-1\) gives:

(8)\[\mathbf{C'}=\mathbf{C}\frac{\alpha'}{\alpha}\]

This is exactly equivalent to unpremultiplying the colour by the original alpha value, computing the new alpha value, and premultiplying by the new value.

Splitting Transparent Samples

When \(\alpha=0\) (or alternatively, \(T=1\)), unpremultiplying in (8) would require a division by zero. A special case is derived for \(\alpha=0\). The sample model of the colour provided by many light sources still applies, but there is now no attenuation. Therefore, each light contributes equally to the observed colour \(\mathbf{C}\). The ‘instantaneous’ colour equals the final colour, (assuming sample length is normalised to 1)

(9)\[\begin{split}\begin{aligned} \mathbf{C}&=\int_0^1 \mathbf{c} {\mathop{}\!\textrm{d}}x\\ \mathbf{C}&=\mathbf{c}\end{aligned}\end{split}\]

A subsection \(\mathbf{C'}\) of length \(r\) is given by

\[\begin{split}\begin{aligned} \mathbf{C'}&=\int_0^r \mathbf{c} {\mathop{}\!\textrm{d}}x \nonumber \\ \mathbf{C'}&=r\mathbf{C}\end{aligned}\end{split}\]

That is, if \(\alpha=0\), then scaling a sample’s length by \(r\) scales the colour by the same amount.

Sample Merging

Now, consider combining two samples \(a\) and \(b\) together. This operation is required when “tidying” a deep sample list, which is essential before flattening a deep image into a regular one. We assume that \(z_{\textsf{front}_a}=z_{\textsf{front}_b}\) and \(z_{\textsf{back}_a}=z_{\textsf{back}_b}\) so the lengths of each sample are the same, and they fully overlap in space. If this is not the case, the samples should be subdivided and those subsamples merged separately as described in the document “Interpreting OpenEXR Deep Samples”

The same sample model using discrete light sources is used. However, at each location, there are now two light sources, \(\mathbf{c_a}\) and \(\mathbf{c_b}\), and the light is attenuated by both \(t_a\) and \(t_b\). We can treat this as being attenuated first by \(t_b\), then by \(t_a\), so the total light reaching the end of the sample is:

\[\mathbf{c_c}=(\mathbf{c_a}+\mathbf{c_b})(T_{a}T_b)^{x}\]

By substitution into (6) and then substituting for \(\mathbf{c_a}\) and \(\mathbf{c_b}\) from (7) gives

\[\begin{split}\begin{aligned} \mathbf{C_c}&=\frac{(\mathbf{c_a}+\mathbf{c_b})(T_aT_b-1)}{\log(T_aT_b)}\\ \mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(T_a)}{T_a-1} + \frac{\mathbf{C_b}\log(T_b)}{T_b-1}\right)\left(T_aT_b-1\right)}{\log(T_aT_b)}\\ \mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{-\alpha_a} + \frac{\mathbf{C_b}\log(1-\alpha_b)}{-\alpha_b}\right)\left((1-\alpha_a)(1-\alpha_b)-1\right)}{ \log\left((1-\alpha_a)(1-\alpha_b)\right)}\\\end{aligned}\end{split}\]

Multiplying both parts of the top line by -1 gives

(10)\[\mathbf{C_c}=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} + \frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)\left(1-(1-\alpha_a)(1-\alpha_b)\right)}{ \log\left((1-\alpha_a)(1-\alpha_b)\right)}\]

Note that the combined transmission is given by \(T_c=T_{a}T_{b}\), implying that the combined alpha follows the screen equation (6). Substituting into both top and bottom of (10) gives the formula for final colour:

(11)\[\mathbf{C_c}=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} + \frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)\alpha_c}{\log(1-\alpha_c)}\]

This is the premultiplied combined colour. For the unpremultiplied colour, the \(\alpha_c\) term can be omitted. This gives the transmission weighted average of the unpremultiplied input colours.

Merging when One Sample is Transparent

Now suppose one of the samples (say, \(b\)) has no alpha, so \(\alpha_b=0\) and \(T_b=1\). From (9) \(\mathbf{c_b}=\mathbf{C_b}\). The combined colour \(\mathbf{C_c}\) is attenuated according to \(T_a\), the transmission of \(a\) alone, and the combined colour is given by:

\[\mathbf{c_c} = (\mathbf{c_a}+\mathbf{C_b})(T_a)^{x}\]

and substituting into (6) gives:

\[\begin{split}\begin{aligned} \mathbf{C_c}&= \frac{(\mathbf{c_a}+\mathbf{C_b})(T_a-1)}{\log(T_a)}\nonumber\\ &=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{-\alpha_a} + \mathbf{C_b}\right)\left((1-\alpha_a)-1\right)}{\log(1-\alpha_a)} \nonumber\\ \mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} - \mathbf{C_b}\right)\alpha_a}{\log(1-\alpha_a)}\end{aligned}\end{split}\]

Merging Two Transparent Samples

Where both samples are transparent, the colours simply add together:

\[\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\]

Merging Solid Samples

For numerical stability, we must give a sensible value for \(\mathbf{C_c}\) when \(\alpha_b=1\). Equation (11) is undefined, since \(\log(1-\alpha_b)=\log(0)=-\infty\). We follow the initial model of light sources in an absorbing material, but now the absorbing material absorbs all light. Thus, we must be observing only its closest light source (that at \(x=0\)) in the solid sample: the sample will absorb light from its other sources. Thus, we can treat \(b\) as an infinitely thin ‘discrete’ sample at the front of the sample, which will block all light behind it; we can simply composite \(b\) over \(a\). Therefore, if \(\alpha_b=1\) and \(\alpha_a<1\), the total observed colour will be \(\mathbf{C_b}\). Transposing \(a\) and \(b\) gives \(\mathbf{C_c}=\mathbf{C_a}\) if \(\alpha_a=1\)

Following the definitions above, when \(\alpha_a=1\) and \(\alpha_b=1\), then \(\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\). However, for stability, it makes sense to define

\[\mathbf{C_c}=\frac{\mathbf{C_a}+\mathbf{C_b}}{2}\]

This approach gives more stable results when there is random sampling error in the depth channel. Assume that the front depth of sample \(A\) differs by some small amount from that of \(B\), so that \(z_a=\delta+z_b\). Only the front sample will be visible: if \(\delta<0\), then \(\mathbf{C_c}=\mathbf{C_a}\); if \(\delta>0\), then \(\mathbf{C_c}=\mathbf{C_b}\). Setting \(\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\) when \(\delta=0\) gives an image which is (potentially) twice as bright. If \(\delta\) is random noise due to sampling, we will get a random pattern of bright pixels whenever \(\delta=0\). Using the mean colour seems a safer option. However, this is not associative: when combining three solid samples \(d\),\(e\) and \(f\) does not give the same result as combining first \(d\) and \(e\), then combining the result with \(f\).

Summary

To clip a sample covering range \(z_\textsf{front}\)\(z_\textsf{back}\) into a subsample \(z_\textsf{front}\)\(z\), or extract a subsample of length \(l'\) from a sample of length \(l\):

\[\begin{split}\begin{aligned} r&=\frac{z-z_\textsf{front}}{z_\textsf{back}-z_\textsf{front}} = \frac{l'}{l}\\ \alpha' &= 1-(1-\alpha)^r\\ \mathbf{C'} &= \begin{cases} r\mathbf{C},& \text{ if } \alpha=0\\ \mathbf{C}\frac{\alpha'}{\alpha},& \text{ otherwise} \end{cases}\end{aligned}\end{split}\]

where \(\alpha\) and \(\alpha'\) are the alpha values of the original sample and the subsample respectively, and \(\mathbf{C}\) and \(\mathbf{C'}\) are the colours of the original sample and the subsample respectively

To merge two samples \(a\) and \(b\) into a combined sample \(c\):

\[\begin{split}\begin{aligned} \alpha_c &= 1-(1-\alpha_a)(1-\alpha_b)\\ \mathbf{C_c} &= \begin{cases} \mathbf{C_a} + \mathbf{C_b}, & \text{if } \alpha_a=0 \text { and } \alpha_b=0\\ \mathbf{C_a},& \text{if } \alpha_a=1 \text{ and } \alpha_b <1 \\ \mathbf{C_b},& \text{if } \alpha_b=1 \text{ and } \alpha_a <1 \\ (\mathbf{C_a} + \mathbf{C_b})/2, & \text{if } \alpha_a=1 \text { and } \alpha=1\\ \alpha_a\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} - \mathbf{C_b}\right)/\log(1-\alpha_a),& \text {if } 0<\alpha_a<1 \text { and } \alpha_b=0 \\ \alpha_b\left(\frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b} - \mathbf{C_a}\right)/\log(1-\alpha_b),& \text {if } 0<\alpha_b<1 \text { and } \alpha_a=0 \\ \alpha_c\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} + \frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)/\log(1-\alpha_c),& \text{otherwise} \end{cases}\end{aligned}\end{split}\]