Luminosity Masks and Sepia Toning

Part 2

In Part 1, "Tutorial: Luminosity Masks and Sepia Toning," I introduced the topic of luminosity masks and explained how they can help produce handsome sepia tones. Because of the gaudy and extreme sepia effects I've seen some people produce, my philosophy has been to give users a fairly "safe" sepia effect as a starting point. It goes without saying that by choosing a different fill color or by moving the sliders in the Hue/Saturation adjustment layer, you can finely tune or grossly alter the effect.

In Part 2, I illustrate a layer setup that will let you go from an uncorrected color image to a sepia tone using non-destructive adjustment layers. I also talk a little about grayscale conversion and the Channel Mixer. Both tutorials are based on Photoshop 5.5, though many of the concepts also apply to other versions.

In Part 1, I simplified some concepts in order to bring out the main points. One such simplification was to start with a pre-corrected grayscale image which would undergo further tweaking during the sepia toning process. Some purists might complain that editing the image twice causes needless degradation. If non-destructive editing is your priority, you can set up layers as follows to go from an uncorrected color image to a finished sepia tone:

[List is from bottom layer up. Blend modes are Normal except for Layer 5, which should be set to Color blend mode. Opacities are all 100%.]

1. Background (uncorrected color image in RGB or CYMK)

2. Levels adjustment layer.

3. Curves adjustment layer.

4. Channel Mixer (with "Monochrome" checked).

5. Image layer filled with brown: R162/G138/B101, and a layer mask based on the inverted luminosity values of the background image after correction. Layer 5 should be set to the Color blend mode.

6. Hue/Saturation adjustment layer with Colorize unchecked.

The best toned images come from grayscales which, if originally in color, were carefully prepared for conversion while still in color. Dan Margolis devotes a chapter to the art of grayscale conversion in his excellent book, "Professional Photoshop 5." He suggests that before converting to grayscale, you work with curves on individual color channels, boosting contrast between areas of similar color so that such areas will not look muddy when converted to grayscale. (If the grayscale looks muddy, the sepia tone, duotone, or other toning effect will also look muddy.)

Layers 1-4 in my setup above do not pretend to do all that Dan Margolis would. But Levels, Curves, and the Channel Mixer are sometimes enough to produce a handsome grayscale. Best of all, these three tools can all be implemented as adjustment layers, so the entire composition can be fine-tuned interactively and non-destructively.

While working with only Layers 1-3 visible, you're still working in color. Layer 4, the Channel Mixer adjustment layer, is where the grayscale conversion happens. In case the Channel Mixer is new to you, when the "Monochrome" box is checked, Channel Mixer lets you mix a custom grayscale using different percentages of the color channels. In RGB mode, setting the RGB sliders to 60/30/10 should simulate what the image would look like if converted to Grayscale mode by Photoshop. Setting the sliders to 34/33/33 should simulate the effect of using Desaturate instead of a mode change. But the beauty of the Channel Mixer is that it lets you work with each unique image according the how good the individual color channels are. For example, a video grab from footage shot in incandescent lighting on a consumer camcorder might show a noisy or blurry red channel. Photoshop's default Grayscale conversion would use 60% of the red channel, accentuating the problem. But using Channel Mixer, you might get a better result by using more of the green channel.

Before doing any correction, it's helpful to go into the channels palette and inspect each color channel individually. If none of the RGB channels looks too appealing, try converting to CYMK or Lab mode. Sherry London, one of the authors of "Photoshop 5 In Depth," is a big fan of changing to Lab mode, clicking on the Lightness channel, changing to Grayscale mode, and clicking OK when asked to discard the other channels. By using only the Lightness channel, you can sometimes salvage a good grayscale from an image with bad color noise.

But for now, we're catering to the non-destructive purist who wants an efficient "production layer" that gives good toning results with a minimum of fuss. We work with Layers 1-3 in color to correct the image. Since we're moving towards grayscale, we needn't be afraid to use steep curves which push the colors to extremes. We don't care if the color looks bad, as long as we get a good grayscale down the line.

In Layer 4, we use the Channel Mixer to produce our grayscale. Layer 5 is an image layer containing a brown color fill, with the layer blend mode set to Color. A layer mask is added, and the layer mask is filled with an inverted luminosity mask based on the background image after correction. This requires some explanation:

In the Part 1 tutorial, we worked from a pre-corrected background image. Therefore, in the image layer with the color fill, it was easy to place a copy of the inverted luminosity mask from the Background layer into the layer mask. However, in the present tutorial the Background image is assumed to be uncorrected. We probably don't want to use it for anything in its uncorrected state. What we want is to work on Layers 1-4 until we're satisfied with our grayscale. Then we should send a merged copy, inverted, of Layers 1-4 into the layer mask for Layer 5. Happily, Image: Apply Image will accommodate us, as it did in Part 1. To get this to work right, we have to click on the layer mask for Layer 5 so that it's selected. Then turn off the eye icons for all but Layers 1-4. With only Layers 1-4 visible, we go to Image: Apply Image:

Source: [the current document], Layer: Merged, Channel: RGB, Invert: checked

Target: [You cannot change the target. It should currently say Layer Mask]

Blending: Normal, Opacity 100%, Mask: unhecked. Click OK.

This fills the Layer 5 layer mask with a merged copy, inverted, of Layers 1-4, giving us our luminosity mask for the sepia effect. Layer 6 simply adds a Hue/Saturation adjustment layer which we can use to change the color of the toning effect. Since the image was already colorized in Layer 5, we do not need to check the Colorize box. Keeping Colorize unchecked will allow us to make finer adjustments and will also keep the Hue/Saturation adjustment layer from "fighting" the layer mask in Layer 5.

Some might question why we even need Layer 5. Can't we just use the Hue/Saturation adjustment layer (with "Colorize" checked and Color blend mode) to directly colorize the grayscale output from the Channel Mixer? This is possible. If you're working in low memory and need to hold down file size, you can do without Layer 5. You may find that you sacrifice some control. Do the following:

1. Starting from the six-layer version, drag Layer 5 to the trash. You should see the image in grayscale again.

2. Change the blend mode of the Hue/Saturation adjustment layer to Color.

3. Double-click on the Hue/Saturation adjustment layer to access the settings. Check the "Colorize" box. Set the HSL sliders to 37/19/+30. You should now see a sepia tone which is nearly identical to our six-layer version.

Using this 5-layer setup, we get a smaller file size and a pretty good sepia tone. What will happen if we add a layer mask to the Hue/Saturation adjustment layer? Do the following:

1. Click on the Hue/Saturation adjustment layer to select it. Turn it's eye icon off.

2. With only Layers 1-4 visible, go to Image: Apply Image. Use the same settings as before:

Source: [the current document], Layer: Merged, Channel: RGB, Invert: checked

Target: [You cannot change the target. It should currently say Layer Mask]

Blending: Normal, Opacity 100%, Mask: unhecked. Click OK.

3. Turn the eye icon back on for the Hue/Saturation adjustment layer.

We've just filled the Hue/Saturation adjustment layer's layer mask with the familiar inverted luminosity mask. This has clamped down on the sepia effect quite a bit, especially in light areas. To try and match the effect we've achieved in the past, do the following:

1. Double-click on the Hue/Saturation adjustment layer to access the settings. With "Colorize" still checked, set the HSL sliders to 37/31/+10.

2. With the Hue/Saturation adjustment layer's layer mask selected, choose Image: Adjust: Levels. Change the Output Levels from 0/255 to 34/255.

Our sepia tone should now look similar to the effect we've achieved in the past. Even though we've cut down on file size, I prefer to use the six-layer method where memory permits. With either method, once you're close to the effect you like, you can fine tune any of the adjustment layers to get you the final distance. You can also edit the layer mask to control how much color is absorbed by different tones in the image. Editing of the layer mask is destructive, but if you mess it up you can always use Image: Apply Image to send a fresh copy of the inverted luminosity mask into the layer mask.

Tutorial Copyright © 2001 Outcast125, Used by permission of author

Luminosity Masks and Sepia Toning Part 1

If you have a tutorial relevant to photo restoration, retouching, or manipulation, please email me with details and we'll either publish it here or link to it.

Discuss this tutorial here