Complex hair selection

[bart_hickman]

Originally Posted by Ken45140

Bart: You are the master! Do you mind showing me the settings on your Channel Mixer dialog boxes or just write them in text.
Ken

You're too generous!

This extraction has to be done in sections because the background/foreground relationship isn't uniform. The image is also noisy so it's a good idea to do some NR prior to starting. I did three major sections (left, top, right), plus a little bit of cleanup between sections.

Starting with the top first, do the following steps:

-A quadruple screening of the image to brighten up the hair relative to the background wall--you basically did this already.

-Make a small selection (~5x5 pixels) on her forehead--doesn't matter where. (This initial selection is due to a peculiarity in the select color range tool--it needs a seed selection to begin. I've already complained to Corel about it.)

-Make sure the screened image is the active layer and zoom in to about 200-300%. Selections->modify->select color range. Now move your eyedropper (1x1 pixel sampling) carefully over the center of a bright strand of hair and sample it. This becomes the comparison color and the select-color-range tool will select colors similar to this sample. Adjust the tolerance and softness to reasonably capture the hair strands you want while only getting a small amount of background. The 1st attachment shows how the preview window could look after tweaking. Don't copy my numbers exactly--they'll vary a bit depending on exactly where you sampled. Since you can see the selection in real time, simply adjust it to look good--I err towards capturing some background in order to get more foreground. This is because the defringing steps will remove the excess background anyway. Click okay. You should now see the "walking-ants" selection marquee.

-Right mouse click over the active layer and choose "new adjustment layer->curves". This pops up a curve tool. Make a curve roughly like I show in the 2nd attachment. This boosts the brightness on our selection from the previous step as you can see in the before/after panes. Click okay. You'll notice in the layer palette that the curve adjust layer you created has the alpha channel from your selection applied to it (see 3rd attachment.) The brightness boost applies more strongly to the brighter parts of the alpha channel as you see in the 2nd attachment.

-Invert the selection.

-Create another curve adjustment layer--this one will have the inverse alpha channel of the first one. Make a curve like the one in the 4th attachment. This darkens the background. Play with the curve a bit to see what it's doing--sometimes a linear curve isn't the best choice--I just used it here for simplicity. For a linear curve like this, you could have used the levels adjustment layer. (I'm an engineer by profession, so the curve is much more intuitive to me .) Click okay.

What we have done is create a mask for roughly the top third of her hair. The sides need a separate procedure including some hand-brushing the create some hair on the left side because there's no usable contrast. Notice I didn't use the channel mixer--that's because the foreground/background was already well-distinguished by luminance. This will not be the case for the sides. I'll do the sides in coming posts as well as how to combine the three sections.

Bart

[bart_hickman]

Now for the right side. It has less distinction in luminance, so we need to mix channels to help this. In this region, the background is strong in R & G and weak in B. The foreground (hair strands) is slightly strong in R, very strong in G, and medium in B. So compare the relative strengths and weaknesses between the two.

Rf < Rb
Gf > Gb
Bf > Bb

On top of that, the foreground luminance is greater than the background luminance.

To translate all of this into only luminance, we accentuate these differences. So I pop open a channel mixer. Red will get turned down (darkens the background more than foreground), Green and blue get turned up (brightens the foreground more than background.) I could write a formula for exactly what the sliders should be, but it's easier to just start with the general directions and go by sight after that. The 1st (and only) attachment shows how the channel mixer looks and how the hair strands have been somewhat accentuated. Let's leave this for now and do the left side of the head (next post).

Bart

[bart_hickman]

(Addendum: the attachments didn't come out in the correct order for some reason.)

Now for the left side. What a mess. There's no difference in luminance here at all and a tiny difference in hue and saturation. Looking at the RGB as we did for the right side, the hair strands are strong in red and weak in blue compared to the background. So pop up a channel mixer and crank red up and blue down to make the most of this distinction. I adjusted green to get things centered up. The 1st attachment shows how the channel mixer preview looks. I managed to bring out some contrast, so now do a curve layer (shown in the 2nd attachment.) I'm done--it's clear at this point that the noise is larger than the hair strands so I'm going to have to finish this one up with a hair brush. We'll do that along with the rest of the clean up after everything is combined.

To combine, we simply pick the best from each of the three sections and black out the alpha channel of the adjustment layers that weren't the best. The 3rd attachment shows my layer palette so far. In the group called "Mask 1, top, left, right" are all of my adjustment layers thus far. There are a couple of extras I didn't talk about--but you've seen at least one example of every trick thus far and I'm trying to keep this thing minimized.

You also see in the 3rd attachment a layer group called "Mask 1, combine". This has the three merged results from the three previous sections. I'll simply flip around through the three sections and keep the best parts from each one by selectively erasing. In some cases I even end up with some mixing between sections.

After some soft erasing from each of the sections, "Mask 1, combine", looks like the 4th attachment. The top and right look good--I'll just need some burning to fix those. The right side merely provides guidance on how to use a brush to re-create the hair strands over there. We're not done, but I think you can see the total mask emerging. Now I'll merge this and start the final phase of the Mask 1...next post (I'll try to finish up tomorrow.)

Incidentally, this all seems cumbersome, but it's really not--if I weren't explaining it, I wouldn't be saving all of these steps and I'd only be about 15 minutes into it at this point.

Bart

[Ken45140]

Bart: Amazing!! I am just done with part 1 and am needing to take a break. This is teaching me so much.

You don't spell it out (because you are so used to it) but creating layer groups ahead of time seems essential to turning the group layer into a mask. This is true isn't it? The create mask choice is greyed out when selecting the curves adj layer. Second, I found I had to do some painting on the mask itself as I never got a "clean" one even after color choicing and curves adj. Practice should make this better. But I finally see the approach.

I'll report back after parts 2 and 3 and after more iterations on the technique.

Many thanks.

Ken

[bart_hickman]

I do have the image and the subsequent adjustment layers all together in a group--yes. PSP creates a group automatically when you create a mask because that's nearly always what you want to do anyway.

It's not essential that you create the group ahead of time--you could do the work first and then dump everything into a group later as well. Even when I don't *need* a group, I use them to organize things. When you get to part 3, I posted a picture of the layer palette to that point to show the groupings.

Bart

[bart_hickman]

When we left off, I had combined the best of several mixing/curving exercizes to arrive at the beginnings of a mask. I looked at it tonight and realized I don't like the far right side, so I did minute of fiddling around using some layer differencing to get the slightly revised fig 1 below. I'm not going to delve into those particular details because it's just more variations on what you've seen already (ie., mixing/curving/adjustment layer masking).

1. Do a copy merge of the "Mask 1, combine" and paste it into a new raster layer.

2. Create a new raster layer on top of that and start painting white in the face area and black in the background area away from hair strands.

3. Copy-merge/paste to new layer that and did some dodging and burning in between hairs to finish it off. This gives fig 2.

4. Looking at the left side, we see our very grainy and dim extraction of the nearly-invisible hair strands. I just caved in and used the fine hair brush and painted some strokes of somewhat transparent white and then finished up with light smudging to smooth it out a bit. I don't feel too guilty about this part because the hair really is nearly invisible. The only reason we can see anything resembling hair is psychological in nature--we see more than what is actually there. Fig 3 shows how it looks after I brushed in some new hair.

5. So that's it--we've created what I call Mask 1. It's time to apply it to our original image. I like to put the Mask 1 extraction on the worst possible new background. Since the old background was sort of black, I choose the exact opposite--something very bright. I just created a new raster layer (it's called raster 1 in the attached layer depiction) with the new color.

6. Now put Mask 1 into a mask layer. Select the original image and create a mask (show all) on it. Then go back to the Mask 1 group (see layer palette attachment), select all, copy, and then go back to the Mask you just created and do "paste into selection" which transfers Mask 1 into the actual mask layer. (This, BTW, is how you can paste the luminance from any selection into a mask or adjustment layer.)

This is what I did in fig 4.

It looks a bit...well...off. Part of the problem is the original is VERY dark and it tends to look unnatural in front of such a bright background. The other part is the fringe effect folks complain about--but in this case the fringe is dark--ie., the old background color still remains mixed into the hair color near the edges of the mask. In this case, the old background color is nearly black. In part 5, we'll get rid of it...mostly. As I originally said, this image is at the limits of what is possible.

Fig 5 (layer palette) shows the new layers we created in this part.

Bart

[Ken45140]

Bart: I say it again, this is great work. I have it all captured in an Excel spreadsheet and am working to practice with it. I have numerous real-world images of my grandson which demand its use (including one just like this example, if you can believe it, with dark background right behind the hair area---if nothing else, by learning and applying this technique, you learn to more carefully compose your photos in order to avoid having to apply this technique ).

I am not sure how many PSP users read messages here, but you've given them a gem of a tool/approach. PS users have probably been passing right by, although it seems like most of the principles you describe are applicable across platforms, even if the specific steps and layers, etc are not.

I appreciate your sharing.

Ken

[bart_hickman]

Time to de-fringe and re-fringe.

1. Defringing. For most backgrounds, I would remove the fringing at this point by burning or multiplying the fringe areas by the inverse of the background. The inverse of the background in this case is nearly white--ie., when the background is black, you don't have to remove the fringe--it's already gone because black is 0,0,0. That is the one and only thing about this extraction that's "easy" (although the step we saved is a trivial one.)

Fig 1 shows the latest way I've been doing this. The de-fringing mask is simply the inverse of Mask 1--saves a lot of trouble. The de-fringing group is wedged in between the original and Mask 1. I usually use burn blend mode for defringing. However, it's sometimes worthwhile to try multiply as well. Kind of depends on how aggressive you were with masking. For this technique, you want to be slightly non-aggressive in your masking--ie., preserve details at the expense of having more fringing. I think you'll get the hang of it with practice (I'm slowly getting the hang of it myself.) It's nice having this flexibility--you can choose between multiply and burn, and you can adjust the opacity slider as well (although I usually have this one at 100%).

Note that "synth old background" layer--that's the original image that's been cloned to recreate the background in the fringe area. This is how the background generally has to be created in situations where the background has different colors. If it is a plain color, then you can just flood fill with that color. Of course if it's a plain color and there's decent contrast, you shouldn't be using this method--you should be using the background eraser.

As I mentioned, this defringing has negligable effect because I'm essentially burning the fringe with white--which of course is a no-op. I only include it for completeness.

2. RE-fringing. Now we add back in the correct fringing for this background (pale yellow). Fig 2 show the re-fringing group--good news again. Mask 2 is repeated here--it's simply the inverse of Mask 1. Mask 2 is applied to another copy of the new background and the blend mode of the resultant group is set to "screen". Screen (and dodge) tends to mix a new color in (whereas burn and multiply tend to de-mix colors). I haven't found dodge to be appropriate yet, but you never know. I usually slide the opacity slider around until the refringing looks about right. The nice thing about this method is once you've set these sliders for the most difficult background, it's about the correct answer for any other background you will try.

3. Fig 3 shows how the extraction looks now. The blotchy messiness is mostly gone now, but the fringing still looks sort of dark. That's because the foreground object is dark. You could screen some more background onto it, but the real solution is to boost the effective EV of the foreground object. Fig 2 shows where to add the boosting curve. By putting it here, it boosts everything associated with the extracted object and has no impact on the new background. Consequently you can vary the amount of boost quite a lot and it still looks natural. Fig 4 shows how it looks after EV boosting.

4. If you want further cleanness (ie., web-publishing), you can turn up the opacity of the re-fringe group and/or use a low opacity, very soft white brush and paint around the edges of the hair on Mask 2 in the re-fringing group--this lets more of the new background show through the hair. What's nice here is if you screw up, you can just go grab another copy of Mask 1 and invert it to start over again.

If you will eventually be using a "realistic" background anyway, that sort of cleanup might not be needed. On top of that most images with any decent amount of contrast will come out looking almost perfect at this point. But even this image looks pretty good on real backgrounds (Fig 5 shows some examples).

Link to fig 5:
http://home.comcast.net/~zumbari/fig...5_examples.jpg

That's it!
Bart

[Ken45140]

I am resurrecting this thread because I need some basic help on PS7 masks and layers... (no question is too dumb here, only the unasked ones---right?).

I have been trying to increase my PS7 skills in terms of extractions and selections and have been trying to eliminate "fringing". Bart Hickman, who has contributed a lot to my learning in PSP10, worked up an example of fringe removal, and he put it here (http://tinyurl.com/brwt8). He did this in terms of PSP layer stack/palette, and I pretty much understand it that way. In trying to accomplish a similar (same?) result using PS7 and its layer pallete, I am frustrated and cannot.

The PSP layer system works pretty intutively and masked layers can be used with ones above to create composite or blended results. The key is that the mask is on its own layer.

What is throwing me off is that the PS7 layer system puts the mask ON THE SAME LAYER, and I do not know how to set it up to work equivalently as the PSP layer/mask combination.

Here is a word example to go with the figures below (or in Barts tutorial shown above).

I mask out the hair and put it in front of a black background; the hair "whispy" areas have a blue fringe.

I apply a channel mixer mask and adjust so that all that is showing is the fringed hair. I want this to be a mask.

I sample the color of the fringed color and fill a layer with this color and invert it.

I want to multiply blend this inverted color LAYER, masked by the channel mixed fringe area with the original fringed image LAYER, which will then eliminate the blue fringe.

I cannot do it. I have studied (and tried) clipping groups and layer sets (remember, PS7 and not PSCS), and can not find the combination of mask, layer, group, set, or ??? to accomplish the removal of the blue fringe from the hair.

This is complicated to explain and I don't blame anyone for passing by, but if anyone can set me straight, I would be grateful.

Thanks,

Ken

[Swampy]

>>>I sample the color of the fringed color and fill a layer with this color and invert it.

>>>>I want to multiply blend this inverted color LAYER, masked by the channel mixed fringe area with the original fringed image LAYER, which will then eliminate the blue fringe.

-------
Try loading the mask of the fringe area (Option click on the Layer's mask). With that selection still active, select the inverted blue filled layer and click on the "apply mask" button at the bottom of the layer's palette.

This should load the same mask selection into the blue layer.

[Ed_L]

I'm glad I came in late on this one. Instead of putting in a lot of time trying to get it right, now I can just try the great techniques posted here. Thanks for doing all the hard work!

Ed

[bart_hickman]

Hi Ken,

The mask is the biggest difference between PSP and PS. Once you get that figured out, the rest is cake.

The first attachment shows the generic translation from PSP mask to PS mask. To apply a mask to the active layer, PSP creates a mask layer on above the active layer, then puts the two into a group. So while I agree with you the PSP method is more intuitive, it's much less compact.

Figure 4 from that tutorial will then simply look like the second attachment with the "Remove old fringe" set to multiply.

I did this already and discovered it doesn't quite work as well in PS as it does in PSP. There's something different about the way the two programs are doing multiply with masks.

No matter. PS has the linear light blend mode--change the blend mode of the "Remove old fringe" to linear light. Then adjust the opacity up or down to make it look good--50% seems about right. This is kind of nice because it means you have +/-50% of fudge range to work with.

I finished up the method and the layer palette is shown in the 3rd attachment (corresponds to figure 7 in the tutorial.) Still use the screen blend mode for that.

One more important thing: Layer groups default to "pass through" blend mode in Photoshop. Be sure to change it to "normal".

Bart

[Ken45140]

Hey Bart: I was hoping you would respond.

Sorry for my denseness, but it seems like a key step is missing.

In figure 4 from your tutorial, you create Mask2 by using a Channel mixer to isolate the blue fringies. Where is that in what you posted above? Your attachment of the PS set shows the Mask on the inverted blue layer as if it just popped out of nowhere. It has to be created from a channel mix adj layer, yes? Its even shown in your Figure 3. I even have it on a PS adj layer but ..... so, I just created a channel mask by using the magic wand on it, inverting it, and saving it as a channel (called Mask2). Now I'm stuck on getting this combined with a filled layer of the browish inverted blue color (your "remove old fringe" layer).

What am I missing??? (or is a step or two missing from your comments?)

Thanks for helping.

Ken

Edit: I reread Swampy's note and tried that and got something directionally correct it seems, but the quality of the Mask2 I created with the magic wand, and saved as a channel which I load back, really sucks....so the resulting image is a combination of some blue some brown and some hair....so, it seems like I still need to understand what you are showing me.

[bart_hickman]

Originally Posted by Ken45140

In figure 4 from your tutorial, you create Mask2 by using a Channel mixer to isolate the blue fringies. Where is that in what you posted above? Your attachment of the PS set shows the Mask on the inverted blue layer as if it just popped out of nowhere.

In fig 4 I'm not creating Mask 2--I'm using it to remove the fringe. I don't see any channel mixers in that figure.

Mask 2 itself is created up in fig 2 "The making of Mask 2". In PSP, I would have simply raster-copied the "Mask 2 construction" group and pasted it into the "mask 2" mask layer. In PS, I alt-select the mask, then paste which puts the clipboard into the mask.

It has to be created from a channel mix adj layer, yes?

YES! At least that's my preferred method.

Its even shown in your Figure 3.

Figure 3 just shows Mask 2. Figure 2 has the channel mixer. See comment above.

I even have it on a PS adj layer but ..... so, I just created a channel mask by using the magic wand on it, inverting it, and saving it as a channel (called Mask2).

Aaaack! Not the magic wand.

To copy a raster image into a mask, do the following (in this case, you actually want to do a copy-merge of a group and paste that into a mask)
1. Alt-click the eyeball for the Mask 2 construction group (you should now see Mask 2)
2. Ctrl-A, then Ctrl-shift-c. (you've now got Mask 2 in the clipboard)
3. Alt-click the eyeball again to return layer visibilities to previous states.
4. Alt-click the mask thumbnail that will become Mask 2.
5. Ctrl-V (puts the clipboard into the Mask).

Here's a cheat sheet for all the basic mask operations in Photoshop (at least the one's I use.)
http://home.comcast.net/~zumbari/Tut...heet_sheet.pdf

Now I'm stuck on getting this combined with a filled layer of the browish inverted blue color (your "remove old fringe" layer).

What am I missing??? (or is a step or two missing from your comments?)

Woohoohhahaahaaaaa (evil laugh) There are always steps missing. I prefer to teach using riddles--sort of like Merlin.

I think one of your sticking points is you're trying to combine the creation of Mask 2 into the defringing group and that's not what I was doing (I don't think it's even possible). The group where I create Mask 2 is completely separate from the group that does that actual defringe/refringe operation.

I've attached a layer palette showing *almost* the entire layer stackup. I say "almost" because the "fringy object on black background" also requires a couple of layers to create. I didn't include this for PSP because the layer stackup was way too big. Note how the Mask 1 and Mask 2 construction groups are down at the bottom underneath the new background and thus not contributing directly to the final image. They are simply a scratch-pad area used to calculate the masks. Once I'm done with that, they aren't needed anymore.

Two final notes:
-I don't think you want to use fill layers here. Instead use a raster layer. Just sample the original background color, then fill the raster layer, then invert it. In many cases, you won't have a uniform background color anyway.
-The mask computation is the hard part and I usually iterate two or three times for both Mask 1 and Mask 2.

Bart