Go Back   RetouchPRO > Technique > Photo Retouching
Register Blogs FAQ Site Nav Search Today's Posts Mark Forums Read

Photo Retouching "Improving" photos, post-production, correction, etc.

Masking: the edge

Thread Tools
Old 10-03-2008, 05:55 PM
skydog's Avatar
skydog skydog is offline
Senior Member
Join Date: Jan 2006
Location: Carolina
Posts: 1,294
Masking: the edge

When masking or cutting out an object (like a person) to place in a new background, I need some pointers to improve the blend of the edge of the cutout with the background. I'm not getting a smooth transition. I tried Layer>matting...and not getting the results I'd like using this. I know I've seen a tutorial on this in the past but no longer know the tips that were provided. thanks
Reply With Quote top
Old 10-03-2008, 06:20 PM
pixelzombie pixelzombie is offline
Senior Member
Join Date: Jan 2006
Location: chicago
Posts: 1,098
Re: Masking: the edge

try the blur tool in conjunction with the burn tool on the layer mask, i sometimes use a curve move to get the edges to come in and then selective blurring from there...
Reply With Quote top
Old 10-03-2008, 10:11 PM
TommyO's Avatar
TommyO TommyO is offline
Senior Member
Join Date: Aug 2007
Location: NC, USA
Posts: 1,211
Re: Masking: the edge

I've seen many methods over the years, all work well. A common one is similar to pixelzombie's suggestion. It involves using Modify>Border. First, Ctrl-click the layer icon of your subject. This will load the transparency of the layer. Then, click Select>Modify>Border and use a value of roughly 5 pixels (depending on the density of your image of course). Hit Ctrl-J to copy this to a new layer. Then, apply a low value Gaussian Blur to that layer and adjust the opacity to your liking.

This should soften just the edge, as is often the case in real image edges of people, etc. Other objects, like piano's, don't usually need this.

And of course, don't forget the shadow. That is usually what throws our eyes off.
Reply With Quote top
Old 10-04-2008, 06:38 AM
edgework edgework is offline
Senior Member
Join Date: Jul 2004
Location: Brooklyn
Posts: 494
Re: Masking: the edge

The problem is always with the transition pixels between masked object and old background. Leave them in, you have a dark or light halo. Pull in your mask edge, you have a scissors cut-out. And a global solution, such as blurring and tightening with levels ignores the fact that the edge for something like jeans is going to require different treatment than hair.

Leave the halo. Mask as best you can but don't try to cut so deeply into the silhouette that you destroy the natural edge..

Cut out enough of the new background to cover the masked object and dupe it to a new layer on top.

If the halo is lighter than the background, set this new layer to multiply. if the halo is darker, set it to screen.

Group to the masked object and hide with a layer mask.

Use a small brush at low opacity, peferably a tablet, and paint white into the mask along the halo.

It's pretty easy to "creep up" on the pixels with gradual strokes, bringing them to the same level as the background. And, considering that it actually works, it's not that time consuming.

Usually it's the best way to capture flyaway hair, fuzzy sweaters and any pixels that would naturally blend into the background.
Reply With Quote top
Old 10-04-2008, 07:29 AM
nevie nevie is offline
Junior Member
Join Date: Sep 2007
Posts: 23
Re: Masking: the edge

Whenever I make a selection I feather the edges, that usually helps to keep transitions pretty smooth. Or maybe try refine edges and work with your selection to get the best transition.
Reply With Quote top
Old 10-04-2008, 07:34 AM
jagerman's Avatar
jagerman jagerman is offline
Join Date: Jan 2008
Location: Atlanta, GA
Posts: 76
Re: Masking: the edge

Once you have your base mask. Command>click on the mask to make it into a selection. Then head to the "select" pull down menu and start playing with commands like contract, expand, smooth and inverse. Brushing areas back in and out till it is nice and clean. No feathering. Once I drop the image into my base image, blur the mask a small amount (.5-1) until the edges blend naturally.

Extra tip - keep your unfeathered (no blur) mask and add a second mask and blur that one. Turn your original mask off (shift>click). That way if you over do it you can always go back.

Hair, smoke, bubbles etc.. is a different process.

Good Luck, -W
Reply With Quote top
Old 10-04-2008, 05:05 PM
cricket1961's Avatar
cricket1961 cricket1961 is offline
Senior Member
Join Date: Apr 2005
Location: Flower Mound, Texas
Posts: 488
Blog Entries: 38
Re: Masking: the edge

You can try loading the selection for the existing mask, inverse the selection, and then loading the original selection back in using intersect. Sometimes a very slight feathering, say .2 or.3 done once at the time of your choosing, can help.

This should give you a good mask for any overlapping area to use a curves adjustment layer to lighten or darken the area for the halo.

Its also a good way to use the resulting selection to do a copy all and paste and then doing a slight blur of your choice.

Hope it helps Sky.

Reply With Quote top
Old 10-04-2008, 06:31 PM
Markzebra's Avatar
Markzebra Markzebra is offline
Senior Member
Join Date: Aug 2001
Location: London, UK
Posts: 718
Re: Masking: the edge

Yes, you need to burn or dodge the edge pixels - no mask is every really good enough
Reply With Quote top
Old 10-06-2008, 05:42 AM
skydog's Avatar
skydog skydog is offline
Senior Member
Join Date: Jan 2006
Location: Carolina
Posts: 1,294
Re: Masking: the edge

thanks for the pointers
Reply With Quote top
Old 10-07-2008, 09:38 AM
Karlington Karlington is offline
Join Date: Sep 2008
Location: Gothenburg, Sweden
Posts: 50
Re: Masking: the edge

I recommend watching Deke McClellands Lynda-tuts on Channels & Masks. His way of cutting out an object is extremley effective and timesaving (depending on how precise you want your cut out to be). He uses a channel (that as you know works as an alpha channel) and tweeks it by using levels, painting with overlay etc to get a high quality mask. Perfect for jeans as well as hair – all at once! Use it all the time!
Reply With Quote top

  RetouchPRO > Technique > Photo Retouching

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Anyone want to practice their masking skills? duwayne Photo Retouching 20 10-05-2008 09:05 AM
masking hair EDWINWRG RP Tutorials 12 09-26-2008 12:56 AM
skin masking techniques KR1156 Photo Retouching 20 09-10-2008 08:31 AM
Any tricks or tips for masking hair? AliasAngel Photo Retouching 15 03-11-2008 05:52 PM
masking issues jadams007 Photo Retouching 6 11-27-2007 09:08 PM

All times are GMT -6. The time now is 06:41 AM.

Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
SEO by vBSEO ©2011, Crawlability, Inc.
Copyright © 2016 Doug Nelson. All Rights Reserved