RetouchPRO

Go Back   RetouchPRO > Technique > Photo Retouching
Register Blogs FAQ Site Nav Search Today's Posts Mark Forums Read


Photo Retouching "Improving" photos, post-production, correction, etc.

Repeated pattern from cloning

Reply
 
Thread Tools
  #11  
Old 01-10-2016, 02:25 AM
Eigil Skovgaard Eigil Skovgaard is offline
Junior Member
 
Join Date: Jun 2010
Location: At the moment Scania, southern Sweden.
Posts: 24
Re: Repeated pattern from cloning

Me again,

Well, apparently the novelty value has worn off - just as I posted links to the images in question. I'll maintain the links for further a week, then you (webmaster) could as well remove them from this tread as they will only cause frustration.

The point of interest (at least to me) was, that after I had posted the images, I could suddenly observe a repeated pattern in the smaller version of the final image (to the right). To me it confirms the risk of tunnel vision when working with the same image for a while. And it keeps the idea of an automated recognition of repeated patterns alive. All just to share some thoughts - possibly inspire a brilliant programmer.
Thanks for the feed back - both of you ;O)

I have noticed, that it is primarily Repairman and Klev that have answered my 4-5 questions in the short time I have attended this forum. So, are we really a forum with a broad collective know-how. Or have I introduced an area of photography that fits badly into the practised idea of "retouch" for this site?
Reply With Quote top
  #12  
Old 01-10-2016, 11:04 AM
John Wheeler's Avatar
John Wheeler John Wheeler is offline
Senior Member
 
Join Date: May 2011
Location: Colorado
Posts: 583
Re: Repeated pattern from cloning

Hi Eigil
This may be To Much Information in my post yet here are my thoughts.
- Many holidays start at the time of your posting and some don't like to jump in late to a thread.
- Your question was pretty technical and quite specific in regards to a tool of interest (not too many tool developers here) while there are many retouchers that could help identify if the image looks realistic or edited. A specific tool question may not get as many responses while a question about if anyone can see the cloning/edits might have gotten a bigger response. That may not be what you wanted
- However, if that is what you wanted, here is a hybrid answer on one way that I use to look for subtle unusal parts of an image to examine if it has been "photoshopped" (stop here if this is of no interest to you).

What I do on occasion is look at the color channels individually. Then after that I look at a luminosity rendition, a couple types of saturation map renditions, and the a Hue map rendition. To me, it points out several areas of your image that come in to question through none may have to do with cloning:
1) Your original image after editing
2) Saturation Map (using Sat Blend math)
Hedge and house has some unusual hot spots. Also, higher saturation under the eves.
3) Saturation Map (Sat as defined by color picker)
Similar to #2
4) Hue map
I did not find anything unusual here
5) Luminosity Map
I did not find anything unusual here

I know this is not what you specifically asked for as I was answering a broader question of how to spot PSed areas in the image, yet this is just another angle of possible areas of the image that might be "off" and worth examination.

Nice image and restoration/repair work BTW
Attached Images
File Type: jpg Final-Edited-Image.jpg (702.8 KB, 11 views)
File Type: jpg Saturation-(Blend-math).jpg (330.4 KB, 10 views)
File Type: jpg Saturation-(Color-Picker-math).jpg (397.5 KB, 10 views)
File Type: jpg Hue-map-at-max-Sat.jpg (935.0 KB, 10 views)
File Type: jpg Luminosity.jpg (404.9 KB, 10 views)
Reply With Quote top
  #13  
Old 01-10-2016, 09:03 PM
Eigil Skovgaard Eigil Skovgaard is offline
Junior Member
 
Join Date: Jun 2010
Location: At the moment Scania, southern Sweden.
Posts: 24
Re: Repeated pattern from cloning

Hi John,
Interesting examination. I have examined the hot spot issue, which is due to a single Hue/Saturation layer clipped to the hedge - and thrown in without really thinking - to repair what appeared to be a kind of pre-death in the foliage, possibly a reminiscence from a masking plug-in. When I examined the original hedge-image, the foliage is vital and green. So this part needs a re-touch. Good observation. And I found the HSB/HSL options under Filter-Other, which has not called upon my attention until now.

And yes, I am still open for a good prescription for localizing repeated patterns (before the audience do it), because those patterns are photoshop-tattletales if any.

Another question: What is your policy regarding linking to images. Do you prefere uploads to your site to avoid broken links in time, or do you accept images from "dead" treads to disappear - leaving dead links?
In this case I used Dropbox which have a limited capacity - and external servers mean less consolidated documentation for the RetouchPro discussions.
A common consideration is: Which file size is sufficient to reveal the beauty of my images and which is not ;O)
Reply With Quote top
  #14  
Old 01-10-2016, 10:07 PM
John Wheeler's Avatar
John Wheeler John Wheeler is offline
Senior Member
 
Join Date: May 2011
Location: Colorado
Posts: 583
Re: Repeated pattern from cloning

Doug Nelson is the owner of this site (I am just a participant). Here is the link to the posting policies and you can upload an image or link an image: http://www.retouchpro.com/forums/faq...b3_attachments
I personally think it is best to have the image with the site for long term availability on the site yet even if you pay the annual fee to be a patron of this site, the max upload attachment can only be 1MB. So if you need more than 1MB, it is a moot point and you need to link the image in.

Always a judgement call for needed file resolution and quality. You don't need as much just for display than if you need to share an image to have or demonstrate more extreme adjustments. Just a judgement call for the poster to make.
Reply With Quote top
  #15  
Old 01-11-2016, 02:54 AM
klev klev is offline
Senior Member
 
Join Date: Jun 2010
Posts: 1,109
Re: Repeated pattern from cloning

Quote:
Originally Posted by Eigil Skovgaard View Post
Hi Klev,

Thanks for explaining your workflow.

To further illustrate my case, I attach 4 images:

1. The original image from 2012. The house was better lit in this shot, but the surroundings were ugly, so I masked the house out for later use.
https://www.dropbox.com/s/a7gz93i55g...=e&n=511226638
2. The follow up from 2015. The surroundings were a lot better due to more foliage on the trees. A constant wind rendered the foliage a bit blurred, which facilitated focus on the main subject. Unfortunately a modern board fence had been added, and I had to fill in the reddish marked areas with cloned material.
https://www.dropbox.com/s/1gqe7t53un...=e&n=511226638
3. The final hybrid from December 2015.
https://www.dropbox.com/s/gpg6me31g5...=e&n=511226638
4. An example of how an anti-clone-wiz could anonymize those clones.
https://www.dropbox.com/s/txpjeip22s...=e&n=511226638

Please forgive the much reduced quality due to image compression. I have not sharpened further.
Ah I meant to respond to this. I forgot. Sometimes I put things aside if I want to view the images and take a little time to think over the response.

First off you couldn't just open an old image and have a clone wizard run on it. That's an exponential problem. On some images it would not end if you left your computer running for a year or possibly even a decade. Exponential problems grow quickly. I'm exaggerating a little, but not as much as you might think with a brute force implementation.

Pattern recognition works based on approximation using trained classifiers and inferential techniques such as segmentation and arc-length to detect writing. Facial recognition relies heavily on markov random fields. I considered all of this in my first response, but I omitted the technical jargon.

See the paper poisson image editing for an example of smart cloning techniques.

In the case of your image, I would have solved it in photoshop like a puzzle. I would first make the mask and then fit pieces inside the mask. The mask would be held at a fixed position. A little burn and dodge work can blend things in nicely.

Looking closely at the actual work, I appreciate your concern for details. I don't think your problem is cloning at all. The hedge portion that you added in just looks out of place. Its outline is a bit off. Overall color/brightness/lighting direction are just slightly off relative to its surroundings. This concerns me more. There are also parts that appear sort of smeared. You get this because you appear to have used a soft clone brush on the area. Believe me, hedges are difficult to retouch and take practice.

I think when you do this you should try to get complete leaves and move a nice section, then fix brightness and lighting on it. Add another as necessary until it is filled as you want it. As you noted earlier in the thread, Adobe has attempted to address this with content aware fill. It's a much more sensible approach than trying to randomize matching data.
Reply With Quote top
  #16  
Old 01-11-2016, 10:30 AM
Eigil Skovgaard Eigil Skovgaard is offline
Junior Member
 
Join Date: Jun 2010
Location: At the moment Scania, southern Sweden.
Posts: 24
Re: Repeated pattern from cloning

Hi Klev,

Imagine the state of horror here when I opened the suggested article and realised that two of the authors of "Poison Image Editing" had already passed away ... Raise the bridge and the firewall ...! - the command echoed between the walls of my office.
Then I realized that the signs † and ‡ are used as footnote markers - and the word Poisson means: "a discrete probability distribution".

From Poisson Image Editing (p. 315):
The seamless cloning tool thus obtained ensures the compliance of source and destination boundaries. It can be uses to conceal undesirable image features or to insert new elements in an image, but with much more flexibility and ease than with conventional cloning, as illustrated ...

Well, "conventional cloning" from some time ago. The article is from 2003. Twelve years in this area of development might have made some of the cloning approaches described obsolete. I took the Path tool from PS CC 2015 and in one pass replaced the smiling face of Mona Lisa (poor thing) with the royal face of Leonora Christina, a daughter of the Danish King Christian the IV (1588-1648), the latter in b/w. Chr. IV by the way is famous in all Denmark as the "warrior king" (he also created nice buildings). But most of the time he blindly attacked anybody with stronger armies and managed to minimize Denmark to the current fraction of the firmer empire ...! It's true.
Back to the Path tool: The immediate result is impressing. Most of the fine pattern from the cracked painting is merged in. Only the central parts of the path would need further retouch to simulate the crackling. I think, that the techniques documented in this article has already been implemented in PS and other image editing systems in 2015.

But is cloning the primary problem with a search-algorithm for a repeating pattern wizard? I suggested earlier, that the area to be searched could be limited by a selection - and a few parameters to prevent the calculations to go exponential. Let's say, that the sniffer part of the wizard is instructed to look for patterns within a selected area of 400 x 400 px, with a "nose-size" of 100 x 100 px. It will save the first square from each sweep as its current search-pattern. Then it will search in a normal reading pattern with the current size as the increment. If it finds no match - and no much-alike pattern - the next sweep will be offset with 5 px increments to reach all targets with sufficient precision. If it finds much alike patterns, the search pattern is flagged with a "hit". When the selected area is sweeped as a whole, the flagged targets are replaced with randomly created patterns based on the hits. This is not heavy computing, just scanning and randomizing. Only calculations to find the grade of similarity between the search-pattern of 10000 px (in this case) and the current target of similar size would call for more advanced math.

My final image:
"Lighting a bit off, smeared areas and the hedge not properly defined ..."
I totally agree and will make a new approach using your fixed mask proposal.
According to The Photographer's Ephemeris (a nice little sun and moon adviser) the lighting between the two takes differs this much (around here):
The 20th of January 2012 at 3.23 PM, angle of sun, horizontally: 223.3°, vertically: 4.8°.
The 21st of May 2015 at 2.09 PM, angle of sun, horizontally: 206.5°, vertically: 52.5°
Difference, respectively h: 17° and v: 47° (app.).
The direction of the takes was aimed to be the same, about 240°. The quality of the light was very different. First low and rather diffuse lighting, then higher and more directional.
My hope is, that without knowing the difference of lighting on the building vs. the green sourroundings and the hedge, this impression of difference will not emerge .... :O. (and because, I am reluctant to shoot a third image of the same subject.)

Well, Klev, this should be all for now. Before this tread goes exponential, I will rest my case and hurry back to the drawing-board.

Appreciate your proposals.
Reply With Quote top
  #17  
Old 01-11-2016, 03:50 PM
klev klev is offline
Senior Member
 
Join Date: Jun 2010
Posts: 1,109
Re: Repeated pattern from cloning

Quote:
Originally Posted by Eigil Skovgaard View Post
Hi Klev,

Imagine the state of horror here when I opened the suggested article and realised that two of the authors of "Poison Image Editing" had already passed away ... Raise the bridge and the firewall ...! - the command echoed between the walls of my office.
Then I realized that the signs † and ‡ are used as footnote markers - and the word Poisson means: "a discrete probability distribution".
Ah I didn't know if you would read it, but the paper has some nice visuals. I didn't know that one of the authors had passed away.


Quote:
Originally Posted by Eigil Skovgaard View Post
Hi Klev,



My final image:
"Lighting a bit off, smeared areas and the hedge not properly defined ..."
I totally agree and will make a new approach using your fixed mask proposal.
According to The Photographer's Ephemeris (a nice little sun and moon adviser) the lighting between the two takes differs this much (around here):
The 20th of January 2012 at 3.23 PM, angle of sun, horizontally: 223.3°, vertically: 4.8°.
The 21st of May 2015 at 2.09 PM, angle of sun, horizontally: 206.5°, vertically: 52.5°
Difference, respectively h: 17° and v: 47° (app.).
The direction of the takes was aimed to be the same, about 240°. The quality of the light was very different. First low and rather diffuse lighting, then higher and more directional.
My hope is, that without knowing the difference of lighting on the building vs. the green sourroundings and the hedge, this impression of difference will not emerge .... :O. (and because, I am reluctant to shoot a third image of the same subject.)
I don't think you need a third image. I've built up a sense over time for what will make things not feel pasted in. Even if it's a tough one, you can usually get it to the point where someone will not be drawn to a reconstructed area.

Quote:
Originally Posted by Eigil Skovgaard View Post
But is cloning the primary problem with a search-algorithm for a repeating pattern wizard? I suggested earlier, that the area to be searched could be limited by a selection - and a few parameters to prevent the calculations to go exponential. Let's say, that the sniffer part of the wizard is instructed to look for patterns within a selected area of 400 x 400 px, with a "nose-size" of 100 x 100 px. It will save the first square from each sweep as its current search-pattern. Then it will search in a normal reading pattern with the current size as the increment. If it finds no match - and no much-alike pattern - the next sweep will be offset with 5 px increments to reach all targets with sufficient precision. If it finds much alike patterns, the search pattern is flagged with a "hit". When the selected area is sweeped as a whole, the flagged targets are replaced with randomly created patterns based on the hits. This is not heavy computing, just scanning and randomizing. Only calculations to find the grade of similarity between the search-pattern of 10000 px (in this case) and the current target of similar size would call for more advanced math.
Search is actually the primary problem. You wouldn't want to assume identical pixel values, because that may not happen. You would probably be looking at estimated gradient domain as a preliminary indicator using some kind of finite difference method. This can be estimated using a 3 x 3 grid, because that's the smallest range that will allow you examine pixels before and after a given center. Even if we considered rotated or reflected patterns as different, you would have to do the following.

-Estimate gradient domain of image (although you don't require an anisotropic kernel if you're not expecting to pair against rotated or reflected patterns).

-Compare against gradient for every other section of the same size.

-Store record of those that match and remove from future consideration.

-Move onto next section of pixels. Repeat.

What if you have overlapping patterns? Do you merge them and look for the union of the two patterns or handle them both distinctly. This is also just for one size obviously. You don't have to re-estimate gradients, but it becomes exponential if you don't stop past a pattern of some known size. It's also not just the computation that gets you. This will have a lot of cache misses, which increase the time required by a large factor.

This stuff become a mess of edge cases very quickly, which is why it's approximated. If you want to know where you messed up on cloning, you would want an original reference. Otherwise you're back to approximations as the only reasonable method, in which case you may get false positives / negatives on something that runs in interactive time.

I can point you to a few references on color science and image processing if you like. I know one that is gentle enough for a start. You might enjoy some some of the material in something like Gonzalez and Woods. I would suggest a back edition, inter-library loan, or visiting an academic library to save money. Some of the really modern ones build on that, but they require a pretty extreme comfort level with reading mathematics.

Some of it is easy to misread due to notation where scalar quantities (numbers) and vectors (eg <0,0,255> use the same notation. It's worth noting that points in an rgb gamut are described using vector notation, but they don't really work like vectors. That makes it even more confusing.
Reply With Quote top
  #18  
Old 01-12-2016, 12:58 AM
Eigil Skovgaard Eigil Skovgaard is offline
Junior Member
 
Join Date: Jun 2010
Location: At the moment Scania, southern Sweden.
Posts: 24
Re: Repeated pattern from cloning

Hi Klev,
It's obvious, that you see this problem from the mathematical point of view, and of course the math would have to be lined up before anything could happen. To me its just an idea, an if the idea is interesting enough to the right people, the math will be done and the code implemented some day, otherwise the idea will vanish.
Regarding further studies in this area; my math is at the "aged" highschool level, i.e. widely forgotten though I find the mathematical "house" interesting and the area of digital imaging in particular. I made a choice in the past century. Resources, energy, time etc. must be used on the area of primary interest, and to me practical photography on a fairly educated level has the highest priority. I was deep into the Zone System in the 70'es, when a certain A.A. had reached Denmark. I developed a personal system to a practical work around of the tedious exposing and developing for Normal, minus 1, plus 1 etc and was able to extract the N- and over- and under-overexposure data for a particular film and process from a set of density-curves on a graph paper. That's how deep, I want to go. Like to keep it on a workshop level. I have made tons of completely boring pictures, super calibrated, just to examine if Zone III and VII showed the right amount of details. All that ended, when the s-curve became digital and made any shape possible. I love it and really don't miss the chemical environment for a second. Now I want to make images worth seeing outside my "darkroom", and I appreciate all the help I can get from automated tools - and sharing people.
Only, one tool is still missing in PS ,O)
Reply With Quote top
  #19  
Old 01-12-2016, 05:11 AM
klev klev is offline
Senior Member
 
Join Date: Jun 2010
Posts: 1,109
Re: Repeated pattern from cloning

Quote:
Originally Posted by Eigil Skovgaard View Post
Hi Klev,
It's obvious, that you see this problem from the mathematical point of view, and of course the math would have to be lined up before anything could happen. To me its just an idea, an if the idea is interesting enough to the right people, the math will be done and the code implemented some day, otherwise the idea will vanish.
I get you. I was thinking about how you would go about it. It occurred as undesirable to me, because naive cloning is just copying. I don't see how you avoid this without doing something to alter the cloned data, such as others have tried.

When I thought about it, you could still find ways to cut down problems like this. If you're start off with the smallest acceptable size and find matching patterns, you can then check how many contiguous pixels can be added to each while still containing a match. You also have to find a way to hash the individual pixel values. Otherwise you end up comparing each set of pixels to every other set of the same size.

This can miss things due to the gradient issue I mentioned before.
You also run into edge cases such as what if 2 patches match and a subset of one also matches another smaller patch? I think designing around the major edge cases, then finding ways to test for the smaller ones would be annoying, which is why I think it should help you avoid the garbage patterns in the first place.

It's still complicated, and like I said I prefer other approaches. It's just that would actually converge. You're also correct that I have somewhat of a mathematical obsession, but you were asking about image processing. Implementation details don't involve complicated math, but image processing uses the results from a lot of complicated mathematical theorems.
Reply With Quote top
  #20  
Old 01-13-2016, 12:59 AM
Eigil Skovgaard Eigil Skovgaard is offline
Junior Member
 
Join Date: Jun 2010
Location: At the moment Scania, southern Sweden.
Posts: 24
Re: Repeated pattern from cloning

Hi Klev,

Yes, it would be possible to work around endless calculations through a shortcut here and there. Reality is often a sum of compromises.

One of the limiting parameters could be to establish the critical ratio between observation distance and pattern size. This could include knowledge of human perception. Eye-scanning details on the screen at 300 to 400 percent is dangerous, as we know, because the repeating pattern just outside the screen is not repeated in the visible image segment. First time I observed some kind of repetition in my replacement of foliage in the image in question was in the 2400 px x y image at 1:1 - when it was already shared, and I made an final check, just to be sure (wrong point in time).

Btw - closer examined! the impression of annoying repetition is not on the leaf to leaf - or clusters of leaf - level, but in the repeating shadow-areas between the same two branches of leaf. This repetition would never be detected with a fixed size of a "sniffer". The search pattern would have to widen through a series of enlargements relative to the start-out size. Luckely computers work fast ...
Etc, etc.
I will end these considerations for now.

Not obsessed ... I would use the expression passionate. I worked with a guy next office which was what I would call obsessed by mathematical thinking - in the sense that he could not let go and deal with other problems. Bread and butter problems for example. We worked on the same accounting system for an insurance company, and his low throughput of design and code finally got him fired. In the meantime he had developed and coded (during his working hours) a numbering system that he used as a kind of database. Even before this happened, he had told me, that he was generally out of work for the very same reason, he could not let the math go for more than a few sleeping hours.
Passion is (in my opinion) obsession in a controlled form - and necessary to widen the general level of knowledge without sending the passionate or visionary person to the finger-paint academy in the middle of the process ;O)
Reply With Quote top
Reply

  RetouchPRO > Technique > Photo Retouching


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Batch removal of pattern from scanned images Madison Poster Image Help 9 12-29-2014 10:48 PM
Removing Embossed Paper Pattern AdvencedGuest Photo Restoration 11 07-24-2013 03:04 PM
texture pattern michael79f Image Help 3 10-09-2012 02:00 AM
replicating a difficult pattern & removing shadow ali_bahman Photo Retouching 1 02-10-2012 11:33 AM
Stray hairs over pattern timmy1729 Image Help 7 08-06-2009 06:54 PM


All times are GMT -6. The time now is 09:51 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
SEO by vBSEO ©2011, Crawlability, Inc.
Copyright © 2016 Doug Nelson. All Rights Reserved