Repeated pattern from cloning

[Eigil Skovgaard]

"The Garbage Wizard found 13 incidents of 1 repeated pattern in your image - do you want the wizard to randomize the 12 of them?"

- or something like that.

Is anybody aware of the existense of such a great software that can scan an image and mark every repetition, that is not previously marked as allowed?

I want it!

[klev]

Re: Repeated pattern from cloning

That would be a terrible piece of software for a few reasons. First pattern recognition methods are based on approximation and similarity of transitions. They generally don't look for an exact match in pixel values, but you will find that with some cloning.

It's not actually clear that randomizing pixels would be an improvement. If they no longer fit the surrounding pixels, it will look worse. This can be solved using some kind of inpainting, but it would probably be better to just bake improvements in this area into tools commonly used to resolve inpainting problems rather than clean up messy retouching work at the end.

If you're looking for something more like a way to clean up annoying dust on a scan or litter on a beach, that's more an issue of object recognition and inpainting.

I think that is more of an interesting topic, but there isn't any software that does a really good job of it at a general level.

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Hi Klev,

Yes, terrible to make, but usefull if a gifted programmer could make it happen.

When I wrote "randomize" it was not in the ultimate sense of the word but with Content Aware replacement in mind. Photoshop's CA does a fairly good job on simple surfaces, but often the user must facilitate the utility to get a new pattern closer to the intended prolonged image into the problem areal.
I imagine the following behaviour from a - Undo Repeated Pattern Wiz:

The software finds and marks areas containing repeated patterns bases on a few (or not so few) parameters chosen up front. Fx, selecting a part of the image - selecting a max radius in pixels for the longest match in any direction - selection of prohibited areas within the subcropped area, etc.

After some thinking ..., the wiz would mark the problem areas (those within the chosen radius), and the user could choose between an automated content aware replacement, or a manual doctoring of the problems - spot by spot, i.e. "try to repair this incident ... ok, cancel the last one, I will do this myself."

That's the overall dream.

[klev]

Re: Repeated pattern from cloning

After some thinking ..., the wiz would mark the problem areas (those within the chosen radius), and the user could choose between an automated content aware replacement, or a manual doctoring of the problems - spot by spot, i.e. "try to repair this incident ... ok, cancel the last one, I will do this myself."

That's the overall dream.

That makes much more sense, although it's still a very rough idea. Content aware fill is just a type of inpainting. I think you want a way to apply some sort of inpainting to similar regions once the user provides a sample of what is to be considered "bad". Take for example a hard wood floor where you want to remove all of the knots. The user selects one using some paint or circle mechanic. The program replaces not only this one but looks for any others that are similar to it.

You should let go of the misguided notion of a gifted programmer here. Image processing is a very mathematical topic. You can have a mathematically deep topic and ugly code.

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Hi Klev,
You are describing another use of the same "machine". I think there could be af few more.
My concern was primarily to fix repeated patterns from my own cloning. After some time with cloned foliage in the same picture, it becomes hard to spot repetitions, I get less focused - even "blind", as the problem is SO boring. THEN the machine would be nice to have as a nifty plug-in to Photoshop.
I understand, that such a product represents heavy thinking, calculation, writing algorithms etc., and I had hoped to find a creator or user already using or about to create something like it. I feared that this was not the case, as nothing came up on the Internet. To be sure, I asked here. If anybody knew about the matter, it had to be here.

[klev]

Re: Repeated pattern from cloning

Hi Klev,
You are describing another use of the same "machine". I think there could be af few more.
My concern was primarily to fix repeated patterns from my own cloning. After some time with cloned foliage in the same picture, it becomes hard to spot repetitions, I get less focused - even "blind", as the problem is SO boring. THEN the machine would be nice to have as a nifty plug-in to Photoshop.

In that case you're searching for the wrong tool. This is a partly solved problem, but your approach would be arguably worse than existing ones. First you don't want something that analyzes it at the end. This means that you may have sampled from some of those earlier repetitious areas with further cloning. The other problem is that this wouldn't be implemented in a precise manner. A developer would have to find a way to approximate this, because a brute force approach would just run too long and can't be easily converted to a gpu based implementation due to inter-dependencies.

Content aware fill (a type of inpainting) and healing brush tools address the same problem, and I believe they provide a much cleaner solution than this.

I think on your end you would probably benefit from an improved approach to retouching, because cloning should never be that big of a deal. It's useful for getting rid of specific things such as dust, acne, stray hairs, sometimes very small background objects. Repetitious mistakes should generally be small enough that no one can spot them in the final result.

You may be making this far more difficult on yourself than what is really necessary. I haven't seen your work, so the only tips I can give you are somewhat general.

I would say consider what can be solved with burn and dodge methods rather than cloning. If you're practiced with them, they should take less time.

Don't zoom in too far. You shouldn't have to go past 100% most of the time. If you're using a small tablet relative to your screen size or you have a shaky hand, you might address more things at 200%. I usually do clipping paths at 100-200% depending on the subject matter, and they require more magnification than most things. I might zoom in to check something, but I don't stay there. It makes me lose perspective.

You should never tune out. Many images look a lot worse than they actually are when you're starting out. This can lead to a lot of wasted hours, which makes people tune out. You should be strategic with the work wherever possible. I personally avoid things like cloning out every piece of hair on a woman's cheek. It makes more sense to start with the most noticeable ones and continue only until the remaining ones are not problematic. Results look better, and less cloning leaves less room for mistakes.

I'm just guessing at the moment, but these are very common problems. There are tools that partly address them, but so far a good approach still delivers better results.

[Repairman]

Re: Repeated pattern from cloning

'because cloning should never be that big of a deal'.

That's how I look at it too. Once you get to the stage that your cloning leaves patterns you should consider whether it is the right tool to use in the first place. This applies to other tools/brush strokes etc that leave 'trails'. You either have the wrong tool or are not fastidious enough in the execution.

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Hi Klev and Repairman,

You need the background ...

Let me describe the current project from which this fantasy derived.

I like to record older buildings for my own pleasusre; not in a special artistic way but in a way that is technical sound. I exclude interfering modern accessories in the attempt to close in on the true historical look. The style of older buildings is often superior to newer creations. Expensive materials and working hours forbid the artwork from 100 years ago. Furthermore the generally architectural concept has been simplified, not always to the better.

I photographed "Sandskogens Värdshus" (The Sandforest Pub) for the first time January 2012. It is a Swedish wooden house with an interesting design in the details. As long as I remember the main color has been dark beige to yellow (fx 205,166,123) and the ornamental color green (fx 61,87,67). The house is situated in an area with trees on a sandy soil close to the Baltic Sea near Ystad. I shot the house from the roadside with typical Swedish trees in the background, pine, spruce, birch and different kind of bushes. In the front an almost naked hedge. It was a bright day. The sky was clear blue without a single cloud.

Back in Photoshop I removed a Tv-antenna from the chimney and an industrial ventilation system growing out of the roof. The hedge in the foreground was not dense enough to cover up for yellow and otherwise distracting building materials and it was impossible to clone through the hundreds of tiny branches of the hedge. So in a way I got stuck in my plan to cover up all untidy stuff around the house, and I decided to go back - some day - to shoot a new hedge.

I returned May 2015. The hedge was fine, but behind it was added an ugly brown board fence, which - I found out later - was impossible to separate from the leaves in the top of the hedge. The project rested for a while ...

Recently I found the time to proceed. I found out, that my first shot was the better regarding the house itself, but the latter (years later) was more attractive regarding the natural surroundings. So I masked the house out of the old environment and placed it in the more pleasing one. Unfortunately several ugly "improvements" had been added outdoor to the left and right of the building and a few "square-meters" had to be replaced with new content. The only plausible way was to extend the bushes, i.e. clone of the existing foliage. I used the pattern tool for the replacement and covered the edges with a weak clone-stamp.

The hedge had a new top painted in with a foliage brush which looks surprisingly fine.
Some construction waste was cloned away from the roof and the small security stickers were removed from the windows. The sky needed a few drifting clouds. Finally a small light and contrast adjustment and an "invisible" vignette. Done.
Of course all retouching have been made in adjustment layers.

Can I see repeated patterns like from dragging the clone-stamp several times over the same source area? No. It is not that kind of repeated pattern that bothers me. But I know ... that clones exist within the foliage, and another person might be able to spot them. With a Remove Repeating Pattern Wiz I could rest the case!

The result looks fine. But the cloning knowledge nags me.

Feel free to suggest specific ways of painting in new background without getting this problem.

[klev]

Re: Repeated pattern from cloning

I can't give you a precise strategy without seeing it. I could guess what it looked like, but my guesses are usually pretty bad.

In general hedges are always difficult. The same goes for trees and hair in certain lighting situations. I employ one or more of several strategies when it comes to difficult object removal.

I may mask the area that I intend to rebuild and assign that mask to a layer group before doing anything else. This gives me well defined and enforced boundaries for any work done to that area. It helps if I am trying to fit something to that area.

I may rely on color correction rather than a complete replacement if it is workable. This works in the case of a color you don't like behind one that you do. Selective color combined with masking often works here. Imprecise masking isn't too much of a problem if selective color doesn't generate too many undesirable side effects.

Content aware fill can be used for some of that, but I would probably pick a different strategy. Cloning knowledge shouldn't nag you. Sure it's a reuse of detail, but it doesn't matter and randomizing that could actually make it worse. I suspect what you actually want is a slight perturbation of the pattern, but this would still potentially draw attention to the work if it significantly impacts the gradient domain of the cloned region.

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Hi Klev,

Thanks for explaining your workflow.

To further illustrate my case, I attach 4 images:

1. The original image from 2012. The house was better lit in this shot, but the surroundings were ugly, so I masked the house out for later use.

https://www.dropbox.com/s/a7gz93i55gjp61d/1-sandskogens-vardshus-2012-2400xy.jpg?oref=e&n=511226638

2. The follow up from 2015. The surroundings were a lot better due to more foliage on the trees. A constant wind rendered the foliage a bit blurred, which facilitated focus on the main subject. Unfortunately a modern board fence had been added, and I had to fill in the reddish marked areas with cloned material.

https://www.dropbox.com/s/1gqe7t53unnskuq/2-sandskogens-vardshus-2015-2400xy.jpg?oref=e&n=511226638

3. The final hybrid from December 2015.

https://www.dropbox.com/s/gpg6me31g5fv7cl/3-final-sandskogens-vardshus-2012-plus-2015-2400xy.jpg?oref=e&n=511226638

4. An example of how an anti-clone-wiz could anonymize those clones.

https://www.dropbox.com/s/txpjeip22ssw47o/example-of-how-the-wizard-could-manage-cloning-2400xy.jpg?oref=e&n=511226638

Please forgive the much reduced quality due to image compression. I have not sharpened further.

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Me again,

Well, apparently the novelty value has worn off - just as I posted links to the images in question. I'll maintain the links for further a week, then you (webmaster) could as well remove them from this tread as they will only cause frustration.

The point of interest (at least to me) was, that after I had posted the images, I could suddenly observe a repeated pattern in the smaller version of the final image (to the right). To me it confirms the risk of tunnel vision when working with the same image for a while. And it keeps the idea of an automated recognition of repeated patterns alive. All just to share some thoughts - possibly inspire a brilliant programmer.
Thanks for the feed back - both of you ;O)

I have noticed, that it is primarily Repairman and Klev that have answered my 4-5 questions in the short time I have attended this forum. So, are we really a forum with a broad collective know-how. Or have I introduced an area of photography that fits badly into the practised idea of "retouch" for this site?

[John Wheeler]

Re: Repeated pattern from cloning

Hi Eigil
This may be To Much Information in my post yet here are my thoughts.
- Many holidays start at the time of your posting and some don't like to jump in late to a thread.
- Your question was pretty technical and quite specific in regards to a tool of interest (not too many tool developers here) while there are many retouchers that could help identify if the image looks realistic or edited. A specific tool question may not get as many responses while a question about if anyone can see the cloning/edits might have gotten a bigger response. That may not be what you wanted
- However, if that is what you wanted, here is a hybrid answer on one way that I use to look for subtle unusal parts of an image to examine if it has been "photoshopped" (stop here if this is of no interest to you).

What I do on occasion is look at the color channels individually. Then after that I look at a luminosity rendition, a couple types of saturation map renditions, and the a Hue map rendition. To me, it points out several areas of your image that come in to question through none may have to do with cloning:
1) Your original image after editing
2) Saturation Map (using Sat Blend math)
Hedge and house has some unusual hot spots. Also, higher saturation under the eves.
3) Saturation Map (Sat as defined by color picker)
Similar to #2
4) Hue map
I did not find anything unusual here
5) Luminosity Map
I did not find anything unusual here

I know this is not what you specifically asked for as I was answering a broader question of how to spot PSed areas in the image, yet this is just another angle of possible areas of the image that might be "off" and worth examination.

Nice image and restoration/repair work BTW

[Eigil Skovgaard]

Re: Repeated pattern from cloning

Hi John,
Interesting examination. I have examined the hot spot issue, which is due to a single Hue/Saturation layer clipped to the hedge - and thrown in without really thinking - to repair what appeared to be a kind of pre-death in the foliage, possibly a reminiscence from a masking plug-in. When I examined the original hedge-image, the foliage is vital and green. So this part needs a re-touch. Good observation. And I found the HSB/HSL options under Filter-Other, which has not called upon my attention until now.

And yes, I am still open for a good prescription for localizing repeated patterns (before the audience do it), because those patterns are photoshop-tattletales if any.

Another question: What is your policy regarding linking to images. Do you prefere uploads to your site to avoid broken links in time, or do you accept images from "dead" treads to disappear - leaving dead links?
In this case I used Dropbox which have a limited capacity - and external servers mean less consolidated documentation for the RetouchPro discussions.
A common consideration is: Which file size is sufficient to reveal the beauty of my images and which is not ;O)

[John Wheeler]

Re: Repeated pattern from cloning

Doug Nelson is the owner of this site (I am just a participant). Here is the link to the posting policies and you can upload an image or link an image: http://www.retouchpro.com/forums/faq...b3_attachments
I personally think it is best to have the image with the site for long term availability on the site yet even if you pay the annual fee to be a patron of this site, the max upload attachment can only be 1MB. So if you need more than 1MB, it is a moot point and you need to link the image in.

Always a judgement call for needed file resolution and quality. You don't need as much just for display than if you need to share an image to have or demonstrate more extreme adjustments. Just a judgement call for the poster to make.

[klev]

Re: Repeated pattern from cloning

Hi Klev,

Thanks for explaining your workflow.

To further illustrate my case, I attach 4 images:

1. The original image from 2012. The house was better lit in this shot, but the surroundings were ugly, so I masked the house out for later use.

https://www.dropbox.com/s/a7gz93i55gjp61d/1-sandskogens-vardshus-2012-2400xy.jpg?oref=e&n=511226638

2. The follow up from 2015. The surroundings were a lot better due to more foliage on the trees. A constant wind rendered the foliage a bit blurred, which facilitated focus on the main subject. Unfortunately a modern board fence had been added, and I had to fill in the reddish marked areas with cloned material.

https://www.dropbox.com/s/1gqe7t53unnskuq/2-sandskogens-vardshus-2015-2400xy.jpg?oref=e&n=511226638

3. The final hybrid from December 2015.

https://www.dropbox.com/s/gpg6me31g5fv7cl/3-final-sandskogens-vardshus-2012-plus-2015-2400xy.jpg?oref=e&n=511226638

4. An example of how an anti-clone-wiz could anonymize those clones.

https://www.dropbox.com/s/txpjeip22ssw47o/example-of-how-the-wizard-could-manage-cloning-2400xy.jpg?oref=e&n=511226638

Please forgive the much reduced quality due to image compression. I have not sharpened further.

Ah I meant to respond to this. I forgot. Sometimes I put things aside if I want to view the images and take a little time to think over the response.

First off you couldn't just open an old image and have a clone wizard run on it. That's an exponential problem. On some images it would not end if you left your computer running for a year or possibly even a decade. Exponential problems grow quickly. I'm exaggerating a little, but not as much as you might think with a brute force implementation.

Pattern recognition works based on approximation using trained classifiers and inferential techniques such as segmentation and arc-length to detect writing. Facial recognition relies heavily on markov random fields. I considered all of this in my first response, but I omitted the technical jargon.

See the paper poisson image editing for an example of smart cloning techniques.

In the case of your image, I would have solved it in photoshop like a puzzle. I would first make the mask and then fit pieces inside the mask. The mask would be held at a fixed position. A little burn and dodge work can blend things in nicely.

Looking closely at the actual work, I appreciate your concern for details. I don't think your problem is cloning at all. The hedge portion that you added in just looks out of place. Its outline is a bit off. Overall color/brightness/lighting direction are just slightly off relative to its surroundings. This concerns me more. There are also parts that appear sort of smeared. You get this because you appear to have used a soft clone brush on the area. Believe me, hedges are difficult to retouch and take practice.

I think when you do this you should try to get complete leaves and move a nice section, then fix brightness and lighting on it. Add another as necessary until it is filled as you want it. As you noted earlier in the thread, Adobe has attempted to address this with content aware fill. It's a much more sensible approach than trying to randomize matching data.