FFT and Golf Balls

[jcr6]

You can get pattern noise in any of the channels, especially the blue channel since it has a higher gain setting due to the lower sensitivity of a CCD or CMOS chip. In fact, you probably will notice that different channels have different Nyquist limits because of the interpolation in the Bayer pattern on the sensor.

Your technique will leave behind pattern noise that originates in a given channel (although since the Luminosity equation weights 60% for green, most of the correction will take place in the green channel, but only about 16% in the blue channel).

Furthermore, it won't work with a scanned color halftone.

Look, the whole reason you're even considering processing only one channel is because of the hack that stores the phase information in the color channels -- there really is no place to store the complex FFT information within the image itself. Fovea gets around this by storing high precision data in a buffer on disk. And I'm not even addressing the color problem -- I tell people to do one channel at a time. Ideally, the RGB FFT code should only work in 16-bits because of the extra precision.

It is a fact, however, that the Power Spectra of the three channels are different, that the PSF (Point Spread Functions) of the different wavelengths through your optics can be different, and that trying a one-size-fits-all approach is fundamentally flawed.

That it works in this case is fine, but only processing luminance (or brightness or even the K channel - I don't care which) leaves color artifacts behind. I care about the extra bits in the image, both from the point of view of RAW Camera Workflow (extra bits means better definition, color constancy, etc.) and Science (quantitative image analysis).

To say that the luminance can be affected by removing a noise pattern in a channel is disingenuous. Of course it can, especially if a pattern is being removed. If you don't mess with the DC term (the center of the cross), the overall luminance will not be affected.

So, I think that for now this workflow will work, but it has serious limits. Once we have a better method for dealing with color everyone will benefit.

[byRo]

jcr6, I of course agree with everything that you have written here (and, yes I do care about blue channel noise - I'm a very caring person) however I had understood that you would like to get some feedback from your photo-retouching "guinea-pigs". So...

- The problem that we have to deal with here, and in various other threads, is not channel noise but instead the interference of the texture of the paper used for the original prints when these are scanned. (As Camaraken pointed out, this is a problem only for scanning)
- In medical / scientific / astronomic images the attention to individual channels may be fundamentel, for photos to be viewed by human eyes the colour information is only secondary. (vide JPEG, colour TV..)

You had asked earlier about which tools would be useful for restorers / retouchers - my first thoughts would be: an easy-to-use FFT filter (because this texture problem comes up very often) and the deconvolution - focus filter. But as very few folks around here are maths geeks, the most important part would be a friendly user interface.

Rô

[jcr6]

I guess I am a math geek then.

2 weeks ago I kuldnt spel injeneer. Now I are 1.

You're right that in this case, there does not appear to be a color refringence problem at this scale, so the luminance trick is a good one.

It turns out that patterns come from a number of sources, including your camera (and scanner!) chip. So how do we solve the larger problem?

[byRo]

First my apologies to Camaraken for the hi-jack. Seems like everytime FFT comes up there's a lot to talk about.

A question for Chris..
I've seen in other places that the FFT can only be applied when the image is exactly sized to multiples of two (as in FoveaPro, too). However, the Alex Chirakov version doesn't have that restriction. Is he being fiendishly clever - or is he cheating??

Rô

[jcr6]

Sorry, Cameraken -- we're just having too much fun at your expense

The first Fourier transforms were DFT's - Discrete Fourier Transforms and they would work with any size data set. They were also slow. Then along came the FFT, the Fast Fourier Transform that relied upon a major simplification in the math but also required that the dataset be a power-of-two in size. Then along came a further observation that you could use any prime factor (not just 2), so your dataset could be a multiple of any collection of prime numbers. (Effectively each of these prime stages is a DFT -- larger ones are very slow.)

FFTW and a number of other algorithms use prime factorization and Fourier decomposition in order to achieve speed. If your image size is a nice set of multiples (2's, 3's, 5's, ...) the code will run very fast. If your image is prime in both dimensions or a multiple of large prime numbers, prepare yourself for a long wait.

There are number of other solutions for mean padding, zero padding, sine weighted interpolation, etc. for inserting a small image into a larger one that is a power-of-two in order to take benefit of the power-of-two performance speed up. My FFT is actually a split radix 2,4,8 and will run very fast on both Mac and PC hardware. For 16-bit images it even uses double precision. And, finally, I don't have to license it from anybody and it doesn't fall under the GNU/GPL license. There are some that do have restrictions. I hope that RGB FFT isn't one of those.

Ultimately, I was lazy when I chose the power-of-two requirement. I didn't want to deal with the arbitrary size where 4097 (17x241) might be very slow and 4096 (2^12) would be very fast. Also, in an odd-width FFT, it isn't exactly clear which position the DC term will be placed in. So, I've got a lot of issues.

There is a lot of history for dealing with power-of-two dimensions, so I chose the more conservative route. (Like I said, I'm a LAZY programmer.)

-Chris

[Gary Richardson]

Thanks Cameraken and Ro. Suspected it was some sort of compression system, but didn't know which, shouldn't have any problems now.

Spoke too fast, went to the site, and for some reason had trouble downloading, guess I'll try again later.

Wonder why they used such an obscure compression system to compress such a useful tool in.

[byRo]

Thanks Chris, a very clear explication.

I tried the Alex Chirakov filter on a 1024*2048 canvas filled with noise, and a 1023*2047 one too.

Didn't take any longer, but the result was quite different - the first didn't have any central lines at all, but the second did have strong lines.

Rô

[Cameraken]

ByRo and jcr6
No need to apologise. You are very welcome. Although I must admit you are starting to go over my head a bit.
As I mentioned in an earlier post All I look for is a tool that works and clear instructions on how to use it.

My problems started when I upgraded from the RGB > HSL method (which worked) to the “new” method.

byRo has now provided clear instructions and I think I’m happy again.

I have (Hopefully) taken most of the comment/problems and condensed them to a mini tutorial.
Please let me know what you think of my next posting


Gary
Rar files are very common. In fact they can have a better compression rate than zip files and are very often accompanied with a recovery record which can help repair a corrupted file. I would keep Winrar. You will need it again.
I have had a browser pointed at their web site all night without problem. If you’re having trouble I have added my AIM to my profile. Let me know.

Ken

[Cameraken]

Texture Removal
How to remove the paper pattern from a scanned image

Using Alex Chirakov filter RGB FFT

The Filter may be downloaded Here
http://www.pages.drexel.edu/~avc25/a...April_2005.rar

You will need WinRar to open the compressed ".rar" file
It's shareware, but doesn't stop working after the time limit
Download Winrar Here
http://www.rarlab.com/download.htm

When you've got it open in WinRar, open the "bin" folder and extract the two ".8bf" filters
to your PS filters folder (typically C:\Program Files\Adobe\Photoshop 7.0\Plug-Ins\Filters\)

Restart Photoshop and FFT RGB, and IFFT RGB should appear in the Filters list under the category "Fourier Transform"

That’s it for the Install - Now For its Use



1) Duplicate original image, right click on title bar and click Duplicate (that's a new image, not just a layer):
2) On the new image, get Luminosity (New Layer (<Ctrl><J>), Edit>Fill - 50% grey, color, 100%.);
3) Flatten all, <Alt><L><F> (FFT does not understand layers);
4) Run FFT RGB;
5) Clone out stars in the Red Channel;
6) Select all channels again (easy to forget this bit), and run IFFT RGB;
7) Slide this layer on top of the original image (with <Shift> pressed, to align);
8) Set this new layer's mode to Luminosity.

That’s It.

Follow these steps carefully and they become easy after a couple of times
If you are not sure what to clone out then follow the steps but just clone out some of the stars
You can always go back to your luminosity image and clone some more out to build up a stack of layers above the original. Then choose which to keep or Discard

Points to watch
Step 2) colour is set in the edit > Fill box under Blending
Step 5) Don’t touch the big star in the centre

The attached picture numbers correspond to the step numbers

[Cameraken]

Continued 1

[Cameraken]

Continued 2

[byRo]

Nice work, Ken.
Like I say (or maybe it was Doug) - the best way to learn something is to write a tutorial about it.

Rô

[Duv]

Mmmm. I'm finding with the original posted image that I'm getting texture showing thru from the base layer. Luminosity looked pretty clean. Is the texture too strong or something else I can do?

Dave

[Gary Richardson]

Hi Ken, still having problems getting Winrar to download. Can get the home page OK, download page OK, but can't get the Winrar file to download. Browser just times out. Have tried using both Firefox and IE.

[Cameraken]

Gary
Email me camerashop@aol.com
I'll send you the extractec filters
Ken

[krishna]

Hi,

Though the technicalities of FFT is beyond my comprehension, I would like to mention two softwares which I found useful for editing out the unwanted patterns/textures; Astra Image and ImageJ. FFT editing can done only in grayscale mode. The interface is similar to what Cameraken has attached above.

Two samples of fft edited picture is also being attached. The first one has been done in AstraImage with plain FFT editing. The second, done in ImageJ has gone through adjusting of Brightness/Contrast and applying of Median filter at a setting of 2.

--
Krishna

[Cameraken]

Duv
I agree with you I can’t quite get results as good as jcr6. Perhaps this is the difference between 0$ and 799$
As I said right at the start I thought jcr6 provided the best results
Floras has obviously been manipulated so its hard to compare.

Perhaps someone will have the time to compare FFT RGB with FoveaPro FFT
We may have to rely on byRo and jcr6 for the answers.

As I pointed out this picture is a tiny portion of a much larger image and it was given to me rolled up. I suppose that an invisible crease or stretch in the paper could very easily spoil the mathematics of removing a fixed pattern.

We used to mount a lot of our studio photographs using Ademco dry mounting tissue and an Ademco press. We used to use a tacking iron to tack the photo into place before placing it in the press. I am sure that this process would now cause a problem for FFT. As there will be parts of the print with a slightly different texture


Ken

[jcr6]

Rô:

1023 (presumably the horizontal direction) is 3 x 11 x 31, but that would run at nearly full speed since a whole line should fit in the L1 cache on your computer. 2047 is 23 x 89, but there is already a large delay in vertical accesses through the image, so you might not see a delay that is significantly worse than memory accesses already are. (Isn't caching fun!) Perhaps we should do benchmarks and compare.

Low Pass and High Pass Filters

I need to read Alex's document, but here are a couple of quick tricks you can try:

1) After performing the FFT, make a ROUND selection starting at the center.
2) Feather the selection. (try it with a large number and with a small number)
3) Press the Delete key. (This will make a high pass filter because we've just deleted the low frequencies)
4) Perform the inverse FFT

-or-

1) as before
2) as before (you might try a larger feather radius for this test)
3) Select Inverse (now the high frequencies are selected, not the lows in the center)
4) Press the Delete key. (This will make a low pass filter because we've just wiped out the high frequencies.)
5) Perform the inverse FFT

Notes: You can also try going into the QuickMask and making a radial gradient from the center as a method of proportionately selecting frequencies. If you have one that goes from 0 (not selected) in the center to 255 (fully selected) at the outer edge, you've created something very much like an "Ideal Inverse" and that can be useful for contrast adjustment.

You might also try using the Adjust->Levels function on a selection to increase/decrease the relative strength of a selection. By using Delete or Levels, you only affect the magnitude of a frequency. I would recommend AGAINST using the clone tool because then you're remapping stuff from another frequency and probably making extra junk in your image.

Time to read Alex's notes...

Yup.. He's using FFTW. There are license issues with that. I can't sell a product based on it. It is, however, extremely fast even with non-power-of-two dimensions.

Also, because the channels are only 8-bit (unless you're specifically choosing 16-bit) your image will get noiser just from the FFT->IFFT transform. Try it.

[jcr6]

One of the things that we're going to have to decide here is what should be in a package that does FFTs. I don't want you guys to be paying $800 for something that you're going to use maybe 5 filters out of 177. That isn't cool. I would rather wrap some nice UI around the kinds of functions that you will do and make it part of Optipix ?4? ($150, but *discounted* to all of you.)

I do have ideas along those lines, but am reluctant to share them yet because they will influence your thinking and I'd like to see what y'all come up with.

I will also provide some FoveaPro serial numbers to this group because we cannot get very far in the discussion once your 3-week demo period has run out.

[byRo]

Originally Posted by jcr6

I would recommend AGAINST using the clone tool because then you're remapping stuff from another frequency and probably making extra junk in your image.

This part always had me a bit worried, I've always done it this way, but I'm not easy about it.

I did think once that I should fill with black, then I thunk again that this would be creating new noise - seeing as the original un-textured image would not have little black spots all over the FFT image. Maybe 50% grey? Dunno. In the end cloning (with blending set to Darken) seems the easiest way out.

In practical terms, again, FFT won't usually take out all the texture, and as Camaraken pointed out a little random fluctuation could throw off the FFT completely. So, after going through FFT an image will almost certainly need some more touching up.

What I have recommended has been to take out the worst of the texture with FFT and then use some more conventional techniques (as beautifully explained in Flora's tutorial) to finish up.

- Didn't know that 1023 factored easily! Oops!!
- I had tried manipulating the FFT space before, but always ended up with a semi-transparent gooey mess. Will try again.
- Is that really a ROUND selection? I had imagined that it would have to be oval, in the proportions of the canvas.

Rô
PS Now you've put up a photo, people are going to think we're brothers!

[jcr6]

Black means zero -- that there is no power to that frequency or that that particular sine wave should not be present. You should erase to black. 50% gray would be a pretty significant amplitude for that frequency. You'd essentially be ADDING a pattern.

Since I'm a mac person, I haven't tried Alex's plugins. I presume that they don't work in 16-bits? That's unfortunate because you loose a lot of (useful/important) precision by only working in 8-.

One of the reasons that you can get into trouble in Fourier space is just that gooey mess you've been talking about. The ONLY safe operation that you can perform by hand on a power spectrum is to change the amplitude (AND NOT THE PHASE) of a frequency. This means lighten or darken. I suspect that a better workflow would be:

1) Do the FFT
2) Go into QuickMask
3) Mark the spikes (stars, whatever you call them) with a paintbrush in the mask -- use a fairly large brush, one that is big enough to completely remove the spike and a little bit of the neighborhood. Also use a brush with soft edges. A hard edge will cause ringing artifacts.
4) exit QuickMask
5) Use Levels to reduce the value to black of the spikes. This will preserve the color information (e.g. the Phase)
6) Do the iFFT

-Chris

P.S. We have a lot of bare forehead between us. -- Should I make my image be B&W?

[Cameraken]

Krisna
Thanks for the suggestions. It took some finding but I found them
Astra Image
This is available here
http://www.phasespace.com.au/download.htm
This Program can be used Free of charge but with save disabled until you have paid $AU49.95 for it

The next Program looks much more promising.
Image J It is part of The Image Processing Toolkit
Available Here for Free
http://rsb.info.nih.gov/ij/download.html
ImageJ runs on Linux, Mac OS 9, Mac OS X and Windows
The Docs are here
http://rsb.info.nih.gov/ij/docs/
And the bit about FFT is here
http://rsb.info.nih.gov/ij/docs/menus/process.html#fft

It is Not a plugin but a complete program

Just one problem Krishna I can’t get it to work
I loaded my image
Process > FFT > FFT
Painted out the stars with the Paintbrush tool in Black
Process > FFT > InverseFFT

I suppose I could have a Bad copy but the results came back exactly the same as they went in.

Krishna. Does that sound OK.
If it does I’ll try downloading it again

Ken

[krishna]

Originally Posted by Cameraken

Thanks for the suggestions. It took some finding but I found them
Astra Image
Ken

I find this software extremely easy to edit the FFT. Moreover the algorithms this programme uses for re-focusing blurred pictures works excellent, sometimes.

Originally Posted by Cameraken

Just one problem Krishna I can’t get it to work
I loaded my image
Process > FFT > FFT
Painted out the stars with the Paintbrush tool in Black
Process > FFT > InverseFFT

I suppose I could have a Bad copy but the results came back exactly the same as they went in.

Krishna. Does that sound OK.
If it does I’ll try downloading it again

Ken

Ken, I am surprised at this result. First I thought may be the FFT is not working because of you doing it in RGB mode. So far I have been doing it in grayscale mode. Today, thanks to you, I tried on the colour version of the figure and it worked. The method you used is correct but I am unable to say what is going wrong. Could you check the version of Java running on your machine?

--
thanks
Krishna

[Duv]

Originally Posted by jcr6

Black means zero -- that there is no power to that frequency or that that particular sine wave should not be present. You should erase to black. 50% gray would be a pretty significant amplitude for that frequency. You'd essentially be ADDING a pattern.

I made the attached on Friday and discounted it because of the banding around the edges but given the recent discussions concerning cloning, erasing to black, et al, maybe there's some value. Apart from the outside banding the image to me looks as good or better than Chris's.
All I did was use the Lasso around each of the "stars", deleted, then ran Ifft.
Is there a simple reason to explain the perimeter banding?

Dave

[Caitlin]

Originally Posted by Duv

All I did was use the Lasso around each of the "stars", deleted, then ran Ifft.

Dave, when you say 'delete' what do you mean exactly?

[jcr6]

Duv: The problem with the lasso method is that you have a sharp cutoff of where there is data to where there isn't. If you feathered your selection (by, say 2 pixels) before you deleted, and made the selection correspondingly larger, I'd say you've got a good method.

That problem you're seeing is well known. Very well known. This is why in mine when people provide pass masks (essentially a quickmask), we "suggest" that they blur them first. This smooths out the step function in frequency space, which in turn prevents ringing.

This is also why I suggested painting in the Quickmask with a soft brush -- you get soft edges to the holes in the frequency map.

[Cameraken]

Krishna

I uninstalled ij134-jdk15-setup.exe and downloaded ij133-setup.exe now its working fine.
Astral Image needs a greyscale image but image J works in full colour.
Image J will remain on my PC as a quick easy option
Thanks again for the suggestion.
There also appears to be a lot of other useful tools included which I’ll look at later.

Jcr6. You should look at image J. It may be of interest (and it works on a mac as its Java)

Byro
Image J paints out the stars in Black
Astral Image darkens them
Jcr6 recommends Black
It looks like Darker or Black and not cloning is the correct method

Another interesting point
Image J – when I give it my sample image (which is 322 X 442 pixels)
The FFT is 512 x 512 so it’s automatically taken it to a square power of 2

Try This
Image J
Load picture
Process > FFT >FFT
Now before painting And with FFT selected
Process > Enhance Contrast > Saturated Pixels 1%
Good Eh!
Now you see stars that were not visible before
Paint out all the stars except the big one in the middle
Process > FFT > InverseFFT

Ok the picture still isn’t perfect But I believe this is the best this version of FFT will do.
If anybody has got better then I think they have done further manipulation.

Does anyone want to discuss which version of FFT is best?
Byro. Is Alex Chirakov FFT version the best? Or is it the only one you found?

These pictures are a straight before and after with no additional manipulation

Ken

[Duv]

Ken, I still am seeing residual golfballs but not sure why cause I used a similar method. Thanks to Chris's suggestion, I 2 pixeled my Lasso and got these results. Still a bit of "Ringing" at top left but otherwise I'm a pretty happy camper. This was using FFT RGB. Still haven't figured out Chris's method. Well, going to work on some other texture problems.

Caitlin, sorry, when I selected a "star" I just hit the delete button and moved to the next.

Cheers
Dave

[Flora]

Been away a few days and missed vital information about FFT .... Gosh I have a lot to catch up now!!!

Originally Posted by Cameraken

Floras has obviously been manipulated so its hard to compare.

... If for 'manipulated' you intend run through FFT/IFFT ... yes .. but no other kind of 'manipulation was done in the middle (grey) picture attached to my previous post.
I've only used the 'old' HSB Alex Chirokov FFT plug-in for Photoshop to get that result .... for what I can see it has become 'old fashioned' already but, while I read all this new informations, here is what my HSB channels look like ...

First I 'healed' out the stars in the Blue Channel (figure1), then in the Green Channel (figure 2) and finally I 'cleaned' the 'composite' RGB (figure 3).

One last thing .... I found that the 'banding around the edges' appears whenever the the cross' arms are involved in the 'cleaning' ...

[Cameraken]

The fast Fourier transform (FFT) is a discrete Fourier transform algorithm which reduces the number of computations needed for N points From 2N^2 to 2NlgN…….
Bla Bla Bla.
Our tests performed on on a variety of platforms, show that FFTW's performance is typically superior to that of other publicly available FFT software, and is even competitive with vendor-tuned codes. In contrast to vendor-tuned codes, however, FFTW'………..
Bla Bla Bla.
Hence the name, "FFTW," which stands for the somewhat whimsical title of "Fastest Fourier Transform in the West."
http://www.fftw.org/

OK FFTW is meant to be the best so lets see if we can improve on my last post.

Duv
Your first post was better. We’re going backwards. Have you upgraded to FFT RGB recently? You have missed a couple of stars on the horizontal line but I’m beginning to think there is something fundamentally wrong with byro’s steps. And which I put in my mini tutorial.

Using Alex Chirokov FFT RGB
I very carefully painted out all the stars with a soft brush on a layer mask, changed it back to a selection and used levels to darken to black. The results were not as good as I just got from Image J.
As I said at the beginning my problems started when I ‘Upgraded’ from FFT HSL to FFT RGB.

Flora
Hi, and my apologies. In your picture you had removed the stains etc so I assumed you may have cleaned it further.

If that’s the case then We Are doing something fundamentally wrong Or FFT RGB is not working as well as FFT HSL.

Ken