Lightroom: How exactly does Lightroom CC merge DNG files?

[Doug Nelson]

Andrew Rodney corrected me in another thread for my faux pas in my recent Lightroom CC RetouchPRO LIVE show.

I said in the show that Lightroom CC generates DNG files from HDR Merge and Panorama Merge and that was fabulous because you basically had this big honking Raw file to work with.

Andrew pointed out that I was almost correct in that it does generate DNG files but they were not precisely the same data as multiple Raw files embedded in a wrapper. It was something something linear something.

So what is it precisely you get in a merged DNG from Lightroom? Is it some of the Raw data (and which part) or is it pixels like in a PSD, or something else?

Feel free to geek out on me, as I really want to know precisely what is in that merged DNG and how it differs from the original Raw data.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

Additionally, how is it better/worse than a 16bit TIFF?

[Doug Nelson]

Re: How exactly does Lightroom CC merge DNG files?

Some empirical observations:

I'm experimenting with the Raw files Steve used in the show (doing some timing tests to compare my no-OpenCL video card times with his modern card). I found that his original Raw files were 19.7MB each. 3 HDR merged files made a DNG that was 78.6MB. However, when I chose Export, the new DNG was 40.6MB.

Additionally, I noticed an option in the export prefs to include original Raw data, though I don't know if that's applicable to HDR Merge.

So perhaps Lightroom CC has 2 ways of handling merged Raw files: one way for the native file creation and another way for exporting? Could everyone be correct under different circumstances here?

[andrewrodney]

Re: How exactly does Lightroom CC merge DNG files?

DNG like TIFF is a flexible container. It's based on TIFF of which Adobe owns both. You can have raw data, partiality rendered or fully rendered data within a DNG. You can save a JPEG as a DNG in Lightroom, that isn't raw data in there.

Pano (and HDR) converts a raw image to Linear DNG (which is kind of raw) and then does the DNG Pano blend. This blend is 16-bit Linear DNG.

I don't know if any of this is different than older versions of ACR which supported this before LR6.

[Doug Nelson]

Re: How exactly does Lightroom CC merge DNG files?

I tried the following experiment and it failed. But it was still interesting and might be fruitful with some option changes in Lightroom.

The Adobe DNG Converter has an extraction function. I have 2 DNG files, one of which I explicitly told to embed the original Raw data.

I ran the extractor on both files and came back "no files were extracted".

More fiddling required. Feel free to experiment on your own and report back any positive results.

[andrewrodney]

Re: How exactly does Lightroom CC merge DNG files?

That function is to extract the original proprietary raw file IF you set the DNG to embed it when converting initially.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

Andrew, in a practical sense, is there an advantage to a linearized DNG over a 16bit TIF?

The TIF is more flexible, can be used in/by more programs, so what advantage might the Lin DNG confer to make it worthwhile?

[andrewrodney]

Re: How exactly does Lightroom CC merge DNG files?

I don't know how to get linearized from raw into a TIFF. I'm sure it can be done. Getting linearized DNG is just a setting. I don't think it matters. It's the data that is important, not the container. TIFF and DNG are cousins. This would be a good question for Chris Cox or some one inside Adobe.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

I don't understand Linearized DNG, and I guess I'm wondering if I need to or want to. I just hear it's kinda closer to raw than a baked-in file, but not all the way raw. And I'm wondering how useful this nuanced difference between a Lin DNG and high bit TIF is.

As an example in a workflow, I was excited to see that Helicon Focus can take raw images (in my case a focus stack) and render them as a merged DNG. I was excited by this, thinking I could then take this kinda raw merged DNG into C1 for processing, assuming C1 would surely work on the "wonderful" open source raw format, but it does not. To do what I want instead requires outputting a high bit TIF to work on in C1.

That C1 can process a TIF is an advantage to the TIF. What I'm wondering is what I'm losing by using a TIF vs a DNG. What can the Lin DNG do (be it C1 or elsewhere) that the high bit TIF can't?

If the answer is "nothing" I need not be concerned with that format at all as it would yield only negatives and no positives.

[andrewrodney]

Re: How exactly does Lightroom CC merge DNG files?

Again, what do you mean by High bit TIFF. The Linear DNG data is also high bit. It's scene referred not output referred, there's no white balance applied AFAIK (that's the bit about being more raw). You could put exactly the same kind of data in a TIFF. TIFF and DNG are containers, what's inside is what you want to know about.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

Andy, if what's inside of it is so arcane that even the likes of you and Schewe, etc, barely know what it is, I'm thinking I don't need to know either.

I'm all about getting the work done. If there's indeterminate/minimal benefit to the internals, and negative benefit to the packaging, I'm over it. I'll be sticking with TIFs.

I appreciate the dialog, though. Thanks.

[andrewrodney]

Re: How exactly does Lightroom CC merge DNG files?

Andy, if what's inside of it is so arcane that even the likes of you and Schewe, etc, barely know what it is, I'm thinking I don't need to know either.

It's not at all arcane! It's linear scene referred data without WB applied. It's not as raw as the original DNG's or the proprietary raws prior to the merge. It's partially processed. DNG doesn't necessarily equal raw! Neither does TIFF. There's nothing arcane about it and in fact, now that you know the differences using HDR/Pano merge upon the original data, it should be even less arcane if you actually think about it.

I'll be sticking with TIFs.

TIFF is again just a container for rendered data. Saying you're sticking with TIFF means you're not doing any HDR/Pano merges which is fine with me. The linear scene referred data could just as easily be saved in the TIFF container but it's stored in the DNG container, so what? Makes zero difference.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

It's not at all arcane! It's linear scene referred data without WB applied. It's not as raw as the original DNG's or the proprietary raws prior to the merge. It's partially processed. DNG doesn't necessarily equal raw! Neither does TIFF. There's nothing arcane about it and in fact, now that you know the differences using HDR/Pano merge upon the original data, it should be even less arcane if you actually think about it.

I get you. Saying it was a good question for Chris Cox threw me, but I guess you thought I was asking something else.

TIFF is again just a container for rendered data. Saying you're sticking with TIFF means you're not doing any HDR/Pano merges which is fine with me. The linear scene referred data could just as easily be saved in the TIFF container but it's stored in the DNG container, so what? Makes zero difference.

I don't expect you or anyone else to care what I do. I simply meant that once I understood a linear DNG to be of limited utility outside of the Adobe ecosystem we didn't have to pursue the details within it ad nauseum.

Listen, don't sweat it. I'm just frustrated with Adobe's raw processing engine and looking for a more elegant workflow that might bridge or transcend other processors. Thought this linear DNG might be it, but it's not. I'll figure something else out.

[klev]

Re: How exactly does Lightroom CC merge DNG files?

Listen, don't sweat it. I'm just frustrated with Adobe's raw processing engine and looking for a more elegant workflow that might bridge or transcend other processors. Thought this linear DNG might be it, but it's not. I'll figure something else out.

I've actually been looking at darktable. If I get the time I might even try to contribute to it. It's still a bit unstable on OSX.

[Flashtones]

Re: How exactly does Lightroom CC merge DNG files?

I've actually been looking at darktable. If I get the time I might even try to contribute to it. It's still a bit unstable on OSX.

That would be good of you. I'm just looking at it now, on your recommendation. Pretty rich feature set. I'll certainly give it a spin.