Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...help!

[timmonson]

I am getting a Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii. The corruption only seems to happen at the point that I save the file on my backup drive after processing in Lightroom...here is the order that I process my photos to give some background:

1. Shot in RAW in Camera and saved to various SanDisk and Transcend CF cards (30MB/s, 133x, 60MB/s) (no corruption yet)

2. Transferred via Transcent CF Card reader (TS-RDP8K) to Macbook file folder. (no corruption yet)

3. Copied files are then added into Lightroom 5 (without moving) after creating a new Catalog (no corruption yet)

4. Photos go through my standard Lightroom editing process (some photos I take into Photoshop and edit them there, some I do not...this leaves CR2's mixed with PSD files in the original file folder (no corruption yet)

5. I then copy the finished Catalog to an external harddrive (USB 3.0 WD My **bleep** Ultra 2 TB) (this is where the corruption finally shows up...the strange thing is that it is hit and miss...some Catalogs get corrupted while others are perfectly fine...the corruption only happens to the CR2 files not the PSD's)

6. My original thought was that this was being caused due to a bad cable, which could still be the issue but I've witnessed this happen on 2 different usb 3.0 cables. Perhaps the issue is tied in with the 3.0 USB? (I have also attached a screenshot after importing to Photomatix to show that the problem isn't just in the preview image but at the actual RAW CR2 level)

I have hundreds of garbled/corrupted CR2 files in my archive now with no apparent way to recover them...I need some help from someone! I haven't been able to find any other occurances of this online so any help or feedback would be greatly appreciated. Thanks!

Oh, for any CR2 experts, here is a link to download my original CR2 file: https://www.dropbox.com/s/nsfivvgrmyloa90/IMG_3440.CR2

[RobertTheFat]

---

@timmonson wrote:

I am getting a Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii. The corruption only seems to happen at the point that I save the file on my backup drive after processing in Lightroom...here is the order that I process my photos to give some background:

1. Shot in RAW in Camera and saved to various SanDisk and Transcend CF cards (30MB/s, 133x, 60MB/s) (no corruption yet)

2. Transferred via Transcent CF Card reader (TS-RDP8K) to Macbook file folder. (no corruption yet)

3. Copied files are then added into Lightroom 5 (without moving) after creating a new Catalog (no corruption yet)

4. Photos go through my standard Lightroom editing process (some photos I take into Photoshop and edit them there, some I do not...this leaves CR2's mixed with PSD files in the original file folder (no corruption yet)

5. I then copy the finished Catalog to an external harddrive (USB 3.0 WD My **bleep** Ultra 2 TB) (this is where the corruption finally shows up...the strange thing is that it is hit and miss...some Catalogs get corrupted while others are perfectly fine...the corruption only happens to the CR2 files not the PSD's)

6. My original thought was that this was being caused due to a bad cable, which could still be the issue but I've witnessed this happen on 2 different usb 3.0 cables. Perhaps the issue is tied in with the 3.0 USB? (I have also attached a screenshot after importing to Photomatix to show that the problem isn't just in the preview image but at the actual RAW CR2 level)

I have hundreds of garbled/corrupted CR2 files in my archive now with no apparent way to recover them...I need some help from someone! I haven't been able to find any other occurances of this online so any help or feedback would be greatly appreciated. Thanks!

Oh, for any CR2 experts, here is a link to download my original CR2 file: https://www.dropbox.com/s/nsfivvgrmyloa90/IMG_3440.CR2

---

A Lightroom catalog is a database, isn't it? Databases are apt to contain relative references and position-dependent pointers and can't necessarily be simply copied to another drive. When you say "copy", are you actually using an approved Lightroom export utility, or are you just copying the database files at the operating system level? It's also possible (although somewhat unlikely, one hopes) that Lightroom is appending data to the CR2 files that makes them unreadable by any program other than Lightroom. In that case a Lightroom-specific program to remove the extra data would have to be part of the export process.

I don't know anything about Lightroom, so I could be all wet. But I do know something about computers and a little about databases, which is why I dare to comment.

[timmonson]

Hey Robert, thanks for the reply. I am simply using finder to drag and drop copy the Lightroom Catalog file folder to the external hard drive. All of my RAW images are contained inside the database folder so the paths don't mess up. This has worked for me for years until I have gotten this new WD drive for Christmas...for some reason I have a hunch that it has to do with the External Drive itself. I have reformatted it multiple times trying Mac OS Extended and MS DOS fat formats and the issue still comes up over and over again.

Maybe if I export a new catalog to the new drive that may fix it if the issue you mention is the cause...I'm simply stumped since what I'm doing is the exact thing that backup software would do with the catalog...no?

[RobertTheFat]

---

@timmonson wrote:

Hey Robert, thanks for the reply. I am simply using finder to drag and drop copy the Lightroom Catalog file folder to the external hard drive. All of my RAW images are contained inside the database folder so the paths don't mess up. This has worked for me for years until I have gotten this new WD drive for Christmas...for some reason I have a hunch that it has to do with the External Drive itself. I have reformatted it multiple times trying Mac OS Extended and MS DOS fat formats and the issue still comes up over and over again.

Maybe if I export a new catalog to the new drive that may fix it if the issue you mention is the cause...I'm simply stumped since what I'm doing is the exact thing that backup software would do with the catalog...no?

---

Usually no. If, say, you were going to back up a SQL database with NetBackup, you'd have to use NetBackup's SQL-specific client-side software on the SQL server. Alternatively, you could do a SQL backup into a flat file, then use the regular NetBackup client-side software to pass that file along to the backup server. But you can't just take the raw database files and send them to the backup server and expect them to be usable to restore the database to its proper state.

In this case you may be right about the problem being in the external drive. Probably you should start by making sure that your new backup drive is formatted exactly as your old one was.

[Darren3651]

Please see my latest post. It is not a Lightroom problem, it is the Canon Codec that is corrupting images randomly.

[TCampbell]

Timmonson... 

I am concerned that the corruption is original to the file and not the result of processing in LIghtroom.  

I followed your link to the Dropbox image, downloaded this, and then opened it using Canon's DPP (Digital Photo Professional).  DPP is Canon's supplied software which can process RAW images.

My question is:  Is this the original "straight out of the camera" copy of the file... or is this what happens to the file AFTER it's been copied to the external drive in step 5?

It's important to recognize that when you shoot a RAW image, there's the "real" image... and then there's also an embedded "thumbnail" or "preview" image.  

When many programs open a RAW, the instantly display the thumbnail/preview image on screen while they continue to read and decode the RAW file.  When the RAW file is loaded, it replaces the preview version on the screen.

When I open your image, I see the non-corrupt image... only for that brief second or two... and then it's replaced whith an image that looks like the corrupt image you've included in your post below.

So... if the image you placed on Dropbox is indeed an original "straight ouf of the camera" image (and was never written to the suspect drive)... then it is corrupt.

My first inclination would be to (a) visually inspect the CF card slot in the camera using a light to inspect the pins at the back of your card slot for signs of damage.  Sometimes it's possible to get a bent pin.  I *suspect* the camera would have reported an error if there had been a bent pin... but as I'm not an engineer working for Canon, I don't know how much testing the camera does before accepting that a card is inserted.

Also, (b) test the camera using a different (and preferably new) CF card which has been formatted by the camera with the "low level" option box enabled.  

[ebiggs1]

It certainly looks like a corrupt card. I would get a brand new top quality card and try everything again.  I doubt seriously it is the hard drive.

Could be your card reader, too.

[Darren3651]

No, it is 100% not a card problem

[Darren3651]

Please see my post below, this is an increasing problem and a bug in the RAW decoding codec.

[Darren3651]

This is not a Lightroom problem OR a memory problem. This is a Canon DPP problem! Both versions 3 and 4 since the update are corrupting images. I have experience of this on 3 different computers with 2 different 5D mkII's and 6 different memory cards.

The only work-around I currently have is to make a complete copy of your raw files before you even start DPP or Lightroom. Then only open your copy files. If you experience corruption replace that corrupt file with your original which will be OK.

Canon really needs to sort this problem out, not Adobe, not Microsoft!