topic Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he in EOS DSLR & Mirrorless Cameras

Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...help!

timmonson — Tue, 29 Jul 2014 18:13:16 GMT

I am getting a Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii. The corruption only seems to happen at the point that I save the file on my backup drive after processing in Lightroom...here is the order that I process my photos to give some background:

1. Shot in RAW in Camera and saved to various SanDisk and Transcend CF cards (30MB/s, 133x, 60MB/s) (no corruption yet)

2. Transferred via Transcent CF Card reader (TS-RDP8K) to Macbook file folder. (no corruption yet)

3. Copied files are then added into Lightroom 5 (without moving) after creating a new Catalog (no corruption yet)

4. Photos go through my standard Lightroom editing process (some photos I take into Photoshop and edit them there, some I do not...this leaves CR2's mixed with PSD files in the original file folder (no corruption yet)

5. I then copy the finished Catalog to an external harddrive (USB 3.0 WD My **bleep** Ultra 2 TB) (this is where the corruption finally shows up...the strange thing is that it is hit and miss...some Catalogs get corrupted while others are perfectly fine...the corruption only happens to the CR2 files not the PSD's)

6. My original thought was that this was being caused due to a bad cable, which could still be the issue but I've witnessed this happen on 2 different usb 3.0 cables. Perhaps the issue is tied in with the 3.0 USB? (I have also attached a screenshot after importing to Photomatix to show that the problem isn't just in the preview image but at the actual RAW CR2 level)

I have hundreds of garbled/corrupted CR2 files in my archive now with no apparent way to recover them...I need some help from someone! I haven't been able to find any other occurances of this online so any help or feedback would be greatly appreciated. Thanks!

Oh, for any CR2 experts, here is a link to download my original CR2 file: https://www.dropbox.com/s/nsfivvgrmyloa90/IMG_3440.CR2

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

RobertTheFat — Tue, 29 Jul 2014 19:26:06 GMT

@timmonson wrote:

I am getting a Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii. The corruption only seems to happen at the point that I save the file on my backup drive after processing in Lightroom...here is the order that I process my photos to give some background:

1. Shot in RAW in Camera and saved to various SanDisk and Transcend CF cards (30MB/s, 133x, 60MB/s) (no corruption yet)
2. Transferred via Transcent CF Card reader (TS-RDP8K) to Macbook file folder. (no corruption yet)
3. Copied files are then added into Lightroom 5 (without moving) after creating a new Catalog (no corruption yet)
4. Photos go through my standard Lightroom editing process (some photos I take into Photoshop and edit them there, some I do not...this leaves CR2's mixed with PSD files in the original file folder (no corruption yet)
5. I then copy the finished Catalog to an external harddrive (USB 3.0 WD My **bleep** Ultra 2 TB) (this is where the corruption finally shows up...the strange thing is that it is hit and miss...some Catalogs get corrupted while others are perfectly fine...the corruption only happens to the CR2 files not the PSD's)
6. My original thought was that this was being caused due to a bad cable, which could still be the issue but I've witnessed this happen on 2 different usb 3.0 cables. Perhaps the issue is tied in with the 3.0 USB? (I have also attached a screenshot after importing to Photomatix to show that the problem isn't just in the preview image but at the actual RAW CR2 level)

I have hundreds of garbled/corrupted CR2 files in my archive now with no apparent way to recover them...I need some help from someone! I haven't been able to find any other occurances of this online so any help or feedback would be greatly appreciated. Thanks!

Oh, for any CR2 experts, here is a link to download my original CR2 file: https://www.dropbox.com/s/nsfivvgrmyloa90/IMG_3440.CR2

A Lightroom catalog is a database, isn't it? Databases are apt to contain relative references and position-dependent pointers and can't necessarily be simply copied to another drive. When you say "copy", are you actually using an approved Lightroom export utility, or are you just copying the database files at the operating system level? It's also possible (although somewhat unlikely, one hopes) that Lightroom is appending data to the CR2 files that makes them unreadable by any program other than Lightroom. In that case a Lightroom-specific program to remove the extra data would have to be part of the export process.

I don't know anything about Lightroom, so I could be all wet. But I do know something about computers and a little about databases, which is why I dare to comment.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

timmonson — Tue, 29 Jul 2014 19:42:57 GMT

Hey Robert, thanks for the reply. I am simply using finder to drag and drop copy the Lightroom Catalog file folder to the external hard drive. All of my RAW images are contained inside the database folder so the paths don't mess up. This has worked for me for years until I have gotten this new WD drive for Christmas...for some reason I have a hunch that it has to do with the External Drive itself. I have reformatted it multiple times trying Mac OS Extended and MS DOS fat formats and the issue still comes up over and over again.

Maybe if I export a new catalog to the new drive that may fix it if the issue you mention is the cause...I'm simply stumped since what I'm doing is the exact thing that backup software would do with the catalog...no?

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

RobertTheFat — Tue, 29 Jul 2014 20:01:15 GMT

@timmonson wrote:
Hey Robert, thanks for the reply. I am simply using finder to drag and drop copy the Lightroom Catalog file folder to the external hard drive. All of my RAW images are contained inside the database folder so the paths don't mess up. This has worked for me for years until I have gotten this new WD drive for Christmas...for some reason I have a hunch that it has to do with the External Drive itself. I have reformatted it multiple times trying Mac OS Extended and MS DOS fat formats and the issue still comes up over and over again.

Maybe if I export a new catalog to the new drive that may fix it if the issue you mention is the cause...I'm simply stumped since what I'm doing is the exact thing that backup software would do with the catalog...no?

Usually no. If, say, you were going to back up a SQL database with NetBackup, you'd have to use NetBackup's SQL-specific client-side software on the SQL server. Alternatively, you could do a SQL backup into a flat file, then use the regular NetBackup client-side software to pass that file along to the backup server. But you can't just take the raw database files and send them to the backup server and expect them to be usable to restore the database to its proper state.

In this case you may be right about the problem being in the external drive. Probably you should start by making sure that your new backup drive is formatted exactly as your old one was.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

TCampbell — Wed, 30 Jul 2014 00:03:48 GMT

Timmonson...

I am concerned that the corruption is original to the file and not the result of processing in LIghtroom.

I followed your link to the Dropbox image, downloaded this, and then opened it using Canon's DPP (Digital Photo Professional). DPP is Canon's supplied software which can process RAW images.

My question is: Is this the original "straight out of the camera" copy of the file... or is this what happens to the file AFTER it's been copied to the external drive in step 5?

It's important to recognize that when you shoot a RAW image, there's the "real" image... and then there's also an embedded "thumbnail" or "preview" image.

When many programs open a RAW, the instantly display the thumbnail/preview image on screen while they continue to read and decode the RAW file. When the RAW file is loaded, it replaces the preview version on the screen.

When I open your image, I see the non-corrupt image... only for that brief second or two... and then it's replaced whith an image that looks like the corrupt image you've included in your post below.

So... if the image you placed on Dropbox is indeed an original "straight ouf of the camera" image (and was never written to the suspect drive)... then it is corrupt.

My first inclination would be to (a) visually inspect the CF card slot in the camera using a light to inspect the pins at the back of your card slot for signs of damage. Sometimes it's possible to get a bent pin. I *suspect* the camera would have reported an error if there had been a bent pin... but as I'm not an engineer working for Canon, I don't know how much testing the camera does before accepting that a card is inserted.

Also, (b) test the camera using a different (and preferably new) CF card which has been formatted by the camera with the "low level" option box enabled.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

ebiggs1 — Wed, 30 Jul 2014 13:22:30 GMT

It certainly looks like a corrupt card. I would get a brand new top quality card and try everything again. I doubt seriously it is the hard drive.

Could be your card reader, too.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

Darren3651 — Mon, 20 Jul 2015 20:54:51 GMT

This is not a Lightroom problem OR a memory problem. This is a Canon DPP problem! Both versions 3 and 4 since the update are corrupting images. I have experience of this on 3 different computers with 2 different 5D mkII's and 6 different memory cards.

The only work-around I currently have is to make a complete copy of your raw files before you even start DPP or Lightroom. Then only open your copy files. If you experience corruption replace that corrupt file with your original which will be OK.

Canon really needs to sort this problem out, not Adobe, not Microsoft!

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

Darren3651 — Mon, 20 Jul 2015 20:56:00 GMT

No, it is 100% not a card problem

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

Darren3651 — Mon, 20 Jul 2015 20:56:58 GMT

Please see my post below, this is an increasing problem and a bug in the RAW decoding codec.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

Darren3651 — Mon, 20 Jul 2015 20:57:46 GMT

Please see my latest post. It is not a Lightroom problem, it is the Canon Codec that is corrupting images randomly.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

ebiggs1 — Mon, 20 Jul 2015 21:15:04 GMT

"Canon really needs to sort this problem out, not Adobe, not Microsoft!"

It could be a Mac problem. If you are on a Macbook, why would we think it a MS problem. I don't have a 5D Mk II any longer but I do use DPP4 and DPP 3 on a Win box latest update, and see no problem.

I have used LR since there was numbers.

If you have acess to a Windoze machine, try it on Win 8.1. I would bet it will work.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

RobertTheFat — Mon, 20 Jul 2015 21:27:19 GMT

@ebiggs1 wrote:
"Canon really needs to sort this problem out, not Adobe, not Microsoft!"

It could be a Mac problem. If you are on a Macbook, why would we think it a MS problem. I don't have a 5D Mk II any longer but I do use DPP4 and DPP 3 on a Win box latest update, and see no problem.
I have used LR since there was numbers.

If you have acess to a Windoze machine, try it on Win 8.1. I would bet it will work.

I'm hard-pressed to see it as a Mac problem. The Mac operating system is what it is, and the Canon software has to accept that. It's possible that the Mac's interface specs are deficient or that there's a bug in the Mac OS, but DPP should contain sufficient checks to allow it to maintain the integrity of its RAW files, no matter what the OS does. For example, it's OK if DPP can't manage to save an updated RAW file in a form the OS will accept, but not OK for it to save the file garbled.

But you're right; it can't be a Microsoft problem in this case.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

ebiggs1 — Tue, 21 Jul 2015 19:17:36 GMT

Never-the-less I will not be surprised if it is a Mac OS issue. No matter how it gets saved. I have wittnessed too many Mac/ software problems to be surprised to see another one.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

lensboard — Fri, 02 Oct 2015 02:18:32 GMT

I am familiar with this, and it has happened to me a few times. It has nothing to do with mac or windows. I own both and use both. Both have corrupted. I think it happens when dragging files via usb to another drive locally or on the network. Now I use rsync when moving weddings from my pc to my drobo on my mac, and have had no problem. If I just use the normal dragging method I get corruption (not every time). I think it has to do with the file transfer method. This has also happened with video, so it is not just canon raw files. I don't really know what is happening, but I always use rsync from mac to pc, and I always build previews after moving, and verify no corruption before deleting original.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

DuaneIndeed — Thu, 10 Dec 2015 18:19:26 GMT

See my other post: http://community.usa.canon.com/t5/EOS/5d-mk-III-RAW-issue/m-p/158284

This is a Canon Raw driver issue. It started after I updated DPP and the EOS Utility this morning. Now, DPP, Lightroom, and Photoshop are have the same issue!!! Canon, please fix!!!!!

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

joyohjoy — Mon, 14 Dec 2015 05:59:56 GMT

Greaat... So I'm googling stuff about corrupted files because I have RAW files that look exactly the same. It started last week and I was using a brand new card. I bought another brand new one and it did the same. The thumbnails and jpegs are fine. After a few minutes in LR, the RAWS start showing corrupt. I'm getting about 1% corrupt. My cards are high-dolla and brand schpankin' new. I'm going to call my computer girl tomorrow to ask her about this.

I shot an event last night with both my nikon and my canon. I just put my canon's CF card into my Pixel flash no-bend pins to usb adapter. As always, I selected all the files with ctrl+A copied with ctrl+C and pasted to my big external hard drive with ctrl+V. I then repeated this process to my second hard drive that is a bit smaller. ( I use an annual hard drive and then a huge one that holds several year's work on it.) I then synchronized the file for dec 2015 in LR. The thumb files looked fine. I worked in LR a few minutes just to do some caparisons to figure out which files I would pull from Canon vs my Nikon. They have different strengths, you know. Then my thumbnails started converting to show corruption. As it did last time, my jpegs are fine. None of the Nikon files are corrupted. Those were imported the same way, except they came from an SD card inserted directly into my laptop.

The thing that makes me wonder about it being the hard drive going bad on my computer is that I have tried opening a corrupted RAW file in PS, (I knew it was corrupted from the LR view). and I tried opening it two ways: One from the annual hard drive. The other from my big hard drive. They are both corrupt and the blocky pattern is exactly the same.

It just seems to me, (a non-computer person) that if it was a hard drive issue, the other file wouldn't be corrupt the same way. It would be more random? But I'm not up on all that stuff.

Here's another clue... I just tried putting my CF card into a brand new pixel flash card reader and plugged it directly into my laptop. I went to ps and opened the same file, (directly this time. I didn't copy it or save it.) that was corrupted before and it is just fine. Things that make you go..hummm... so maybe my old pixel flash card reader is buggy and I need to toss it?

I also just put the new pixel flash into my usb port expander that I used the first time. I put my CF card in it. I did the copy and paste thing and created a new file called "dec 2015 test". I opened that in LR and the file that should be corrupted, isn't.

I should also add, I have no idea what people are talking about when they say they have updated Canon DPP or whatever else. I've never updated anything.

Please share your thoughts! 🙂 Thank you! 🙂

Update: I just checked amazon's reviews on the "pixel flash usb 3.0 no-bend pins" and someone has posted a negative review with a picture of a corrupt picture file, (looks similar to mine). The customer is complaining that it was a card reader issue.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

TCampbell — Mon, 14 Dec 2015 16:01:03 GMT

When the camera saves a "RAW" image, there are two images... one is a "preview" image (it's actually a JPEG embedded within the RAW" and then of course there is the RAW data itself.

When the camera takes the image, what it initially gets is the RAW data (that's the data constructed by doing a read-out from the imaging sensor) and it will use this data to create the jpeg PREVIEW (if you are saving a RAW file) or it will use this data to create the JPEG if you are saving a JPEG (and JPEGs get the extra post-processing steps of setting things like white balance, picture style, etc.) Ultimately all this gets written out to the memory card.

That means it is possible to that if there's a problem writing to the card, then the part of the file that contains the JPEG preview might not be corrupt, but if the RAW file could be corrupt elsewhere. This would explain why an image opens and is fine... but only for a moment... and then shows corruption.

Duane's problem is different. Duanne almost certainly has a filesystem issue (possibly a failing drive). This is because in Duanne's situation... he had images shot some time back in history and those images were imported from the camera and were fine... remained on the filesystem... and then at a later date the images ultimately became corrupted (that's not a camera issue.) (I personally had the same issue Duanne describes and my issue was, in fact, a failing hard drive.)

I've already sorted out (in the other thread) that Adobe doesn't use Canon's RAW de-mosaicing engine (in DPP) -- they use their own and you can open images in Photoshop or Lightroom even if you don't have any Canon software installed at all. Installing Canon DPP wouldn't have any effect on Adobe software.

As for card readers... I use the Lexar USB 3.0 Dual-Slot reader and I have no issues with any of my Macs and also with any version of the operating system.

Here's a link to that particular reader (at the Lexar website -- but you can find this reader available from lots of vendors):

[Mod note: Removed per FORUM GUIDELINES]

I more recently switched to a different reader which worked fine with Yosemite and then stopped working when I upgraded to El Capitan. Correspondence with the company resulted in identifying a firmware fix for the reader and they sent their customers a new firmware file and instructions on how to update the firmware embedded within the reader -- and now it works fine. I'm a bit hesitant to mention that brand because I don't think it's reliability is on par with the Lexar brand reader -- which seems to be quite solid.

Of course if you really do have a reader problem, then bypassing the reader by downloading the images via the USB cable connetction (connected directly to your computer and not via a USB hub) should get around the problem. If you see the same problem when using the USB cable then it's improbable that the card reader is the culprit.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

ebiggs1 — Mon, 14 Dec 2015 19:05:48 GMT

"That means it is possible to that if there's a problem writing to the card, then the part of the file that contains the JPEG preview might not be corrupt, but if the RAW file could be corrupt elsewhere."

I really doubt this. Think about it for a minute.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

lensboard — Mon, 14 Dec 2015 19:26:11 GMT

I gave the answer already. It occurs in the file transfer. You can use a free SFV checker to conirm exact transfer. I do this when I have a second shooter transfer at weddings. Once in a while there is corruption, and I catch it right there. Google SFV checker.

Also, try ans transfer outside of LR, then import into LR.

Re: Corrupted/Garbled RAW (CR2) file when shooting with my Canon 5D MKii to a Macbook using LR5...he

joyohjoy — Wed, 16 Dec 2015 04:12:02 GMT

Sorry, lensboard. I didn't mean to skim over your initial comment, it just went over my head. I'm extremely non-techy, so I didn't know what sfv was. I googled it and wiki'ed it, but I'm still looking for the handle on the cup.