Memory leak in Canon DPP 4.10.20.1

[SimonSan]

I'd been having "insufficient memory" problems with DPP 4.9.20* so hoped to see those go away with this latest update. (Confirmed via help menu as 4.10.20.1)

Unfortunately, the problem appears to be persisting so I decided to try to track down which actions caused the app to consume and not properly release memory. Although I saw some smaller leaks in other places, I think this process below highlights it best:

1. Open DPP to a blank folder.

Task manager shows DPP using 221MB.

2. Open a folder with 44 24MP CR3 images in it, click each image in turn, return to the empty folder.

(Please note that I'm not making any changes - not even rating the images.)

Task manager shows DPP using 2667MB.

(Okay, so maybe DPP is caching images / data? If so, re-opening the same folder shouldn't cause any increase in the amount of RAM unless some of it is being needlessly duplicated.)

3. Repeat step 2.

Task manager shows DPP using 5106MB

(Doesn't look like caching...)

4. Repeat step 2.

(Note that DPP started to slow down during step 4 and it felt slow moving thru the images.)

Task manager shows DPP using 7441MB.

5. Give up as the numbers show no sign of stopping so this is either a memory leak or the worst cache ever.

6. Check on forums then ask if anyone knows a solution to this. (Hopefully!)

* = I have a 32GB RAM PC running Win10 1809 and a 4K monitor running at 150dpi. I have DPP set to show the folder window, vertical thumbnails at 384*384 and tool pallete / histogram on the right, and have also set it to show the applied recipe when previewing images. In order to speed things up, I've set DPP to open all CR3 files at 25% of their proper size.

** = I've tried enabling the following Win10 compatibility options - Win7 compatibility mode and changing the high DPI between app / system with no luck. The only good thing I've found is that the quick preview mode might not suffer from this problem, though I didn't test it as fully. (It used to so I switched away from using it.)

[cicopo]

i'm self taught re the computer side of things but you haven't said whether you are running 32 bit or 64 bit. From my own experience running Photoshop CS 6 you need to run in 64 bit or you can't access all the memory you have.

[SimonSan]

To cicopo: Just a quick reply - thanks for your attempt to help.

In response, my system is 64bit and I'm not concerned with DPP using all 32GB of RAM as long as it continues to work smoothly. What I am concerned with is that it is requesting memory to do something, then requesting more memory to do the same thing without returning the previously requested amount. If you look at my first post, it is using over 2GB, then 5GB, then 7GB of RAM just to let me look at the same 44 images 3 times in a row. I gave up trying it a 4th, 5th time as I thought that the leak was clear.

Although this process might seem artificial to you, that is what I would do (without returning to the empty folder) if I was trying to decide which of those 44 images I would spend time processing (and money) storing.

[wq9nsc]

I think that the problem must be machine, graphics card, and operating system version/revision dependent.  I have noticed earlier versions of DPP having a very bad memory leak but in my experience if 4.10.20.1 has one it is very minor.  

Last night I spent about 3 hours processing just under 600 photos from my 1DX and 1DX M2 bodies shot during three soccer games.  I shoot in RAW and then apply a standard recipe based upon shooting conditions before using the quick check window to review and rate images followed by cropping, adjustment, and finally conversion of the target files into jpg.  With standard soccer photos, DPP does everything I need and I don't use Adobe.

I have been curious about how the latest version of DPP interacts with the system so I leave the task manager performance monitor running.  The maximum memory usage I saw peaked at 3.2 gigs but dropped back quickly to its normal 1 to 2 gig range and never went into the memory leak runaway I saw with some earlier versions of DPP.  I am using a NVIDIA RTX 2060 graphics card with 6 gigs of dedicated memory but graphics card dedicated memory usage stayed at 1.3 gig never increasing and it didn't use any of the shared video memory. The GPU 3D engine showed around 15% loading during auto gamma adjustment but although DPP is somewhat slow in doing auto gamma after also running digital lens correction (up to 15 seconds for some images in my experience) it never heavily loads the system processors or GPU processors during this time period so I am not sure exactly what the hold up is with this delay.

I am using a dual Xeon processor workstation running Win 10 64 bit pro with 256 GB of RAM and although none of the versions of DPP came close to creating a true shortage of memory some had glitches that triggered an out of memory error when only 5 percent was in use for all processes and I had one version that was up to 25 GB used before I shut it down so that version qualified for the memory leak hall of fame..

Rodger

[SimonSan]

To wq9nsc - thanks for your reply. I can't rule out drivers (Intel chipset, nvidia video) or av (windows 10 built in) at the moment but I will try testing in a virtual pc when I have time in order to do so. To me, what I am seeing is that the process outlined - browsing and clicking multiple images with DPP set as mentioned on a high DPI 4K screen - shows signs of a memory leak. If you would like to see this for yourself then please could you try following the same with 40+ images in one folder and look at how much RAM each pass uses? (I'll add that I didn't notice any problems using small amounts of images in the quick check view last night, but then that's a different process so it's to be expected that it might not show the same problem.)

Simon

[wq9nsc]

Hi Simon,

I just checked with a folder of images I shot two weeks ago with 1,152 RAW images total in the folder and the memory leak didn't appear for me.  I have EOS transfer where it drops the files in a folder by shooting date (both the 1DX II and 1DX files go into the same sub-folder by date) and any JPG or TIF conversions go into separate subdirectories of the main directory.  I reapplied a slightly modified recipe, did a brief review and rating doing quick check, and converted a few more to JPG and the memory leak issue didn't show up.

I am glad I usually don't have that many images to review at a time because I killed most of a Sunday after that tournament along with a few hours Monday.  As my daughter's high school teachers would say, "too much screen time!".

Rodger

[SimonSan]

Hi Wq9nsc,

Thanks for letting me know what you tried, I think that I've found the cause! 🙂

(In short, don't set CR3 files to open at 25% - maybe only true if DPP is set to 100%.)

To view this:

Use CR3 files.

Confirm that DPP is set to show CR3 files at 100%. (Which it is by default.)

Open a folder with multiple (in my case, 34) CR3 files and nothing else.

Set those images to open at 25% (because my PC only has a i5-7500 3.4GHz) via the tool pallete.

Close DPP.

TO SEE FAULT:

Reopen DPP to empty folder.

RAM Usage now at 200MB ish. (4K screen, now running at 200% dpi.)

Click into the folder with 34 images.

Click each image in turn quickly. (Well, you might be able to take your time but I didn't have time to try that.)

Click back into the empty folder.

Notice RAM usage now at over 1GB.

Click into the folder with 34 images.

Click each image in turn quickly. (Well, you might be able to take your time but I didn't have time to try that.)

Click back into the empty folder.

Notice RAM usage now at over 2GB.

To confirm fix (hopefully):

Close DPP.

Set all (34) images to open at 100% via the tool pallete.

Close DPP.

Repeat the steps in "TO SEE FAULT".

Notice RAM usage back to 200MB ish each time you return to empty folder.

(This is what should happen ordinarily. I don't mind DPP using 20GB of RAM to show my photos as long as it works and doesn't need 40/60/80GB each successive time.)

I confirmed the fix twice and saw the fault twice but will try a few more times to get exact figures.

Thanks for your replies - I'll let Canon know assuming the fix continues to work once I've got the exact figures.

Simon

[SimonSan]

Firstly, sorry about the caps. m(_ _)m

Here's the results of my last check, checking DPP ram usage in task manager and starting with all CR3 images set to display at 25% via the tool pallete.

OPEN TO EMPTY FOLDER

207MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES, RETURN TO EMPTY
1065MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES, RETURN TO EMPTY
1919MB
OPEN FOLDER 2 (BY MISTAKE), CLICK THRU 12 IMAGES, RETURN TO EMPTY (USAGE EXCEEDED 3GB)
2265MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES, RETURN TO EMPTY (USAGE EXCEEDED 4GB)
3401MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES A BIT SLOWER, RETURN TO EMPTY (USAGE EXCEEDED 5.6GB)
4722MB
(Clicking a bit slower increases chance that you catch the processing when it has grabbed the most memory - when the next image starts processing (some of) that extra ram isn't released.)

And now to prepare to check if the fix works...

SET ALL 34 IMAGES TO VIEW AT 100%, BROWSE TO EMPTY FOLDER, CLOSE DPP.

Then...

OPEN TO EMPTY
205MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES, RETURN TO EMPTY
230MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES, RETURN TO EMPTY
233MB
OPEN FOLDER 1, CLICK THRU 34 IMAGES A BIT SLOWER, RETURN TO EMPTY
234MB

As you can see, DPP is releasing memory properly (as far as we can tell) and a 29MB increase isn't anything to be worried about given the 4500MB (and rising) increase seen when CR3s are set to 25%.

I'll forward this to Canon as soon as I can.

Simojn