Digital Photo Professional - New release 4.21.30

[March411]

For those that use it it's available here: Digital Photo Professional 

When you use the link it should recognize your OS and supply the correct download. There isn't much in the way of release notes but they can be found here: Digital Photo Professional 4.21.30 for Windows 

I have loaded it and taken it for a spin. Noting much to note although it does appear to load images faster. I always had an issue where the photo would load and then a delay to focus, that time has been reduced.. 

[SignifDigits]

Thank you Marc.

[wq9nsc]

Thanks Marc,

I downloaded it a couple of days ago and I noticed it still has one anomaly that has been present in all of the recent releases of DPP (at least on my Windows systems).  If you click on an image and choose the X1 button to go to 100%, the enlarged section is rendered instantaneously BUT if you scroll to another image while X1 is active, there is about a 2 second delay while it goes from its coarse resolution to fine resolution.  This is with CR3 files, the older CR2 files from my 1DX II bodies don't show this glitch. 

It is annoying when I want to look at several files in series at 100% and it is actually faster to toggle back to regular before clicking on the next file and choosing X1.

In general, I am happy with the responsiveness of the current version of DPP and besides that glitch when scrolling with X1/100% active, the only really slow operation is using the stamp tool which takes a few seconds before the image can be edited.  DPP has definitely gotten better over the years but it still needs work AND according to my system monitor it isn't doing anything useful with my Nvidia GPUs.  Neither actively editing nor batch processing results in any GPU activity attributable to DPP.  During batch processing of CR3 files it is taking about 6 seconds per file which I believe is marginally slower than the previous version.  Peak CPU load is 10-11% for an instant (less than a second) and stays below 5% for the rest of the file time so not sure what is going on with that in the current version.  In the prior version it would spike at around 15-20% and then stay at 6 to 7% CPU load for the remainder of each image file.

Rodger

[johnrmoyer]

I hope some of this might be helpful or interesting.

I run DPP on a 2019 iMac with intel CPU and have tried it on a Windows 10 virtual machine using qemu on my Debian Linux desktop. I have not bothered with giving GPU access to the Windows virtual machine. To me, the biggest advantage to having a GPU is the addtional RAM that is not shared with the CPU. More RAM and more CPU cores are needed without the GPU.

On the iMac, DPP often uses 300-400% while all 6 CPU cores at 100% would be 600%. DPP seems to me to be constrained by memory access and by disk speed. ( I have held a grudge against Nvidia firmware since I was writing BIOS code in 1998, but I am still on their developer email list. I do not have high expectations for GPU calculations. )

On my iMac, I kept increasing the amount of RAM until performance was acceptable. I have 96GB now. Newer Mac computers do not have RAM that can be upgraded. 

I give the Windows Virtual machine running on Debian 6 CPU cores and 16GB RAM. Before my EOS R5 CR3 files, it was 2 CPU cores and 12GB RAM. But, Debian caches in RAM almost the entire virtual hard drive used by Windows and that speeds up DPP. It seems to me likely that DPP would benefit from a rewrite of the disk caching code.

On the iMac, I needed a larger backup drive, but the 14TB external USB drive is slow to spin up. Whenever timemachine backup starts, there is a delay. I have excluded the DPP cache from timemachine backups.

For viewing multiple images in DPP, the Quick Check works best for me. I often will view the eye and beak of a bird at 400% to decide whether that image will be worth editing. It seems to me that Quick Check does not use the DPP disk cache or spend time applying the recipe.

The most time consuming parts of applying the recipe in DPP seem to me to be digital lens optimizer, DPRAW processing, and actions that cause the color curve to change.

[wq9nsc]

John,

This is a screen grab from the task monitor showing DPP processing a CR3 file to jpg.  Activity is primarily from DPP although Firefox along with Outlook and enterprise Outlook were also running.  So CPU activity peaked at about 10% briefly for that file.

I haven't had a chance to do a long editing session with this latest version to see if it still has the memory leak issue that has been part of DPP for decades but I expect that it does.  After a little work, the DPP process is using a little over 1 GB of memory but if it holds true to form that will grow to 8 to 10 GB over a couple of hours of use; the "fix" is to exit and restart regularly during a long editing session because DPP becomes noticeably sluggish as its resident size increases.

This slowdown occurs regardless of available memory.  I am using a HP Z8 G5 with twin Intel 28 core Xeon processors and each has 256 GB of RAM installed.  Also installed are two Nvidia Ada 4000 workstation cards with 20 GB of DDR6 and 6,144 Cuda cores per card.  My video editing software makes good use of the Nvidia capabilities but not DPP.

DPP resides on one HP Zdrive SSD sitting on the processor bus and the image and output files are on a second Zdrive.  DPP is sufficiently responsive on this machine but it does a lousy job of utilizing available resources and likely still becomes extremely sluggish during long editing sessions.  From a typical sports event, I will have around 1,200 to 1,500 images from 3 bodies in the image director for that event and it takes a few hours to work through all of them.

Rodger

[johnrmoyer]

Disclaimer: most of what I know about computer architecture and performance is from ancient times, but some things never change. I hope some of this might be helpful anyway.

Have you tried adjusting the size of the DPP disk cache? Maybe make it large enough to hold all of the photos for the editing session? Or size it to hold all of the photos in one subdirectory? Too large or too small seems to me to slow DPP down. Too many images in one directory seems to me to slow DPP down. Does Windows scan each image for viruses?

I have read about but not tried the Ubuntu Windows subsystem. If that really works and if the camera models are old enough to be supported, then gphoto2 and Darktable might be good options for previewing a large number of photos. Bottlenecks usually include RAM speed, CPU ram cache (but with 28 cores that will be large), filesystem type, disk buffering and caching, and database lookup. There is nothing one can do about database lookup in DPP except to divide the photos into smaller subdirectories (maybe by time of day?). There is nothing one can do about Windows file systems. Debian Linux works very well in a VMWare virtual machine on Windows and has drivers for that purpose. Files can be shared by mounting Windows shares using cifs. In the old days I got permission to run the free to download VMWare Player on the Windows machine where I worked because some of the systems we supported ran Linux.

SSD speeds will likely be limited by the speed of the interface. 

Some things I do not know: 

Does Windows know how to use the performance cores instead of the low power cores? Does Windows know how to migrate a process from a core that is limited by high temperature to a cooler core to get higher performance?

[Jkarl]

I wonder if you have ever used the Sysinternals tools. There are a number of process diagnostic tools in the suite that I have used to find problems with both hardware and software. This discussion has inspired me to check how bad DPP is preforming and where the problems are.

Sysinternals suite can be downloaded from the Microsoft website for both x86 (Intel and AMD) and Arm chips.

Go to Microsoft.com and search for Sysinternals.

[wq9nsc]

Thanks for your comments John!

DPP in Windows, at least with my configuration, doesn't seem to use its available disk cache.  The Z turbo drives sit on the processor bus with a 6.5 GB/S read and 5 GB/S write so I doubt if they are slowing things down. Windows doesn't scan the files for virus before reading in DPP.

The most annoying bottleneck I see is when scrolling through files in X1 / 100% mode.  No delay when activating X1 in a file, only when browsing in that setting so it has to be a DPP issue.  I can toggle out of X1, select the next file, and select X1 again with no delay but if I just switch files in X1 there are a couple of seconds for it to adjust.

The batch processing is reasonably fast, average 4 to 5 seconds per CR3 file and much faster for older CR2 files.  The only concern is it really doesn't make much use of available processing power.

I ordered the workstation configured with both Linux and Win 11 Pro from HP because some of the other software I use has Linux specific versions and it is my preferred operating system.  I tried running DPP in a "Windows window" within Linux shortly after I bought this setup but it wasn't an improvement over running native in Win 11.  I have some free time later this summer once I finish some big landscaping projects so I will experiment further with it.

The Xeon processors are designed for enterprise/workstation system use so all 28 cores per CPU are high performance cores.  I suspect Windows knows how to switch cores if thermal limiting occurs but the Z8 G5 is typically used for video rendering, complex modeling, and other intensive activities and the thermal design is intended for 24/7 full load use.  The CPU heat sinks are vapor phase cooling types and the CPUs and memory modules are in a ducted cooling system.  In my use, the CPU fans have never gone above level 3 of their 7 available levels (and that has only happened rendering some very large 4K video files, DPP never does enough to generate any heat to move the cooling system from its base level 1).

This configuration of the Z8 G5 is really overkill for DPP but I need its capabilities for some consulting work that I do so it also gets used as a high end image processing workstation.

Rodger

[wq9nsc]

Thanks Karl,

I have used that tool but had forgotten all about it!  Another experiment for later.

I suspect any issue is DPP specific because I run much more demanding math modeling and stats software on the workstation and only DPP has performance that I would call disappointing.  Although it probably wouldn't make a significant performance difference, I wish that Canon would have DPP ported to a Linux native version.  There are only two pieces of software I run that absolutely need to run in Windows; DPP is one and the other isn't critical to me.  With Win 11 headed towards even great AI integration (when MS still hasn't made Win 11 into a solid base OS), I don't have high hopes for it but with more EU countries moving government stuff off of Windows onto Linux I am hopeful that sooner rather than later Canon does a much needed "clean sheet" revision of DPP and provides a Linux native version at that same point.

Rodger

[SignifDigits]

I have an OLED monitor on a second GPU in my Asus ProArt where I use DPP.   It does take advantage of that GPU for display.  I see up to 11% usage at peak - most times much less.  On HDDs DPP is slow, using NAS is decrepit/unusable, and SSD performance is acceptable.  It uses a lot memory - 1.2GB or so.  It has a few quirks (Dust/Stamp function slow to load) but is a good value for the money.  Even though it uses a lot of memory it seems to play nice with other memory hogs.  I generally don't run multiple apps while running DPP, but have sometimes.

I'm curious to know if anyone uses and pays for the Neural Network Image Processing Tool and, if so, what their experience is, especially with noise reduction.  I have a NPU on the laptop and wonder if it will use that or the GPU or just the CPU.  Topaz is clearly a winner but I'm not paying $500 for the privilege to satisfy my low-volume needs.  That, and I don't mind some noise - just part of the photographic process IMO.