Bug 397207 - Too many open files, when reverse geocoding many images
Summary: Too many open files, when reverse geocoding many images
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Geolocation-ReverseGeoCoding (other bugs)
Version First Reported In: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-08-06 11:01 UTC by Kristian Karl
Modified: 2018-08-07 09:54 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed/Implemented In: 6.0.0
Sentry Crash Report:


Attachments
Proposed patch (2.18 KB, patch)
2018-08-06 11:01 UTC, Kristian Karl
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kristian Karl 2018-08-06 11:01:24 UTC
Created attachment 114319 [details]
Proposed patch

Running reverse geocoding on a large set of images, generates error:
GLib-ERROR **: Creating pipes for GWakeup: Too many open files


Actions
=======
1) In the Geolocation Editor, select the Reverse Geocoding tab.
2) Select a large set of images. I used 5,000 images.
3) Select Open Street Map as the service
4) Click on Apply reverse geocding button

Expected result
===============
All images with coordinates are reversed geocoded.

Actual result
=============
After around 1,0000 requests, digikam crashes.
First I got a SSL Handshake failed, error message dialog.
Then when I exit the Editor, digikam crashes with:
GLib-ERROR **: Creating pipes for GWakeup: Too many open files

Additional info
===============
Running the same scenario again, and tracking number of open files using:
 watch -n1 "ls -la /proc/$(pidof digikam)/fd | wc -l"
I could see that the number was increasing, until I ran into the same problem.

A similar issue (https://github.com/albertlauncher/albert/issues/634) was fixed by https://github.com/albertlauncher/albert/commit/70e3662a8e3a08137073b5932dbd7ddee136ff65

When I tested their approach on digikam, digigam did not crash running the same scenario.

Given the above I propose the attached fix.
Comment 1 Maik Qualmann 2018-08-06 11:37:27 UTC
Thanks for the error report and patch, yes that is a wrong use of QNetworkAccessManager in our code. Thousands of QNetworkAccessManager are created. I'll fix it tonight, but different.

Maik
Comment 2 Maik Qualmann 2018-08-06 20:26:11 UTC
Git commit f0cb9459229d84c6c96158a9fb8e5016ae36b3c7 by Maik Qualmann.
Committed on 06/08/2018 at 20:24.
Pushed by mqualmann into branch 'master'.

fix use of QNetworkAccessManager
FIXED-IN: 6.0.0

M  +2    -1    NEWS
M  +16   -18   core/utilities/geolocation/editor/backends/backend-geonames-rg.cpp
M  +16   -20   core/utilities/geolocation/editor/backends/backend-geonamesUS-rg.cpp
M  +14   -17   core/utilities/geolocation/editor/backends/backend-osm-rg.cpp
M  +13   -15   core/utilities/geolocation/editor/searches/searchbackend.cpp
M  +14   -13   core/utilities/geolocation/geoiface/lookup/lookupaltitudegeonames.cpp

https://commits.kde.org/digikam/f0cb9459229d84c6c96158a9fb8e5016ae36b3c7
Comment 3 Kristian Karl 2018-08-07 09:54:23 UTC
I verified the fix.
Working well. Thanks!