Bug 396671 - google photo: Cannot download more than 1000 photos at once
Summary: google photo: Cannot download more than 1000 photos at once
Status: RESOLVED NOT A BUG
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-WebService-Google (show other bugs)
Version: 5.9.0
Platform: Fedora RPMs Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-07-19 16:10 UTC by Alexander Potashev
Modified: 2023-10-12 05:39 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.2.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Potashev 2018-07-19 16:10:19 UTC
Cannot download more than 1000 photos at once from Google Photo. When there are more than 1000 photos in the selected album (e.g. 1235 photos), only 1000 are donwloaded. Google Photo albums can contain up to 2000 photos.
Comment 1 caulier.gilles 2018-08-14 14:55:48 UTC
Thanh,

Do you know if current Google API have a limitation to 1000 items by upload session ?

Gilles
Comment 2 caulier.gilles 2019-07-27 12:50:39 UTC
Please test with 6.2.0 pre-release bundles available here :

https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 3 Alexander Potashev 2019-07-31 08:11:41 UTC
The appimage failed to run here on Fedora 30 because of this problem:

https://github.com/AppImage/pkg2appimage/issues/336

Command line output:
  [aspotashev@candy Downloads]$ ./digikam-6.2.0-20190730T044304-qtwebkit-x86-64.appimage 
  -- digiKam Linux AppImage Bundle
  -- Use 'help' as CLI argument to know all available options for digiKam application.
  -- Note: to integrate this bundle to your desktop, use AppImageLauncher.
  libudev.so.0 
  -- Preloading shared libs: 
  digikam: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory
  [aspotashev@candy Downloads]$

I will hopefully try to build Digikam from source this week.
Comment 4 caulier.gilles 2019-07-31 10:01:04 UTC
Try to just install the Fedora RPM including the libnsl.so.1 library.

As you can see here :

https://cgit.kde.org/digikam.git/tree/project/bundles/appimage/04-build-appimage.sh#n295

This library is excluded from AppImage build script. This rules is copied from the official AppImage script here :

https://github.com/probonopd/linuxdeployqt/blob/master/tools/linuxdeployqt/excludelist.h

libnsl is the public client interface library for NIS(YP) and NIS+. It's about the Yellow Page dictionary used on the network. This library is removed from the bundle about possible security issue and update. It's always better to have this library taken from the host system.

Gilles Caulier
Comment 5 Alexander Potashev 2019-07-31 10:33:28 UTC
I couldn't login to my Google Photo account in the respective digiKam plugin. After I accept permissions to be given to digiKam in my browser (Google Chrome), the browser is redirected to http://127.0.0.1:8000/?code=4/.....&scope=profile%20https://www.googleapis.com/auth/photoslibrary%20https://www.googleapis.com/auth/photoslibrary.readonly.appcreateddata%20https://www.googleapis.com/auth/photoslibrary.sharing%20https://www.googleapis.com/auth/userinfo.profile
and there is no more instructions to follow to finalize connection between digiKam and Google Photo.

When I click the button to reload the list of albums in the "Import from Google Photos Service" dialog, I get the following error on the command line:

  unknown: list albums
  unknown: url for list albums QUrl("https://photoslibrary.googleapis.com/v1/albums")
  unknown: reply error: QNetworkReply::NetworkError(NoError) - "Unknown error"
  unknown: parseResponseListAlbums
Comment 6 Alexander Potashev 2019-07-31 10:36:24 UTC
I had to install the package libnsl that contains libnsl.so.1 on Fedora 30.
Comment 7 Maik Qualmann 2019-07-31 20:13:11 UTC
That nothing happened after login is normal, the import tool is automatically logged in, the browser can be closed. The output in the console is normal, there is no error (QNetworkReply::NetworkError(NoError)). Their albums should be available.

Maik
Comment 8 caulier.gilles 2022-01-10 07:51:59 UTC
Hi Alexander and happy new year,

Can you check if problem remain with digiKam 7.5.0 pre-release bundle available
here :

https://files.kde.org/digikam/

Thanks in advance

Gilles Caulier
Comment 9 Maik Qualmann 2022-06-20 17:10:43 UTC
Git commit 979c3f318fea3d4b14d525c2d45d10f801beccd9 by Maik Qualmann.
Committed on 20/06/2022 at 17:09.
Pushed by mqualmann into branch 'master'.

port Google Export Tool to QOAuth2AuthorizationCodeFlow
Related: bug 440156, bug 435384, bug 415365, bug 419956, bug 411521
FIXED-IN: 8.0.0

M  +3    -1    NEWS
M  +2    -1    core/cmake/macros/MacroDPlugins.cmake
M  +1    -0    core/cmake/rules/RulesQtFramework.cmake
M  +2    -1    core/dplugins/generic/webservices/google/CMakeLists.txt
M  +11   -31   core/dplugins/generic/webservices/google/gdrive/gdtalker.cpp
M  +35   -63   core/dplugins/generic/webservices/google/gphoto/gptalker.cpp
M  +77   -83   core/dplugins/generic/webservices/google/gstalkerbase.cpp
M  +8    -9    core/dplugins/generic/webservices/google/gstalkerbase.h
M  +44   -1    core/libs/dplugins/webservices/wstoolutils.cpp
M  +10   -0    core/libs/dplugins/webservices/wstoolutils.h

https://invent.kde.org/graphics/digikam/commit/979c3f318fea3d4b14d525c2d45d10f801beccd9
Comment 10 Maik Qualmann 2023-02-15 12:36:45 UTC
*** Bug 465758 has been marked as a duplicate of this bug. ***
Comment 11 caulier.gilles 2023-05-03 08:06:13 UTC
@Alexander Potashev

digiKam 8.0.0 is out. This entry still valid with this release ?

Best regards

Gilles Caulier
Comment 12 caulier.gilles 2023-05-03 08:14:46 UTC
Maik,

If there is a limitation of 1000 items max to upload at one time, this must be annotated somewhere in online documentation.

Gilles
Comment 13 caulier.gilles 2023-05-07 13:29:14 UTC
Git commit d3ee50926bafdaa7298bad3c677518a20eefb288 by Gilles Caulier.
Committed on 07/05/2023 at 13:28.
Pushed by cgilles into branch 'master'.

digiKam online documentation: Google tool limit to 1000 item at once while downloading files.

M  +2    -0    export_tools/google_export.rst

https://invent.kde.org/documentation/digikam-doc/commit/d3ee50926bafdaa7298bad3c677518a20eefb288
Comment 14 caulier.gilles 2023-05-07 13:31:03 UTC
Maik,

I don't think we can do more fixes in the file, as the limitation come from the Google side... Close it ?

Gilles