Bug 472494

Summary: Export to Google Photos ends up with "too many requests" error
Product: [Applications] digikam Reporter: Ilgaz Öcal <ilgaz>
Component: Plugin-WebService-GoogleAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: antonrosenfeld, caulier.gilles, connect.daniel, ikding, ishanto78, mchesleyjohnson, metzpinguin, patrik.gfeller, rgiw, rt2sbaaj8
Priority: NOR    
Version: 8.3.0   
Target Milestone: ---   
Platform: macOS (DMG)   
OS: macOS   
See Also: https://bugs.kde.org/show_bug.cgi?id=400187
Latest Commit: Version Fixed In: 8.2.0
Sentry Crash Report:

Description Ilgaz Öcal 2023-07-22 10:00:46 UTC
SUMMARY
***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***


STEPS TO REPRODUCE
1. Run Digikam
2. Select some photos, 5-8 would be OK
3. Export to Google Photos
4. Login to your Google Photos account with the opened Web Browser
5. Allow Digikam to perform displayed actions

WHAT HAPPENS

Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests

WHAT SHOULD HAPPEN

Photos should be uploaded to Google Photos

OBSERVED RESULT


EXPECTED RESULT


SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION

On Reddit some users suggest that we should be able to enter our own API key to prevent the error from happening. See the comments at (archived) https://web.archive.org/web/20230722095942/https://www.reddit.com/r/FOSSPhotography/comments/151k574/digikam_export_to_google_photos_too_many_requests/?context=3
Comment 1 Ilgaz Öcal 2023-07-22 10:18:05 UTC
Duplicate of https://bugs.kde.org/show_bug.cgi?id=400187 ?
Comment 2 caulier.gilles 2023-07-22 10:44:18 UTC

*** This bug has been marked as a duplicate of bug 400187 ***
Comment 3 Maik Qualmann 2023-07-22 11:52:58 UTC
The cause here is that Google has blocked the API key at the moment, we sent a corresponding response to Google to clarify it. But it will take a few days.

Maik
Comment 4 Maik Qualmann 2023-07-24 08:13:14 UTC
*** Bug 472567 has been marked as a duplicate of this bug. ***
Comment 5 Maik Qualmann 2023-08-06 14:08:12 UTC
*** Bug 473073 has been marked as a duplicate of this bug. ***
Comment 6 Maik Qualmann 2023-08-10 06:12:36 UTC
Our Google API key has been reactivated. I urge all users not to use the digiKam Google Export Plugin for massive uploads and downloads. Keep in mind that the "free" volume provided by Google is what all digiKam users want to use. If you want to download your images, Google offers the corresponding functions on the Google Photos website. There you can request ZIP files (the creation of which can take hours or days) for which you will then receive a download link.

Maik
Comment 7 I-Kang Ding 2023-12-29 00:55:34 UTC
Hello, as of 2023-12-28 this problem seems to have resurfaced again. The error message I am getting is "Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests". This is happening on both Digicam 7.4.0 and Digikam 8.2.0. I am using Mac OSX version.
Comment 8 Maik Qualmann 2023-12-29 07:27:20 UTC
Our Google API key was blocked again due to automatically detected violations of the Google Terms of Service. I have just made a corresponding request for activation. By asking Google to let us know exactly what violations have occurred. If it turns out that users are using our Google Photo plugin in a way that violates Goggle's terms of use, we will have to remove the plugin.

Maik
Comment 9 caulier.gilles 2023-12-29 07:40:23 UTC
Hi Maik,

Providing a specific google API key settings for the plugin can be a solution ?
Gilles
Comment 10 Maik Qualmann 2023-12-29 09:03:20 UTC
A Google API key is very different from a map key.
For a Google Photo key you need:
- A website
- You must confirm that you have access to the website
- There must be a privacy statement on the website.
- you must know and request the exact rights that the plugin requires
- you must submit a video of how the plugin works
- and much more...

It won't work because, unlike the map key, the Google Service API key is only ever assigned to one app.

It is also possible that our key is used by third parties, so we need to "hide" it better. I will definitely change the key so that the Google plugin will no longer work for all digiKam versions smaller than the current one.

Maik
Comment 11 Maik Qualmann 2024-01-02 09:28:25 UTC
*** Bug 479294 has been marked as a duplicate of this bug. ***
Comment 12 mchesleyjohnson 2024-01-05 18:00:42 UTC
Getting the same problem today:

Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests

This after authenticating my account w/ Google and allowing needed permissions for Digicam
Comment 13 I-Kang Ding 2024-01-07 20:02:17 UTC
The Google Photo upload in KDE seems to be working for me now - Mac OSX, digicam version 8.2.0. 

Build date: 11/29/23 10:44 AM (target: Debug)
Revision: d9b5f5bcf81df10de7b3b33b1fca1cd189eab79a
Branch: HEAD
Comment 14 Maik Qualmann 2024-01-07 20:18:41 UTC
Unfortunately, we only receive a standard message from Google that the key has been reactivated, without any information why. We should ensure that the terms of use are adhered to. Brave new AI world, probably even Google doesn't know what the AI does...

Maik
Comment 15 Reinhold Graf 2024-02-14 15:49:24 UTC
I get the same error since some days.
Try to upload 1 image and immediately after calling the Export to Google Photos, the error appears.

"Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests"
Comment 16 Reinhold Graf 2024-02-14 15:50:01 UTC
(In reply to Reinhold Graf from comment #15)
> I get the same error since some days.
> Try to upload 1 image and immediately after calling the Export to Google
> Photos, the error appears.
> 
> "Error transferring
> https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied:
> Too Many Requests"

Just to mention, I'm on Windows 10 with DigiKam 8.2.0.
Comment 17 caulier.gilles 2024-02-14 15:54:06 UTC
You can try with the 8.3.0 pre-release available here :

https://files.kde.org/digikam/

... but i fears that the problem is relevant of Google, as they want more and more money... Typically we will need to pay for a generic account to use with Google web service, but of course it's NOT acceptable to do it open source software. Google is a giant and we will never give money to this company.

Gilles Caulier
Comment 18 Maik Qualmann 2024-02-14 16:26:29 UTC
Our API key has been deactivated again, I have to write another objection. Since I don't know which Google policies we are violating, it will be difficult to do anything about it.

Maik
Comment 19 caulier.gilles 2024-02-14 17:13:59 UTC
Maik,

it's probably an IA which operate this action. I cannot imagine a team checking all Google accounts around the world.

Perhaps others project has a similar problem with Google, an public discussion available in forums ?

Best

Gilles
Comment 20 Reinhold Graf 2024-02-14 19:31:46 UTC
Thanks for the information.
For sure, I agree with you, not to feed the beast ;)
There is still the workaround of direct a manual upload :)
Comment 21 Ilgaz Öcal 2024-02-15 05:32:00 UTC
Without going into non-technical discussion, can't the user get a personal API key, enter that key in some field in the plugin settings and use it in an amateur (non-pro load) manner?

FYI if one uses a non de-googlefied Android like Lineage+microG, they already give Google full access to their data, "as root".

What happens is: KDE's target market which isn't GNU-HURD, especially paying ones from Third World countries (thanks to localized pricing) are being victimized. Disabling a small time user grabbing 10 images from their Google Photos won't really hurt Google, it will only hurt long term goals of KDE project. They will simply use Google Photos Web and get more profiled.
Comment 22 caulier.gilles 2024-02-15 05:55:09 UTC
Hi,

About the API key, i thinking about this feature already implemented in GoogleMaps support from digiKam, but he explain me that for Google Photo this work around is different in the Google strategy. To resume, it's not simple as to use the same API key everywhere for all Google web services.

I feeling that Google plan for the future is more AI like everywhere. You want to use legacy web services, you need to pay. Free to use Web service will be reduced in the long time. This is my pure viewpoint of course here...

Gilles Caulier
Comment 23 caulier.gilles 2024-02-15 05:58:44 UTC
More personal viewpoints here about Google :

- You use Chrome with Google web service, there is no limitation, but Google scan and store your Web activity to make money.
- You use a non Google application to use Web service : you need to pay of course.

It's simple, money is money. The free to use game is over.

Gilles Caulier
Comment 24 Maik Qualmann 2024-02-15 07:20:28 UTC
With a Google Photo API key, it is not possible for a user to have a personal one. See my Comment 10.

Maik
Comment 25 Reinhold Graf 2024-02-15 16:45:15 UTC
(In reply to caulier.gilles from comment #17)
> You can try with the 8.3.0 pre-release available here :
> 
> https://files.kde.org/digikam/
> 
> ... but i fears that the problem is relevant of Google, as they want more
> and more money... Typically we will need to pay for a generic account to use
> with Google web service, but of course it's NOT acceptable to do it open
> source software. Google is a giant and we will never give money to this
> company.
> 
> Gilles Caulier

Just tried it with 8.3.0 - it gives the same error.
Comment 26 Maik Qualmann 2024-02-17 12:04:35 UTC
Git commit 09858c9d57f92818695d8b238805811432414d34 by Maik Qualmann.
Committed on 17/02/2024 at 12:03.
Pushed by mqualmann into branch 'master'.

update Google Photo icon to the current one from 2020

M  +1    -1    core/dplugins/generic/webservices/google/gsplugin.cpp
M  +-    --    core/dplugins/generic/webservices/google/icons/gphoto/16-apps-dk-googlephoto.png
M  +-    --    core/dplugins/generic/webservices/google/icons/gphoto/22-apps-dk-googlephoto.png
M  +-    --    core/dplugins/generic/webservices/google/icons/gphoto/32-apps-dk-googlephoto.png
M  +-    --    core/dplugins/generic/webservices/google/icons/gphoto/48-apps-dk-googlephoto.png
M  +-    --    core/dplugins/generic/webservices/google/icons/gphoto/sc-apps-dk-googlephoto.svgz

https://invent.kde.org/graphics/digikam/-/commit/09858c9d57f92818695d8b238805811432414d34
Comment 27 Maik Qualmann 2024-02-24 21:35:03 UTC
*** Bug 481787 has been marked as a duplicate of this bug. ***
Comment 28 Maik Qualmann 2024-03-02 20:12:55 UTC
Google Photo Export will only be available again from digiKam-8.3.0. I'll close here for now.

Maik
Comment 29 I-Kang Ding 2024-04-25 04:34:16 UTC
I noticed that the "Too Many Requests" error appears again on the Digicam 8.3.0. I am using MacOS platform. The error messages are as below:

"Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests"

I know this problem is from the google side and there's little we can do other than relaying this info to google so that they can unlock the API key used by DigiKam. Thank you in advance for whatever you can do.
Comment 30 Maik Qualmann 2024-04-25 05:46:35 UTC
It is always the AUP-3 guideline that we are said to have violated. Does anyone have any ideas about what exactly  the Google AI might not like about our Google Export Tool?

https://developers.google.com/photos/library/guides/acceptable-use#permission

Maik
Comment 31 caulier.gilles 2024-04-25 06:02:40 UTC
Hi Maik,

I read the AUP-3 section of the document, and i do not understand why the Google stuff considerate that the digiKam uploader violate something.

Without a real explanation given by a real human, it will be difficult to fix something...

Gilles Caulier
Comment 32 Maik Qualmann 2024-04-25 06:18:23 UTC
Hi Gilles,

I do not understand it either. I filled out the appeal again. Google also requested a review of our OAuth client, which I sent again with additional information that was not previously asked for, e.g. link to the program documentation.
Wait...

Maik
Comment 33 Maik Qualmann 2024-05-09 11:49:57 UTC
*** Bug 486807 has been marked as a duplicate of this bug. ***
Comment 34 Maik Qualmann 2024-05-09 11:54:30 UTC
As a quick update, our OAuth client has already been confirmed by Google. All that remains is to wait for the objection about the blocking of our API key due to a suspected violation of the AUP-3 guideline.

Maik
Comment 35 Maik Qualmann 2024-05-16 06:02:35 UTC
For now, our API key has been reactivated.

Maik
Comment 36 I-Kang Ding 2024-05-16 06:03:41 UTC
Thank you very much, Maik Qualmann!
Comment 37 I-Kang Ding 2024-07-20 19:23:41 UTC
Hello, as of 2024-07-20 this problem seems to have resurfaced again. The error message I am getting is "Error transferring https://photoslibrary.googleapis.com/v1/albums?pageSize=50 - server replied: Too Many Requests". This is happening on both Digicam 8.3.0 (Mac OSX version).

I know this problem is from the google side and there's little we can do other than relaying this info to google so that they can unlock the API key used by DigiKam. Thank you in advance for whatever you can do.
Comment 38 Maik Qualmann 2024-07-21 06:17:04 UTC
It continues here: Bug 490397...

Maik