Bug 410787

Summary: can't export to Google photo
Product: [Applications] digikam Reporter: Philippe ROUBACH <philippe.roubach>
Component: Plugin-WebService-GoogleAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, metzpinguin
Priority: NOR    
Version: 7.0.0   
Target Milestone: ---   
Platform: openSUSE   
OS: Linux   
Latest Commit: Version Fixed In: 7.4.0
Sentry Crash Report:
Attachments: "photos" in google photos
my album in google photos

Description Philippe ROUBACH 2019-08-10 11:57:17 UTC
SUMMARY


STEPS TO REPRODUCE
1. I created anew album then check it is well created in google photo site.
2. I launch transfer of 243 photos to google photo.
3. at the end i get a message "host requires authentication"

OBSERVED RESULT

i go to google photo
not any of the 246 photos, album is empty

EXPECTED RESULT

246 photos transferred to google photos

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: openSuse Argon 15.1
KDE Plasma Version: 5.16.3
KDE Frameworks Version: 5.60.0
Qt Version: 5.13.0

ADDITIONAL INFORMATION

kde apps 19.04.3
Comment 1 Maik Qualmann 2019-08-10 15:39:01 UTC
The Google API has a upload limit of 50 images per batch. I think we are doing it right. As a test I had already tried 70 images in one piece. What happens if you stay under 50 images?

Maik
Comment 2 Philippe ROUBACH 2019-08-10 16:40:16 UTC
9 photos selected => 9 photos transferred

48 photos selected => 48 photos transferred
Comment 3 Maik Qualmann 2019-08-10 19:43:53 UTC
I have now uploaded a set of small photos (250) and a set of large photos (250) where the upload took about an hour, I could not reproduce the problem.

I already use the pre-release version digiKam-6.3.0 where we have fixed an error, that the upload fails if there are quotes in the metadata of the images of title or description. Is that possibly the case with you?

Maik
Comment 4 Philippe ROUBACH 2019-08-11 08:29:22 UTC
68 photos selected => no photo at google photo.

I wonder if it is a delay problem. My internet link is slow, an upload about 954 kbit/s (119 kB/s)
Comment 5 Philippe ROUBACH 2019-08-11 08:34:25 UTC
The most complicated description is "Parc Montsouris - Général San Martin".
All photos of this type are well transferred.
Comment 6 Philippe ROUBACH 2019-08-11 08:42:24 UTC
What do you mean by "quote" ? These symbols ' or " ?

If yes there are quotes in metadata of all photos. For example 

GPS longitude 2deg 20' 17"
Comment 7 Maik Qualmann 2019-08-11 09:39:47 UTC
In digiKam-6.2.0 make images with a " in the metadata of title or description field problems.

How long does the upload take if it does not work? Unfortunately, the doc of the Google API does not provide information on how long an upload token is valid.
For explanation:
First, all images are uploaded, so they are not yet in the target album, but in the Google Cloud. We get an upload token and save it.
After uploading all images, the images are now moved to the selected album with the upload token. Title and description information added, etc.

Maik
Comment 8 Philippe ROUBACH 2019-08-11 10:48:29 UTC
to transfer 246 photos the estimated time is 2 h.

it would be useful to enrich dialog with user with estimated time like any copy or transfer procedure.

user must be informed when transfer is complete then he can go to google photo to see the album.

User must be informed that 246/246 does not mark the end of the transfer procedure. The button "close" must be in grey till the end or before to close the window you must ask for user if he confirms because there is a procedure running.

Also it would be useful to add an "interrupt" and a "cancel" button.


Thanks
Comment 9 Philippe ROUBACH 2019-08-11 11:55:11 UTC
i retried a transfer of 246 photos.

after getting 246/246 i get the authentication message.

i wait for 30 mn then check :

- progress indicator is still on all 246 thumbnails
- at google photo the album is empty
Comment 10 Philippe ROUBACH 2019-08-11 12:02:25 UTC
in album list if i click on "load again" i get the authentication message.
then
i reconnect to my google account
then
i can refresh the album list.
Comment 11 Philippe ROUBACH 2019-08-11 12:03:56 UTC
Progress indicator on all thumbnails is still there and album still empty.
Comment 12 Philippe ROUBACH 2019-08-11 14:48:42 UTC
i tried to transfer by batch of 48.

at the fifth batch at the moment i get 48/48 i get an error message :

Error transferring https://photoslibrary.googleapis.com/v1/mediaItems:batchCreate - server replied: Bad Request

progress indicator on thumbnails stays spinning.

at google photo then photos was not added.
Comment 13 Maik Qualmann 2019-08-11 15:54:16 UTC
(In reply to Philippe ROUBACH from comment #12)
> i tried to transfer by batch of 48.
> 
> at the fifth batch at the moment i get 48/48 i get an error message :
> 
> Error transferring
> https://photoslibrary.googleapis.com/v1/mediaItems:batchCreate - server
> replied: Bad Request
> 
> progress indicator on thumbnails stays spinning.
> 
> at google photo then photos was not added.

This looks like the bug in digiKam-6.2.0. We wait until you have tested pre-release 6.3.0.

Maik
Comment 14 caulier.gilles 2019-08-11 16:14:59 UTC
Maik, 

Perhaps a tip somewhere in plugin settings widget can help users to prevent future problems about Google download limitations.

Gilles
Comment 15 Philippe ROUBACH 2019-08-12 12:23:56 UTC
today i tried to tansfer the last 48 photos then it fails as previous transfer.

i am going to finish transfer with google photo import tool.
Comment 16 caulier.gilles 2020-01-15 08:47:55 UTC
Today, next digiKam 7.0.0 beta2 64 bits pre-release AppImage bundle will be
published with last Qt5.14 and QtWebEngine by default (instead QtWebkit).

Please test and give us a feedback to see if problem remain on your computer.

https://files.kde.org/digikam/

Thanks in advance

Giles Caulier
Comment 17 Philippe ROUBACH 2020-01-16 07:47:59 UTC
7.00 beta 2

It remains a problem: photos and video are well in the good Google photo album, but they don't appear in "google photo > photos > today".
Comment 18 caulier.gilles 2020-01-16 07:52:00 UTC
Thanks Philippe for the feedback.

Perhaps it depend of the Exif date parsed from image by Google when importing files ? If date is not today...

Gilles
Comment 19 Philippe ROUBACH 2020-01-16 07:55:05 UTC
(In reply to caulier.gilles from comment #18)
> Thanks Philippe for the feedback.
> 
> Perhaps it depend of the Exif date parsed from image by Google when
> importing files ? If date is not today...
> 
> Gilles

no, today refers to the date of the import i never imported the photos same day i took them.
Comment 20 Maik Qualmann 2020-01-16 11:38:26 UTC
The virtual albums like "Today" are managed by Google and we have no influence on that. Google reads the Exif date from the images and sorts them accordingly. The upload date is not interesting. If you use the Time Adjust Tool to change an image to today's date and upload it, it will also be displayed under "Today".

Maik
Comment 21 Philippe ROUBACH 2020-01-16 11:41:10 UTC
if we import photo from google photo with google import tool then imported photos are in the album and in "today"
Comment 22 Philippe ROUBACH 2020-01-16 11:44:39 UTC
i must be accurate for diagnostic.

only the last selected photo was put in "today"
Comment 23 Maik Qualmann 2020-01-16 12:23:31 UTC
Sorry I don't understand that now. Are you talking about Google import tool, so download? In digiKam you cannot select individual images when importing, but only entire albums. And what should the import have to do with the "Today" view on Google Photo?

Maik
Comment 24 Philippe ROUBACH 2020-01-17 08:02:03 UTC
ok i define again the problem.

go to photo.google.com
use import tool
photos are put in album and in "today"

in the past with digikam
use export tool
photos are put in album and in "today"

with digikam 7 beta 2
use export tool
photos are put in album and only one is put in today.

When exporting photos if only one photo is put in "today"
then
hypothesis 1 : google api gets a problem.
hypothesis 2 : digikam export tool gets a problem.

did you try to reproduce phenomenon ?
Comment 25 Philippe ROUBACH 2020-02-15 09:50:33 UTC
I confirm the problem at comment #24

With digikam 6.4.0 there is no problem. When with digikam you export photos to a Google album then photos are copied in the "today" album and in the user album.
Comment 26 Maik Qualmann 2020-02-15 10:16:29 UTC
There are no relevant changes in the upload between 6.4.0 and 7.0.0. I don't know if you mean another virtual "Today" album than I do. But we have no real influence on the "Today" album. A image will appear in the "Today" album if the date IN the metadata of the image is today. If you upload an "old" image, it will not be displayed in "Today". Instead, it is inserted into Google's timeline on the day it was created.

Please send me a image that will be inserted with 6.4.0 in the album and in "Today". And a image that is not inserted in "Today" in 7.0.0.

Maik
Comment 27 Philippe ROUBACH 2020-02-15 10:34:18 UTC
I try to be more accurate.

Today 15/02/2019 i export 2 photos (1/1/20020) and 4 photos (25/12/2019) to an album with digikam 6.4.0

this photos are copied in:

- "Photos" with titles "Mer. 1 Janv. Arcachon" and "Mer. 25 Déc. 2019 Arcachon"

see photos.png capture

- in album "pascal_noel_2019_12_25"

see album.png capture

if i use digikam 7.0.0 then no photo is copied in "photos". Photos are only copied to album.
Comment 28 Philippe ROUBACH 2020-02-15 10:37:10 UTC
Created attachment 126053 [details]
"photos" in google photos
Comment 29 Philippe ROUBACH 2020-02-15 10:37:52 UTC
Created attachment 126055 [details]
my album in google photos
Comment 30 Maik Qualmann 2020-02-15 20:24:05 UTC
I cannot understand your problem. Maybe Gilles can test it too. I have no difference when I upload images with digiKam-6.4.0 or digiKam-7.0.0 (both AppImages). A image from today appears under "Today", the image from January under January. No matter which version.

"Today" in Google is not about the upload date, but the creation date of the image. We only upload the image to the corresponding album, everything else is interpreted by Google. It doesn't change when I upload it via the web interface.

Maik
Comment 31 caulier.gilles 2020-02-15 20:54:08 UTC
I just tested all step that Maik explain, and i cannot reproduce the problem here.

Perhaps you can take a screencast of all operations that you perform to see if an important step is missing from your explaination.

Also, perhaps the console output can help to understand all you google workflow.

Gilles Caulier
Comment 32 Philippe ROUBACH 2020-02-16 09:36:47 UTC
Ok i investigated about the problem.

There is a delay sometime for a photo to be added in google photos "photos". That is why i said photo is only copied in album.

If a photo is already in the Google photos "photos" it is not added in it, only in the album. That is why i said photo is only copied in album.

Conclusion : we must use new photos to test digikam export to google photos and wait some delay.
Comment 33 Maik Qualmann 2020-02-17 12:05:47 UTC
Ok, that makes sense. You have a lot of albums and probably a lot of images on Google Photo. The Google servers will take a while to insert the images into the timeline.

Maik
Comment 34 Philippe ROUBACH 2021-08-04 08:46:58 UTC
no more problem