Bug 346053 - Add Google PhotoSphere support through EquiRectangular projection
Summary: Add Google PhotoSphere support through EquiRectangular projection
Status: REOPENED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Generic-Panorama (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-04-10 19:54 UTC by giggi1999
Modified: 2016-07-05 03:51 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description giggi1999 2015-04-10 19:54:07 UTC
Add the support for edit the google  Photo Sphere XMP Metadata
The full description: https://developers.google.com/photo-sphere/metadata/
That's a useful add-on for the panoramic images.

Currently under Linux don't exists any serious applications for editing these metadata.


Reproducible: Always
Comment 1 Alan Pater 2015-04-24 01:09:48 UTC
I'll take a look at adding support for this to exiv2.
Comment 2 caulier.gilles 2015-05-10 07:22:42 UTC
So this is clearly an UPSTREAM REPORT to Exiv2...

Gilles Caulier
Comment 3 Alan Pater 2015-05-10 17:17:50 UTC
Upstream support for reading and writing these properties has been added to the exiv2 library. 

              http://dev.exiv2.org/issues/1063

In digikam, there is the Panorama Tool, part of kipi-plugins. I haven't looked at that tool nor at what metadata properties it uses.
Comment 4 caulier.gilles 2015-05-10 17:31:29 UTC
If you is interested by metadata set by Panorama tool, look code here, it's simple :

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/panorama/tasks/copyfilestask.cpp#L159

In this method, we write metadata info to created pano image. In line 159, we plug Google+ Pano settings to show image with right effect on web interface of social network.

A similar code can be done at this place to play with PhotoSphere info.

We use KPMetadata class which is a wrapper of libkexiv2 for kipi tools (libkexiv2 is Exiv2 Qt interface from digiKam/KDE)

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/common/libkipiplugins/tools/kpmetadata.h

Gilles
Comment 5 Alan Pater 2015-05-10 21:09:15 UTC
Cool. Photo Sphere is Google+ Pano, they are the same thing (GPano) under different names. So the kipi Panorama Tool already uses Photo Sphere XMP Metadata. I think Hugin also uses Photo Sphere.

So, as well as those gui apps for working with Panoramic images, we now also have support to edit this metadata from the command line!
Comment 6 caulier.gilles 2015-05-11 08:29:14 UTC
Git commit 87d2fb608d634b4c140a81a2ed080f1872256019 by Gilles Caulier.
Committed on 11/05/2015 at 08:28.
Pushed by cgilles into branch 'master'.

enable PhotoSphere metadata support since Exiv2 support this XMP namespace
FIXED-IN: 4.11.0

M  +4    -1    NEWS
M  +38   -29   panorama/importwizard/intropage.cpp
M  +6    -4    panorama/importwizard/intropage.h
M  +2    -1    panorama/tasks/copyfilestask.cpp

http://commits.kde.org/kipi-plugins/87d2fb608d634b4c140a81a2ed080f1872256019
Comment 7 caulier.gilles 2015-05-11 14:26:53 UTC
The implementation to generate equirectangular projection with hugin CLI tool is not implemented in Panorama processing tasks.

I fixed all point to plug metadata about PhotoSphere xmp namespace. but image generated and loading into G+ is badly rendered because it's not in right projection.

Benjamin,

The question still open : how to generate equirectangular projection with Hugin CLI tools ? The current code in OptimisationTask drop -s option for autooptimiser CLI tool which don't create equirectangular projection.

Any suggestion ?

Gilles
Comment 8 caulier.gilles 2015-05-11 14:28:35 UTC
Git commit a4b98570170daa801d4cebf0c611631dc710f1b3 by Gilles Caulier.
Committed on 11/05/2015 at 14:27.
Pushed by cgilles into branch 'master'.

disable PhotoSphere support. We need to found a way to generate equirectangular projection with Hugin CLI tools

M  +10   -7    panorama/importwizard/importwizarddlg.cpp
M  +9    -15   panorama/importwizard/intropage.cpp
M  +12   -12   panorama/manager/actionthread.cpp
M  +4    -0    panorama/manager/actionthread.h
M  +48   -15   panorama/manager/manager.cpp
M  +13   -13   panorama/manager/manager.h
M  +19   -21   panorama/tasks/copyfilestask.cpp
M  +3    -3    panorama/tasks/copyfilestask.h
M  +28   -15   panorama/tasks/optimisationtask.cpp
M  +4    -5    panorama/tasks/optimisationtask.h

http://commits.kde.org/kipi-plugins/a4b98570170daa801d4cebf0c611631dc710f1b3
Comment 9 Benjamin Girault 2015-05-11 22:20:34 UTC
Your last commit is messy between the cosmetic changes and the actual disabling of PhotoSphere you enabled too quickly (it was disabled for a reason after all ;-) ), and hard to read. I cannot guarantee you didn't break anything in the process.

I called it "Google+ Metadata" for the lack of a better name. PhotoSphere is not explicit to the user, and PhotoSphere was used only on G+ at the time. If the name is more transparent nowadays, I don't mind changing the name of the checkbox, but if PhotoSphere is only used by G+ (maybe Google Image now?), then it's not a good idea to use "PhotoSphere metadata"...

As for your question, if the PhotoSphere checkbox is checked, this triggers in CreatePtoTask::run the PTOType::Project::EQUIRECTANGULAR setting. In OptimisationTask::run the "-s" argument to "autooptimiser" is used to optimise the projection. This parameter is disabled when PhotoSphere is checked, like you said, such that the projection set in CreatePtoTask::run should be used.

Everything should be in place to tell the Hugin CLI tools to use the equirectangular projection. If I remember correctly, I disabled it because I couldn't get the G+ 3D viewer to be triggered when submitting panoramas. I won't have the time to dive into it anytime soon, so if anyone is interested, I suggest checking the projection type in the various pto files created during the process (if the projection is indeed the issue):
 - Manager::cpFindPtoData().project.projection
 - Manager::cpCleanPtoData().project.projection
 - Manager::autoOptimisePtoData().project.projection
 - Manager::viewAndCropOptimisePtoData().project.projection
 - ( Manager::previewPtoData().project.projection )
 - ( Manager::panoPtoData().project.projection )
The check can be done in the importwizard/* classes quite easily.
Comment 10 Alan Pater 2015-05-11 22:58:24 UTC
Regarding the name, "Photo Sphere XMP Metadata" appears to be the official name of the schema according to https://developers.google.com/photo-sphere/metadata/

Many if not most XMP schemas could use a more user friendly name. But if every application chooses a different name, that will not make life any easier for any user. For better or worse, best to stick to the official name.
Comment 11 caulier.gilles 2015-05-12 04:35:57 UTC
Benjamin,

>Your last commit is messy between the cosmetic changes and the actual disabling of >PhotoSphere you enabled too quickly (it was disabled for a reason after all ;-) ), and hard to >read. I cannot guarantee you didn't break anything in the process.

No. I don't break anything.

>Everything should be in place to tell the Hugin CLI tools to use the equirectangular projection. If >I remember correctly, I disabled it because I couldn't get the G+ 3D viewer to be triggered when >submitting panoramas.

The G+ equirectangular detection is fixed with my commit. In fact the syntax used to register Photophere metadata in XMP was wrong.

I tried to render a pano in equirectangular projection. G+ detect it and switch to Photosphere viewer on Web interface. But as projection look badly generated by autooptimiser (it's not really a real equirectangular projection), the pano view in G+ is weird.

So i suspect that something is missing somewhere about projection

Gilles
Comment 12 Benjamin Girault 2015-05-12 10:10:44 UTC
Alan: I've looked at the Google's page. Photo Sphere seems to be the Android app to generate panoramas. So maybe a title like "Google Photo Sphere panorama", without the "XMP metadata" since the panorama projection has to be set accordingly (and maybe the output should be a JPEG)?

Gilles: The projection is one thing, but not the only possible culprit. Have you checked the projection in the pto files like I suggested? Also, it seems that Hugin is setting the Exif tag "User Comment" with the projection (among other informations). That may be quicker to determine if the projection is indeed the culprit.

I somehow doubt that a very mature piece of software like Hugin generates a not "real equirectangular projection", especially for such a basic projection. Either your photos are not stitched correctly (for example because of parallax errors from the photo shoot), or the 3D viewer does not understand correctly how to map the pixels in the space (the most likely scenario).

My strongest suspect at the moment is the field of view, rather than the projection. Google's 3D viewer assume by default that the FOV is 360x180. That means that the projection is a rectangle and the horizontal span is 360° and the vertical span 180°. If it's not, the 3D viewer tries anyway and the panorama does not show correctly. In that case, several key XMP fields have to be set with the correct panorama size (it were a 360x180), FOV, and crop (at least). This is the complex part of setting the Show Photo XMP metadata correctly, and maybe the reason I deactivated the code (I slowly begin to remember the complexity of the task).
Comment 13 caulier.gilles 2015-05-12 10:42:42 UTC
>Gilles: The projection is one thing, but not the only possible culprit. Have you checked the >projection in the pto files like I suggested?

No, because in commented code, i don't see any lines about this operation.

Perhaps i forget something when i un-commented code.

>I somehow doubt that a very mature piece of software like Hugin generates a not "real >equirectangular projection", especially for such a basic projection. 

And Hugin support this kind of projection as it's written in documentation. This is not the problem. The real question is how to activate it with CLI tools.

>Either your photos are not stitched correctly (for example because of parallax errors from the >photo shoot), or the 3D viewer does not understand correctly how to map the pixels in the >space (the most likely scenario).

No. Panorama is simple : 3 JPEG without any difficulty to stitch. But it's not a large panorama (aka 360°)

>My strongest suspect at the moment is the field of view, rather than the projection. Google's >3D viewer assume by default that the FOV is 360x180. That means that the projection is a >rectangle and the horizontal span is 360° and the vertical span 180°. If it's not, the 3D viewer >tries anyway and the panorama does not show correctly. In that case, several key XMP fields >have to be set with the correct panorama size (it were a 360x180), FOV, and crop (at least). >This is the complex part of setting the Show Photo XMP metadata correctly, and maybe the >reason I deactivated the code (I slowly begin to remember the complexity of the task).

So in first, i must to try with a 360° pano. I have some sample at home.

For a reduced FOV, there is a computation to add to set right XMP tag for photoshere viewer. Can be complex to deal...

Gilles
Comment 14 Benjamin Girault 2015-05-12 13:45:27 UTC
(In reply to Gilles Caulier from comment #13)
> >Gilles: The projection is one thing, but not the only possible culprit. Have you checked the >projection in the pto files like I suggested?
> 
> No, because in commented code, i don't see any lines about this operation.

There is indeed no commented code for this, but it's a really simple thing to do. For example, for autoOptimisePtoData, you can add something like
kDebug() << d->mngr->autoOptimisePtoData().project.projection;
at line 304 of importwizard/optimizepage.cpp.

> >I somehow doubt that a very mature piece of software like Hugin generates a not "real >equirectangular projection", especially for such a basic projection. 
> 
> And Hugin support this kind of projection as it's written in documentation.
> This is not the problem. The real question is how to activate it with CLI
> tools.

Like I said, activating the equirectangular is done through the pto file (hugin project file), and already taken care of in the code. Every option to generate a given panorama is within the associated pto file. The parameters to the CLI tools are only controlling how to modify the pto files.

> >My strongest suspect at the moment is the field of view, rather than the projection. Google's >3D viewer assume by default that the FOV is 360x180. That means that the projection is a >rectangle and the horizontal span is 360° and the vertical span 180°. If it's not, the 3D viewer >tries anyway and the panorama does not show correctly. In that case, several key XMP fields >have to be set with the correct panorama size (it were a 360x180), FOV, and crop (at least). >This is the complex part of setting the Show Photo XMP metadata correctly, and maybe the >reason I deactivated the code (I slowly begin to remember the complexity of the task).
> 
> So in first, i must to try with a 360° pano. I have some sample at home.

Yes, this is ideal. (Actually, it should be a 360x180 panorama, not just 360.)