Bug 206842 - New tool to export to Wikimedia Commons
Summary: New tool to export to Wikimedia Commons
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-WebService-MediaWiki (show other bugs)
Version: 2.6.0
Platform: Mandriva RPMs Unspecified
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-09-09 11:32 UTC by Peter Potrowl
Modified: 2018-01-30 21:26 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In: 2.0.0
Sentry Crash Report:


Attachments
Patch to clear the problem with mediwiki plugin (3.16 KB, patch)
2012-02-14 17:20 UTC, PARTHASARATHY
Details
Patch to clear the problem with mediwiki plugin (3.93 KB, patch)
2012-02-15 11:59 UTC, PARTHASARATHY
Details
Patch to clear the problem with mediwiki plugin + changed the urlentry type. (8.99 KB, patch)
2012-02-21 17:41 UTC, PARTHASARATHY
Details
Patch to clear the problem with mediwiki plugin + changed the urlentry type. (10.67 KB, patch)
2012-02-21 17:51 UTC, PARTHASARATHY
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Potrowl 2009-09-09 11:32:15 UTC
Version:            (using KDE 4.2.4)
Installed from:    Mandriva RPMs

digiKam should be able to export photos and descriptions to Wikimedia Commons (and possibly to any wiki), as it is for Flickr or Facebook, for instance.

That would be a great gain of time. Currently, tools like Commonist (http://djini.de/software/commonist/) can upload a lot of images on wikis, but we are still forced to copy/paste the descriptions/location/categories from digiKam to that tool.

Other possible improvements could be:
* automatic generation of location wiki tags {{location dec|1.234567|7.654321}} from the coordinates
* wiki categories suggestion from the tags
Comment 1 Guillaume Paumier 2009-09-09 11:40:47 UTC
Work is currently underway regarding the integration of an upload tool to Wikimedia Commons as part of the KIPI plugins. See http://commons.wikimedia.org/wiki/User:Guillom/KIPI

See also bug 143978
Comment 2 caulier.gilles 2011-02-02 17:57:43 UTC
*** Bug 265202 has been marked as a duplicate of this bug. ***
Comment 3 caulier.gilles 2011-02-02 18:02:22 UTC
I work with Guilaume Paumier on this subject during KDE-Imaging coding sprint 2009. 

Some code have been done, to create a new plugin, but it's so far uncomplete. At least, you can use this code to continue, based on Facebook export tool.

Look code here :

http://websvn.kde.org/branches/work/~cgilles/wikimedia/

Gilles Caulier
Comment 4 Bussenot 2011-02-09 16:53:05 UTC
Hello,

We had a look to your facebook code but there are no comments in the source code. We would know what classes we have to modify and what classes we have to re-use.
We also need some information about your facebook classes. 

IUP ISI Team
Comment 5 caulier.gilles 2011-02-09 17:31:37 UTC
I CC Luka Renko who is author of FaceBook tool. He is the right guy to guide you in this tool classes.

Gilles Caulier
Comment 6 Bussenot 2011-02-16 13:51:47 UTC
Hello,
We would like to know how works metadata in Digikam. Moreover, we want to extract and send them to Commons.Wikimedia.
Best regards,
The IUP ISI team.
Comment 7 caulier.gilles 2011-02-16 14:08:58 UTC
What do you mean by Metadata ?

Rating? Tags? Exif/XMP properties ?

Gilles Caulier
Comment 8 Guillaume Paumier 2011-02-16 14:29:09 UTC
The main metadata relevant for Wikimedia Commons are:
1. Image description (in multiple languages, if available)
2. Author's name
3. Copyright information
4. Date created
5. (optional) Geolocation
6. (optional) Tags, to match categories on Wikimedia Commons

This information can be stored in the digiKam database, or in the EXIF/IPTC/XMP metadata, whether the user has chosen to write metadata to the files or not.

The tool should extract this information from the database or the files, then include and format them appropriately to build the file description page on Wikimedia Commons.

Robin, I suggest you look at similar KIPI plugins (such as the facebook or flickr plugins) to see what classes & methods are used to access this information.
Comment 9 caulier.gilles 2011-02-16 14:41:08 UTC
About code to extract digiKam DB information, look in digiKam kipi interface :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/kipiiface/kipiimageinfo.cpp#L125

In fact this metadata are named Attributes in this interface. Not all digiKam metadata are exported to plugin, but it's very easy to handle new one. The container in digiKam is Imageinfo :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/database/imageinfo.h

An example how to play with this attributes into a plugin :

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/picasawebexport/picasawebwindow.cpp#L446

Gilles Caulier
Comment 10 caulier.gilles 2011-04-13 13:24:54 UTC
Guillaume,

As you have said me recently, Mediawiki plugin cannot be used in production now due to a wrong way to talk with mediawiki web service.

It work (just tested) but it use a non standard way... Right ?

This require some changes in source code (libmediawiki ?)

Can you give me a plan of fix to see if i can enable this tool in next kipi-plugins 2.0.0 ?

Thanks in advance

Gilles Caulier
Comment 11 Guillaume Paumier 2011-05-11 21:16:11 UTC
(In reply to comment #10)
> 
> It work (just tested) but it use a non standard way... Right ?

Not exactly. The protocol used is fine. The main problem is with missing information.

When you upload to Wikimedia Commons, the following information is mandatory for each upload:
1. Description of the topic of the picture (what is it about?) (and ideally, the language in which the description is provided)
2. Author and license (copyright information)
3. Categories (i.e. classification)

Point 2. is the most important, and is implemented almost perfectly. No work is needed for now.

Point 1. is implemented half-way: The user is able to provide a description when they open the export window to Commons, but it's not practical when you're uploading multiple images. If you want to provide a different description for each image (which is usually the case), you need to upload them one by one, which kind of defeats the purpose of a mass uploader plugin.

So what is needed for point 1. is the ability to automatically pull the description from the digikam database or the file metadata. Digikam provides a very good interface to add descriptions to a picture, and in several languages (which is ideal for Wikimedia Commons). So we just need to pull that information and use it when we build the image description page for Commons. Although this is not a blocker, it doesn't really make sense to release without this feature imho.

The real blocker is point 3.: the plugin currently offers no way for the user to provide categories for the images they upload. If we release the tool as is and people start uploading their images without categories, they'll get a lot of hostile messages from the Commons community who'll have to clean up their uploads. The digikam / kipi people will get blamed because the plugin simply doesn't allow the users to add categories.

There are two ways to fix this issue.
* One way is to do something similar to what we have for the description: add a field in the window of the uploader, and apply the categories in that field to all the uploads. This solution suffers from the same flaws as explained for the descriptions: usually, the files you upload need different categories. Which leads me to the other solution...
* Use special tags as categories. The solution is not ideal but it's the best possible if we want to release the plugin soon (and I think it's good enough for now). Digikam has an awesome built-in tagging interface. Let the user use it to indicate the Commons categories they want for each picture. All you need to do is make these category-tags special in some way. For example, you (as developers) can decide that Commons categories will be the tags that have the "Category:" prefix. Or that they're the tags that have the "Wikimedia Commons categories" parent tag. In any case, the user can add these tags to their images (using the names of existing Commons categories that they'll find on Commons), and when they upload the files, you simply read the special tags assigned to each image. Done.

Also, each picture that is missing either a description, or an author, or a license, or at least one category shouldn't be uploaded. I realize this is a bit cumbersome, but it's the way things work for now on Commons. Maybe in a future evolution of the plugin, we can offer a better interface to explain this and encourage the user to provide the information.

Another point: the plugin is technically able to upload to any MediaWiki-powered site, but the interface doesn't provide a way to specify that wiki. I recommend to advertise the plugin as a "Wikimedia Commons export plugin" for now, and rename the appropriate files/folders. I completely agree that it'd be nice to provide a generic export plugin to any MediaWiki site, but for now we don't have that.

For the same reason, I recommend to remove the drop-down menu in the export window that lets the user choose between the test Wikipedia, the French Wikipedia and the English Wikipedia as target for the upload. Those were fine for testing, but for production use, we should only upload to Wikimedia Commons, not to any of the three wikis currently in the drop-down menu.

There are other things that should be improved, but the list above summarizes the most important items. For me, it was easier to list everything here, but feel free to break the list into several bugs if you find it more convenient. 

Last, here's a list of easy things to fix for which I can provide patches in the next few days:
* The description page that is generated has a few issues, mostly due to extra spaces.
* We should add a tracking category to know which images were upload with the kipi plugin.

If something is unclear in what I listed here, please let me know in the follow-up comments. I'll do my best to answer in a timely manner.
Comment 12 caulier.gilles 2011-05-11 23:02:34 UTC
Guillaume (Hormière),

It sound possible to fix these issues reported by Guillame Paumier, before 2.0.0 final release planed arounf july ?

Gilles Caulier
Comment 13 caulier.gilles 2011-07-11 06:55:56 UTC
To all wikimedia developers,

I don't receive any feedback since Guillaume Paumier comment #11

Kipiplugins and digiKam 2.0.0 final release are planed in few weeks.

- I must to know if i lets Mediawiki Export tool disabled for this release ? 
- Do you plan to work on it in a near future ? 

Best

Gilles Caulier
Comment 14 Yoms 2011-07-11 07:18:35 UTC
I did not planed to work in a near future on Mediawiki Export plugin, only on Libmediawiki if the library need some fix and improvement.

Best

Guillaume Hormière.
Comment 15 PARTHASARATHY 2012-02-14 09:20:32 UTC
https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/annotate/mediawiki/wikimediajob.cpp#L140


In the buildwikitext function, what is the syntax for adding the list of categories before uploading to wiki?
Comment 16 caulier.gilles 2012-02-14 11:05:48 UTC
Guillaume (Paumier),

Look #15, this question is for you...

Gilles Caulier
Comment 17 Guillaume Paumier 2012-02-14 13:10:53 UTC
(In reply to comment #15)
> https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/annotate/mediawiki/wikimediajob.cpp#L140
> 
> 
> In the buildwikitext function, what is the syntax for adding the list of
> categories before uploading to wiki?

It would probably be something like:
text.append( "\n[[Category:Foobar]]");

Where Foobar is the name of the category, and everything in a loop to add all the categories
Comment 18 PARTHASARATHY 2012-02-14 17:20:52 UTC
Created attachment 68795 [details]
Patch to clear the problem with mediwiki plugin

1) Removed the Description edit box and fetching description from kpimageinfo class.

2) Fetching the tags from kpimageinfo class and appending it in the buildwikitext function.
Comment 19 caulier.gilles 2012-02-14 17:24:11 UTC
Guillame,

This patch go to the right way for you ? Please take a look.

Thanks in advance

Gilles Caulier
Comment 20 PARTHASARATHY 2012-02-15 11:59:55 UTC
Created attachment 68817 [details]
Patch to clear the problem with mediwiki plugin

1) Description from kpimageinfo.h
2) Tags with "wikimedia" in them become the categories for wikimedia.
Comment 21 PARTHASARATHY 2012-02-15 12:01:28 UTC
Guillaume,

I have attached a patch, please test it and confirm.
Comment 22 caulier.gilles 2012-02-15 12:03:14 UTC
Guillaume,

Patch from Parthasarathy is enough to enable mediawiki kipi plugin for production ?

Gilles Caulier
Comment 23 Guillaume Paumier 2012-02-16 12:10:10 UTC
(In reply to comment #21)
> Guillaume,
> 
> I have attached a patch, please test it and confirm.

Thank you! I'll do that shortly and follow up on this bug.
Comment 24 PARTHASARATHY 2012-02-21 17:41:27 UTC
Created attachment 68983 [details]
Patch to clear the problem with mediwiki plugin + changed the urlentry type.

1) Categories for mediawiki upload.
2) Description from imageinfo
3) KurlCombobox for urlediting.
Comment 25 PARTHASARATHY 2012-02-21 17:51:28 UTC
Created attachment 68986 [details]
Patch to clear the problem with mediwiki plugin + changed the urlentry type.

1) Description from imageinfo
2) Categories for mediawiki
3) Kurlcomborequester for entering url
Comment 26 caulier.gilles 2012-02-23 15:24:31 UTC
Git commit 72f7fa88be6b4a742b1696ef930851428f534359 by Gilles Caulier.
Committed on 23/02/2012 at 16:19.
Pushed by cgilles into branch 'master'.

apply patch #68986 from Parthasarathy Gopavarapu + few improvement of code about gui layout and coding style.
wiki url history is saved/restored between sessions.
All points listed by Guillaume Paumier from comment #11 of bug #206842 are implemented.

M  +1    -2    CMakeLists.txt
M  +9    -8    mediawiki/wikimediajob.cpp
M  +9    -5    mediawiki/wikimediajob.h
M  +81   -64   mediawiki/wmwidget.cpp
M  +26   -22   mediawiki/wmwidget.h
M  +50   -30   mediawiki/wmwindow.cpp
M  +21   -15   mediawiki/wmwindow.h

http://commits.kde.org/kipi-plugins/72f7fa88be6b4a742b1696ef930851428f534359
Comment 27 caulier.gilles 2012-02-23 15:27:53 UTC
To Guillame (Paumier),

With my last commit, plugin is fixed following your recommendations, listed into #11.

Thanks to PARTHASARATHY to contribute.

Plugin is now enabled by default, and will be available for next 2.6.0-beta2 release, planed into 2 weeks.

Please test current implementation to see if nothing have been forget before to publish it in production.

Thanks in advance

Gilles Caulier
Comment 28 caulier.gilles 2012-03-16 15:59:09 UTC
I close this file. all is implemented. Plugin is now open for bug report

Gilles Caulier