Bug 341752 - Resource manager doesn't import it's own bundle proper.
Summary: Resource manager doesn't import it's own bundle proper.
Status: RESOLVED FIXED
Alias: None
Product: krita
Classification: Applications
Component: General (show other bugs)
Version: 2.9 Beta
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Krita Bugs
URL:
Keywords:
: 324333 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-12-10 17:45 UTC by wolthera
Modified: 2015-05-01 13:39 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
A kpp bundle. (1.90 MB, application/x-krita)
2014-12-10 17:46 UTC, wolthera
Details
Testing bundles (1.97 MB, application/zip)
2014-12-19 21:00 UTC, wolthera
Details

Note You need to log in before you can comment on or make changes to this bug.
Description wolthera 2014-12-10 17:45:19 UTC
The resource manager fails to import it's own .kpps. Seems to be reading them as PNGs

Will attach a bundle after report.

Reproducible: Always
Comment 1 wolthera 2014-12-10 17:46:55 UTC
Created attachment 89910 [details]
A kpp bundle.

And attached is a paintop preset bundle so you can see this for yourself.
Comment 2 wolthera 2014-12-19 20:03:39 UTC
Okay, so this briefly seemed to work with brush-tips, but alas, it was not meant to be, as upon deactivating and activating again, the same error occured: libpng error: IDAT: CRC error

However. And this is the interesting part. For some reason Krita thinks all the resources were made in february 2106. In the mata-data it's right, but the rest is apparantly from 2106. This may be why they don't open.

To wit, the errors look something like this:
libpng error: IDAT: CRC error
Failed to load "paintoppresets/Calligraphy_45degrees.kpp" from bundle "/home/wolthera/.kde/share/apps/krita/bundles/Inking_Pack.bundle" 
etc.
Comment 3 wolthera 2014-12-19 21:00:37 UTC
Created attachment 90056 [details]
Testing bundles

Okay, so I went and changed around the kpp bundle so it would have proper dates(just copied from krita's preset folder.) and then I manually recalculated the md5checksums. This didn't help, but I've added the bundle anyway.

I then checked all other resource types:

workspace, gradients, patterns, palettes -> Work fine, both import and activate/deactivate. Their resource bundle image doesn't get read though.

brush-tips -> Works fine on IMPORT, but not on activate/deactivate and new session.

paintop presets-> Don't work in any setting. 

Everything has been added as a zip in attachments.
Comment 4 wolthera 2014-12-19 21:19:54 UTC
my apologies, none of the resource bundles get activated again, because they don't get un-blacklisted upon reinstall.

Why kpps are so special is still a mystery however.
Do notice that both the thumbnail as well as the .kpps get the same libpng error. So there's something weird with the resource thumbnail as well

I'll stop spamming for tonight.
Comment 5 wolthera 2014-12-20 12:46:08 UTC
Okay, so...

1. That the .kpp gets considered to have too many idats makes sense, because kpp uses the extra idats to store paintopsettings.
2. That the preview does as well, doesn't make any sense.

When it comes to paintops, this is the offending line:
if (!res->loadFromDevice(resourceStore->device())) {

Which brings us to 
KisPaintOpPreset::loadFromDevice(QIODevice *dev)

Where we find that.
if (!reader.read(&img)) {
doesn't work.(Doesn't decode the png.)
Comment 6 wolthera 2014-12-20 14:57:00 UTC
boud(OpenSuse): Works.
Slangkamp(kubuntu): Same errors.
Amadiro(Archlinux): Works but does get this error krita(7300)/kio (KDirModel) KDirModelPrivate::_k_slotDeleteItems: No node found for item that was just removed: KUrl("file:///home/amadiro/Downloads/pmlfax.py")

Amadiro(Windows 8): Doesn't work.
Smjert(Debian-testing): Same errors
Comment 7 Sven Langkamp 2014-12-20 16:26:52 UTC
Git commit 39d8d310874e23cfb57760c69e44b85666b63f1e by Sven Langkamp.
Committed on 20/12/2014 at 16:24.
Pushed by langkamp into branch 'calligra/2.9'.

workaround for loading presets from bundles on some systems

M  +3    -1    krita/plugins/extensions/resourcemanager/resourcebundle.cpp

http://commits.kde.org/calligra/39d8d310874e23cfb57760c69e44b85666b63f1e
Comment 8 wolthera 2014-12-21 12:13:59 UTC
Okay, final list of things wrong with the resource manager:

1. The preview isn't loaded due to similar issues.
2. The tags aren't added onto my system. It may be because my tags already exists and Krita doesn't understand how to merge the existing tags with the new tags.
2b. They also only get tags upon restarting krita, but not upon import.
3. Activating/Deactivating doesn't work, because upon deactivating bundles get added to the blacklist but not removed.
4. Edit window doesn't understand which brushes are already in a pack.
5. For some reason the zip file stores everything as being made in 2106 or 1970. The latter I understand somewhat, considering that's the starting year of computer time counting.
6. The description window in brush management should have a scrollbar. My own read-me's are pretty long. I imagine the same for others.
Comment 9 wolthera 2014-12-21 12:46:01 UTC
Git commit b589559480917df94e116215b928c91202fc06d5 by Wolthera van Hovell.
Committed on 21/12/2014 at 12:45.
Pushed by woltherav into branch 'calligra/2.9'.

Using Sven's workaround to get resource bundle pngs working.

We can now see them!

Feel free to suggest alternative workarounds. I am very inexperienced
with IO.

M  +4    -2    krita/plugins/extensions/resourcemanager/resourcebundle.cpp

http://commits.kde.org/calligra/b589559480917df94e116215b928c91202fc06d5
Comment 10 wolthera 2014-12-21 18:20:48 UTC
Fixed the unblacklisting/enabling of resource bundles: http://quickgit.kde.org/?p=calligra.git&a=commit&h=4867a3d977f9bfccc515bf66b474fb8d6f319b5b

1. The tags aren't added onto my system. It may be because my tags already exists and Krita doesn't understand how to merge the existing tags with the new tags.
1b. They also only get tags upon restarting krita, but not upon import.
2. MD5sums in the manifest are not the MD5sums of the actual included presets, may be related to #4, this results in the preset not being removed from the resource server upon deinstall, because Krita can't find a match between md5sums.
3. Edit window doesn't understand which brushes are already in a pack.
4. For some reason the zip file stores everything as being made in 2106 or 1970. The latter I understand somewhat, considering that's the starting year of computer time counting.
5. The description window in brush management should have a scrollbar. My own read-me's are pretty long. I imagine the same for others.
Comment 11 wolthera 2014-12-23 15:24:12 UTC
Git commit 3286d52f67c65f340c776a29a10cb20544f76e14 by Wolthera van Hovell.
Committed on 23/12/2014 at 15:23.
Pushed by woltherav into branch 'calligra/2.9'.

Fix the loading of tags from bundles.

Turns out the reading of tags was pointed at the wrong element, this
resulted in said tags being completely empty instead of the tag in the
manifest.

M  +3    -3    krita/plugins/extensions/resourcemanager/resourcebundle_manifest.cpp

http://commits.kde.org/calligra/3286d52f67c65f340c776a29a10cb20544f76e14
Comment 12 wolthera 2014-12-23 16:23:24 UTC
Git commit 5588176101023968f06715daf8f2f185a4c2d6f0 by Wolthera van Hovell.
Committed on 23/12/2014 at 16:22.
Pushed by woltherav into branch 'calligra/2.9'.

Prevent the resource manager from installing a resource twice

Well, specifically, this prevents the resource manager from installing a
resource which has the same name as a resource existing already on the
related resource server.

This can happen, amongst others, if you modify a bundle resource and
save it. Or if you install a bundle with resources that you already had
installed. This prevents the resource manager from overwriting both
cases.
If necessary we'll think up something different, but this works for now.

tl;dr: No duplicated resources.

M  +72   -23   krita/plugins/extensions/resourcemanager/resourcebundle.cpp

http://commits.kde.org/calligra/5588176101023968f06715daf8f2f185a4c2d6f0
Comment 13 wolthera 2014-12-23 18:57:13 UTC
Git commit a56da8b3c3ddc868e937f5fbf39c271d0d98d2f5 by Wolthera van Hovell.
Committed on 23/12/2014 at 18:48.
Pushed by woltherav into branch 'calligra/2.9'.

Make the resource manager track the md5 sums of what it installs.

The MD5sums inside the manifest of bundles are nonsense.

This commit makes sure that Krita will remember the MD5sums of the
resources it installs, and uses those same values to deinstall them.

Furthermore, it compares them to the values inside the manifest, and
will return which ones are incorrect as a warning message.

M  +83   -17   krita/plugins/extensions/resourcemanager/resourcebundle.cpp
M  +8    -0    krita/plugins/extensions/resourcemanager/resourcebundle.h

http://commits.kde.org/calligra/a56da8b3c3ddc868e937f5fbf39c271d0d98d2f5
Comment 14 Halla Rempt 2015-02-14 10:54:06 UTC
*** Bug 324333 has been marked as a duplicate of this bug. ***
Comment 15 Halla Rempt 2015-02-16 09:43:47 UTC
(gdb) bt
#0  0x00007fffef18c187 in raise () at /lib64/libc.so.6
#1  0x00007fffef18d538 in abort () at /lib64/libc.so.6
#2  0x00007ffff04882b4 in qt_message_output(QtMsgType, char const*) (msgType=msgType@entry=QtFatalMsg, buf=<optimized out>) at global/qglobal.cpp:2359
#3  0x00007ffff0488439 in qt_message(QtMsgType, const char *, typedef __va_list_tag __va_list_tag *) (msgType=msgType@entry=QtFatalMsg, msg=msg@entry=0x7ffff05f5858 "ASSERT: \"%s\" in file %s, line %d", ap=ap@entry=0x7fffffffb248) at global/qglobal.cpp:2405
#4  0x00007ffff0488c44 in qFatal(char const*, ...) (msg=msg@entry=0x7ffff05f5858 "ASSERT: \"%s\" in file %s, line %d") at global/qglobal.cpp:2588
#5  0x00007ffff0488c8a in qt_assert(char const*, char const*, int) (assertion=assertion@entry=0x7ffff7a4160d "preset->valid()", file=file@entry=0x7ffff7a41570 "/home/boud/kde/src/2.9/krita/ui/kis_canvas_resource_provider.cpp", line=line@entry=176) at global/qglobal.cpp:2054
#6  0x00007ffff761464b in KisCanvasResourceProvider::setPaintOpPreset(KisSharedPtr<KisPaintOpPreset>) (this=0x2549a00, preset=...) at /home/boud/kde/src/2.9/krita/ui/kis_canvas_resource_provider.cpp:176
#7  0x00007ffff76cc9a6 in KisPaintopBox::setCurrentPaintop(KoID const&, KisSharedPtr<KisPaintOpPreset>) (this=this@entry=0x26a8460, paintop=..., preset=...)
    at /home/boud/kde/src/2.9/krita/ui/kis_paintop_box.cc:464
#8  0x00007ffff76cd0d1 in KisPaintopBox::setCurrentPaintopAndReload(KoID const&, KisSharedPtr<KisPaintOpPreset>) (this=this@entry=0x26a8460, paintop=..., preset=...)
    at /home/boud/kde/src/2.9/krita/ui/kis_paintop_box.cc:416
#9  0x00007ffff76d26c9 in KisPaintopBox::resourceSelected(KoResource*) (this=this@entry=0x26a8460, resource=0x624c240) at /home/boud/kde/src/2.9/krita/ui/kis_paintop_box.cc:388
#10 0x00007ffff7544f10 in KisPaintopBox::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (_o=0x26a8460, _c=<optimized out>, _id=3, _a=0x7fffffffb7c0) at krita/ui/moc_kis_paintop_box.cpp:111
#11 0x00007ffff05a81fa in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (sender=sender@entry=0x2c11920, m=m@entry=0x7ffff7d44660 <KisPaintOpPresetsChooserPopup::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7fffffffb7c0) at kernel/qobject.cpp:3576
#12 0x00007ffff7540daf in KisPaintOpPresetsChooserPopup::resourceSelected(KoResource*) (this=this@entry=0x2c11920, _t1=0x624c240) at krita/ui/moc_kis_paintop_presets_chooser_popup.cpp:112
#13 0x00007ffff75461b0 in KisPaintOpPresetsChooserPopup::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (_o=0x2c11920, _c=<optimized out>, _id=<optimized out>, _a=0x7fffffffb960)
    at krita/ui/moc_kis_paintop_presets_chooser_popup.cpp:59
#14 0x00007ffff05a81fa in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (sender=0x2adee90, m=m@entry=0x7ffff7d58800 <KisPresetChooser::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7fffffffb960) at kernel/qobject.cpp:3576
#15 0x00007ffff781dc9f in KisPresetChooser::resourceSelected(KoResource*) (this=<optimized out>, _t1=0x624c240) at krita/ui/kis_preset_chooser.moc:105
#16 0x00007ffff781dcfe in KisPresetChooser::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (_o=0x2adee90, _c=32520, _id=0, _a=0x7fffffffbae0) at krita/ui/kis_preset_chooser.moc:54
#17 0x00007ffff05a81fa in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (sender=sender@entry=0x2c12920, m=m@entry=0x7ffff4fac380 <KoResourceItemChooser::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7fffffffbae0) at kernel/qobject.cpp:3576
#18 0x00007ffff4d320ff in KoResourceItemChooser::resourceSelected(KoResource*) (this=this@entry=0x2c12920, _t1=_t1@entry=0x624c240) at libs/widgets/KoResourceItemChooser.moc:114
#19 0x00007ffff4d32549 in KoResourceItemChooser::activated(QModelIndex const&) (this=0x2c12920) at /home/boud/kde/src/2.9/libs/widgets/KoResourceItemChooser.cpp:396
#20 0x00007ffff05a81fa in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (sender=sender@entry=0x2c57640, m=m@entry=0x7ffff4facce0 <KoResourceItemView::staticMetaObject>, local_signal_index=local_signal_index@entry=0, argv=argv@entry=0x7fffffffbc30) at kernel/qobject.cpp:3576
#21 0x00007ffff4d3658d in KoResourceItemView::selectionChanged(QItemSelection const&, QItemSelection const&) (_t1=..., this=0x2c57640) at libs/widgets/KoResourceItemView.moc:101
#22 0x00007ffff4d3658d in KoResourceItemView::selectionChanged(QItemSelection const&, QItemSelection const&) (this=0x2c57640, selected=...) at /home/boud/kde/src/2.9/libs/widgets/KoResourceItemView.cpp:70
#23 0x00007ffff05a81fa in QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (sender=0x2c3f6a0, m=<optimized out>, local_signal_index=<optimized out>, argv=0x7fffffffbd70)
    at kernel/qobject.cpp:3576
#24 0x00007ffff14e6087 in QItemSelectionModel::selectionChanged(QItemSelection const&, QItemSelection const&) () at /usr/lib64/libQtGui.so.4
#25 0x00007ffff14eb6c1 in QItemSelectionModel::emitSelectionChanged(QItemSelection const&, QItemSelection const&) () at /usr/lib64/libQtGui.so.4
#26 0x00007ffff14ebe93 in QItemSelectionModel::select(QItemSelection const&, QFlags<QItemSelectionModel::SelectionFlag>) () at /usr/lib64/libQtGui.so.4
#27 0x00007ffff14c55aa in QTableView::setSelection(QRect const&, QFlags<QItemSelectionModel::SelectionFlag>) () at /usr/lib64/libQtGui.so.4
#28 0x00007ffff149860a in QAbstractItemView::mousePressEvent(QMouseEvent*) () at /usr/lib64/libQtGui.so.4
#29 0x00007ffff0ff1a7b in QWidget::event(QEvent*) () at /usr/lib64/libQtGui.so.4
#30 0x00007ffff138e5ee in QFrame::event(QEvent*) () at /usr/lib64/libQtGui.so.4
#31 0x00007ffff149de13 in QAbstractItemView::viewportEvent(QEvent*) () at /usr/lib64/libQtGui.so.4
#32 0x00007ffff4d36699 in KoResourceItemView::viewportEvent(QEvent*) (this=0x2c57640, event=0x7fffffffc8f0) at /home/boud/kde/src/2.9/libs/widgets/KoResourceItemView.cpp:60
#33 0x00007ffff0594416 in QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*) (this=<optimized out>, receiver=0x2c4fe00, event=0x7fffffffc8f0) at kernel/qcoreapplication.cpp:1063
#34 0x00007ffff0fa274c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib64/libQtGui.so.4
#35 0x00007ffff0fa8dea in QApplication::notify(QObject*, QEvent*) () at /usr/lib64/libQtGui.so.4
#36 0x00007ffff7880f47 in KisApplication::notify(QObject*, QEvent*) (this=<optimized out>, receiver=0x2c4fe00, event=0x7fffffffc8f0) at /home/boud/kde/src/2.9/krita/ui/KisApplication.cpp:488
#37 0x00007ffff05942ad in QCoreApplication::notifyInternal(QObject*, QEvent*) (this=0x7fffffffd120, receiver=0x2c4fe00, event=0x7fffffffc8f0) at kernel/qcoreapplication.cpp:953
#38 0x00007ffff0fa85e3 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool) () at /usr/lib64/libQtGui.so.4
#39 0x00007ffff1019c9b in  () at /usr/lib64/libQtGui.so.4
#40 0x00007ffff101870c in QApplication::x11ProcessEvent(_XEvent*) () at /usr/lib64/libQtGui.so.4
#41 0x00007ffff103ff30 in  () at /usr/lib64/libQtGui.so.4
#42 0x00007ffff0592e6f in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) (this=this@entry=0x7fffffffced0, flags=...) at kernel/qeventloop.cpp:149
#43 0x00007ffff0593165 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) (this=this@entry=0x7fffffffced0, flags=...) at kernel/qeventloop.cpp:204
#44 0x00007ffff05985b9 in QCoreApplication::exec() () at kernel/qcoreapplication.cpp:1225
#45 0x0000000000402a71 in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at /home/boud/kde/src/2.9/krita/main.cc:182
Comment 16 Halla Rempt 2015-02-20 10:07:10 UTC
Git commit 2ea04558b38976ccdad2598c939fa19fea7ecc38 by Boudewijn Rempt.
Committed on 20/02/2015 at 10:04.
Pushed by rempt into branch 'calligra/2.9'.

The KisPaintopPreset::load() method broke when trying to reload a preset
that's in a bundle, and that method is called whenever we activate a
preset unless dirty presets is on.

It's probably worth thinking whether it's a good idea to hit the disk
on every preset change...

Note: this makes krita/image depend on kostore, which is in koodf. Which
is not ideal.

M  +2    -2    krita/image/CMakeLists.txt
M  +42   -9    krita/image/brushengine/kis_paintop_preset.cpp

http://commits.kde.org/calligra/2ea04558b38976ccdad2598c939fa19fea7ecc38
Comment 17 Friedrich W. H. Kossebau 2015-02-23 01:38:38 UTC
> 4. For some reason the zip file stores everything as being made in 2106 or 1970. The latter I understand somewhat, considering that's the starting year of computer time counting.

Those strange timestamps in the zip file are because KoStore just uses the default values "UnknownTime" of KZip when opening a new file to add to the zip file. And this ends up as a bogus value in the file metadata.
But it should not be a matter, as the timestamps in the metadata on the zip file level are not used actually in KoStore.
Still might be nicer to simply use the current datetime when creating the archive, given this is not leaking unwanted info, as this information is also stored in meta.xml file.

Found a mistake in KZip looking into this BTW ;) https://git.reviewboard.kde.org/r/122682/

The timestamps in the first attachment "A kpp bundle" though surprise me, 4 different ones:
the 2016 is expected, but the 1970 and especially the two 2014 have me still scratching my had to find the reason in our code. Is it possible this zip file was modifed manually after being created with Krita?
Comment 18 americo 2015-03-08 12:19:08 UTC
Don't see the own .bundle created with krita 2.9 - ubuntu gnome 14.04 > dimula ppa
But if try open via double click them krita returns the message is:
Could not open /home/jag/Documents/prova.bundle
Reason: Invalid document: no file 'maindoc.xml'.
Comment 19 americo 2015-03-08 12:30:34 UTC
(In reply to americo from comment #18)
> Don't see the own .bundle created with krita 2.9 - ubuntu gnome 14.04 >
> dimula ppa
> But if try open via double click them krita returns the message is:
> Could not open /home/jag/Documents/prova.bundle
> Reason: Invalid document: no file 'maindoc.xml'.

I open the zip archive and we have a date wrong... in my case is 07 feb 2106 :-)
Comment 20 Halla Rempt 2015-05-01 13:39:11 UTC
File dates are fixed now, for all that I can tell.