Bug 269993 - Subdirectories in separated + independant databases
Summary: Subdirectories in separated + independant databases
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Albums (show other bugs)
Version: 2.0.0
Platform: Ubuntu Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-04-03 12:22 UTC by Axel Krebs
Modified: 2022-01-24 16:12 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 7.6.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Axel Krebs 2011-04-03 12:22:45 UTC
Version:           2.0.0 (using KDE 4.6.1) 
OS:                Linux

As much as I understand, digiKam is able to use several _different_ paths for pictures (even on external usb or nfs).

Still _all_ information is kept at the selected location centrally:
- digikam4.db
- thumbnails-digikam4.db

This seems quite risky to me: the more pictures, the larger these databases. 

I suggest for each path _independent_ databases. 

Therefore we get 3 pathes with 3 times, e.g.:
- digikam4_1.db, thumbnails-digikam4_1.db
- digikam4_2.db, thumbnails-digikam4_2.db
- digikam4_3.db, thumbnails-digikam4_3.db

Whenever one wants to deal with a picture, the related database will be loaded automatically.

I think, this separation into "sub-databases" could improve reliability, stability and offer infinite(!!) large  pic collections. Besides, loading times and memory needs would be better. 

Reproducible: Always




P.S.: Maybe my suggestion is related to problems described in [Bug 269989] New: Several crashes, unclear background ( picsize "15GB"?, thumbnails-digikam.db 3,5GB, memory 4GB? )

https://bugs.kde.org/show_bug.cgi?id=269989

           Summary: Several crashes, unclear backround ( picsize "15GB"?,
                    thumbnails-digikam.db 3,5GB, memory 4GB? )
           Product: digikam
           Version: 2.0.0
          Platform: Ubuntu Packages
        OS/Version: Linux
            Status: UNCONFIRMED
          Severity: crash
          Priority: NOR
         Component: general
        AssignedTo: digikam-devel@kde.org
        ReportedBy: axel.krebs@t-online.de


Version:           2.0.0 (using KDE 4.6.1) 
OS:                Linux

I experienced several sudden crashes since yesterday; no delay, following
bug-tracking-system no "useful information".

Found huge file with 15,9 GB size.
Thumbnail-digkam.db is 3,5 GB, while memory holds 4 GB.

Do these details indicate (hardware- or software-)limits??

Reproducible: Always

Steps to Reproduce:
Just re-open digiKam, starting to do anything. Sometimes only waiting may
trigger crash.



DigiKam:
Version 2.0.0-beta4
Unter KDE 4.6.1 (4.6.1)

Hardware:
cpu:        AMD Athlon(tm) II X4 640 Processor
memory:     4000 MB

Statistics:
digiKam version 2.0.0-beta4
AVI:         59
GIF:         99
JPG:      57853
PNG:        650
RAW-CR2:    629
RAW-CRW:  10978
RAW-DNG:      4
RAW-NEF:  26249
TIFF:       700
Gesamtzahl der Einträge: 97221
Alben:     1510
Stichwörter: 44
Datenbanktreiber: QSQLITE

Components:
digiKam version 2.0.0-beta4
Exiv2 kann in JP2 speichern: Ja
Exiv2 kann in JPEG speichern: Ja
Exiv2 kann in PGF speichern: Ja
Exiv2 kann in PNG speichern: Ja
Exiv2 kann in TIFF speichern: Ja
Exiv2 unterstützt XMP-Metadaten: Ja
LibCImg:         130
LibClapack: internal library
LibExiv2:     0.21.1
LibJPEG:          62
LibJasper:   1.900.1
LibKDE:        4.6.1 (4.6.1)
LibKExiv2:     2.0.0
LibKMap:       2.0.0
LibKdcraw:     2.0.0
LibLCMS:         118
LibPGF:      6.09.44 - internal library
LibPNG:       1.2.44
LibQt:         4.7.0
LibRaw:       0.13.2
LibTIFF: LIBTIFF, Version 3.9.4 Copyright (c) 1988-1996 Sam Leffler Copyright
(c) 1991-1996 Silicon Graphics, Inc.
Marble Widget:  0.11.0 (Stable Release)
Parallelisiertes Entfernen von Mosaikmustern: Ja
Datenbanktreiber: QSQLITE
LibGphoto2:  2.4.10.1
LibKface:       2.0.0
LibKipi:        1.2.0
LibOpenCV:      2.1.0
Libface:          0.2
Comment 1 Axel Krebs 2011-10-30 09:30:43 UTC
(In reply to comment #0)
> Version:           2.0.0 (using KDE 4.6.1) 
> OS:                Linux
> 
> As much as I understand, digiKam is able to use several _different_ paths for
> pictures (even on external usb or nfs).
> 
> Still _all_ information is kept at the selected location centrally:
> - digikam4.db
> - thumbnails-digikam4.db
> 
> This seems quite risky to me: the more pictures, the larger these databases. 
> 
> I suggest for each path _independent_ databases. 
> 
> Therefore we get 3 pathes with 3 times, e.g.:
> - digikam4_1.db, thumbnails-digikam4_1.db
> - digikam4_2.db, thumbnails-digikam4_2.db
> - digikam4_3.db, thumbnails-digikam4_3.db
> 
> Whenever one wants to deal with a picture, the related database will be loaded
> automatically.
> 
> I think, this separation into "sub-databases" could improve reliability,
> stability and offer infinite(!!) large  pic collections. Besides, loading times
> and memory needs would be better. 
> 
> Reproducible: Always
> 
> 
> 
> 
> P.S.: Maybe my suggestion is related to problems described in [Bug 269989] New:
> Several crashes, unclear background ( picsize "15GB"?, thumbnails-digikam.db
> 3,5GB, memory 4GB? )
> 
> https://bugs.kde.org/show_bug.cgi?id=269989
> 
>            Summary: Several crashes, unclear backround ( picsize "15GB"?,
>                     thumbnails-digikam.db 3,5GB, memory 4GB? )
>            Product: digikam
>            Version: 2.0.0
>           Platform: Ubuntu Packages
>         OS/Version: Linux
>             Status: UNCONFIRMED
>           Severity: crash
>           Priority: NOR
>          Component: general
>         AssignedTo: digikam-devel@kde.org
>         ReportedBy: axel.krebs@t-online.de
> 
> 
> Version:           2.0.0 (using KDE 4.6.1) 
> OS:                Linux
> 
> I experienced several sudden crashes since yesterday; no delay, following
> bug-tracking-system no "useful information".
> 
> Found huge file with 15,9 GB size.
> Thumbnail-digkam.db is 3,5 GB, while memory holds 4 GB.
> 
> Do these details indicate (hardware- or software-)limits??
> 
> Reproducible: Always
> 
> Steps to Reproduce:
> Just re-open digiKam, starting to do anything. Sometimes only waiting may
> trigger crash.
> 
> 
> 
> DigiKam:
> Version 2.0.0-beta4
> Unter KDE 4.6.1 (4.6.1)
> 
> Hardware:
> cpu:        AMD Athlon(tm) II X4 640 Processor
> memory:     4000 MB
> 
> Statistics:
> digiKam version 2.0.0-beta4
> AVI:         59
> GIF:         99
> JPG:      57853
> PNG:        650
> RAW-CR2:    629
> RAW-CRW:  10978
> RAW-DNG:      4
> RAW-NEF:  26249
> TIFF:       700
> Gesamtzahl der Einträge: 97221
> Alben:     1510
> Stichwörter: 44
> Datenbanktreiber: QSQLITE
> 
> Components:
> digiKam version 2.0.0-beta4
> Exiv2 kann in JP2 speichern: Ja
> Exiv2 kann in JPEG speichern: Ja
> Exiv2 kann in PGF speichern: Ja
> Exiv2 kann in PNG speichern: Ja
> Exiv2 kann in TIFF speichern: Ja
> Exiv2 unterstützt XMP-Metadaten: Ja
> LibCImg:         130
> LibClapack: internal library
> LibExiv2:     0.21.1
> LibJPEG:          62
> LibJasper:   1.900.1
> LibKDE:        4.6.1 (4.6.1)
> LibKExiv2:     2.0.0
> LibKMap:       2.0.0
> LibKdcraw:     2.0.0
> LibLCMS:         118
> LibPGF:      6.09.44 - internal library
> LibPNG:       1.2.44
> LibQt:         4.7.0
> LibRaw:       0.13.2
> LibTIFF: LIBTIFF, Version 3.9.4 Copyright (c) 1988-1996 Sam Leffler Copyright
> (c) 1991-1996 Silicon Graphics, Inc.
> Marble Widget:  0.11.0 (Stable Release)
> Parallelisiertes Entfernen von Mosaikmustern: Ja
> Datenbanktreiber: QSQLITE
> LibGphoto2:  2.4.10.1
> LibKface:       2.0.0
> LibKipi:        1.2.0
> LibOpenCV:      2.1.0
> Libface:          0.2

Is there no one for remarks?

I think, handling of large datasets is a crucial matter. 

Are there considerations going on to adress my suggestions?

I believe, the most secure handling would be keeping every pic and every metadata is independent files(!?). I fear the moment, when digiKam database won't open anymore...

Axel
Comment 2 caulier.gilles 2014-08-07 07:42:32 UTC

*** This bug has been marked as a duplicate of bug 338034 ***
Comment 3 caulier.gilles 2022-01-24 16:12:45 UTC
Fixed in https://bugs.kde.org/show_bug.cgi?id=338034