Bug 478674

Summary: Find new items at startup very slow
Product: [Applications] digikam Reporter: Gerd Mann <gerd.mann>
Component: Database-ScanAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: major CC: atoms, caulier.gilles, iwannaberich, metzpinguin
Priority: NOR    
Version: 8.2.0   
Target Milestone: ---   
Platform: Appimage   
OS: Linux   
Latest Commit: Version Fixed In: 8.3.0
Sentry Crash Report:
Attachments: Log file from appimage digiKam 8.2

Description Gerd Mann 2023-12-18 04:34:13 UTC
After switching from Digikam V8.1 to V8.2 (both appimage) startup is very slow. At startup the function "Find new items" is evoked. In V8.1 (and previous versions) this process took less then a minute. In V8.2 it takes >40 minutes.

When changing settings to "Miscellaneous Settings -> Fast Scan -> Checked" the scan is much faster.
However, after adding new images to the collection and activating "Refresh" the images are not show for a long period.
With this behaviour V8.2 is almost not usable. 

My configuration:
Ubuntu 22.04
Digikam database on a local SSD.
Image collection (>140k images) on a NAS.
SQLite

***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***


STEPS TO REPRODUCE
1. Start Digikam
2. Process "Find new items" started automatically

OBSERVED RESULT
Process "Find new items" finished after >40 minutes

EXPECTED RESULT
Process "Find new items" should be finished after  <=1 minute

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: Ubuntu 22.04
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Comment 1 caulier.gilles 2023-12-18 06:25:46 UTC
What's the Qt version used by digiKam ? 5 or 6 ? Go to Help Component info dialog for details.

Gilles Caulier
Comment 2 caulier.gilles 2023-12-18 06:27:02 UTC
Also did you use Snap package ?

Can you run digiKam from a console after to enable debug trace in Setup/Miscs/System section ?

Gilles Caulier
Comment 3 Gerd Mann 2023-12-18 07:10:26 UTC
digikam version 8.2.0
OpenCV Configuration: 
Threads features: 
Parallel framework: pthreads
Number of Threads: 16
OpenCL platform: 
AMD Accelerated Parallel Processing: 
dGPU: gfx1031 (OpenCL 2.0 )
OpenCL Device extensions: 
cl_khr_subgroups: Yes
cl_khr_local_int32_extended_atomics: Yes
cl_khr_local_int32_base_atomics: Yes
cl_khr_int64_extended_atomics: Yes
cl_khr_int64_base_atomics: Yes
cl_khr_image2d_from_buffer: Yes
cl_khr_global_int32_extended_atomics: Yes
cl_khr_global_int32_base_atomics: Yes
cl_khr_gl_sharing: Yes
cl_khr_fp64: Yes
cl_khr_fp16: Yes
cl_khr_depth_images: Yes
cl_khr_byte_addressable_store: Yes
cl_khr_3d_image_writes: Yes
cl_amd_media_ops2: Yes
cl_amd_media_ops: Yes
cl_amd_device_attribute_query: Yes
cl_amd_copy_buffer_p2p: Yes
cl_amd_assembly_program: Yes
OpenCL Device: 
Version: OpenCL 2.0 
Type: dGPU
Preferred vector width short: 2
Preferred vector width long: 1
Preferred vector width int: 1
Preferred vector width half: 1
Preferred vector width float: 1
Preferred vector width double: 1
Preferred vector width char: 4
Name: gfx1031
Max work group size: 256
Max memory allocation size: 10 GB 191 MB 204 KB 816 B
Local memory size: 64 KB
Host unified memory: No
Has AMD Fft: No
Has AMD Blas: No
Half support: Yes
Driver version: 3590.0 (HSA1.1,LC)
Double support: Yes
Compute units: 20
Address bits: 64
Hardware features: 
SSSE3: Yes
SSE4.2: Yes
SSE4.1: Yes
SSE3: Yes
SSE2: Yes
SSE: Yes
POPCNT: Yes
MMX: Yes
FP16: Yes
FMA3: Yes
AVX2: Yes
AVX: Yes
Manifests: 
O2 Snapshoot 2022-02-10: 
o2: 99902cc37e083a8311c1f8eee918e93c93cbc937
LIBRAW Snapshoot 2023-11-21: 
libraw: 41506ef73e33c5e0c3b94608cce2481d92003d5d
LIBLQR Snapshoot 2023-11-06: 
liblqr: 7b88c05cec2f0e60125e1cfcad18c79721a2b27c
LENSFUN Snapshoot 2023-11-29: 
lensfun: 7d72e1a552f66d03cd8bea8d4921dc6129eb7e5f
HEIF Snapshoot 2023-10-26: 
heif: 06f6acfe735379b9c3bcf865285f6dba80611e90
EXIV2 Snapshoot 2023-11-29: 
exiv2: 5c91bfc8b5bd2227bf0e2a6607feb85a4020064e
Libraries: 
XMP SDK: 5.6.0
Qt WebEngine version: 5.15.11
Qt Framework: 5.15.11
Marble: 23.8.1
Libx265: 2.6
LibTIFF: 4.0.9
LibRaw: 0.21.1
LibPNG: 1.6.34
LibPGF: 7.21.07
LibOpenCV: 4.8.1
LibLCMS: 2090
LibJPEG: 80
LibJasper: 4.0.0
LibHEIF: 1.15.2
LibGphoto2: 2.5.16
LibCImg: 130
LibAVUtil: 56.70.100
LibAVFormat: 58.76.100
LibAVCodec: 58.134.100
LensFun: 0.3.95-0
KDE Framework: 5.110.0
ImageMagick codecs: 7.1.1
Exiv2: 0.28.1
ExifTool: 12.29
Eigen: 3.3.4
DNG SDK: 1.5.1
FFmpeg Configuration: 
Video Encoders: 
vc2: SMPTE VC-2
prores_ks: Apple ProRes (iCodec Pro)
prores_aw: Apple ProRes
ljpeg: Lossless JPEG
libxvid: libxvidcore MPEG-4 part 2
libx265: libx265 H.265 / HEVC
libx264rgb: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB
libx264: libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
libtheora: libtheora Theora
a64multi5: Multicolor charset for Commodore 64, extended with 5th color (colram)
a64multi: Multicolor charset for Commodore 64
Video Decoders: 
zmbv: Zip Motion Blocks Video
zlib: LCL (LossLess Codec Library) ZLIB
zerocodec: ZeroCodec Lossless Video
yuv4: Uncompressed packed 4:2:0
yop: Psygnosis YOP Video
ylc: YUY2 Lossless Codec
y41p: Uncompressed YUV 4:1:1 12-bit
xwd: XWD (X Window Dump) image
xpm: XPM (X PixMap) image
xl: Miro VideoXL
xface: X-face image
xbm: XBM (X BitMap) image
xbin: eXtended BINary text
xan_wc4: Wing Commander IV / Xxan
xan_wc3: Wing Commander III / Xan
wrapped_avframe: AVPacket to AVFrame passthrough
wnv1: Winnov WNV1
wmv3image: Windows Media Video 9 Image
wmv3: Windows Media Video 9
wmv2: Windows Media Video 8
wmv1: Windows Media Video 7
webp: WebP image
wcmv: WinCAM Motion Video
vqavideo: Westwood Studios VQA (Vector Quantized Animation) video
vp9_v4l2m2m: V4L2 mem2mem VP9 decoder wrapper
vp9: Google VP9
vp8_v4l2m2m: V4L2 mem2mem VP8 decoder wrapper
vp8: On2 VP8
vp7: On2 VP7
vp6f: On2 VP6 (Flash version)
vp6a: On2 VP6 (Flash version, with alpha channel)
vp6: On2 VP6
vp5: On2 VP5
vp4: On2 VP4
vp3: On2 VP3
vmnc: VMware Screen Codec / VMware Video
vmdvideo: Sierra VMD video
vcr1: ATI VCR1
vc1image: Windows Media Video 9 Image v2
vc1_v4l2m2m: V4L2 mem2mem VC1 decoder wrapper
vc1: SMPTE VC-1
vble: VBLE Lossless Codec
vb: Beam Software VB
v410: Uncompressed 4:4:4 10-bit
v408: Uncompressed packed QT 4:4:4:4
v308: Uncompressed packed 4:4:4
v210x: Uncompressed 4:2:2 10-bit
v210: Uncompressed 4:2:2 10-bit
utvideo: Ut Video
ultimotion: IBM UltiMotion
txd: Renderware TXD (TeXture Dictionary) image
tscc2: TechSmith Screen Codec 2
truemotion2rt: Duck TrueMotion 2.0 Real Time
truemotion2: Duck TrueMotion 2.0
truemotion1: Duck TrueMotion 1.0
tmv: 8088flex TMV
tiff: TIFF image
tiertexseqvideo: Tiertex Limited SEQ video
thp: Nintendo Gamecube THP video
theora: Theora
tdsc: TDSC
targa_y216: Pinnacle TARGA CineWave YUV16
targa: Truevision Targa image
svq3: Sorenson Vector Quantizer 3 / Sorenson Video 3 / SVQ3
svq1: Sorenson Vector Quantizer 1 / Sorenson Video 1 / SVQ1
sunrast: Sun Rasterfile image
srgc: Screen Recorder Gold Codec
speedhq: NewTek SpeedHQ
sp5x: Sunplus JPEG (SP5X)
snow: Snow
smvjpeg: SMV JPEG
smc: QuickTime Graphics (SMC)
smackvid: Smacker video
simbiosis_imx: Simbiosis Interactive IMX Video
sheervideo: BitJazz SheerVideo
sgirle: Silicon Graphics RLE 8-bit video
sgi: SGI image
sga: Digital Pictures SGA Video
screenpresso: Screenpresso
scpr: ScreenPressor
sanm: LucasArts SANM/Smush video
rv40: RealVideo 4.0
rv30: RealVideo 3.0
rv20: RealVideo 2.0
rv10: RealVideo 1.0
rscc: innoHeim/Rsupport Screen Capture Codec
rpza: QuickTime video (RPZA)
roqvideo: id RoQ video
rl2: RL2 video
rawvideo: raw video
rasc: RemotelyAnywhere Screen Capture
r210: Uncompressed RGB 10-bit
r10k: AJA Kona 10-bit RGB Codec
qtrle: QuickTime Animation (RLE) video
qpeg: Q-team QPEG
qdraw: Apple QuickDraw
ptx: V.Flash PTX image
psd: Photoshop PSD file
prosumer: Brooktree ProSumer Video
prores: Apple ProRes (iCodec Pro)
ppm: PPM (Portable PixelMap) image
png: PNG (Portable Network Graphics) image
pixlet: Apple Pixlet
pictor: Pictor/PC Paint
photocd: Kodak Photo CD
pgx: PGX (JPEG2000 Test Format)
pgmyuv: PGMYUV (Portable GrayMap YUV) image
pgm: PGM (Portable GrayMap) image
pfm: PFM (Portable FloatMap) image
pcx: PC Paintbrush PCX image
pbm: PBM (Portable BitMap) image
pam: PAM (Portable AnyMap) image
paf_video: Amazing Studio Packed Animation File Video
nuv: NuppelVideo/RTJPEG
notchlc: NotchLC
mxpeg: Mobotix MxPEG video
mwsc: MatchWare Screen Capture Codec
mvha: MidiVid Archive Codec
mvdv: MidiVid VQ
mvc2: Silicon Graphics Motion Video Compressor 2
mvc1: Silicon Graphics Motion Video Compressor 1
mv30: MidiVid 3.0
mts2: MS Expression Encoder Screen
mszh: LCL (LossLess Codec Library) MSZH
msvideo1: Microsoft Video 1
mss2: MS Windows Media Video V9 Screen
mss1: MS Screen 1
msrle: Microsoft RLE
msp2: Microsoft Paint (MSP) version 2
msmpeg4v2: MPEG-4 part 2 Microsoft variant version 2
msmpeg4v1: MPEG-4 part 2 Microsoft variant version 1
msmpeg4: MPEG-4 part 2 Microsoft variant version 3
mscc: Mandsoft Screen Capture Codec
msa1: MS ATC Screen
mpegvideo: MPEG-1 video
mpeg4_v4l2m2m: V4L2 mem2mem MPEG4 decoder wrapper
mpeg4: MPEG-4 part 2
mpeg2video: MPEG-2 video
mpeg2_v4l2m2m: V4L2 mem2mem MPEG2 decoder wrapper
mpeg1video: MPEG-1 video
mpeg1_v4l2m2m: V4L2 mem2mem MPEG1 decoder wrapper
motionpixels: Motion Pixels video
mobiclip: MobiClip Video
mmvideo: American Laser Games MM Video
mjpegb: Apple MJPEG-B
mjpeg: MJPEG (Motion JPEG)
mimic: Mimic
mdec: Sony PlayStation MDEC (Motion DECoder)
magicyuv: MagicYUV video
m101: Matrox Uncompressed SD
lscr: LEAD Screen Capture
loco: LOCO
libvpx-vp9: libvpx VP9
libvpx: libvpx VP8
libaom-av1: libaom AV1
lagarith: Lagarith lossless
kmvc: Karl Morton's video codec
kgv1: Kega Game Video
jv: Bitmap Brothers JV video
jpegls: JPEG-LS
jpeg2000: JPEG 2000
ipu: IPU Video
interplayvideo: Interplay MVE video
indeo5: Intel Indeo Video Interactive 5
indeo4: Intel Indeo Video Interactive 4
indeo3: Intel Indeo 3
indeo2: Intel Indeo 2
imm5: Infinity IMM5
imm4: Infinity IMM4
iff: IFF ACBM/ANIM/DEEP/ILBM/PBM/RGB8/RGBN
idf: iCEDraw text
idcinvideo: id Quake II CIN video
hymt: HuffYUV MT
huffyuv: Huffyuv / HuffYUV
hqx: Canopus HQX
hq_hqa: Canopus HQ/HQA
hnm4video: HNM 4 video
hevc_v4l2m2m: V4L2 mem2mem HEVC decoder wrapper
hevc: HEVC (High Efficiency Video Coding)
hap: Vidvox Hap
h264_v4l2m2m: V4L2 mem2mem H.264 decoder wrapper
h264: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
h263p: H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2
h263i: Intel H.263
h263_v4l2m2m: V4L2 mem2mem H.263 decoder wrapper
h263: H.263 / H.263-1996, H.263+ / H.263-1998 / H.263 version 2
h261: H.261
gif: GIF (Graphics Interchange Format)
gdv: Gremlin Digital Video
g2m: Go2Meeting
frwu: Forward Uncompressed
fraps: Fraps
fmvc: FM Screen Capture Codec
flv: FLV / Sorenson Spark / Sorenson H.263 (Flash Video)
flic: Autodesk Animator Flic video
flashsv2: Flash Screen Video v2
flashsv: Flash Screen Video v1
fits: Flexible Image Transport System
fic: Mirillis FIC
ffvhuff: Huffyuv FFmpeg variant
ffv1: FFmpeg video codec #1
exr: OpenEXR image
escape130: Escape 130
escape124: Escape 124
eatqi: Electronic Arts TQI Video
eatgv: Electronic Arts TGV video
eatgq: Electronic Arts TGQ video
eamad: Electronic Arts Madcow Video
eacmv: Electronic Arts CMV video
dxv: Resolume DXV
dxtory: Dxtory
dxa: Feeble Files/ScummVM DXA
dvvideo: DV (Digital Video)
dsicinvideo: Delphine Software International CIN video
dpx: DPX (Digital Picture Exchange) image
dnxhd: VC3/DNxHD
dirac: BBC Dirac VC-2
dfa: Chronomaster DFA
dds: DirectDraw Surface image decoder
cyuv: Creative YUV (CYUV)
cri: Cintel RAW
cpia: CPiA video format
cllc: Canopus Lossless Codec
cljr: Cirrus Logic AccuPak
clearvideo: Iterated Systems ClearVideo
cinepak: Cinepak
cfhd: GoPro CineForm HD
cdxl: Commodore CDXL video
cdtoons: CDToons video
cdgraphics: CD Graphics video
cavs: Chinese AVS (Audio Video Standard) (AVS1-P2, JiZhun profile)
camtasia: TechSmith Screen Capture Codec
camstudio: CamStudio
c93: Interplay C93
brender_pix: BRender PIX image
bmv_video: Discworld II BMV video
bmp: BMP (Windows and OS/2 bitmap)
bitpacked: Bitpacked
bintext: Binary text
binkvideo: Bink video
bfi: Brute Force & Ignorance
bethsoftvid: Bethesda VID video
ayuv: Uncompressed packed MS 4:4:4:4
avui: Avid Meridien Uncompressed
avs: AVS (Audio Video Standard) video
avrp: Avid 1:1 10-bit RGB Packer
avrn: Avid AVI Codec
av1: Alliance for Open Media AV1
aura2: Auravision Aura 2
aura: Auravision AURA
asv2: ASUS V2
asv1: ASUS V1
argo: Argonaut Games Video
arbc: Gryphon's Anim Compressor
apng: APNG (Animated Portable Network Graphics) image
ansi: ASCII/ANSI art
anm: Deluxe Paint Animation
amv: AMV Video
alias_pix: Alias/Wavefront PIX image
aic: Apple Intermediate Codec
agm: Amuse Graphics Movie
aasc: Autodesk RLE
8bps: QuickTime 8BPS video
4xm: 4X Movie
012v: Uncompressed 4:2:2 10-bit
Supported Extensions: 
y4m: YUV4MPEG pipe
xml: WebM DASH Manifest
wv: raw WavPack
wtv: Windows Television (WTV)
webp: WebP
webm: WebM
wav: WAV / WAVE (Waveform Audio)
w64: Sony Wave64
vtt: WebVTT subtitle
voc: Creative Voice
vob: MPEG-2 PS (VOB)
vc1: raw VC-1 video
vag: Simon & Schuster Interactive VAG
uw: PCM unsigned 16-bit little-endian
ul: PCM mu-law
ub: PCM unsigned 8-bit
ttml: TTML subtitle
tta: TTA (True Audio)
thd: raw TrueHD
swf: SWF (ShockWave Flash)
sw: PCM signed 16-bit little-endian
sup: raw HDMV Presentation Graphic Stream subtitles
sub: MicroDVD subtitle format
srt: SubRip subtitle
spx: Ogg Speex
spdif: IEC 61937 (used on S/PDIF - IEC958)
sox: SoX native
scc: Scenarist Closed Captions
sb: PCM signed 8-bit
rso: Lego Mindstorms RSO
roq: raw id RoQ
rcv: VC-1 test bitstream
opus: Ogg Opus
oma: Sony OpenMG audio
ogv: Ogg Video
ogg: Ogg
oga: Ogg Audio
nut: NUT
mxf: MXF (Material eXchange Format) Operational Pattern Atom
mpd: DASH Muxer
mp4: MP4 (MPEG-4 Part 14)
mp3: MP3 (MPEG audio layer 3)
mov: QuickTime / MOV
mmf: Yamaha SMAF
mlp: raw MLP
mkv: Matroska
mka: Matroska Audio
mjpg: MIME multipart JPEG
m4v: raw MPEG-4 video
m3u8: Apple HTTP Live Streaming
m2v: raw MPEG-2 video
lrc: LRC lyrics
lbc: iLBC storage
ivf: On2 IVF
ico: Microsoft Windows ICO
h263: raw H.263
h261: raw H.261
gxf: GXF (General eXchange Format)
gsm: raw GSM
gif: CompuServe Graphics Interchange Format (GIF)
g722: raw G.722
flv: FLV (Flash Video)
flm: Adobe Filmstrip
flac: raw FLAC
fits: Flexible Image Transport System
ffmeta: FFmpeg metadata in text
f4v: F4V Adobe Flash Video
eac3: raw E-AC-3
dvd: MPEG-2 PS (DVD VOB)
dv: DV (Digital Video)
dts: raw DTS
cpk: Sega FILM / CPK
chk: WebM Chunk Muxer
cavs: raw Chinese AVS (Audio Video Standard) video
caf: Apple CAF (Core Audio Format)
c2: codec2 .c2 muxer
bit: G.729 BIT file format
avi: AVI (Audio Video Interleaved)
au: Sun AU
ast: AST (Audio Stream)
aptxhd: raw aptX HD (Audio Processing Technology for Bluetooth)
aptx: raw aptX (Audio Processing Technology for Bluetooth)
apng: Animated Portable Network Graphics
apm: Ubisoft Rayman 2 APM
amv: AMV
amr: 3GPP AMR
al: PCM A-law
adx: CRI ADX
ac3: raw AC-3
3gp: 3GP (3GPP file format)
3g2: 3GP2 (3GPP2 file format)
302: D-Cinema audio
Audio Encoders: 
sonicls: Sonic lossless
mp2fixed: MP2 fixed point (MPEG audio layer 2)
libmp3lame: libmp3lame MP3 (MPEG audio layer 3)
Audio Decoders: 
xma2: Xbox Media Audio 2
xma1: Xbox Media Audio 1
xan_dpcm: DPCM Xan
ws_snd1: Westwood Audio (SND1)
wmavoice: Windows Media Audio Voice
wmav2: Windows Media Audio 2
wmav1: Windows Media Audio 1
wmapro: Windows Media Audio 9 Professional
wmalossless: Windows Media Audio Lossless
wavpack: WavPack
wavesynth: Wave synthesis pseudo-codec
vorbis: Vorbis
vmdaudio: Sierra VMD audio
twinvq: VQF TwinVQ
tta: TTA (True Audio)
truespeech: DSP Group TrueSpeech
truehd: TrueHD
tak: TAK (Tom's lossless Audio Kompressor)
sonic: Sonic
sol_dpcm: DPCM Sol
smackaud: Smacker audio
siren: Siren
sipr: RealAudio SIPR / ACELP.NET
shorten: Shorten
sdx2_dpcm: DPCM Squareroot-Delta-Exact
sbc: SBC (low-complexity subband codec)
s302m: SMPTE 302M
roq_dpcm: DPCM id RoQ
real_288: RealAudio 2.0 (28.8K)
real_144: RealAudio 1.0 (14.4K)
ralf: RealAudio Lossless
qdmc: QDesign Music Codec 1
qdm2: QDesign Music Codec 2
qcelp: QCELP / PureVoice
pcm_vidc: PCM Archimedes VIDC
pcm_u8: PCM unsigned 8-bit
pcm_u32le: PCM unsigned 32-bit little-endian
pcm_u32be: PCM unsigned 32-bit big-endian
pcm_u24le: PCM unsigned 24-bit little-endian
pcm_u24be: PCM unsigned 24-bit big-endian
pcm_u16le: PCM unsigned 16-bit little-endian
pcm_u16be: PCM unsigned 16-bit big-endian
pcm_sga: PCM SGA
pcm_s8_planar: PCM signed 8-bit planar
pcm_s8: PCM signed 8-bit
pcm_s64le: PCM signed 64-bit little-endian
pcm_s64be: PCM signed 64-bit big-endian
pcm_s32le_planar: PCM signed 32-bit little-endian planar
pcm_s32le: PCM signed 32-bit little-endian
pcm_s32be: PCM signed 32-bit big-endian
pcm_s24le_planar: PCM signed 24-bit little-endian planar
pcm_s24le: PCM signed 24-bit little-endian
pcm_s24daud: PCM D-Cinema audio signed 24-bit
pcm_s24be: PCM signed 24-bit big-endian
pcm_s16le_planar: PCM signed 16-bit little-endian planar
pcm_s16le: PCM signed 16-bit little-endian
pcm_s16be_planar: PCM signed 16-bit big-endian planar
pcm_s16be: PCM signed 16-bit big-endian
pcm_mulaw: PCM mu-law / G.711 mu-law
pcm_lxf: PCM signed 20-bit little-endian planar
pcm_f64le: PCM 64-bit floating point little-endian
pcm_f64be: PCM 64-bit floating point big-endian
pcm_f32le: PCM 32-bit floating point little-endian
pcm_f32be: PCM 32-bit floating point big-endian
pcm_f24le: PCM 24.0 floating point little-endian
pcm_f16le: PCM 16.8 floating point little-endian
pcm_dvd: PCM signed 16|20|24-bit big-endian for DVD media
pcm_bluray: PCM signed 16|20|24-bit big-endian for Blu-ray media
pcm_alaw: PCM A-law / G.711 A-law
paf_audio: Amazing Studio Packed Animation File Audio
opus: Opus
on2avc: On2 Audio for Video Codec
nellymoser: Nellymoser Asao
mpc8: Musepack SV8
mpc7: Musepack SV7
mp3on4float: MP3onMP4
mp3on4: MP3onMP4
mp3float: MP3 (MPEG audio layer 3)
mp3adufloat: ADU (Application Data Unit) MP3 (MPEG audio layer 3)
mp3adu: ADU (Application Data Unit) MP3 (MPEG audio layer 3)
mp3: MP3 (MPEG audio layer 3)
mp2float: MP2 (MPEG audio layer 2)
mp2: MP2 (MPEG audio layer 2)
mp1float: MP1 (MPEG audio layer 1)
mp1: MP1 (MPEG audio layer 1)
mlp: MLP (Meridian Lossless Packing)
metasound: Voxware MetaSound
mace6: MACE (Macintosh Audio Compression/Expansion) 6:1
mace3: MACE (Macintosh Audio Compression/Expansion) 3:1
libvorbis: libvorbis
libspeex: libspeex Speex
libopus: libopus Opus
libopencore_amrwb: OpenCORE AMR-WB (Adaptive Multi-Rate Wide-Band)
libopencore_amrnb: OpenCORE AMR-NB (Adaptive Multi-Rate Narrow-Band)
libfdk_aac: Fraunhofer FDK AAC
interplayacm: Interplay ACM
interplay_dpcm: DPCM Interplay
imc: IMC (Intel Music Coder)
ilbc: iLBC (Internet Low Bitrate Codec)
iac: IAC (Indeo Audio Coder)
hcom: HCOM Audio
hca: CRI HCA
gsm_ms: GSM Microsoft variant
gsm: GSM
gremlin_dpcm: DPCM Gremlin
g729: G.729
g726le: G.726 ADPCM little-endian
g726: G.726 ADPCM
g723_1: G.723.1
g722: G.722 ADPCM
flac: FLAC (Free Lossless Audio Codec)
fastaudio: MobiClip FastAudio
evrc: EVRC (Enhanced Variable Rate Codec)
eac3: ATSC A/52B (AC-3, E-AC-3)
dvaudio: Ulead DV Audio
dst: DST (Digital Stream Transfer)
dss_sp: Digital Speech Standard - Standard Play mode (DSS SP)
dsicinaudio: Delphine Software International CIN audio
dsd_msbf_planar: DSD (Direct Stream Digital), most significant bit first, planar
dsd_msbf: DSD (Direct Stream Digital), most significant bit first
dsd_lsbf_planar: DSD (Direct Stream Digital), least significant bit first, planar
dsd_lsbf: DSD (Direct Stream Digital), least significant bit first
dolby_e: Dolby E
derf_dpcm: DPCM Xilam DERF
dca: DCA (DTS Coherent Acoustics)
cook: Cook / Cooker / Gecko (RealAudio G2)
comfortnoise: RFC 3389 comfort noise generator
bmv_audio: Discworld II BMV audio
binkaudio_rdft: Bink Audio (RDFT)
binkaudio_dct: Bink Audio (DCT)
atrac9: ATRAC9 (Adaptive TRansform Acoustic Coding 9)
atrac3plusal: ATRAC3+ AL (Adaptive TRansform Acoustic Coding 3+ Advanced Lossless)
atrac3plus: ATRAC3+ (Adaptive TRansform Acoustic Coding 3+)
atrac3al: ATRAC3 AL (Adaptive TRansform Acoustic Coding 3 Advanced Lossless)
atrac3: ATRAC3 (Adaptive TRansform Acoustic Coding 3)
atrac1: ATRAC1 (Adaptive TRansform Acoustic Coding)
aptx_hd: aptX HD (Audio Processing Technology for Bluetooth)
aptx: aptX (Audio Processing Technology for Bluetooth)
ape: Monkey's Audio
amrwb: AMR-WB (Adaptive Multi-Rate WideBand)
amrnb: AMR-NB (Adaptive Multi-Rate NarrowBand)
als: MPEG-4 Audio Lossless Coding (ALS)
alac: ALAC (Apple Lossless Audio Codec)
adpcm_zork: ADPCM Zork
adpcm_yamaha: ADPCM Yamaha
adpcm_xa: ADPCM CDROM XA
adpcm_vima: LucasArts VIMA audio
adpcm_thp_le: ADPCM Nintendo THP (little-endian)
adpcm_thp: ADPCM Nintendo THP
adpcm_swf: ADPCM Shockwave Flash
adpcm_sbpro_4: ADPCM Sound Blaster Pro 4-bit
adpcm_sbpro_3: ADPCM Sound Blaster Pro 2.6-bit
adpcm_sbpro_2: ADPCM Sound Blaster Pro 2-bit
adpcm_psx: ADPCM Playstation
adpcm_mtaf: ADPCM MTAF
adpcm_ms: ADPCM Microsoft
adpcm_ima_ws: ADPCM IMA Westwood
adpcm_ima_wav: ADPCM IMA WAV
adpcm_ima_ssi: ADPCM IMA Simon & Schuster Interactive
adpcm_ima_smjpeg: ADPCM IMA Loki SDL MJPEG
adpcm_ima_rad: ADPCM IMA Radical
adpcm_ima_qt: ADPCM IMA QuickTime
adpcm_ima_oki: ADPCM IMA Dialogic OKI
adpcm_ima_mtf: ADPCM IMA Capcom's MT Framework
adpcm_ima_moflex: ADPCM IMA MobiClip MOFLEX
adpcm_ima_iss: ADPCM IMA Funcom ISS
adpcm_ima_ea_sead: ADPCM IMA Electronic Arts SEAD
adpcm_ima_ea_eacs: ADPCM IMA Electronic Arts EACS
adpcm_ima_dk4: ADPCM IMA Duck DK4
adpcm_ima_dk3: ADPCM IMA Duck DK3
adpcm_ima_dat4: ADPCM IMA Eurocom DAT4
adpcm_ima_cunning: ADPCM IMA Cunning Developments
adpcm_ima_apm: ADPCM IMA Ubisoft APM
adpcm_ima_apc: ADPCM IMA CRYO APC
adpcm_ima_amv: ADPCM IMA AMV
adpcm_ima_alp: ADPCM IMA High Voltage Software ALP
adpcm_ea_xas: ADPCM Electronic Arts XAS
adpcm_ea_r3: ADPCM Electronic Arts R3
adpcm_ea_r2: ADPCM Electronic Arts R2
adpcm_ea_r1: ADPCM Electronic Arts R1
adpcm_ea_maxis_xa: ADPCM Electronic Arts Maxis CDROM XA
adpcm_ea: ADPCM Electronic Arts
adpcm_dtk: ADPCM Nintendo Gamecube DTK
adpcm_ct: ADPCM Creative Technology
adpcm_argo: ADPCM Argonaut Games
adpcm_aica: ADPCM Yamaha AICA
adpcm_agm: ADPCM AmuseGraphics Movie
adpcm_afc: ADPCM Nintendo Gamecube AFC
adpcm_adx: SEGA CRI ADX ADPCM
adpcm_4xm: ADPCM 4X Movie
acelp.kelvin: Sipro ACELP.KELVIN
ac3_fixed: ATSC A/52A (AC-3)
ac3: ATSC A/52A (AC-3)
aac_latm: AAC LATM (Advanced Audio Coding LATM syntax)
aac_fixed: AAC (Advanced Audio Coding)
aac: AAC (Advanced Audio Coding)
8svx_fib: 8SVX fibonacci
8svx_exp: 8SVX exponential
Features: 
Spell-Checking support: Yes
Rajce support: Yes
Qt Multimedia support: Yes
Parallelized demosaicing: Yes
Panorama support: Yes
Memory available: 31.3 GiB
Media player support: Yes
Image cache size: 1.0 GiB
HTML Gallery support: Yes
HEIF writing support: Yes
HEIF reading support: Yes
Exiv2 supports XMP metadata: Yes
Exiv2 supports JPEG-XL metadata: Yes
Exiv2 supports Base Media: Yes
DBus support: No
Database backend: QSQLITE
CPU cores: 16
Calendar support: Yes
Baloo support: No
AddressBook support: No
Comment 4 Gerd Mann 2023-12-18 07:15:09 UTC
(In reply to caulier.gilles from comment #2)
> Also did you use Snap package ?
> 
> Can you run digiKam from a console after to enable debug trace in
> Setup/Miscs/System section ?
> 
> Gilles Caulier

Via Snap only digiKam 8.1 is available. I'm using the appimage.
Comment 5 Maik Qualmann 2023-12-18 07:26:21 UTC
A test here shows no change in the scan time compared to a digiKam-8.0.0 AppImage version. I'll test it with an even larger network collection tonight.
Do you use sidecars?

Maik
Comment 6 Maik Qualmann 2023-12-18 07:40:57 UTC
Sorry Marc if I add you to this bug report. You have a slow network collection. Have you noticed a slowdown in scan time in digiKam-8.2.0?

Maik
Comment 7 Gerd Mann 2023-12-18 08:07:38 UTC
(In reply to Maik Qualmann from comment #5)
> A test here shows no change in the scan time compared to a digiKam-8.0.0
> AppImage version. I'll test it with an even larger network collection
> tonight.
> Do you use sidecars?
> 
> Maik

Yes, I am using the xmp-sidecars. For most photos I do some processing in Darktable and Gimp. At least Darktable is writing to the sidecars, too.
Comment 8 Gerd Mann 2023-12-18 08:10:30 UTC
An other user reports similar problems. Cf. following thread:

https://discuss.pixls.us/t/digikam-8-2-scanning-for-new-entries-very-slow/40851/14

However, the user reports: "I’m not using a NAS, everything is on a local SSD in my case. So that can’t be the only reason."
Comment 9 Gerd Mann 2023-12-18 09:08:33 UTC
Created attachment 164270 [details]
Log file from appimage digiKam 8.2

I hope the attached log file helps.
Comment 10 Maik Qualmann 2023-12-18 09:30:17 UTC
I think I know the problem, files with sidecar keep getting rescanned due to a small change in digiKam-8.2.0. This was the change:

https://invent.kde.org/graphics/digikam/-/commit/4cfbcbdc0c87bd958f4fa3d2715cefd88932c02b

However, I forgot to make this change in the item scanner itself, so the modification date is not updated.
As a test, you can change the option in the metadata settings for timestamp updating. The first start will be slow, then the following ones will be faster again.

Depending on the time stamp update setting, the scanning speed has always been slightly different.

Maik
Comment 11 Gerd Mann 2023-12-18 09:36:22 UTC
Trying your proposal just now. It will take some time before I can report the results.
Thanks a lot. DigiKam is a great tool.
Comment 12 Gerd Mann 2023-12-18 11:38:52 UTC
The initial scan with the proposed setting took quite some time...
However, after the scan in V8.2 works now as expected. Instead of >40 minutes its finished in 30 seconds.
Thanks a lot.
Comment 13 Maik Qualmann 2023-12-18 12:01:50 UTC
Thanks for testing and reporting. I'll fix the problem tonight.

Maik
Comment 14 Gerd Mann 2023-12-18 12:15:08 UTC
Thank you, I appreciate that very much.

Gerd
Comment 15 Maik Qualmann 2023-12-18 17:57:38 UTC
Git commit e31a5e2f641d66723b38c6e061be8b03f204b957 by Maik Qualmann.
Committed on 18/12/2023 at 18:56.
Pushed by mqualmann into branch 'master'.

fix repeated scanning of items with sidecar
FIXED-IN: 8.3.0

M  +1    -1    NEWS
M  +2    -2    core/libs/database/collection/collectionscanner_scan.cpp
M  +1    -2    core/libs/database/item/scanner/itemscanner.cpp

https://invent.kde.org/graphics/digikam/-/commit/e31a5e2f641d66723b38c6e061be8b03f204b957
Comment 16 Maik Qualmann 2023-12-18 20:51:24 UTC
Git commit 68f5ca7e74e30da50215126659ba8bdb1f941508 by Maik Qualmann.
Committed on 18/12/2023 at 21:50.
Pushed by mqualmann into branch 'master'.

faster initial scan when sidecars are enabled

M  +2    -1    core/libs/database/collection/collectionscanner.h
M  +52   -19   core/libs/database/collection/collectionscanner_scan.cpp

https://invent.kde.org/graphics/digikam/-/commit/68f5ca7e74e30da50215126659ba8bdb1f941508
Comment 17 Maik Qualmann 2023-12-18 20:55:36 UTC
@Gerd Mann, if an AppImage with this change is available, I would be interested to know how it changed the scan time for you. The AppImage date must be greater than December 17, 2023.

https://files.kde.org/digikam/

Maik
Comment 18 Gerd Mann 2023-12-18 23:38:46 UTC
@Maik: I'll test and report results as soon as the new appimage is available.

Gerd
Comment 19 caulier.gilles 2023-12-19 06:05:54 UTC
@gerd,

The new AppImage will be online in one hour at usual place. I start the build few minutes ago.

Best

Gilles Caulier
Comment 20 Gerd Mann 2023-12-19 07:45:38 UTC
@Gilles:
 I just downloaded and tested the new appimage. With "Fast scan" and "Update file modification timestamp..." switched off the scan takes 43 seconds (collection size 140k images). That is faster than V8.1.
It guess this resolves the issue in 8.2.

My sincere thanks to you and Maik.
Comment 21 caulier.gilles 2023-12-19 07:58:11 UTC
Great, thanks for the feedback.

Gilles Caulier
Comment 22 Marshalleq 2024-01-11 01:32:45 UTC
I'm on Mac as my primary desktop these days and loved digikam on linux. I have been experiencing the exact same issue.  Glad to hear it's fixed in 8.3 as I nearly gave up on Digikam (figuring it was a Mac specific issue).  In my opinion this should be back ported to 8.2 or the whole 8.2 pulled (especially when the main page brags about how easy digikam is able to work with large libraries) but also because this gives new people trying out digikam a really bad impression.

For anyone else coming here, I've managed to get an 8.3.0 version from: https://files.kde.org/digikam/
Comment 23 Marshalleq 2024-01-11 02:03:10 UTC
Sadly this has actually not fixed the problem for me.  Calculating the time it's taken to get to 2%, I think I'm up for an hour or two.  This is with fast scan on and using 8.3 10/1/24 7:08PM build.

I take it since I'm on Mac I should open a new ticket rather than continue this one.

Will do that.