Bug 426807 - Out of range changedate prevents importing music folder
Summary: Out of range changedate prevents importing music folder
Status: RESOLVED FIXED
Alias: None
Product: amarok
Classification: Applications
Component: general (other bugs)
Version First Reported In: 2.9.0
Platform: Other Linux
: NOR normal
Target Milestone: kf5
Assignee: Amarok Developers
URL:
Keywords:
: 491321 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-09-21 10:42 UTC by thilo
Modified: 2025-03-15 11:52 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description thilo 2020-09-21 10:42:08 UTC
SUMMARY

Im getting an error while scanning my music folder.


STEPS TO REPRODUCE
1.  Have following folders with music in them:
$ stat * -t
AM 4096 8 41ed 1000 1000 fe00 3309571 2 0 0 0 -3600 1600607230 1600607229 4096
Tranquility Base Hotel & Casino 4096 8 41ed 1000 1000 fe00 3309570 2 0 0 0 -3600 1600607088 1600607087 4096

2. have your collection directory point to a root which includes these folders
3. use tools > update collections  

OBSERVED RESULT
 crashes with: 
MySQLe query failed! (1264) Out of range value for column 'changedate' at row 1 on INSERT INTO directories(deviceid,changedate,dir) VALUES (3,4294963696,'Music/Arctic Monkeys/AM');
MySQLe query failed! (1264) Out of range value for column 'changedate' at row 1 on INSERT INTO directories(deviceid,changedate,dir) VALUES (3,4294963696,'Music/Arctic Monkeys/Tranquility Base Hotel & Casino');

the songs from these folders are not in the library

EXPECTED RESULT
should add the songs to the collection without error

SOFTWARE/OS VERSIONS
archlinux 5.8.10-arch1-1 #1 SMP PREEMPT Thu, 17 Sep 2020 18:01:06 +0000 x86_64 GNU/Linux
KDE Plasma Version: plasmashell 5.19.5
Qt: 5.15.1
KDE Frameworks: 5.74.0
Comment 1 thilo 2020-09-21 10:44:16 UTC
Additional info:
$  stat * -tf
AM 2c6731a50772e064 255 ef53 4096 4096 29277940 7414595 6095503 7454720 6240792
Tranquility Base Hotel & Casino 2c6731a50772e064 255 ef53 4096 4096 29277940 7414595 6095503 7454720 6240792
Comment 2 thilo 2020-09-21 10:47:24 UTC
A 'touch *' fixes the issue
Comment 3 Heiko Becker 2020-09-21 11:13:40 UTC
stat without "-t"  would have been nicer, but assuming you have the same implementation as I do, the change date of the file is the unix epoch minus 3600 seconds, which overflows somewhere to 4294963696 secs since the epoch, which is a bit in the future: "Sun  7 Feb 06:28:16 CET 2106"

1) Obviously this should be handled more robustly and 2) it also looks like Amarok has a year 2038 problem because the maximum value of mysql's INTEGER is 2147483647.
Comment 4 Raphaël G. 2024-02-15 22:01:19 UTC
SUMMARY
I set amarok to use my mariadb server at localhost.

I get a sql error in the /var/lib/mysql/sql_error.log when amarok update the collection:
2024-02-15 22:14:22 amarok[amarok] @ localhost [] ERROR 1264: Out of range value for column 'changedate' at row 1 : INSERT INTO directories(deviceid,changedate,dir) VALUES (1,4294963696,'home/<user>/Musique/<path>')

STEPS TO REPRODUCE
1.  Create directory with such timestamps
2.  Update collection
3.  See error in sql error log

FIX SUGGESTION
Change changedate from int(11) to datetime ?

OBSERVED RESULT
ERROR 1264: Out of range value for column 'changedate' at row 1 : INSERT INTO directories(deviceid,changedate,dir) VALUES (1,4294963696,'home/<user>/Musique/<path>')

EXPECTED RESULT
No error

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
LC_ALL=C l -ldh '/home/<user>/Musique/<path>':
drwxr-xr-x 1 <user> <user> 3.8K Dec 31  1969

LC_ALL=C stat '/home/<user>/Musique/<path>'
  File: /home/<user>/Musique/<path>
  Size: 3872            Blocks: 0          IO Block: 4096   directory
Device: 0,39    Inode: 270513      Links: 1
Access: (0755/drwxr-xr-x)  Uid: ( <uid>/  <user>)   Gid: ( <uid>/  <user>)
Access: 2024-02-15 21:54:41.937300905 +0000
Modify: 1969-12-31 23:00:00.000000000 +0000
Change: 2024-01-12 18:40:54.232067829 +0000
 Birth: 2024-01-12 16:40:52.050847188 +0000
Comment 5 Raphaël G. 2024-02-15 22:03:37 UTC
Same bug as https://bugs.kde.org/show_bug.cgi?id=405241 too, which was invalidly closed as invalid.

Amarok should support out of timestamp range creation/modification dates.
Comment 6 Tuomas Nurmi 2024-08-06 19:14:15 UTC
*** Bug 491321 has been marked as a duplicate of this bug. ***
Comment 7 Bug Janitor Service 2025-01-05 12:35:24 UTC
A possibly relevant merge request was started @ https://invent.kde.org/multimedia/amarok/-/merge_requests/134
Comment 8 Tuomas Nurmi 2025-03-15 11:52:47 UTC
Git commit 0e7509d0a8fc6e70ee3b60f1ec5db9080c3c524c by Tuomas Nurmi.
Committed on 15/03/2025 at 11:38.
Pushed by nurmi into branch 'master'.

Add some database bug related tests that fail for now

Database should be utf8mb4 for complete UTF8 support.
Directory changedates shouldn't hit year 2038 problem.
Related: bug 462268

M  +14   -0    tests/core-impl/collections/db/sql/TestSqlCollectionLocation.cpp
M  +2    -0    tests/core-impl/collections/db/sql/TestSqlCollectionLocation.h
M  +12   -0    tests/core-impl/collections/db/sql/TestSqlTrack.cpp
M  +2    -0    tests/core-impl/collections/db/sql/TestSqlTrack.h

https://invent.kde.org/multimedia/amarok/-/commit/0e7509d0a8fc6e70ee3b60f1ec5db9080c3c524c
Comment 9 Tuomas Nurmi 2025-03-15 11:52:49 UTC
Git commit 0e3770aa01f004a7ae12a11ad3bda5f0ff8b5fa2 by Tuomas Nurmi.
Committed on 15/03/2025 at 11:38.
Pushed by nurmi into branch 'master'.

Convert directories' changedate to use BIGINT in db and quint64 in code

Scanner side of things was already using quint64-based functions, so this is probably
the most coherent path for fixing

M  +1    -0    ChangeLog
M  +2    -2    shared/collectionscanner/Directory.cpp
M  +2    -2    shared/collectionscanner/Directory.h
M  +4    -2    src/core-impl/collections/db/sql/DatabaseUpdater.cpp
M  +1    -1    src/core-impl/collections/db/sql/SqlCollection.cpp
M  +2    -2    src/core-impl/collections/db/sql/SqlRegistry.cpp
M  +1    -1    src/core-impl/collections/db/sql/SqlRegistry.h

https://invent.kde.org/multimedia/amarok/-/commit/0e3770aa01f004a7ae12a11ad3bda5f0ff8b5fa2