Bug 266008 - "Move to collection" deleted most of my music.
Summary: "Move to collection" deleted most of my music.
Status: RESOLVED DUPLICATE of bug 257739
Alias: None
Product: amarok
Classification: Applications
Component: Collections/Local (show other bugs)
Version: 2.3.2
Platform: Gentoo Packages Linux
: HI major
Target Milestone: 2.4.1
Assignee: Amarok Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-02-10 19:26 UTC by nospamformike
Modified: 2012-08-19 16:53 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description nospamformike 2011-02-10 19:26:30 UTC
Version:           2.3.2 (using KDE 4.4.5) 
OS:                Linux

I'll start by summarising the situation I'm in:

I have over 100 gigs of music.  It's the result of me being lazy in sorting it out for ever. Unfortunately I tended in the past to retag files, recode them, forget which CDs i'd ripped and rerip them, trade music collections with friends and family (more than once so I end up getting my collection back too, with whatever changes were made to them then)I recon there are at least 2 copies of each track.  In some cases there are many more copies. fdupes is unfortunately not an option. I have already run it and deleted probably 50 gigs or so.  What's left is harder to deal with.

I thought that the easiest way to remove duplicates would be to make sure the files were properly tagged and this I have used a number of tagging tools on, including picard, puddletag, amarok, mp3diags.  As I've switched from one tool to another, I've taken a complete copy and carried on with the copy.  So I could keep track of the order, I named the directories pass00, pass01 etc.  When I got to pass03, I thought I'd got to a reasonable point to try removing duplicates.

My plan: with the tags correct and consistent, I simply instruct amarok to rename the files in a consistant manner (using move to collection).  Any files with the same tags will get the same filename and path. If I tell amarok not to overwrite existing files, that file should therefore remain in the source directory and either the move would fail and the whole batch would stop (annoying) or the file would be ignored and the rest would be processed (my hope).  Sound reasonable?

The latest was in a directory called pass03.  here is what I did:

rsync -avn pass03/ pass04
mkdir pass05
-tell amarok that pass05 is the only directory for my collection
-in amarok browse to pass04
-right click and choose move to collection
-wait for ages for the organise files dialog to come up
-confirm that pass05 is the collection route
-set "%initial/%artist/%artist - {%album - }{%discnumber - }{%track - }%title" as my renaming string
-check that overwrite existing files is unchecked
-press ok

the next day:
I encountered an error saying it couldn't delete a particular file.  Just 1 file. I pressed ok.

then:
du -ks pass*
103805256       pass00
103805172       pass01
103687388       pass02
102457640       pass03
241040  pass04
574432  pass05

Now, remember that I told amarok not to overwrite files, so of the files that were in pass04, some should be in pass05, with a different name, and some should remain in pass04.  None of them should have disappeared.  My expectation therefore was that the sizes of pass04 and pass05 should add up to that of pass03.  241Megs + 574Megs = 815Megs which is just a smidge smaller than the 100gigs total I was expecting.  Where have all the other files gone?

On closer inspection, pass04 contains no music.  the remaining files are things like albumart and lyrics.  So all music files were deleted from pass04.  This is clearly the wrong result.  Even more wrong is the fact that they haven't ended up in pass05.

Perhaps du can't count? Perhaps I accidentally moved them to the wrong place? A quick glance at the collection in amarok shows a miniscule portion of the number of artists there would normally be- only 28 artists vs the normal hundreds. Another check of the collection settings shows that the only selected directory is pass05.

Reproducible: Didn't try

Steps to Reproduce:
I'm in the midst of trying to reproduce it but I'm sure you'll appreciate that it'll take me a while.

I plan to try the same thing again, then if it happens again, try it in smaller batches. This will likely take me several days but I will report as each test is complete.

Actual Results:  
241 megs of files in pass04 and 574 megs of files in pass05

Expected Results:  
100 gigs of files spread across pass04 and pass05 in unknown proportions.
Comment 1 Myriam Schweingruber 2011-02-10 20:40:44 UTC
Could you please test with the latest Amarok 2.4 version? I suspect this is a duplicate of bug 257739 which is solved already.
Comment 2 nospamformike 2011-02-11 00:54:41 UTC
Thanks for the fast response.

From reading that other bug it sounds like it could be the same.  I did a repeat and I got an error about not being able to delete the same file as previously, though actually fewer files remained the second time.

I'll try a newer version and get back to you.  probably mid next week as I'm going away for a long weekend.
Comment 3 nospamformike 2011-02-17 08:40:07 UTC
Upgrading to 2.4.0 (from gentoo unstable) seems to resolve the issue.  It now appears to stop when it encounters a conflict. Interestingly, the total of the source and destination directory sizes are only approximately preserved.  There appears to have been a loss of 12k somewhere.

I'll investigate later whether there has actually been a file lost or modified.
Comment 4 Myriam Schweingruber 2011-02-19 13:06:12 UTC
Missing files would probably another issue, be this lacking tabs or duplicate tracks. I close this for now, please open a separate bug if you can confirm that issue.

*** This bug has been marked as a duplicate of bug 257739 ***