Bug 482497 - rsync backup plan fills up backup disk when it shouldn't
Summary: rsync backup plan fills up backup disk when it shouldn't
Status: REPORTED
Alias: None
Product: kup
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Simon Persson
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-03-05 21:18 UTC by Nate Graham
Modified: 2024-03-14 19:09 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Backup plan (1.12 KB, text/x-log)
2024-03-05 21:18 UTC, Nate Graham
Details
kuprc file (1.16 KB, text/plain)
2024-03-05 21:19 UTC, Nate Graham
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nate Graham 2024-03-05 21:18:36 UTC
Created attachment 166446 [details]
Backup plan

I've set up an rsync-based backup job using Kup. My home folder is being backed up to an external disk. The size of the home folder's contents are 258.1 GB, and the size of the backup disk is 320 GB.

In the process of running this backup job, the disk filled up. This should not have happened as the source size is smaller than the capacity of the target! Yet somehow it happened anyway.

I'm attaching the kup_plan log file that shows the rsync command it came up with. It looks a bit fishy, with /home/nate being *ex*cluded?
Comment 1 Nate Graham 2024-03-05 21:19:08 UTC
Created attachment 166447 [details]
kuprc file
Comment 2 Nate Graham 2024-03-06 00:45:55 UTC
If it helps, here's a comparison of the rsync command that kup generated which causes this problem, compard to the rsync command that I've been using myself for a few years, formatted with one option per line for readability:

Kup made it based on my selections in the UI, doesn't work:
> rsync
> -avX
> --delete-excluded
> --delete-before
> --info=progress2
> --exclude /nate/.cache
> --exclude /nate/.local/share/Steam
> --exclude /nate/.local/share/Trash
> --exclude /nate/.local/share/baloo
> --exclude /nate/.local/share/shellrunnertest
> --exclude /nate/.steam
> --exclude /nate/.var/app/com.discordapp.Discord/cache
> --exclude /nate/.var/app/com.mojang.Minecraft/cache
> --exclude /nate/.var/app/com.obsproject.Studio/cache
> --exclude /nate/.var/app/im.riot.Riot/cache
> --exclude /nate/.var/app/org.kde.kamoso/cache
> --exclude /nate/.var/app/org.telegram.desktop/cache
> --exclude /nate/kde/build
> --exclude /nate/kde/usr
> /home/nate
> /home/nate/kde/build/kitemmodels
> /home/nate/kde/build/knotifyconfig
> /home/nate/kde/usr/bin
> /run/media/nate/Backup/

Made it myself, works
> rsync
> -avX
> --delete-excluded
> --info=progress2
> --no-i-r
> --exclude=/.cache
> --exclude=/.config/Riot/Cache
> --exclude=/.local/share/TelegramDesktop/tdata/temp
> --exclude=/.local/share/Trash
> --exclude=/.local/share/baloo
> --exclude=/kde/build
> --exclude=/kde/usr
> --exclude=/.steam
> --exclude=/.local/share/Steam
> /home/nate/
> /run/media/nate/Backup/

Things that stand out:
- Kup's command has "delete-before", mine doesn't
- My command has "--no-i-r"; Kup's doesn't
- My command uses relative paths, Kup's uses absolute paths
- My command has only one inclusion path, Kup's has four
Comment 3 Simon Persson 2024-03-09 16:34:39 UTC
Do you want these folders to be included?
> /home/nate/kde/build/kitemmodels
> /home/nate/kde/build/knotifyconfig
> /home/nate/kde/usr/bin

In other words, are you reporting a problem with the folder selection being used not matching with what you have selected?

Do these folders contain enough large files to explain the problem of running out of space?

Is there any problem with folders that should be excluded still getting copied?

Kup is already doing the safest but possibly slower option of deleting everything that should be deleted before copying anything. Meaning that if it can fit on the destination it will work. This --delete-before option implies --no-i-r so that is no actual difference.

I would understand if your version caused problems with running out of space but you are reporting the opposite.
Comment 4 Nate Graham 2024-03-14 19:09:48 UTC
(In reply to Simon Persson from comment #3)
> Do you want these folders to be included?
> > /home/nate/kde/build/kitemmodels
> > /home/nate/kde/build/knotifyconfig
> > /home/nate/kde/usr/bin

I do; they're only excluded because I experienced Bug 483222 and let it exclude them to make the message go away.

> In other words, are you reporting a problem with the folder selection being
> used not matching with what you have selected?
I guess the problem I'm reporting is too much being copied somehow, which causes the target disk to fill up early.

> Do these folders contain enough large files to explain the problem of
> running out of space?
No, they're just small build directories:

du -ch /home/nate/kde/build/kitemmodels | grep total
38M     total

du -ch /home/nate/kde/build/knotifyconfig/ | grep total
14M     total


> Is there any problem with folders that should be excluded still getting
> copied?
I'm not sure. To check I'd have to reproduce the bug again which temporarily renders my backup unusable, so I'm hesitant to do so unless it would help. Would it?