Bug 479357 - Downloads don't work
Summary: Downloads don't work
Status: RESOLVED FIXED
Alias: None
Product: kget
Classification: Applications
Component: general (show other bugs)
Version: 24.01.85
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: KGet authors
URL:
Keywords: qt6
Depends on:
Blocks:
 
Reported: 2024-01-03 14:51 UTC by Nicolas Fella
Modified: 2024-01-29 21:07 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nicolas Fella 2024-01-03 14:51:20 UTC
STEPS TO REPRODUCE
1. Click "New Download"
2. Enter some URL, e.g. https://files.kde.org/neon/images/testing/20240102-0250/neon-testing-20240102-0250.iso
3. Click Ok

OBSERVED RESULT
Download is in "Stalled" state forever

EXPECTED RESULT
Download goes brrr

SOFTWARE/OS VERSIONS
KDE Plasma Version: master
KDE Frameworks Version: master
Qt Version: 6.7
Comment 1 Nicolas Fella 2024-01-03 14:52:42 UTC
This is most likely caused by the KIO HTTP refactoring in KF6.

Debugging I found at least two issues:
- kio_http doesn't emit canResume
- kio_http doesn't emit totalSize
Comment 2 Nicolas Fella 2024-01-03 15:23:25 UTC
processedSize is also not emitted, which likely breaks the progress indicator
Comment 3 Nicolas Fella 2024-01-09 23:50:34 UTC
https://invent.kde.org/frameworks/kio/-/merge_requests/1535 makes it somewhat work.

It's still broken for large files because kio_http doesn't send data() in chunks and instead all at once, which breaks with too much data.

pause/resume also doesn't work
Comment 4 Nicolas Fella 2024-01-10 13:59:32 UTC
Git commit bc78a85e1c8f7e54b0f1c0d1611a9432d5549677 by Nicolas Fella.
Committed on 10/01/2024 at 13:53.
Pushed by nicolasfella into branch 'master'.

[http] Emit processedSize and totalSize

This is needed for download progress tracking in KGet

M  +11   -0    src/kioworkers/http/http.cpp

https://invent.kde.org/frameworks/kio/-/commit/bc78a85e1c8f7e54b0f1c0d1611a9432d5549677
Comment 5 Bug Janitor Service 2024-01-26 00:23:32 UTC
A possibly relevant merge request was started @ https://invent.kde.org/frameworks/kio/-/merge_requests/1543
Comment 6 Nicolas Fella 2024-01-29 20:17:43 UTC
Git commit 153e6359f420bf484d46f73c02a922a2fe32257f by Nicolas Fella.
Committed on 29/01/2024 at 19:58.
Pushed by nicolasfella into branch 'master'.

[workers/http] Read and emit data in chunks

Currently we readAll the whole reply into a QByteArray and pass that to data()

This works fine for small bodies. However when downloading large files (e.g. in KGet)
this results in the whole file being loaded into RAM. Not only can this fill all available
RAM, passing this much data via the worker socket also fails.

Instead process the data in chunks.

To facilitate this we need to slightly change the makeRequest interface to indicate how the
data is to be processed. We have several internal uses cases:

- For requests like GET we want to emit data()
- For some DAV requests like list() we want to process the data outselves
- For some requests like DELETE we don't expect/process any data

Add a DataMode parameter to indicate it.

M  +67   -32   src/kioworkers/http/http.cpp
M  +15   -3    src/kioworkers/http/http.h

https://invent.kde.org/frameworks/kio/-/commit/153e6359f420bf484d46f73c02a922a2fe32257f
Comment 7 Nicolas Fella 2024-01-29 21:07:31 UTC
Downloads work now. Will follow up with some minor bugs