Bug 425154 - Setting an application to open application/octet-stream has unexpected behavior
Summary: Setting an application to open application/octet-stream has unexpected behavior
Status: CONFIRMED
Alias: None
Product: systemsettings
Classification: Applications
Component: kcm_filetypes (show other bugs)
Version: 5.19.4
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: David Faure
URL:
Keywords: usability
Depends on:
Blocks:
 
Reported: 2020-08-09 05:54 UTC by Adam Fontenot
Modified: 2024-05-17 21:14 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adam Fontenot 2020-08-09 05:54:36 UTC
SUMMARY

If you (intentionally or inadvertently) associate a program with the application/octet-stream mimetype, every binary file will also be associated with the program and files not associated with any other program will start opening with it by default. This is unexpected and hard for users to understand and debug.

STEPS TO REPRODUCE
1. Open the file associations KCM
2. Set KWrite to open the application/octet-stream file type

OBSERVED RESULT

KWrite has been added to most other file associations, e.g. video/mp4, which it can't actually open. 

EXPECTED RESULT

KWrite is only shown under the application/octet-stream association in the KCM, and only opens binary files of unknown type, not every file.

SOFTWARE/OS VERSIONS
Linux: 5.7.12
KDE Plasma Version: 5.19.4
KDE Frameworks Version: 5.72.0
Qt Version: 5.15.0

ADDITIONAL INFORMATION

This is actually a pretty common program users have run into in the past, but I don't think it's been reported before. For example [1] and [2] are user reports just for LibreOffice, which was also affecting me (along with KWrite and Okular). This was probably not reported because it's not at all obvious to users that the application/octet-stream association would have this effect. (Notably, neither of the users in the forums I linked solved their problem.)

I suspect what might be happening is that Firefox adds an association to mimeapps.list when a file is opened with a program for the first time, and many users are opening files with types not recognized by Firefox using common programs.

I think the best solution to the problem would be what I suggested above, i.e. to only use the application/octet-stream association to open files with that mimetype (or possibly an unknown mimetype), and not use it as a fallback association for every other mimetype.

[1] https://www.kubuntuforums.net/showthread.php/69391-LibreOffice-Keeps-Reassociating-With-File-Types

[2] https://www.reddit.com/r/kde/comments/5bnchz/question_about_file_associations/
Comment 1 David Faure 2020-08-14 10:10:09 UTC
Not sure this is a bug.

All mimetypes inherit application/octet-stream, as per the shared-mime-info spec.

The scenario you describe makes perfect sense for e.g. okteta (Hex Editor) so that it can be used for all types of files
(not necessarily by default, of course).

> it's not at all obvious to users that the application/octet-stream association would have this effect

I agree with this. Not sure what the solution is though.
Comment 2 Adam Fontenot 2020-08-15 06:49:29 UTC
I agree that it's probably not a bug that other mimetypes inherit application/octet-stream. There are really two problems:

One issue is that this inheritance isn't surfaced to the user in any meaningful way - from the user's point of view it's as though every single mimetype had the program added. Actually, it is possible to remove programs from individual mimetypes. I think this blacklists them, but this is not obvious: people experiencing this bug are reporting it as a program being added to every mimetype! Since that's not really what's happening, this suggests the current presentation of this information is flawed.

The other issue is that random programs can get added to the application/octet-stream mimetype without the user's explicit request for this to happen. I think sometimes the "remember application association" box is checked by default, and if I pipe garbage from /dev/urandom into a file and try to open it, KDE shows it as application/octet-stream. So maybe it can happen that way? Or maybe opening certain files in programs directly from Firefox will lead to the mimetype getting added.

Some possible solutions:

1. In the file associations module all the types appear under a menu with the title "Known Types". Maybe an entirely different section could be added for types that are inherited, call it "Inherited Types"? Or if application/octet-stream is the *only* type that gets inherited according to the spec, add a section of the KCM explicitly for "programs that open all files". 

2. When a user tries to remove an inherited file association, pop up a dialogue: "ApplicationX is currently associated with the application/octet-stream type, which means it will open every file type by default. Would you like to remove this application's association with every file type, or just this one?" [Every File Type] [Just this One] [Cancel] (This is similar to how Google Calendar lets you delete all instances of a repeating event by just trying to delete one of them.)

Obviously the specifics would need to be adapted to whatever the UI / usability folks think is the most useful to the end user.

3. Track down the different ways in which a user (or program) might incorrectly add a file association with application/octet-stream, and try to prevent this from happening without it being obvious to the user. In particular, in KDE applications, setting a program to open *one specific file* that KDE doesn't recognize should not set that program to open every other type of file, including ones that KDE does recognize.

Example case using Okteta: suppose I have some memory dumps with the extension .bin or something. I might have hundreds of these, so I set Okteta to open them with "remember application association" checked. What actually happens: Okteta gets set to open every kind of file. What I probably expected to happen: Okteta gets set to open every file with the extension .bin that *doesn't* have a known magic pattern. At the least it would be good to give a warning: "KDE doesn't recognize this file type. Do you want to set Okteta to open all file types?"

I suspect if Firefox isn't at fault then something similar must be happening with LibreOffice. Maybe at some point I had what I thought was a CSV saved with a .dat file extension or something, but was actually garbage. I set it to open with LibreOffice, and oops! All of my files now open in LibreOffice. 

4. Applications that are only associated with a mimetype because they're inherited should be prioritized *after* applications that are associated directly, and more distant inheritances should be deprioritized accordingly. I can say for sure that this doesn't always happen: if LibreOffice gets added to application/octet-stream, it takes over all my .py files, which is obviously undesirable.
Comment 3 David Faure 2020-08-16 10:09:30 UTC
Thanks for the useful feedback and thought process.

1. There are other inherited mimetypes, many others. See https://specifications.freedesktop.org/shared-mime-info-spec/shared-mime-info-spec-0.18.html and the sub-class-of attribute in /usr/share/mime/packages/freedesktop.org.xml
The spec for choosing which application to start (https://specifications.freedesktop.org/mime-apps-spec/mime-apps-spec-latest.html) does not recommend separating inherited mimetypes from non-inherited ones. In general I think this would be harmful.

2. sounds useful, assuming it is generalized to all cases of inheritance. If I remove kate from application/x-desktop, maybe I just want to stop using kate for desktop files, or maybe I want to stop using kate altogether for all text/plain files. I see the benefit of asking the user which one is wanted.

3. is the real issue IMHO. When the user says "open .bin files with okteta" and the system only finds application/octet-stream for that .bin file, then we should offer creating a new mimetype on the fly for that extension, and only associating that one with okteta. This would avoid getting into the need for the other "solutions", which are mostly workaround to this problem.

4. I thought we already did that. But indeed text/x-python is a bit of a difficult case because of multiple inheritance (it inherits both application/x-executable and text/plain). And the code was incorrect in case of multiple inheritance. The fix is up for review at https://invent.kde.org/frameworks/kservice/-/merge_requests/9
One issue fixed :-)
Comment 4 David Faure 2020-08-17 19:37:20 UTC
Git commit d4c9cb9d0652c1ed7ba504335a50ea6bca7851e5 by David Faure.
Committed on 17/08/2020 at 19:37.
Pushed by dfaure into branch 'master'.

Fix application preference ordering for mimetypes with multiple inheritance.

Associating an app with application/octet-stream would make it preferred
over the app for text/plain (e.g. kate) for text/x-python files.
That's wrong, since text/x-python inherits application/x-executable
and text/plain, so the kate association is more direct.

M  +10   -1    autotests/kmimeassociationstest.cpp
M  +4    -10   src/sycoca/kbuildservicefactory.cpp
M  +1    -1    src/sycoca/kbuildservicefactory_p.h

https://invent.kde.org/frameworks/kservice/commit/d4c9cb9d0652c1ed7ba504335a50ea6bca7851e5
Comment 5 Adam Fontenot 2020-08-18 04:08:14 UTC
I think the big advantage of (2) is that even if a user doesn't understand where the problem came from, or even if KDE isn't at fault (suppose some misbehaving application adds itself to application/octet-stream), the user is likely to solve the problem on their own if we implement this. 

Perhaps instead of (1) there's some way of at least indicating to the user that the association is inherited. As it stands, all the associations look exactly the same and that's confusing, even to power users.

Thanks for the multiple inheritance fix. I look forward to seeing that improvement with the next release.
Comment 6 D. Debnath 2021-10-13 18:01:41 UTC
Ran into this problem. See my experience in this thread just to illustrate the nature of the not at all obvious source of the problem:

https://www.reddit.com/r/kde/comments/q77yjx/okular_is_associated_with_almost_all_file_types/
Comment 7 Adam Fontenot 2021-10-14 03:40:33 UTC
(In reply to D. Debnath from comment #6)
> Ran into this problem. See my experience in this thread just to illustrate
> the nature of the not at all obvious source of the problem:
> 
> https://www.reddit.com/r/kde/comments/q77yjx/
> okular_is_associated_with_almost_all_file_types/

Thanks for the report! Do you think you could clarify a couple of things?

1. Do you experience what I called issue 4 in my earlier comment? To recap: it makes sense (given this issue) that Okular is associated with almost all of your file types. But has Okular been unexpectedly opening files even though you already had another program assigned to open that file type more directly? If so, can you report your Plasma desktop version? Given David's fix, this shouldn't be happening unless you have an old version.

It looks to me from your screenshot that Okular is indeed listed first, despite the fact that it is (I assume) only inherited from octet-stream. 

2. Do you think you would have been able to resolve this problem on your own if the File Associations KCM had done something to make you aware of the root cause? E.g. an indication that the file association for application/x-ipynb+json was inherited *AND* when deleting that file association, a popup message identifying the inherited association (application/otect-stream) and offering to delete it for you?

-----

I might have a look at coding something to resolve the issue in the way I suggested in the next few days, if I have the time. This would then only leave issue 3, as I described it in my earlier comment.
Comment 8 D. Debnath 2021-10-14 04:31:00 UTC
(In reply to Adam Fontenot from comment #7)

> 1. Do you experience what I called issue 4 in my earlier comment? To recap:
> it makes sense (given this issue) that Okular is associated with almost all
> of your file types. But has Okular been unexpectedly opening files even
> though you already had another program assigned to open that file type more
> directly? If so, can you report your Plasma desktop version? Given David's
> fix, this shouldn't be happening unless you have an old version.
> 
> It looks to me from your screenshot that Okular is indeed listed first,
> despite the fact that it is (I assume) only inherited from octet-stream. 

I did experience that issue as you noticed from my screenshot. Okular got automatically prioritized to the highest spot even though more direct associations existed. 

> 2. Do you think you would have been able to resolve this problem on your own
> if the File Associations KCM had done something to make you aware of the
> root cause? E.g. an indication that the file association for
> application/x-ipynb+json was inherited *AND* when deleting that file
> association, a popup message identifying the inherited association
> (application/otect-stream) and offering to delete it for you?

Oh yes absolutely. If the File Associations KCM had given me hints about the chain of inheritance, it would've made the troubleshooting a lot more straightforward and I would've likely succeeded in resolving the issue on my own.

The XDG specification here: https://specifications.freedesktop.org/shared-mime-info-spec/shared-mime-info-spec-0.21.html 

under the "Subclassing" heading notes that:

> All streamable types (ie, everything except the inode/* types) are subclasses of application/octet-stream.

So, application/octet-stream has special meaning, and this information would be helpful to point out when troubleshooting any issues with file associations. However, when a normal user, who isn't having file association related problems, and visits the KCM just to change a file association, this information about octet-stream wouldn't be useful and wouldn't even make much sense to the user. So from a UI/UX perspective, you would have to implement it in a way that still retains a simple and easy to use interface, but presents this extra information to a curious user.


> I might have a look at coding something to resolve the issue in the way I
> suggested in the next few days, if I have the time. This would then only
> leave issue 3, as I described it in my earlier comment.


Is it possible to add comments to the ~/.config/mimeapps.list? If yes, then let's say whenever a programs modifies that file, a comment could be added to it about which program made the modification. Something like:

[Added Associations]
application/octet-stream=org.kde.okular.desktop; # added by /usr/bin/firefox
inode/directory=org.kde.dolphin.desktop;         # added by File Association KCM
x-scheme-handler/http=firefox.desktop;           # added by File Association KCM 
x-scheme-handler/https=firefox.desktop;          # added by File Association KCM
x-scheme-handler/mailto=thunderbird.desktop;     # added by File Association KCM

[Default Applications]
application/epub+zip=calibre-ebook-viewer.desktop; # added by /usr/bin/dolphin

[Removed Associations]
application/epub+zip=okularApplication_epub.desktop; # added by File Association KCM
Comment 9 D. Debnath 2021-10-14 04:34:03 UTC
(In reply to Adam Fontenot from comment #7)

> can you report your Plasma desktop version? Given David's
> fix, this shouldn't be happening unless you have an old version.

Operating System: Arch Linux
KDE Plasma Version: 5.22.4
KDE Frameworks Version: 5.85.0
Qt Version: 5.15.2
Kernel Version: 5.13.13-arch1-1 (64-bit)
Comment 10 Adam Fontenot 2021-10-14 05:17:41 UTC
(In reply to D. Debnath from comment #9)
> (In reply to Adam Fontenot from comment #7)
> 
> > can you report your Plasma desktop version? Given David's
> > fix, this shouldn't be happening unless you have an old version.
> 
> Operating System: Arch Linux
> KDE Plasma Version: 5.22.4
> KDE Frameworks Version: 5.85.0
> Qt Version: 5.15.2
> Kernel Version: 5.13.13-arch1-1 (64-bit)

Thanks. I can replicate here on similar versions.

@David: can you have a look at this? I see the issue with a simple JSON file. An application assigned *directly* to application/json appears at the highest priority, then Okular (which I assigned to open application/octet-stream).

application/json inherits application/javascript, which inherits application/ecmascript, which inherits application/x-executable and text/plain.

I think what is happening is that the mimetype search is going depth-first instead of breadth-first. So the search order (abbreviated) is a/json, a/javascript, a/ecmascript, a/x-executable, a/octet-stream, t/plain, a/octet-stream.

Since there's multiple inheritance, we bottom out *twice* in the search, and crucially we hit octet-stream and pick up its associations *before* we hit text/plain, which is likely to contain the actually reasonable inherited associations.
Comment 11 Oleg 2024-05-17 20:00:13 UTC
I can confirm this issue. For me, Okular and Kate were set for application/octet-stream causing them to be set for every other application. Unsetting them for application/octet-stream fixed the issue.
Comment 12 Adam Fontenot 2024-05-17 21:14:32 UTC
Just a quick clarificatory comment, since this bug has received some recent attention. I believe that all inheritance order issues have been fixed along with Bug 475200, including the issue described by D. Debnath above.

The remaining issues with file associations are:

 * Issue where applications receive an *unintended* low level file association, and this leads to unexpected or unwanted behavior, and the underlying cause of the issue isn't surfaced to the user in a helpful way via the File Associations KCM. (That's this issue.)
 * Issue where certain applications like Okular which provide multiple .desktop files are duplicated in the file association menu, without causing any further harmful effects. (Bug 481473)

If anyone finds any other issues with file associations, a new bug should be filed. If files are still opening in Okular by default despite a more direct association being set for the file type, please provide a test case if possible.