Bug 446877 - Putting country-specific mirrors in /etc/apt/sources.list breaks AppStream
Summary: Putting country-specific mirrors in /etc/apt/sources.list breaks AppStream
Status: RESOLVED DUPLICATE of bug 453571
Alias: None
Product: neon
Classification: KDE Neon
Component: Packages User Edition (show other bugs)
Version: unspecified
Platform: Neon Linux
: NOR normal
Target Milestone: ---
Assignee: Neon Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-12 11:53 UTC by nttkde
Modified: 2022-05-24 13:22 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
discover (41.24 KB, image/png)
2021-12-12 11:53 UTC, nttkde
Details
software sources from Discover (83.19 KB, image/png)
2021-12-15 05:50 UTC, nttkde
Details
appstreamcli refresh --force --verbose (394.22 KB, text/plain)
2021-12-17 14:29 UTC, nttkde
Details

Note You need to log in before you can comment on or make changes to this bug.
Description nttkde 2021-12-12 11:53:20 UTC
Created attachment 144478 [details]
discover

"Featured" page is showing just one item.
See the attached screenshot.

~/.cache/discover/featured-5.9.json is the file that contains the list of featured apps, right?
Its contents are currently:
[
    "appstream://org.kde.krita.desktop",
    "appstream://org.kde.digikam.desktop",
    "appstream://org.kde.kdenlive.desktop",
    "appstream://org.kde.ktorrent.desktop",
    "appstream://org.kde.gcompris.desktop",
    "appstream://org.kde.kmymoney.desktop",
    "appstream://org.kde.kolourpaint.desktop",
    "appstream://blender.desktop",
    "appstream://inkscape.desktop",
    "appstream://org.kde.development"
]



This started happening quite recently, but unfortunately I can't give much more info.
However, I have KDE Neon also installed in a VM but I can't reproduce the bug there. :(

(Could be related to https://bugs.kde.org/show_bug.cgi?id=418971 ?)
Comment 1 nttkde 2021-12-12 11:54:52 UTC
Should I try removing .cache/discover/* to see if it's a cache-related bug?
Comment 2 Patrick Silva 2021-12-12 17:01:22 UTC
I can confirm this bug on neon unstable.
Comment 3 Nate Graham 2021-12-14 18:59:50 UTC
(In reply to nttkde from comment #1)
> Should I try removing .cache/discover/* to see if it's a cache-related bug?
Yes please.
Comment 4 nttkde 2021-12-14 23:36:11 UTC
Deleting .cache/discover/* and rebooting did not fix the issue.
Comment 5 Aleix Pol 2021-12-15 00:43:41 UTC
I assume that your default apps source is still the distro?

Can you check if appstreamcli search kolourpaint still finds your kolourpaint?
Comment 6 nttkde 2021-12-15 05:49:44 UTC
(In reply to Aleix Pol from comment #5)
> I assume that your default apps source is still the distro?
> 
> Can you check if appstreamcli search kolourpaint still finds your
> kolourpaint?

I attached Discover software sources list. Neon distro sources are the default ("oletus").

There's something weird there.
appstreamcli search kolourpaint shows only the Flatpak version:

Tunniste: org.kde.kolourpaint.desktop [desktop-application]
Nimi: KolourPaint
Yhteenveto: Piirto-ohjelma
Bundle: flatpak:app/org.kde.kolourpaint/x86_64/stable
Verkkosivusto: http://www.kolourpaint.org/
Kuvake: org.kde.kolourpaint.png


I can find the kolourpaint deb package in Synaptic though.

Compared to a 'normal' VM Neon installation; there appstreamcli search kolourpaint shows both the deb and flatpak versions.




I tried also opening the appstream links from ~/.cache/discover/featured-5.9.json (as seen on the first message of this bug report) in Discover.
Discover couldn't open some of them:

Couldn't open: appstream://org.kde.krita.desktop
Couldn't open: appstream://org.kde.kmymoney.desktop
Couldn't open: appstream://blender.desktop
Couldn't open: appstream://inkscape.desktop


The 'normal' VM Discover couldn't open the blender and inkscape links but did open the others. It does not show Blender or Inkscape on the Featured page either. (I remember reading that some Ubuntu packages are not supported on Neon and Discover won't show them?)



--------------

I did find another interesting thing: if I click on the only featured software (KDE Software Development Kit) or launch the page directly with 
plasma-discover --application appstream://org.kde.development then its page doesn't load completely, Discover freezes and fills terminal output with lots of 
file:///usr/lib/x86_64-linux-gnu/qt5/qml/QtQuick/Controls.2/org.kde.desktop/ScrollView.qml:98:27: QML ScrollBar: Binding loop detected for property "visible"

If I open some other app I get the message just once; or not at all if the app page doesn't have the horizontal scrollbar under the screenshots.

To reproduce, in your .config/discoverrc set window geometry to something like this:
[Window]
geometry=111,72,977,670
visibility=2

and then try to open the KDE Software Development Kit app page :)

HOWEVER, while that freezes the app page it DOES NOT seem to cause the other featured apps disappear in the Wayland VirtualBox VM where I tried it; and it doesn't make other featured apps to appear if I change the window size so that the app page doesn't anymore freeze on my host machine. So I believe it's unrelated to this bug report but maybe Aleix or Nate has time to look into that, or a separate report can be made.
Comment 7 nttkde 2021-12-15 05:50:45 UTC
Created attachment 144565 [details]
software sources from Discover
Comment 8 Aleix Pol 2021-12-17 00:56:55 UTC
Can you try running "appstreamcli refresh --force" and see if it makes appstream find new things?

The problem you have there is that the appstream database is not populated, so when discover tries to look it up, it cannot.
Comment 9 Aleix Pol 2021-12-17 00:57:50 UTC
The other problem you mentioned is the same as this one.
https://bugs.kde.org/show_bug.cgi?id=446934
Comment 10 nttkde 2021-12-17 14:28:53 UTC
appstreamcli refresh --force did not fix the problem.



appstreamcli status prints:

>AppStreamin tila:
>Versio: 0.15.0
> 
>OS metadata sources:
> Software catalog data
> Group: os-catalog
>  /var/lib/app-info/yaml
>    • YAML:  12
>  /var/cache/app-info/xmls
>    • Collection XML:  0
>  /var/lib/app-info/icons
>    • Iconsets:
>        ubuntu-focal-main
>        neon
>        ubuntu-focal-updates-main
>        ubuntu-focal-universe
>        ubuntu-focal-multiverse
>        ubuntu-focal-updates-universe
>        ubuntu-focal-updates-multiverse
>        ubuntu-focal-backports-main
>        ubuntu-focal-backports-universe
>        ubuntu-focal-security-main
>        ubuntu-focal-security-universe
>        ubuntu-focal-security-multiverse
> 
> Data from locally installed software
> Group: local-metainfo
>  /usr/share/applications
>    • Desktop Entry:  141
>  /usr/share/metainfo
>    • MetaInfo XML:  146
> 
>Other metadata sources:
> Group: flatpak-flathub-x86_64
>  /var/lib/flatpak/appstream/flathub/x86_64/active
>    • Collection XML:  2
>  /var/lib/flatpak/appstream/flathub/x86_64/active/icons
>    • Iconsets:
>        128x128
>        64x64
>        flatpak
> 
>Yhteenveto:
>✔ We have information on 2348 software components.


I have no idea how it works but if it's supposed to find the missing software info in those yaml files, I checked that at least the file linked to /var/lib/app-info/yaml/archive.​neon.​kde.​org_​user_​dists_​focal_​main_​dep11_​Components-​amd64.​yml.​gz is identical between the working and non-working systems (same checksum).

I will attach output of appstreamcli refresh --force --verbose.
If I search the output for eg. kmymoney, it has printed:
>** (appstreamcli:28988): DEBUG: 15:58:15.149: Replaced 'system/package/os/org.kde.kmymoney.desktop/*' with data of higher priority.

>** (appstreamcli:28988): DEBUG: 15:58:15.202: Removed via merge component: system/package/os/org.kde.kmymoney.desktop/*

The working system has this in its output:
>** (appstreamcli:1744): DEBUG: 16:10:54.979: Metadata ignored: Detected colliding IDs: system/package/os/org.kde.kmymoney.desktop/* was already added with a higher priority.
though has 'Removed via merge...' for many other packages.
Comment 11 nttkde 2021-12-17 14:29:52 UTC
Created attachment 144625 [details]
appstreamcli refresh --force --verbose
Comment 12 nttkde 2022-03-06 18:30:51 UTC
As an update to this, I found the reliable way to reproduce this is to change your /etc/apt/sources.list to use country-specific package mirrors, eg.
>deb http://sv.archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
>deb http://security.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
>deb http://sv.archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
or some other country code.
Then run pkcon refresh or apt update and appstream will be borked. Changing it back to the generic mirrors won't fix it.

Would this be rather appstream bug or something Neon packaging specific?
Comment 13 Nate Graham 2022-05-24 13:17:25 UTC
Pretty sure this is a Neon thins; moving there
Comment 14 Jonathan Riddell 2022-05-24 13:22:28 UTC

*** This bug has been marked as a duplicate of bug 453571 ***