Bug 442918

Hi,

Thanks for taking the time to make this suggestion.

First, a small bite of background. When you first open the app, it does not load every message, it only loads the most-recent one for each conversation. When you open a conversation, it loads more messages for that conversation.

The heart of the issue is that the current implementation for collecting the list of conversations on the Android side is (very) slow. When I first wrote the interface, using standard Android SDK calls, it loaded very quickly (~few seconds for my hundreds of conversations). However, it turns out that a vast number of Android manufacturers (Samsung, Huawei... See here: https://bugs.kde.org/show_bug.cgi?id=401677) are using non-standard SMS/MMS implementations.

Given that those non-standard implementations are not documented (and even if they were, there's no way I can support multiple "families" of devices), I chose a "hack" solution where, instead of using the slick, fast interface that Android provides for exactly this purpose, it opens the SMS database and then the MMS database _for each conversation_ and then returns the most-recent one. That's a lot of connections and is pretty much the exact opposite of what any database expert would tell you is optimal.

Now, to your point. Yes, the desktop could cache conversations and present the cache to users. However (without any other changes to the Android implementation), it would take the same amount of time to update that cache as it does today to load messages, meaning the user might happily spend quite awhile responding to what they thought was the most-recent message, only to discover that they've just replied to an old/ancient message.

In my mind, a better fix would be one (or both) of:
- Re-implement my original conversation-getting scheme with some mechanism of failing over to the safe-but-slow implementation we have today. Users of non-standard-Android phones would still be stuck with this slow interface, but at least everyone else wouldn't suffer for their choices.
- Sort conversations by most-recently-replied, send them back in that order, streaming those conversations rather than waiting for all of the conversations to be collected. It would still take a long time to get all the conversations, but most likely a user cares most about the top handful of conversations.

All that is needed is a developer to volunteer their time to implement one of these solutions. I am close to being annoyed enough with the ~20s wait to do something, but it could be a awhile still before I am annoyed enough and have time enough to do the job :)

(As an aside, I don't like dealing with caches. Defining the phone as the only source of truth simplifies the entire design significantly!)

Wow, thanks so much for such a comprehensive reply Simon, that's very helpful. It seems the more I get involved with dev on Android, the more I realise it's an endless series of exceptions, everyone has their own special way of doing a thing, with little advantage gained to the wider ecosystem/other developers/users.

The database access method does indeed sound screwy, not at all efficient or good practice.

> it would take the same amount of time to update that cache as it does today to load messages,

I assume there is no metadata stored anywhere to say "this message/conversation has changed since it was last synced"?. I'm thinking of the model which "unison" uses to store sync information.

Your suggestion of "try the best method, fail to the lowest-common denominator if necessary" sounds like a good step forward, I'd love to volunteer my time, but it's a bit beyond my skills I'm afraid.

Also, it sounds like you might be about to volunteer to write a standards-compliant SMS app, to get around the problem. ;-)

Or at least recommend something which follows the appropriate spec.

I forgot about this bug. However, several months ago I implemented this: https://invent.kde.org/network/kdeconnect-android/-/merge_requests/296. It should be in the latest (early January 2023) KDE Connect Android release and does not require any change on the desktop side.

Conversations are now returned one-by-one, with more-recent conversations returned first. This means the GUI starts being populated nearly instantly, and presumably the more-useful conversations get loaded first since those are likely the most-recent.

There is still plenty of room to improve in this space, but I am going to consider this bug resolved.

(In reply to robin from comment #3)
> Also, it sounds like you might be about to volunteer to write a
> standards-compliant SMS app, to get around the problem. ;-)
> 
> Or at least recommend something which follows the appropriate spec.

Annoyingly, the issue is with non-standard _Android_ implementations. If everyone ran ASOP, there would be no problem. The SMS app which sits on top just uses the interfaces Android provides. There are SMS apps which do everything by themselves (Signal, for example), but that's a special case.