Bug 403375

Summary:	Thumbnail extraction for players when there is no metadata provided
Product:	[Plasma] plasma-browser-integration	Reporter:	aldo.mateli
Component:	general	Assignee:	Kai Uwe Broulik <kde>
Status:	REPORTED ---
Severity:	normal	CC:	flying-sheep
Priority:	NOR
Version First Reported In:	unspecified
Target Milestone:	---
Platform:	Other
OS:	Linux
Latest Commit:		Version Fixed/Implemented In:
Sentry Crash Report:
Attachments:	Patch from git

Description aldo.mateli 2019-01-18 20:13:47 UTC

Created attachment 117549 [details]
Patch from git

This patch adds the ability to extract thumbnails from the page when they are not provided from the MediaSession API in order to provide a richer user experience.

As of now thumbnail extraction is as follows: 

1. If the page has already provided metadata then the artwork specified there will override any image the new functionality finds.
2. YouTube is hard-coded. Since it does not support MediaSession nor provide meta tags there is no way of extracting a good thumbnail without it being an edge case. But since it is by far the largest video website, this will have to do until they improve their situation.
3. It attempts to load an image from the OpenGraph meta og:image property which many websites do provide.
4. It uses the favicon. First attempts to find a "hi-res" one then it uses the standard favicon. Although this might lead to small images intended as favicons being upscaled.


Album with some screenshots from plasma/android: https://imgur.com/a/rYY1FgD

Attached is the diff.

Comment 1 Kai Uwe Broulik 2019-01-21 10:52:49 UTC

I toyed around with og:image but if you have a website that uses Ajax for page navigation and/or has playlists, the tags are never updated when you switch to another video as it's only used by static content extractors/parsers.

Comment 2 aldo.mateli 2019-01-21 18:22:08 UTC

I encountered the og:image issue while testing with SoundCloud, but I don't see a way out of it without hardcoding it, like this patch does for YouTube.

Generally, some image should be some improvement over no image at all, although the original goal of this patch was to simply support YouTube, then I added og:image and favicons while working on it.

The downside of og:image is that users might report the incorrect image as a bug when using services which do not update the meta tags, but in any case YouTube and favicon should be safe to use.

Let me know of any thoughts/concerns so that this can get merged or dropped.

Comment 3 Philipp A. 2019-04-02 10:10:15 UTC

We could use a MutationObserver for that, right?

I also think we should also include support for the poster attribute on video tags:

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/video#attr-poster

Comment 4 Kai Uwe Broulik 2019-04-23 07:36:16 UTC

Btw YouTube is not supported through media sessions API, see Bug 406782