Bug 267065 - ServiceSqlQueryMaker creates inefficient SQL queries
Summary: ServiceSqlQueryMaker creates inefficient SQL queries
Status: RESOLVED UNMAINTAINED
Alias: None
Product: amarok
Classification: Applications
Component: Services/Jamendo (show other bugs)
Version: 2.8.0
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: later
Assignee: Amarok Developers
URL:
Keywords:
Depends on:
Blocks: 235184
  Show dependency treegraph
 
Reported: 2011-02-24 16:48 UTC by Wolfgang Lorenz
Modified: 2024-04-12 16:01 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
patch for testing the behaviour of the SQL collection with genre-less tracks from Jamendo added to the library (1.26 KB, patch)
2013-01-13 14:56 UTC, Wolfgang Lorenz
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Wolfgang Lorenz 2011-02-24 16:48:12 UTC
Version:           2.4.0 (using KDE 4.6.0) 
OS:                Linux

While trying to create a fix for Bug 235184, Amarok hung with an SQL query running, while listing the contents of the newly created genre "Unknown", which contains all albums that have no genre specified (at the moment 28758 albums).

I copied the SQL query and tried it myself:

SELECT DISTINCT jamendo_artists.id, jamendo_artists.name, jamendo_artists.description , jamendo_artists.country, jamendo_artists.photo_url, jamendo_artists.jamendo_url, jamendo_artists.home_url  FROM  jamendo_tracks LEFT JOIN jamendo_albums ON jamendo_tracks.album_id = jamendo_albums.id LEFT JOIN jamendo_artists ON jamendo_albums.artist_id = jamendo_artists.id LEFT JOIN jamendo_genre ON jamendo_genre.album_id = jamendo_albums.id WHERE 1  AND jamendo_genre.name = 'Unknown' GROUP BY jamendo_tracks.id;

The search took about 67 minutes and did a lot of swapping, since mysql doesn't get much memory space for single queries (1MB). The problem here is, that this query uses the table jamendo_tracks which is not needed.

This query delivers the same results after only one second: 

SELECT DISTINCT jamendo_artists.id, jamendo_artists.name, jamendo_artists.description , jamendo_artists.country, jamendo_artists.photo_url, jamendo_artists.jamendo_url, jamendo_artists.home_url  FROM  jamendo_artists LEFT JOIN jamendo_albums ON jamendo_artists.id = jamendo_albums.artist_id LEFT JOIN jamendo_genre ON jamendo_genre.album_id = jamendo_albums.id WHERE 1  AND jamendo_genre.name = 'Unknown' GROUP BY jamendo_artists.id;

The trick is simply to leave out the table jamendo_tracks.

Reproducible: Didn't try

Steps to Reproduce:
* patch the Amarok Jamendo-Service to not ignore albums without genre.
* reload the Jamendo database
* open the genre "unknown" (or "other", or whatever you used)

Actual Results:  
The SQL query takes forever.

Expected Results:  
It should be quicker...
Comment 1 Myriam Schweingruber 2011-02-25 10:57:37 UTC
Could you please submit a patch to http://git.reviewboard.kde.org?
Comment 2 Wolfgang Lorenz 2011-02-25 14:20:24 UTC
Well, there's a problem there, as I have no patch (containing the changed SQL query). I have tested the query, using mysql-query-browser. I could, if you wish, commit the patch, which fills your db with the missing Jamendo entries. But this might leave you with an Amarok, that waits more than an hour for a single query.

Oh... okay... Here you are: https://git.reviewboard.kde.org/r/100742/
Comment 3 Wolfgang Lorenz 2011-05-16 22:21:06 UTC
I just played around a bit and here is another way to query the data (without joining any tables):

SELECT DISTINCT jamendo_artists.id, jamendo_artists.name,
                jamendo_artists.description , jamendo_artists.country,
                jamendo_artists.photo_url, jamendo_artists.jamendo_url,
                jamendo_artists.home_url
FROM jamendo_artists
WHERE jamendo_artists.id IN (
    SELECT jamendo_albums.artist_id
    FROM jamendo_albums
    WHERE jamendo_albums.id IN (
        SELECT jamendo_genre.album_id
        FROM jamendo_genre
        WHERE jamendo_genre.name = 'Unknown'
    )
);
Comment 4 Myriam Schweingruber 2011-05-18 00:49:17 UTC
I think you should take that to the amarok-devel@kde.org mailing list, the developers will be easier to reach than through a bug report.
Comment 5 Ralf Engels 2011-06-01 17:46:39 UTC
Some comments:
1. the review request was discarded because of some issues.
2. leaving out the "tracks" table probably doesn't work because we need to group by track IDs.
3. The grouping by track ids is probably needed to filter out duplicate tracks that appear in several genres.

Comment from me:
I think it would make sense to allow a track being in several genres.
Especially important for my soundtracks that might be "Soundtrack" and "Rock" at the same time.
Comment 6 Myriam Schweingruber 2011-06-04 12:16:46 UTC
This is an automated message from the triager:

Amarok 2.4.1 has been released on May 8 already. Could you please upgrade and test if you can still reproduce this bug?

Without feedback within a month we will close this bug as resolved.

Thank you for your understanding.
Comment 7 Wolfgang Lorenz 2011-06-26 22:39:28 UTC
I just updated Amarok and the problem still exists in 2.4.1.
Comment 8 Myriam Schweingruber 2011-06-26 23:30:03 UTC
Thank you for the feedback.
Comment 9 Myriam Schweingruber 2013-01-11 13:26:33 UTC
There has been a lot of work on the Amarok database since 2.4.1, please test Amarok 2.6.90 aka 2.7 beta or Amarok 2.7, to be released in a few days.
Comment 10 Wolfgang Lorenz 2013-01-13 14:56:30 UTC
Created attachment 76438 [details]
patch for testing the behaviour of the SQL collection with genre-less tracks from Jamendo added to the library

NOTE: the given patch is for testing purposes only and not meant to be applied to the Amarok sources for stable releases – Don't try this at home!

What the appended patch does:
* set genre in Jamendo tracks, that come without a genre to "Unknown-test"
* also, don't ignore genre "Unknown"
* print out running SQL queries with block information

What I did:
* applied patch to Amarok 2.6.90
* ran Amarok with empty configuration and databases
* updated Jamendo track list
* opened genre "Unknown-test"

Result (note delay, at the end):
amarok: BEGIN: QStringList Collections::ServiceSqlQueryMaker::runQuery(const QString&) 
amarok:   running SQL query:  "SELECT DISTINCT jamendo_artists.id, jamendo_artists.name, jamendo_artists.description , jamendo_artists.country, jamendo_artists.photo_url, jamendo_artists.jamendo_url, jamendo_artists.home_url  FROM  jamendo_tracks LEFT JOIN jamendo_albums ON jamendo_tracks.album_id = jamendo_albums.id LEFT JOIN jamendo_artists ON jamendo_albums.artist_id = jamendo_artists.id LEFT JOIN jamendo_genre ON jamendo_genre.album_id = jamendo_albums.id WHERE 1  AND jamendo_genre.name = 'Unknown-test' GROUP BY jamendo_tracks.id;" 
amarok: END__: QStringList Collections::ServiceSqlQueryMaker::runQuery(const QString&) [DELAY Took (quite long) 4.3e+03s] 

Long story made short: The bug still exists.
Comment 11 Myriam Schweingruber 2013-01-13 21:11:22 UTC
Thank you for the feedback
Comment 12 Oskar Jauch 2013-12-03 14:43:25 UTC
Partially reproducible with Amarok 2.8.
Amarok doesn't hung but opening the unknown genre takes to long.
Comment 13 Myriam Schweingruber 2013-12-04 07:49:29 UTC
Mind you, queries on the Jamendo db do take long as it is really huge, and it is a remote service, so that doesn't really help ...
Comment 14 Oskar Jauch 2013-12-04 15:59:37 UTC
Than it isn't reproducible with Amarok 2.8 because it's possible to open the "unknown"-genre section.
Comment 15 Tuomas Nurmi 2024-04-12 16:01:55 UTC
Closing as the old Jamendo API was apparently shut down some years ago and Amarok does not currently have functional Jamendo support.