Bug 277007 - KMail2 full search of emails does not work
Summary: KMail2 full search of emails does not work
Status: RESOLVED FIXED
Alias: None
Product: kmail2
Classification: Applications
Component: general (show other bugs)
Version: 5.3.0
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: kdepim bugs
URL:
Keywords:
: 299840 324189 345054 353318 353671 353675 354406 365801 374952 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-07-03 14:42 UTC by Thomas Berger
Modified: 2017-01-13 14:28 UTC (History)
26 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Berger 2011-07-03 14:42:11 UTC
Version:           2.0.95 (using Devel) 
OS:                Linux

Searching mails via the "full search" feature of KMail2 does always return a empty result.

Reproducible: Didn't try

Steps to Reproduce:
Open the "full search" window and click on "Search" without any keyword.

Actual Results:  
0 results returned immediately 

Expected Results:  
All mails in the folder should be shown up.

I am using the kde-overlay for Gentoo, currently all versions are at 4.6.90
Comment 1 Allen Winter 2011-07-03 15:10:57 UTC
version 2.0.95 is old by now.
you should consider installing KDEPIM 4.6 which includes kmail 2.1.0
Comment 2 Aleksey 2011-07-07 08:33:08 UTC
Search doesn't work in kmail 2.1.0 at all. Using kontact 4.6.0, KDE 4.6.5.
Comment 3 Laurent Montel 2011-07-08 11:46:40 UTC
Are you sure that you have not pb with nepomuk ?
Comment 4 Andreas Kuhl 2011-07-08 14:41:06 UTC
Happens for me, too! Nepomuk is working (indexed search works properly in Dolphin) but searching in KMail via "find messages" and scope "complete message" does return nothing.

Searching the same keyword (which only exists in a mail) via KRunner returns a "akonadi:?item=10977" result from type Resource.

KDE SC 4.6.90 with KMail 2.0.95 on openSUSE 11.4 64bit
Comment 5 Unknown 2011-07-09 11:52:17 UTC
The description of this bug with "steps to reproduce" leaves all fields empty. If you click "search" then, I think it's acceptable that no results are shown, because you're basically not searching for anything. :-)

If you select "Complete Message" contains "something", I'm getting results... although it might take quite some time depending on the amount of e-mails and whether you're using IMAP or POP3. Funny thing is, that it doesn't show anywhere that it's still busy searching. So you might think that it doesn't come up with any results. This is unacceptable behavior, since there's no way of knowing whether it has finished with no results, or it is still busy searching.

So, I'm guessing that the above is what's happening to Andreas Demmer in #4? To confirm this, wait a few minutes after a search.

As for the bug description, it might be a nice feature if one can do a search for everything: that way, one can combine "sent" and "inbox" folders and turn threading on. But that's not part of this bug. :)
Comment 6 Andreas Kuhl 2011-07-10 09:50:00 UTC
No, there is no background activity for me, the search has already finished. And an indexed search should run only a couple of seconds. At least, that is how it worked back in KDE 4.6 already with KMail2: Searching took only a fraction of a second and results appeared. Here I can wait for minutes and no results turn up.
Comment 7 Luis Silva 2011-07-31 20:21:51 UTC
I have the exact same result. I am compiling from git master against kde 4.7.0.
When launching kmail from a command line and doing a search for foo in mail headers, for example I get the following invalid SPARQL querry:
select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector('foo'), ?v2))) as ?_n_f_t_m_ex_ where { { ?r <http://akonadi-project.org/ontologies/aneo#akonadiItemId> ?reqProp1 . ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#messageSubject> ?v2 . FILTER(bif:contains(?v2, "'foo'")) . ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#Email> . } . ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible> ?v1 . FILTER(?v1>0) . }

If instead I search for "foo" in the "Complete message" I again get an invalid querry:
select distinct ?r ?reqProp1 (bif:concat(bif:search_excerpt(bif:vector('foo'), ?v11),bif:search_excerpt(bif:vector('foo'), ?v12),bif:search_excerpt(bif:vector('foo'), ?v14),bif:search_excerpt(bif:vector('foo'), ?v4))) as ?_n_f_t_m_ex_ where { { ?r <http://akonadi-project.org/ontologies/aneo#akonadiItemId> ?reqProp1 . { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#to> ?v2 . ?v2 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEmailAddress> ?v3 . ?v3 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#emailAddress> ?v4 . FILTER(bif:contains(?v4, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#cc> ?v5 . ?v5 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEmailAddress> ?v6 . FILTER(bif:contains(?v4, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#bcc> ?v7 . ?v7 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEmailAddress> ?v8 . FILTER(bif:contains(?v4, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#from> ?v9 . ?v9 <http://www.semanticdesktop.org/ontologies/2007/03/22/nco#hasEmailAddress> ?v10 . FILTER(bif:contains(?v4, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#messageSubject> ?v11 . FILTER(bif:contains(?v11, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#plainTextMessageContent> ?v12 . FILTER(bif:contains(?v12, "'foo'")) . } UNION { ?r <http://www.semanticdesktop.org/ontologies/2007/01/19/nie#isPartOf> ?v13 . ?v13 <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#plainTextMessageContent> ?v14 . FILTER(bif:contains(?v14, "'foo'")) . } . ?r a <http://www.semanticdesktop.org/ontologies/2007/03/22/nmo#Email> . } . ?r <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#userVisible> ?v1 . FILTER(?v1>0) . }

It seems that the problem is in the querry construction.
Comment 8 Laurent Montel 2011-08-01 09:13:24 UTC
I will work on search during this week
It's important to fix it.
Regards
Comment 9 Andreas Kuhl 2011-08-01 09:33:35 UTC
That's good news, thank you! :)
Comment 10 Laurent Montel 2011-08-01 12:04:20 UTC
Yes for complete message query is not good. I will investigate it.
There is still some search type not implemented.
I will implement them soon.
Comment 11 Laurent Montel 2011-08-02 12:32:54 UTC
Ok now search for messages/headers etc. is ok.
Need to improve all search but now it's ok.
fixed in 4.7/master.
Will work to make other search work (for example specific header etc.)
Comment 12 Andreas Kuhl 2011-08-02 12:49:49 UTC
I will not be able to test until KDE SC 4.7.1 is available in openSUSE repos, building KMail from trunk is a tough job.
Comment 13 Laurent Montel 2011-08-02 13:05:12 UTC
not necessary to build now
I think there is still bugs but by default we can search
so it's a good step :)
Comment 14 Luis Silva 2011-08-03 07:57:20 UTC
I can confirm that the latest changes already shopw an improvement. However, kmail is now crashing everitime I perform a given search the first time. Probably a different issue. Should I submit a different bug or just post the backtrace here and keep it all together?
Comment 15 Laurent Montel 2011-08-03 08:01:36 UTC
Yes please create a new bug or give the backtrace
thanks
Comment 16 Luis Silva 2011-08-03 17:13:22 UTC
(In reply to comment #15)
> Yes please create a new bug or give the backtrace
> thanks

Just created Bug 27930 for this. Cheers and thanks.
Comment 17 leo_rockway 2012-06-12 07:38:54 UTC
Search not working on KDE SC 4.9 beta 1.
Comment 18 Luis Silva 2012-06-12 17:22:38 UTC
I can confirm this with both kubuntu-ppa/beta packages and sources compiled from git master.
Comment 19 leo_rockway 2012-07-01 07:40:44 UTC
I can see that search is finding some results using KDE SC 4.9RC1. It's only showing a limited amount of results, though. I searched for an email with a subject I already knew existed; the email is a few months old and it didn't show up in the search. When I search for something that I know exists and is newer then I can get proper results.
In the Maintenance tab of the folder properties I can see that it says the folder was indexed a few days ago.

Anyway, glad to see that searching is making progress. Thank you for this and keep up the good work.
Comment 20 Colin J Thomson 2012-07-16 18:05:37 UTC
I am seeing this on Fedora17 x86_64 - KDE 4.9RC2.
Downgrading virtuoso-opensource to version 6.1.4 and search in Kmail now works again and fast..
Of note the results of my test search that was empty when running 6.1.5 where shown after the downgrade to 6.1.4.
Comment 21 Colin J Thomson 2012-07-31 17:07:15 UTC
An update of virtuoso-opensource to 1:6.1.6-0.1.rc2 seems to have fixed this with my initial testing. KDE 4.9/F17 x86_64
Comment 22 Myriam Schweingruber 2012-10-03 10:09:21 UTC
*** Bug 299840 has been marked as a duplicate of this bug. ***
Comment 23 Parameshwara Bhat 2012-10-03 11:07:33 UTC
search not working for opensuse 12.2 kde 4.8.5 kmail 4.8.5
pbhat@linux-8ddu:~> rpm -qa|grep virtuoso
virtuoso-drivers-6.1.6-3.4.1.i586
soprano-backend-virtuoso-2.7.6-2.1.2.i586
virtuoso-server-6.1.6-3.4.1.i586
Comment 24 Andrew Gaydenko 2012-10-07 15:24:20 UTC
4.9.2 (under Arch) still has the problem.
Comment 25 Luis Silva 2012-10-07 15:34:10 UTC
I can confirm that this is still a problem with 4.9.2 and master.
I was looking at the code and it seems that the problem may even be in the akonadi code that performs the search.
Comment 26 Luis Silva 2012-10-07 15:42:04 UTC
Other issues that may have a common cause:
- Searching for a specific message status like "Important" only ever returns the unread messages.
- Searching for messages arrived in the last three days returns all my messages.
Comment 27 Andrew Gaydenko 2012-10-07 20:00:47 UTC
This is a debug output from all processes with 'mail' context in name. I have started kmail, invoked search dialog (S), printed a letter in the search context field and clicked search button. At the end of fragment below you will find two interesting notes about 

- disabled code, and
- "" (empty string) context field content.

kmail2(26429) KMKernel::openReader:
kmail2(26429) MessageList::StorageModel::StorageModel: Using model: Akonadi::EntityTreeModel
kmail2(26429) Nepomuk::ResourceWatcher::start: Successfully connected to watch service
kmail2(26429) KMail::KMSystemTray::KMSystemTray: Initting systray
kmail2(26429) MessageList::Core::View::resizeEvent: Resize event enter (viewport width is  235 )
kmail2(26429) MessageList::Core::View::resizeEvent: Resize event enter (viewport width is  1108 )
kmail2(26429) KSieveUi::Vacation::Vacation: Vacation: found url " "" "
kmail2(26429) MessageList::Core::View::resizeEvent: Resize event enter (viewport width is  1108 )
kmail2(26429) MailCommon::SearchRuleWidget::setRule: ( ""Subject" <contains> """ )
kmail2(26429) KMail::SearchWindow::SearchWindow: AKONADI PORT: Disabled code in   KMail::SearchWindow::SearchWindow(KMMainWidget*, const Akonadi::Collection&)
Comment 28 Till Schäfer 2013-07-29 14:34:08 UTC
I can confirm that  the advanced search does not find every message which contains a keyword. It seems that only a subset of the results are shown. This may be related to IMAP (see bug: 223813 ). 

another bug that might be related to this one: 258676


using KDE 4.10.5 on Gentoo
Comment 29 Andrew Gaydenko 2013-07-29 14:48:59 UTC
(In reply to comment #28)
> This may be related to IMAP (see bug: 223813 ). 

The issue takes place with POP3 accounts.
Comment 30 Christopher Heiny 2013-07-29 14:50:21 UTC
It also occurs with local mail files and maildirs.
Comment 31 Laurent Montel 2013-09-12 14:03:45 UTC
I made a lot of improvment in 4.11.2 and 4.12
Will continue to improve search
Comment 32 Hussam Al-Tayeb 2014-04-09 22:27:17 UTC
Search works better here after baloo port.
Comment 33 Andrew Gaydenko 2014-04-09 23:18:15 UTC
Hussam, please, would you be a little more elaborate? Frankly, I don't understand "better". I understand "does work" or "doesn't work" :) In other words, can you be absolutely sure current search finds *all* occurrences of the search context among thousands of messages?
Comment 34 Hussam Al-Tayeb 2014-04-10 07:38:53 UTC
On Wednesday 09 April 2014 23:18:15 you wrote:
> https://bugs.kde.org/show_bug.cgi?id=277007
> 
> --- Comment #33 from Andrew Gaydenko <a@gaydenko.com> ---
> Hussam, please, would you be a little more elaborate? Frankly, I don't
> understand "better". I understand "does work" or "doesn't work" :) In other
> words, can you be absolutely sure current search finds *all* occurrences of
> the search context among thousands of messages?

Searching for keywords in email subjects now always works. Previously it 
randomly failed.
Searching for keywords in "Body of Message" now works in emails that are 
filtered to specific folders but not in the top "inbox" folder. This is when I 
do "Tools -> Search" and search only in "Local Folders/inbox" but with 
"Include sub-folders" enabled.
Comment 35 Till Schäfer 2014-05-02 14:19:14 UTC
it is still broken for me in 4.13.0 with baloo backend. when i do a a search in the quickfilter or in the general search dialog over all folders, a mail that is located in my send folder is never found. i tried different matching strings, the from, to and subject, fields. The message is never listed. 

it is much better than before baloo, but is is still not completely working.
Comment 36 Thomas Arend 2014-09-22 18:03:44 UTC
Search and quick filtering doesn't work in 4.14.1

I try to filter after info@twitter.com but get a lot of messages where this string is not included.

Restricting the search  to "From" works. Restricting it to the body should gives a lot none twitter e-mails but no twitter e-mail.

Filtering is still a disaster!

How did you code this? Normally "If a=b then" should work in any programming language. Or do you roll the dice?
Comment 37 Thomas Arend 2014-09-22 18:16:06 UTC
I searched for "info@twitter.com" in the whole message and "vmware" only in the From header field. This should return an empty list but I got 36 of 53 VMWare messages.
Comment 38 Robert Scott 2015-09-28 16:29:29 UTC
I'm having the same problem with 4.14.1 under debian jessie. Immediately returning no search results no matter what I type.

Part of me wants to say this is the last straw with kmail2, but I've said it before too much already. I think the only reason I've been staying with it is because of the convenience of kde's kioslaves. It makes me sad because kmail1 was one of my *favourite* applications.
Comment 39 Ezio Vergine 2015-10-06 09:10:05 UTC
Same issue here:

kmail2 5.0.1
KDE Frameworks 5.14.0
Qt 5.5.0 (built against 5.5.0)

I've only one result for every query I try.
Comment 40 Josep 2016-01-21 14:03:03 UTC
Just the same, with the same configuration as Robert Scott (kmail 4.14.1, Debian Jessie, all KDE packages installed through package system). Searchs finish almost instantly, but always with empty results. Tried selecting search folder manually, include and don't include subfolders and several other options, but the search was always instantaneous and empty.
Comment 41 Denis Kurz 2017-01-13 13:09:41 UTC
*** Bug 345054 has been marked as a duplicate of this bug. ***
Comment 42 Denis Kurz 2017-01-13 13:11:09 UTC
*** Bug 354406 has been marked as a duplicate of this bug. ***
Comment 43 Denis Kurz 2017-01-13 13:12:40 UTC
Confirmed for 5.3.0 in Bug 354406
Comment 44 Denis Kurz 2017-01-13 13:14:10 UTC
*** Bug 365801 has been marked as a duplicate of this bug. ***
Comment 45 Denis Kurz 2017-01-13 13:16:01 UTC
*** Bug 374952 has been marked as a duplicate of this bug. ***
Comment 46 Denis Kurz 2017-01-13 13:18:09 UTC
*** Bug 353671 has been marked as a duplicate of this bug. ***
Comment 47 Denis Kurz 2017-01-13 13:23:56 UTC
*** Bug 353318 has been marked as a duplicate of this bug. ***
Comment 48 Denis Kurz 2017-01-13 13:29:40 UTC
*** Bug 353675 has been marked as a duplicate of this bug. ***
Comment 49 Denis Kurz 2017-01-13 13:47:11 UTC
Bug 353675 probably describes what's going on when this bug occurs. Bug 353675, Comment 3 describes the cure. Daniel committed the fix before the 16.12.1 release, so this version should be unaffected.

I set this bug to fixed for now. If it still happens to you, dear reader, in version 16.12.1 (a.k.a. 5.4.1), please report back and I'll reopen.

If you do, please take care not to mix up this bug with one of the other search-related bugs currently around, since we have other reports for these.

Ideally, you'd make sure that *every* search fails, especially the first one after a reboot (or restart of akonadi/kmail). Not being able to do subsequent searches, or not being able to edit search criteria of existing search folders, or even searching in the message body or using the "does not contain" filter... those are all different bugs that should not be discussed here.

Btw, without Daniel's comment I would mark this as WORKSFORME, because it works for me on 16.12.1. It did not work in older versions in my setup.
Comment 50 Denis Kurz 2017-01-13 14:28:24 UTC
*** Bug 324189 has been marked as a duplicate of this bug. ***