Bug 122437 - htdig not functional in 'Build Search Index' for kde_application_manuals (khelpcenter)
Summary: htdig not functional in 'Build Search Index' for kde_application_manuals (khe...
Status: RESOLVED FIXED
Alias: None
Product: khelpcenter
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Slackware Linux
: NOR normal
Target Milestone: ---
Assignee: Documentation Editorial Team
URL:
Keywords:
: 112518 125192 136043 146579 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-02-21 19:17 UTC by koolniczka
Modified: 2016-03-12 23:26 UTC (History)
10 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description koolniczka 2006-02-21 19:17:23 UTC
Version:            (using KDE KDE 3.5.1)
Installed from:    Slackware Packages
Compiler:          3.3.6 
OS:                Linux

There are similar bug #103266 and #102912 but i wish to report it for the 3.5.1.

Anyway the problem is widespread in many distributions and i experience it on Slack 10.2 with KDE3.5.1.

The kde_application_manuals index building routine is utterly broken. It does not work either with 
htdig 3.1.6 -- fails in the htmerge step
nor with 
htdig 3.2.0b6. -- routine finishes with the errors described above and no usable index is built.

The first problem is that the htdig PATHs hardcoded (sic!) in the khc_docboodig.pl wrapper (called from khelpcenter) are "distro specific" at best (not to say absolutely bogus) and the script doesnt even honor sytem wide paths. Hardcoded assorted locales don't do much good either.

Other problem is that htdig local URL substitution appears to be broken or the use of it in this wrapper isn't appropriate (for either versions), the htdig 3.2.0b6 doesn't return any errors upstream, finishes correctly but increased verbosity revaeals errors such as previously described here 

http://bugs.kde.org/attachment.cgi?id=11431&action=view

and the subsequent search doesn't return any results (although the index is reportedly built). 

Maybe finaly fixing this (reports since 2003?) would be worth a the effort because it simply "Doesn't Work" (tm)
Comment 1 getreal 2006-04-30 00:59:37 UTC
Yes, yes,

Please do fix this.  So very annoying to get that message:

Creating index for 'kde_application_manuals'
htdig failed

Truly, us "stupid" users might adopt Linux more readily if things like the "Documentation" system actually worked out of the box and we didn't have to spend an hour plus tracking down information on a bug that's been know to exist for the last three years. . .

Mandriva user
Comment 2 Eric Pierce 2006-05-14 03:48:34 UTC
Agreed, this has been broken on Gentoo also, for as long as I can remember.
Comment 3 C. Dominik Bódi 2006-06-17 19:02:51 UTC
*** This bug has been confirmed by popular vote. ***
Comment 4 giuseppe 2006-10-11 02:19:49 UTC
Kde 3.5.5 also suffer this bug!
:(
Comment 5 giuseppe 2007-01-14 09:02:30 UTC
This bug is solved on slackware
you need to edit
2 files /opt/kde/bin/khc_htdig.pl
and /opt/kde/bin/khc_docbookdig.pl
and edit the variables,should be like this

my $htdigdata = "/opt/www/htdig/common/";
my $htdigbin = "/opt/www/htdig/bin";
my $kdeprefix = "/opt/kde";

Comment 6 giuseppe 2007-01-19 20:17:39 UTC
I forgot..
and edit the file /opt/kde/bin/khc_docbookdig.pl

my $htsearchpath="/opt/www/htdig/cgi-bin/htsearch";

Htdig MUST BE the latest 3.20b6
Comment 7 Bram Schoenmakers 2007-02-03 20:00:15 UTC
*** Bug 112518 has been marked as a duplicate of this bug. ***
Comment 8 Bram Schoenmakers 2007-02-03 20:01:45 UTC
*** Bug 136043 has been marked as a duplicate of this bug. ***
Comment 9 Bram Schoenmakers 2007-02-03 20:02:26 UTC
*** Bug 125192 has been marked as a duplicate of this bug. ***
Comment 10 Unknown 2007-09-29 16:54:07 UTC
This exists further in 3.5.7 on Kubuntu Gutsy Gibbon.
This is also the reason, why indexing fails when having a different localization then english.

For example: Error-Message for german localization:

INDEXDIR: /home/emu/.kde/share/apps/khelpcenter/index/
FINDCMD: find /usr/share/doc/kde/HTML/en/ -name index.docbook
Creating index for 'kde_application_manuals'
htdig failed
Comment 11 Myriam Schweingruber 2009-05-17 13:32:14 UTC
Is this bug still relevant for KDE4? No working help system would be really bad...
Comment 12 David Faure 2009-09-17 20:56:42 UTC
"htdig failed" means that htdig was not found. More precisely, it's currently looking for /usr/bin/htdig. I added a better error message now (cf bug 146010), but if htdig can come from other directories then the hardcoded /usr/bin/htdig is bad indeed.
The question is: for those of you who have it in e.g. /opt/www/htdig/bin, is that directory in the $PATH? And if not, how is KDE supposed to find it? :-)
Comment 13 David Faure 2009-09-18 02:34:03 UTC
SVN commit 1025097 by dfaure:

Look for htsearch not only in /srv/www/cgi-bin (SUSE-specific?) but also in /usr/lib/cgi-bin
BUG: 126709
CCBUG: 122437


 M  +2 -1      searchengine.cpp  
 M  +9 -9      searchengine.h  
 M  +6 -2      searchhandler.cpp  
 M  +1 -0      searchhandler.h  
 M  +3 -1      searchhandlers/docbook.desktop.cmake  
 M  +3 -1      searchhandlers/htdig.desktop.cmake  
 M  +1 -1      searchhandlers/khc_htsearch.pl  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1025097
Comment 14 David Faure 2009-09-18 02:35:48 UTC
SVN commit 1025099 by dfaure:

Also look in /opt/www/htdig/bin  (for Slackware, according to comment #5)
CCBUG: 122437


 M  +1 -1      docbook.desktop.cmake  
 M  +1 -1      htdig.desktop.cmake  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1025099
Comment 15 David Faure 2009-09-18 19:22:29 UTC
*** Bug 146579 has been marked as a duplicate of this bug. ***
Comment 16 Ilgaar Elseven 2010-08-30 00:39:53 UTC
I got the same problem with OpenSUSE 11.3, I get this output when I try to search for something.

op-Level Documentation
Application Manuals
openSUSE manual: Security Guide Documentation (en)
 Htdig error: 
Unable to read word database file '/home/me/.kde4/share/apps/khelpcenter/index//opensuse-manuals_en.words.db'
Did you run htdig?
Comment 17 Aaron Digulla 2011-01-12 12:16:42 UTC
On Ubunutu 10.10 with KDE 4.5.4, I get no error after running htdig but it also doesn't return any results. When I run htdig, it seems to index something:

htdig: 44 servers seen:
htdig:     amarok:80 1 document
htdig:     ark:80 1 document
htdig:     dolphin:80 1 document
htdig:     dragonplayer:80 1 document
htdig:     gwenview:80 1 document
htdig:     kate:80 1 document
...

but that barely scratches the number help documents I have installed. Since "amarok" was indexed, I searched for two words that I copied from the help document: "music player"

No results. It seems that searching the help is currently broken.
Comment 18 Pino Toscano 2016-03-12 23:06:10 UTC
Git commit 583ed246c75ce2c686957b6e1dc34aaed31206b4 by Pino Toscano.
Committed on 12/03/2016 at 23:01.
Pushed by pino into branch 'master'.

Add Xapian support for indexing and searching

Start using Xapian for indexing and searching the documentation: it
works much better than htp://dig, it provides a C++ API, and it is well
maintained.

Introduce two helper tools to create/update an index, and search on it,
starting from the cache.bz2 files generated by meinproc: this way it is
possible to index the whole text, with the entities already expanded
(something which would not be there when parsing the docbook files
directly). Make use of libxml2 to parse the HTML documents, and get all
the text in them.

Xapian and libxml2 are currently considered as mandatory dependencies:
they are portable, generally available everywhere, and not requiring
themselves too many extra dependencies.
Related: bug 126710, bug 93664, bug 103266, bug 115935, bug 125276, bug 152671, bug 158633, bug 209415, bug 256397, bug 266290

M  +13   -0    CMakeLists.txt
M  +35   -0    searchhandlers/CMakeLists.txt
A  +134  -0    searchhandlers/cachereader.cpp     [License: GPL (v2+)]
A  +49   -0    searchhandlers/cachereader.h     [License: GPL (v2+)]
A  +107  -0    searchhandlers/htmltextdump.cpp     [License: GPL (v2+)]
A  +29   -0    searchhandlers/htmltextdump.h     [License: GPL (v2+)]
A  +5    -0    searchhandlers/xapian.desktop.cmake
A  +126  -0    searchhandlers/xapiancommon.cpp     [License: GPL (v2+)]
A  +50   -0    searchhandlers/xapiancommon.h     [License: GPL (v2+)]
A  +269  -0    searchhandlers/xapianindexer.cpp     [License: GPL (v2+)]
A  +151  -0    searchhandlers/xapiansearch.cpp     [License: GPL (v2+)]

http://commits.kde.org/khelpcenter/583ed246c75ce2c686957b6e1dc34aaed31206b4
Comment 19 Pino Toscano 2016-03-12 23:26:11 UTC
Git commit 163133768d77ae16e36b31d1174d6c60d53cfdce by Pino Toscano.
Committed on 12/03/2016 at 23:01.
Pushed by pino into branch 'master'.

Remove the ht://dig support

ht://dig is basically unmantained for the past decade, and its
integration in khelpcenter has always been a sort of hack (execute
a CGI, and print its resulting HTML text).

Since Xapian is used as better solution, all of the htp://dig
supporting code can be removed completely.
Related: bug 125276, bug 152671, bug 158633, bug 209415, bug 244091, bug 255384, bug 256397, bug 277464, bug 103266, bug 112958

M  +0    -2    CMakeLists.txt
D  +0    -14   README.htdig
M  +0    -14   doc/khelpcenter/index.docbook
M  +0    -9    docmetainfo.cpp
M  +0    -3    docmetainfo.h
D  +0    -61   htmlsearch.cpp
D  +0    -51   htmlsearch.h
D  +0    -47   htmlsearch/CMakeLists.txt
D  +0    -5    htmlsearch/Messages.sh
D  +0    -23   htmlsearch/checked.xpm
D  +0    -470  htmlsearch/htmlsearch.cpp
D  +0    -254  htmlsearch/htmlsearch.desktop
D  +0    -62   htmlsearch/htmlsearch.h
D  +0    -25   htmlsearch/index.cpp
D  +0    -371  htmlsearch/kcmhtmlsearch.cpp
D  +0    -84   htmlsearch/kcmhtmlsearch.h
D  +0    -52   htmlsearch/klangcombo.cpp
D  +0    -47   htmlsearch/klangcombo.h
D  +0    -246  htmlsearch/ktagcombobox.cpp
D  +0    -92   htmlsearch/ktagcombobox.h
D  +0    -6    htmlsearch/long.html
D  +0    -4    htmlsearch/meinproc_wrapper
D  +0    -21   htmlsearch/nomatch.html
D  +0    -89   htmlsearch/progressdialog.cpp
D  +0    -32   htmlsearch/progressdialog.h
D  +0    -1    htmlsearch/short.html
D  +-    --    htmlsearch/star.png
D  +-    --    htmlsearch/star_blank.png
D  +0    -19   htmlsearch/syntax.html
D  +0    -22   htmlsearch/unchecked.xpm
D  +0    -16   htmlsearch/wrapper.html
D  +0    -156  htmlsearchconfig.cpp
D  +0    -58   htmlsearchconfig.h
M  +0    -1    kcmhelpcenter.cpp
M  +0    -2    kcmhelpcenter.h
M  +2    -14   searchhandlers/CMakeLists.txt
D  +0    -8    searchhandlers/docbook.desktop.cmake
D  +0    -8    searchhandlers/htdig.desktop.cmake
D  +0    -2    searchhandlers/htdig_long.html
D  +0    -222  searchhandlers/khc_docbookdig.pl.cmake
D  +0    -148  searchhandlers/khc_htdig.pl.cmake
D  +0    -117  searchhandlers/khc_htsearch.pl
M  +1    -2    tests/CMakeLists.txt

http://commits.kde.org/khelpcenter/163133768d77ae16e36b31d1174d6c60d53cfdce