Bug 274294 - okular's synctex support doesn't work with lua(la)tex generated output
Summary: okular's synctex support doesn't work with lua(la)tex generated output
Alias: None
Product: okular
Classification: Applications
Component: general (show other bugs)
Version: 0.12
Platform: OpenSUSE Linux
: NOR normal (vote)
Target Milestone: ---
Assignee: Okular developers
Depends on:
Reported: 2011-05-27 18:52 UTC by max
Modified: 2011-10-25 20:10 UTC (History)
8 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.7.2

source (8.85 KB, text/tex)
2011-05-27 19:00 UTC, max
the luatex generated synctex output (23.22 KB, application/x-gzip-compressed)
2011-05-27 19:03 UTC, max
the pdftex generated synctex output (22.26 KB, application/x-gzip-compressed)
2011-05-27 19:04 UTC, max
generated pdf (24.63 KB, application/pdf)
2011-05-27 19:05 UTC, max
The test pdf file (10.18 KB, application/pdf)
2011-09-28 20:17 UTC, Philipp Woelfel
synctex file (664 bytes, application/x-gzip)
2011-09-28 20:17 UTC, Philipp Woelfel
LaTeX file (191 bytes, text/x-tex)
2011-09-28 20:18 UTC, Philipp Woelfel

Note You need to log in before you can comment on or make changes to this bug.
Description max 2011-05-27 18:52:43 UTC
Version:           0.12 (using KDE 4.6.0) 
OS:                Linux

Please C.f. 
Jose Aliste from evince said he believes it to be a bug in the synctex parser.

Reproducible: Always

Steps to Reproduce:
generate a tex/latex file of more than one page (easier to see that way) and run lua(la)tex --synctex=1 some.tex
okular --unique some.pdf#src:100some.tex

Actual Results:  
Nothing happens

Expected Results:  
Okular should jump to the line corresponding to line 100 in some.tex

I can provide samples if needed but you can generate them yourself easily
Comment 1 max 2011-05-27 19:00:39 UTC
Created attachment 60394 [details]
Comment 2 max 2011-05-27 19:03:02 UTC
Created attachment 60395 [details]
the luatex generated synctex output
Comment 3 max 2011-05-27 19:04:00 UTC
Created attachment 60396 [details]
the pdftex generated synctex output
Comment 4 max 2011-05-27 19:05:26 UTC
Created attachment 60397 [details]
generated pdf
Comment 5 Jochen Trumpf 2011-05-29 10:20:08 UTC

have a look at the luatex generated synctex output. It has a full path in the source line, but you try to evoke synctex (embedded in okular) with a relative path. That doesn't seem to work. There doesn't seem to be anything else wrong with your synctex output. You can use the synctex command line tool to play with it yourself and you will find that it finds the synctex nodes once you provide a path that matches the source entry. 

Try to set up your latex environment such that it calls okular with the same path as it writes to the synctex file. That should solve your problem.

Comment 6 max 2011-05-29 10:33:46 UTC
Thanks for looking at it. That is of course a possible solution, I guess i will do that until the parser is patched to accept all possible ways to specify it.
Comment 7 Philipp Woelfel 2011-07-21 22:34:27 UTC
The same problem happens with pdflatex in texlive2011. Synctex now produces filenames of the form "<path>/./<filename>". Although these are correct pathnames, okular does not recognize them correctly and does not jump to the specified line. 

IMO the filename parser of okular should be fixed to recognize all syntactically correct pathnames.

The work-around described by Jochen does not work when okular is called from a third-party tool, e.g., kile.

Also, is it possible to change the title of this bug, because the bug is more general than the current title suggests? E.g., "okular's synctex filename parser is broken".
Comment 8 Jochen Trumpf 2011-07-29 05:48:58 UTC
I think you are partly confusing which program is doing what. The synctex file (<name>.synctex.gz) is written by pdflatex (or luatex) and specifies a path to a source file. Pdflatex (or luatex) writes this path and decides whether to use a relative path, an absolute path, or even a path that needs expanding. 

Okular does no parsing of source file paths. It simply passes the path you specify on the commandline to the embedded synctex library which in turn reads the synctex file. It looks like this library only supports exactly matching paths, in particular no path expansion. That is not a problem in okular (although theoretically okular could try to provide a workaround).

My comment above points out that in the absence of better support through the synctex library and unless somebody writes a source file path parser for okular, you will need to configure your latex environment (pdflatex/lualatex and your mechanism for calling okular) to provide matching source file paths. This workaround applies in all cases, including when you are using kile.
Comment 9 Philipp Woelfel 2011-07-30 09:00:53 UTC
Seems to be fixed in okular 0.13, openSuSE RPMs of KDE 4.7. The synctex library seems to be fixed with version 1.15 (according to its changelog).
Comment 10 beoram 2011-07-30 21:49:38 UTC
This is an incredibly annoying problem.

I updated Okular to 0.13 and the "command-line synctex" I'm using reports that it's version 1.15. But I am still experiencing the same trouble trying to use forward search with Okular (both from within Emacs and from the command-line) with pdf files compiled with TeXlive 2011---namely that it doesn't work. 

Has anyone any idea of a solution or a work-around?

(I've tried using the file's full path when calling Okular, and it doesn't make a bit of difference.)
Comment 11 Jochen Trumpf 2011-07-30 22:46:57 UTC
The only suggestion I have is to look at the .synctex.gz file. The source file path is listed in one of the lines labelled "Input" (usually the first). Match that path on the okular commandline (you might have to match it exactly as printed).

According to the synctex changelog, 1.15 supports a prefix of "./" (as used by lua(la)tex) but only on windows platforms, 1.16 supports it on all platforms. Texlive 2011 seems to use a different syntax.

I would appreciate if you could report back on your findings so we can get a better sense of what the actual problem is. Thanks.
Comment 12 Philipp Woelfel 2011-07-31 03:17:05 UTC
I don't know why it works with the openSuSE okular 0.13 rpm. From the source file it seems that it uses synctex 1.9 + several patches. Perhaps one of the patches fixes it, but I don't know which one...
Comment 13 Philipp Woelfel 2011-07-31 20:06:54 UTC
ugh... when it worked I may have used an "old" .synctex.gz file. So it may not be fixed in my okular version after all. Sorry for the confusion.
Comment 14 beoram 2011-07-31 20:13:31 UTC
Jochen is right: if I use the exact name, including "./" in referencing the tex file, I can get it to work from the command line, using a new .synctex.gz file and the new synctex command line program.


"okular --unique my-file.pdf#src:50/path/to/my/tex/file/./my-file.tex"

correctly goes to line 50.

How to write the appropriate command in emacs lisp is a different question, however.
Comment 15 Philipp Woelfel 2011-07-31 22:40:58 UTC
In linux, the synctex filename parser compares just the plain strings using strcmp. For windows it actually compares the filenames character in a loop and gets rid of case sensitivity and treats "/" and "\" equivalently.

So here is a non-developer's dirty fix:

In generators/poppler/synctex/synctex_parser_utils.c, function  _synctex_is_equivalent_file_name I replaced the part between "#else" and "#endif" with with the following:

#       else
        if(SYNCTEX_IS_PATH_SEPARATOR(*lhs)) {/*  lhs points to a path separator */
                if(!SYNCTEX_IS_PATH_SEPARATOR(*rhs)) {/*  but not rhs */
                        return synctex_NO;
                for (; *(lhs + 1) == '.' && SYNCTEX_IS_PATH_SEPARATOR(*(lhs + 2)); lhs += 2);
                for (; *(rhs + 1) == '.' && SYNCTEX_IS_PATH_SEPARATOR(*(rhs + 2)); rhs += 2);
        } else if(SYNCTEX_IS_PATH_SEPARATOR(*rhs)) {/*  rhs points to a path separator but not lhs */
                return synctex_NO;
        } else if (!*lhs) {/*  lhs is at the end of the string */
                return *rhs ? synctex_NO : synctex_YES;
        } else if(!*rhs) {/*  rhs is at the end of the string but not lhs */
                return synctex_NO;
        goto next_character;
/*    return 0 == strcmp(lhs,rhs)?synctex_YES:synctex_NO; */
#       endif

This is just a copy of the windows part of the code minus comparing uppercase symbols plus 2 for-loops taken from the sumatraPDF code (http://blog.kowalczyk.info/software/sumatrapdf/free-pdf-reader.html)

Perhaps it's  not very clean, but it works for me...
Comment 16 Jochen Trumpf 2011-08-01 00:18:57 UTC
Philipp, can I suggest that you report that upstream to the synctex developers? My understanding is that there is a new version 1.16 that already tries to fix this. It would be a shame if your fix would conflict with that. From okular's perspective, I think we want to use upstream synctex unchanged so will probably wait until a newer synctex version is available (this is not my call of course but that of the okular maintainer).
Comment 17 Philipp Woelfel 2011-08-01 02:10:03 UTC
I sent an e-mail to the synctex developer.

As a side note, I think (and I didn't realize this previously) synctex version 1.16 is trying to fix something else, namely to remove leading occurrences of "./" as opposed to arbitrary substrings of the form "./".
Comment 18 jenshnielsen 2011-08-01 15:07:51 UTC
I have found a work around for auctex and okular.

A possible work around in emacs is as follows using okular. 
Change the okular command in Tex View to 

okular --unique %o#src:%n%(dir)./%b

There is no expander to give the current directory as far as I can see. 
But you can define a new expander in Tex Expand List.
I.e go to Customize AUCTeX ... TeXCommand ... Tex Expand List and add one. 
and save:

I used:

Key: %(dir)
Expander: (lambda nil default-directory)
Comment 19 beoram 2011-08-01 15:17:23 UTC
Many thanks, jenshnielsen!  That seems to work!

Comment 20 Oliver Heins 2011-08-02 18:05:46 UTC
jenshnielsen's solution for emacs doesn't work if you have organized your files in subdirectories.  The following, which is based on code from Jochen, works for me:

(add-hook 'LaTeX-mode-hook '(lambda ()
			      (add-to-list 'TeX-expand-list
					   '("%u" Okular-make-url))))
(defun Okular-make-url () (concat
			   (expand-file-name (funcall file (TeX-output-extension) t)
					     (file-name-directory (TeX-master-file)))
			   (expand-file-name (TeX-master-directory))

The okular command in Tex-View then has to be changed to 

okular --unique %u

Comment 21 Jorge Alfaro 2011-09-18 21:15:01 UTC
Oliver's solution works only for directory names without spaces. Is there something that can be done for the ones that have spaces?
Comment 22 Enrique Artal 2011-09-26 13:58:15 UTC
Does anyone know a solution for okular+kile using pdflatex?
Comment 23 Philipp Woelfel 2011-09-26 17:28:11 UTC
Below is a quick workaround that works with kile+okular and probably can accommodate any other <editor>+okular. I only tested it with one TeX file, so it may break with more complicated filenames, but it should be easy to extend.

Now, can you please vote for that bug? I'm surprised that I am the only one who voted for it. It's status is still UNCONFIRMED, which is also suprising.

Workaround for kile:
(1) make sure you have "sed" (should be installed by default on most linux distros)

(2) Create a file, say "<homedir>/bin/fwokular.sh", which the following script:

target=`echo $1 | sed -e "s%\(.*\)/\([^/]*$\)%\1/./\2%"`
okular --unique "$target"

(3) Configure okular's ForwardPDF to execute "<homedir>/bin/fwokular.sh" with the option '%absolute_target'

The shell script just replaces the last occurrence of "/" in '%absolute_target' with "/./" and then calls okular.
Comment 24 Enrique Artal 2011-09-26 20:15:31 UTC
Thanks! It works.

(In reply to comment #23)
> Below is a quick workaround that works with kile+okular and probably can
> accommodate any other <editor>+okular. I only tested it with one TeX file, so
> it may break with more complicated filenames, but it should be easy to extend.
> Now, can you please vote for that bug? I'm surprised that I am the only one who
> voted for it. It's status is still UNCONFIRMED, which is also suprising.
> Workaround for kile:
> (1) make sure you have "sed" (should be installed by default on most linux
> distros)
> (2) Create a file, say "<homedir>/bin/fwokular.sh", which the following script:
> #!/bin/bash
> target=`echo $1 | sed -e "s%\(.*\)/\([^/]*$\)%\1/./\2%"`
> okular --unique "$target"
> (3) Configure okular's ForwardPDF to execute "<homedir>/bin/fwokular.sh" with
> the option '%absolute_target'
> The shell script just replaces the last occurrence of "/" in '%absolute_target'
> with "/./" and then calls okular.
Comment 25 Jochen Trumpf 2011-09-26 21:06:58 UTC
(In reply to comment #23)
> It's status is still UNCONFIRMED, which is also suprising.

Well, it is not really a bug in okular (cf. my comment #8 above). IMHO this needs to be "fixed" in the (upstream) synctex library, and even there it is more of a wish then a bug. Synctex in its current state does not support path expansion (with the exception of a leading './' since version 1.15/1.16). You can't expect a tool to magically do something it does not support. Kile does just that by writing one form of path into the synctex file and then calling the viewer with a different path. You could call this a bug in kile. It is irrelevant that the paths containing '/./' are "valid". They need expansion which is currently not supported. 

I would change the title and/or status of this bug to more closely reflect this but I am not the owner (and not the okular maintainer).
Comment 26 Albert Astals Cid 2011-09-28 13:30:11 UTC
To be honest i lost track of what the bug is about, i don't use latex at all so most of the things you speak about are quite obscure to me but as I read some of the comments it seems to me this is a problem on how the paths are generated by third party tools and if those paths are generated correctly everything works.

Now if there is some other !Okular tool that is able to understand those paths, then it'd be a fault on our part.

So please if there is someone that can summarize what the situation is and give a very exact steps to reproduce the problem (if there is actually one) i can try to fix it.
Comment 27 Philipp Woelfel 2011-09-28 20:17:29 UTC
Created attachment 64048 [details]
The test pdf file
Comment 28 Philipp Woelfel 2011-09-28 20:17:54 UTC
Created attachment 64049 [details]
synctex file
Comment 29 Philipp Woelfel 2011-09-28 20:18:18 UTC
Created attachment 64050 [details]
LaTeX file
Comment 30 Philipp Woelfel 2011-09-28 20:30:54 UTC
Albert, thanks for willing to work on this. Here is how to reproduce the bug: Move the 3 attachments I just created (stest.*) in a directory <dir>

Call (exact syntax and filenames is important):
> okular file:<dir>/stest.pdf#<num><dir>/./stest.tex

where <num> is one of 6,12, or 18. The correct behaviour is that okular shows page <num>/3 (i.e., page 1,2, or 3, resp.)

The bug manifests itself if you call
> okular file:<dir>/stest.pdf#<num><dir>/stest.tex

(The difference is that "/./" has been removed from the path.)
In this case, okular always shows the same page, no matter what <num> is.

The expected behavhiour is, however, that okular should also show page <num>/3.
LaTeX editors (such as kile) call okular using the 2nd syntax, and thus fail to open the correct page.

The bug can be easily fixed in okular itself (see my Comment #15). Essentially, in the file generators/poppler/synctex/synctex_parser_utils.c the method _synctex_is_equivalent_file_name has to be made a bit more intelligent.
Comment 31 Albert Astals Cid 2011-09-28 22:10:46 UTC
saved files in lalala folder
 okular file:lalala/stest.pdf#18lalala/./stest.tex
 okular file:/home/kdeunstable/lalala/stest.pdf#18/home/kdeunstable/lalala/./stest.tex
open okular in the first page
Comment 32 Philipp Woelfel 2011-09-29 00:52:39 UTC
Sorry, my bad. Please gunzip stest.synctex.gz and at the top of stest.synctex, replace the path in the line starting with "Input1" with your path.

Also, when you call okular, add "src:" after "#", like so:
> okular file:<dir>/stest.pdf#src:<num><dir>/./stest.tex
Comment 33 Philipp Woelfel 2011-09-29 15:35:34 UTC
Albert: Just a bit of information how these files are generated. The author of a document creates the .tex file, and then runs LaTeX on it. That creates the output .pdf file together with the .synctex.gz. With editors such as kile the author can point to a certain position in the .tex file, and okular should open the .pdf file at the corresponding page.
Comment 34 Albert Astals Cid 2011-09-30 18:49:50 UTC
Git commit b932ef7e0fbd96bbb04aafca22c1c936847e5fe2 by Albert Astals Cid.
Committed on 30/09/2011 at 20:47.
Pushed by aacid into branch 'KDE/4.7'.

Comparing paths with strcmp is not really smart

I don't know if Dirk has already tagged 4.7.2 or not so it will be in either 4.7.2 or 4.7.3

BUGS: 274294

A  +22   -0    generators/poppler/synctex/patches/08-fix_path_comparison.diff
M  +1    -0    generators/poppler/synctex/patches/series
M  +10   -1    generators/poppler/synctex/synctex_parser_utils.c

Comment 35 Albert Astals Cid 2011-10-25 20:10:59 UTC
It seems it ended up in 4.7.2