Bug 413819 - highlight annotation not carried over to pdfjs
Summary: highlight annotation not carried over to pdfjs
Status: RESOLVED UPSTREAM
Alias: None
Product: okular
Classification: Applications
Component: PDF backend (show other bugs)
Version: 1.8.2
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Okular developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-04 20:20 UTC by ghavamikia
Modified: 2019-11-10 18:54 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
1 (219.23 KB, image/png)
2019-11-06 19:28 UTC, ghavamikia
Details
1 (225.21 KB, image/png)
2019-11-06 19:29 UTC, ghavamikia
Details
2 (352.75 KB, image/png)
2019-11-06 19:31 UTC, ghavamikia
Details
3 (271.43 KB, image/png)
2019-11-06 19:32 UTC, ghavamikia
Details
4 (219.23 KB, image/png)
2019-11-06 19:33 UTC, ghavamikia
Details
pdf (93.71 KB, application/pdf)
2019-11-06 19:34 UTC, ghavamikia
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ghavamikia 2019-11-04 20:20:37 UTC
SUMMARY
Saving a odf that has been highlighted in Okular 1.8.2 on linux and windows, with the yellow highliter, and corresponding note (on double clicking the highighted text) filled in and saved; 
when this file is opened in pdf.js on (in my case) firefox browser, the highlight colour is gone, 
but the corresponding note is retrievable and present. 

Opening the same okular annotated document in both foxit and adobe, preserve both the highlight and associated note as expected.(as a comparison)

Further comparison, highlight annotation and associated note made in either foxit and/or adobe and then opening the file in pdfjs veiwer, preserve both the highlight and associated note as expected,  made in those respective pdf viewers.

Here's the most interesting part... once the above action (highliting and note) is made in okular, as descirbed above, saved and opened in pdfjs, not showing highlight in pdfjs (but note preserved), and then loaded in foxit, to make a new completely separate highlight and note, [nothing to do with the okular annotation] and then reopened in pdfjs, the okular highlight is now magically visible. 

Something in the way foxit saved the annotation has modified the anotation from okular... 

I noted the yellow boxes for the associated note in okular, do not have the word "highlight" in their top corner; which i initially thought was a cosmetic difference but perhaps is related to the was popller and okular handle and save these. 

This is of some importance  to me, not least because it would be nice to have unified behavious across the board, but also I do use annotation software online and the poppler annotation would idealy be carried over for the purposes of continuity. 

Many thanks for your work on this, okular is fantastic. If I had a further comment, it would be around maybe adding a line from higlight to note anotation, like in the proprietary software, useful when many notes opened, and perhaps a smaller yellow note size (in terms of screen realestate), for similar reasons.
Comment 1 Tobias Deiminger 2019-11-05 22:02:26 UTC
From your description I'm quite sure this is the same as already explained in bug 410642 comment 9.

You may want to attach document snapshots here from before and after the "magic" repair operation by Foxit. I'd assume Foxit added an AP entry, which is kind of pregenerated graphics representing the note, unburdening PDF.js from generating it at their side.
Comment 2 ghavamikia 2019-11-06 19:28:54 UTC
Created attachment 123758 [details]
1
Comment 3 ghavamikia 2019-11-06 19:29:24 UTC
Created attachment 123759 [details]
1
Comment 4 ghavamikia 2019-11-06 19:31:47 UTC
Created attachment 123760 [details]
2
Comment 5 ghavamikia 2019-11-06 19:32:40 UTC
Created attachment 123761 [details]
3
Comment 6 ghavamikia 2019-11-06 19:33:53 UTC
Created attachment 123762 [details]
4
Comment 7 ghavamikia 2019-11-06 19:34:35 UTC
Created attachment 123763 [details]
pdf
Comment 8 ghavamikia 2019-11-06 19:41:04 UTC
pls ignore 1st picture, (it's same as 4, I don't know how to remove it)
so i took exactly the same steps as before... and got a completely different this, I couldn't reproduce the same error, but got an even worse one!
opened and saved annotation in okular (see screenshot)
opened same in firefox.. as before cant see the colour but can see the note on hovering (cursor missing on screenshot) (see screenshot)
opened same doc in foxit and made new annotation and saved (see screenshot)
opened this is firefox... now there nothing at all....
opened same in okular... now okular highlight is missing! (see screenshot)
for what it's worth it opens the same in adobe, okular note missing..
so I've attached the pdf...
I worry if I do this again in 24 hours I'll get yet another result, I woish I had sent you the pdf and screenshots from the original bug report now...
Comment 9 Tobias Deiminger 2019-11-07 21:01:11 UTC
(In reply to ghavamikia from comment #8)
> opened same doc in foxit and made new annotation and saved (see screenshot)
> opened this is firefox... now there nothing at all....
> opened same in okular... now okular highlight is missing! (see screenshot)
> for what it's worth it opens the same in adobe, okular note missing..
> so I've attached the pdf...
Uncompressing object streams reveals the following:
- 1st page annotation dictionary (it's something like a TOC for annotations) contains a "null object" where there ought be an indirect reference to your "okular note"
- and, in the whole document there's no occurence of an annotation object with /Contents (okular note)

It quite clearly indicates Foxit has not only made your "okular note" invisible, but it actually deleted it completely.

The first issue (newly created Okular highlights not visible in PDF.js) is very likely because poppler doesn't write AP, and PDF.js fails to display something without AP. It's a known issue in both projects.

> I worry if I do this again in 24 hours I'll get yet another result, I woish
> I had sent you the pdf and screenshots from the original bug report now...

If you're able to reproduce the "deleted annotation" behavior, you should file a bug report to Foxit.

I'd suggest to close the Okular bug here.
- Okular can't do anything about Foxit deleting annotations
- Okular can't do anything about AP. That's up to poppler. I think we're aware over there, shall we open a dedicated poppler issue for it anyway?
Comment 10 ghavamikia 2019-11-07 21:16:01 UTC
Thank you for your time it's much appreciated.
I agree with your plans, except the conclusion about foxit deleting things, firstly because it persists in foxit, and secondly because it opens in mupdf which I downloaded yesterday for a try....
anyway I do appreciate your input since I have no idea about programming and having used okular over the last 8 years I appreciate how much it's developed, I hope it continues!
Comment 11 Tobias Deiminger 2019-11-08 02:30:28 UTC
(In reply to ghavamikia from comment #10)
> it opens in mupdf

You're right, I can confirm mupdf shows the highlight when opening your unmodified attachment 123763 [details]. I looked a bit deeper what's going on with object 74 0 (that's the ID of the annotation object "okular note"):

- mutool:
  page 1 annots = [ 74 0 R 78 0 R 80 0 R 81 0 R 82 0 R 87 0 R ]
  mutool show l3project.pdf 74
74 0 obj << /Contents <FEFF006F006B0075006C006100720020006E006F00740065> /Type /Annot /Subtype /Highlight ... >> endobj

- poppler (via small debug program):
  document->getPage(1)->getAnnotsObject().print();
[74 0 R 78 0 R 80 0 R 81 0 R 82 0 R 87 0 R]
  document->getXRef()->fetch(74, 0).print();
null

- qpdf:
  qpdf --show-object=74 l3project.pdf
null

mutool fetches object 74 0 and returns annotation content.
poppler has the object referenced in annotation array, but can't fetch it, returns null object.
qpdf also can't fetch it, returns null object.

That's actually interesting and might be something we want to fix or workaround in poppler. I'll investigate further.
Comment 12 ghavamikia 2019-11-08 15:11:01 UTC
Thank you. 
if I find similar incongruencies in the future I will post them anywhere in particular?
Comment 13 Tobias Deiminger 2019-11-08 15:43:36 UTC
(In reply to ghavamikia from comment #12)
> Thank you. 
> if I find similar incongruencies in the future I will post them anywhere in
> particular?

PDF parsing and rendering issues can directly go to poppler:
https://gitlab.freedesktop.org/poppler/poppler/issues
But don't worry, reporting to Okular in first place is absolutely fine. Both projects have a close relation and people behind are partially the same.

Anyway thanks for your reports :)
Comment 14 Albert Astals Cid 2019-11-10 01:40:29 UTC
i'm closing this as upstream since nothing okular does here is wrong, if anything at all is wrong it'd be in poppler as Tobias mentioned.

Tobias i take you're on top of this?
Comment 15 Tobias Deiminger 2019-11-10 18:54:26 UTC
(In reply to Albert Astals Cid from comment #14)
> i'm closing this as upstream since nothing okular does here is wrong, if
> anything at all is wrong it'd be in poppler as Tobias mentioned.
> 
> Tobias i take you're on top of this?

Yep, created https://gitlab.freedesktop.org/poppler/poppler/issues/837 and https://gitlab.freedesktop.org/poppler/poppler/issues/839 to keep track.

@ghavamikia: If you can reproduce the steps that lead to "okular note" note even shown in Okular, it would be nice to give more details and snapshots of the PDF file on its way breaking bad. attachment 123763 [details] is in an inconsistent state wrt. generation numbers, whatever software caused it should get its own bug report.