| Summary: | Custom/image stamp annotations are not saved into the PDF file in a way that can be printed or that other readers can see | ||
|---|---|---|---|
| Product: | [Applications] okular | Reporter: | Nate Graham <nate> |
| Component: | general | Assignee: | Okular developers <okular-devel> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | 4aa7f31e, aacid, alighraibeh87, amg1127, burton+bugs.kde.org, dav1dblunk3tt, Denny.Seniazi, dharman, dimitrios.tanis, haxtibal, jens-bugs.kde.org, lopezibanez, oliver.sander, pirminbraun16+bugs.kde.org, putr4.s, simonandric5, valentin |
| Priority: | NOR | Keywords: | usability |
| Version: | 1.0.0 | ||
| Target Milestone: | --- | ||
| Platform: | Ubuntu | ||
| OS: | Linux | ||
| See Also: |
https://bugs.kde.org/show_bug.cgi?id=354894 https://bugs.kde.org/show_bug.cgi?id=415024 https://bugs.kde.org/show_bug.cgi?id=451609 |
||
| Latest Commit: | Version Fixed In: | ||
| Sentry Crash Report: | |||
|
Description
Nate Graham
2017-08-17 23:00:01 UTC
Is this caused 100% by https://bugs.freedesktop.org/show_bug.cgi?id=23108? Or are there other things that need to be done if/once poppler adds appearance streams for rubber stamps? Albert, will the printing case be resolved with https://bugs.freedesktop.org/show_bug.cgi?id=102513? (in combination with https://phabricator.kde.org/D7688, of course) No. Your problem really smells like a problem with annotation handling, and printing is just a symptom of something deeper. You may be able to partially circumvent your printing problem now that we have https://git.reviewboard.kde.org/r/130218/ With this patch, if you select rasterized printing, then Okular will render (almost) right into a QPrinter object, rather than converting the file to postscript and rendering that. I'm fairly confident that this will make your stamps appear on the printout. *** Bug 377567 has been marked as a duplicate of this bug. *** Just had a look into it, this is what I found. Dependent on users choice, Okular sets the annotation dicts /Name entry to standard names ("Approved", "Confidential", ... see PDF 32000-1:2008, chapter 12.5.6.12 Rubber Stamp Annotations, Table 181), but also to non standard names ("okular", "kde", <path_to_your_custom_image>). Conforming readers implement a representation for the standard names (may look a bit different as in Okular, but at least something reasonable is shown). But of course non standard names like "okular", "kde", <some_local_path> are not understood by other readers. A portable way to have such custom stamps would be to add an /AP entry with the full icon representation into the PDF. Unfortunately, Poppler can't generate appearance streams for stamps yet. Note that Poppler #23108 is a bit different, it is about hardcoded default appearance streams, not about generating ones for your custom image.
There's another special thing about stamps. Contrary to most other annotation types, stamps are not rendered by Poppler, but by Okular itself. Okular first looks for /Name in ui/data/stamps.svg (contains "approved", "confidential", "departmental"...). If there's no match, it uses KIconLoader lookup an icon. Not rendering with Poppler probably introduces special handling when printing, haven't checked that yet.
I guess an ideal solution for this bug will require aid from Poppler side (maybe a new API to store raster and vector graphics as appearance stream into the PDF, and some predefined appearance streams for the standard names so that stamps without /AP can be rendered).
Should we split this into a set of separate smaller issues, then? E.g., issues with stamps with standard appearances seem to be quite independent from custom stamps. How about this breakdown? Boils down to the same root causes a bit... Okular #1: Stamp Annotations are not printable (I can confirm this, for both standard and non-standard icon names) Root cause: PDF print requests are forwarded from Okular to the Poppler generator (see PDFGenerator::print). But Poppler can't render stamps for two reasons: -Poppler doesn't implement default appearance streams for standard icon names, so it can't render stamps like "Draft" and "Approved". See [1], "I leave the bug open becuase there are still annotations missing default appearance streams (Free Text annots, rubber stamp, ...)". -Poppler can't generate/render/save an appearance streams for your custom *.svg, *.png, whatever image yet. Displaying them in Okular on screen only works because Okular renders stamps on its own. Okular #2: Custom stamp annotations (i.e. non standard icon name) are not visible in other readers Root cause: If stamp is created by Okular+Poppler, no appearance stream is generated/embedded into the PDF. Other readers should implement a default appearance stream for standard icon names, but for non-standard icon names like "kde", "okular", <path_to_local_file> they can't know what to display. Maybe of interest is this pdf.js (firefox) issue [2]: "Most annotations, even unsupported ones, render just fine in PDF.js because of their appearance stream. However, there exist PDF files with annotations that do not have an appearance stream (even though this is a deprecated practice). In the latter case, PDF.js displays nothing.". There have been some thoughts about handling of appearance streams in Poppler [3], but I don't know the current state and desire there. [1] https://bugs.freedesktop.org/show_bug.cgi?id=23108#c21 [2] https://github.com/mozilla/pdf.js/issues/6810 [3] https://lists.freedesktop.org/archives/poppler/2009-June/004789.html Is it too late to turn this into yet another gsoc project? (In reply to Oliver Sander from comment #9) > Is it too late to turn this into yet another gsoc project? No, the student application deadline is March 27 https://developers.google.com/open-source/gsoc/timeline Can somebody please post this on the gsoc project list for me? Thanks! Project: Improve custom stamp annotation handling Brief explanation: Okular does display stamp annotations, but the support is somewhat incomplete. This particularly shows when trying to use stamp annotations with a custom image. For example, such annotations can be added in Okular, but they cannot be saved to the pdf file in a way that any other pdf viewer can read. Also, they will not appear on print-outs. The underlying reason for this is that Okular renders these stamps itself, rather than relying on the poppler library, which does all other pdf rendering. Goal of this project is therefore to teach poppler how to render stamp annotations, and then make Okular use that new functionality. More details can be found in the bug report [0]. [0] https://bugs.kde.org/show_bug.cgi?id=383651 Expected results: Poppler should render stamp annotations. Annotations should be printable from Okular. Custom stamps inserted via the Okular GUI should be visible in other pdf readers. Knowledge prerequisite: C++, and a bit about the pdf format. You should really add it to the wiki for yourself, *personally* i've like 100 unread emails, half of which are people waiting for me to review their code, so I really don't have time to add it myself. Albert, you are not the only busy person on this planet. Last time I tried I couldn't log in to the wiki page. Now it works all of a sudden. What do I know... Thanks for adding it to the Wiki, Oliver! https://community.kde.org/GSoC/2018/Ideas#Project:_Improve_custom_stamp_annotation_handling *** Bug 394620 has been marked as a duplicate of this bug. *** This bug is related to the issue of default annotation appearance handling. It isn't just "image stamp" handling. There are two bugs here. https://bugs.freedesktop.org/show_bug.cgi?id=23108 https://bugs.freedesktop.org/show_bug.cgi?id=106635 My bug was here but marked as a duplicate of this bug. https://bugs.kde.org/show_bug.cgi?id=394620 I'd like to propose that we create a tracking bug or merge this with a larger bug regarding annotation appearance. What I'm worried about is that if it goes to GSoC or someone else handles it they won't resolve the larger issue of highlight annotations, note annotations, etc. All these are VERY important to resolve now. Right now Okular generates PDFs that aren't usable by other clients and this is dangerous. I agree that this is important and fixing it should be a high priority. We have a number of other bugs relating to this subject: - https://bugs.kde.org/show_bug.cgi?id=275371 - https://bugs.kde.org/show_bug.cgi?id=378186 - https://bugs.kde.org/show_bug.cgi?id=390293 Are any of those more appropriate? There's no real benefit to having a master "annotations don't work in other PDF readers" bug. Better to track the individual issues with individual bugs, IMHO. Thanks Nate. I think one main concern I have is for other users of Okular using annotations. Some of these bugs are rather old and I suspect the issue of highlights and notes not being visible impacts Okular from years ago. If we can't fix these in the next release, I'd like to propose that the annotation functionality be disabled. The reason is that if a user installs Okular, then migrates away from Okular (say they go from Ubuntu to MacOS/Windows) then they lose all their annotations essentially. This is a data loss issue. If the user can't see on the page where his annotations were, in other PDF readers, they're essentially gone. That has been discussed before and rejected, since that would represent a loss of functionality--a great deal of which *does* work in other viewers, and all of which works in Okular itself. Let's focus on fixing the bugs rather than getting into the weeds with a bruising argument about whether the feature is ready for prime time yet. It's clearly not, but turning it off until it is isn't an option. Are you a programmer? Are you able to review any of the poppler patches floating around? I am a programmer. I'm able to review them and will get to them. I'm curious if this is the best solution. If it means just revising the poppler patches and that's the best path for 80% of the functionality then I'm good to go. Also, another solution might just be a warning to the user the the annotations MAY not be viewable in all PDF editors and that we don't control those. Then show them the warning once. My concern is that some of these bugs seem like they won't be readily resolved but a warning might be the best approach here. Data loss can really hit you I know, and I don't necessarily disagree with you. But as a fellow programmer someone who's watched this project for a while and advocated for the same things, let me tell you that disabling the feature or showing a warning is not going to be accepted. It is what it is, sorry. Let's focus on fixing the bugs. Where do you think we should start? Do you have a thought on the poppler default annotation appearance issue? That's probably the greatest bang for the buck. If that fixes 80% of the problems then we should focus a patch there. I've asked for commentary on their bugs. I agree that we should focus on Poppler. If I'm reading https://bugs.freedesktop.org/show_bug.cgi?id=23108#c21 correctly, patches were merged that handled some annotation types, but not all of them. We should probably focus on the remaining annotation types that still need default Poppler appearance streams. I'll probably have to look and see if we can confirm that this is the problem. maybe a test app that generates all the annotation types ... I'm working on a separate project that can export the annotations as JSON from PDF.js and make them machine readable. I might be able to include the colors there so that we could have a unit test or some way to verify that they work. poppler might already have some options for this. Is there any update on this? I agree with Kevin that data loss can be a buzz but when advocating for the bells and whistles of linux and KDE as a pro business environment, sending an annotated pdf that others can't view is even worst (I've just been hit by that). It's been quite a few years since I've coded in C++ and Qt so I can't be of use in coding but I can help with testing if needed. Hello, is there some news about this bug? It is still happening in version 1.6.3. Thanks in advance! (In reply to Valentin Melot from comment #26) > Hello, is there some news about this bug? It is still happening in version > 1.6.3. Thanks in advance! Afaikt there's no news, sorry. @Nate, Kevin: You seem to focus on default appearances. I don't think they would fix this bug, can you explain how? For my understanding we need to extend poppler so that it can generate and save appearance streams into the PDF file in a consistent way: - either generate PDF operations from user image to draw image as paths - or rasterize / convert user image to a sampled PDF image - create AP structure, enter the generated image stream, enter additional resources - update all resource references throughout the document - save new and dirty objects That all has not much to do with default appearances. Even if poppler had hard coded default appearances for stamps, non-Okular readers would not benefit from them unless the appearance is saved into the file, which is currently not possible and which is the toughest part to implement. And, "default" is quite the opposite to "custom" (as in the bug title). Git commit f15e8568a5330b6795b1dd5287a6a1530dc60476 by Simone Gaiarin. Committed on 25/07/2019 at 18:09. Pushed by gaiarin into branch 'master'. General improvements to stamp annotation Summary: Configuration: - Add push button to select custom stamp image - Check if loaded image is usable as stamp or throw error - Keep image proportions in previewer - Move previewer below the combobox to display larger preview Annotation tool: - Keep stamp image proportion in annotation preview (while left mouse button is down) - Adding the annotation with one-click (without holding the left mouse button and dragging) adds the stamp with original proportions Related: bug 370381, bug 383652 FIXED-IN: 1.9.0 Closes T8074 TODO: - [ ] Check if filters in file chooser make sense / propose better alternative - [x] Update doc ( @yurchor will do it after we merge this) Test Plan: >From stamp annotation configuration dialog: - Show a warning regarding limitations of the feature's current implementation - Click push button next to combo box opens a file selector - Selecting a corrupted image file should throw an error - Selecting a good image file shows the preview of the image - Select a horizontal image shows a large clear preview - Select a vertical image file shows a smaller preview without messing up the visual of the config dialog - Input a valid icon name in the combobox and the preview of the icon is shown >From page view, select the stamp annotation with horizontal image file (not squared): - Click and hold. The preview maintains proportions - Single click. The stamp image in the pdf maintains proportions and has the same size of the click and hold preview. - Add an annotation of the Okular custom stamps (internal SVG so treated slightly differently) do not create problems Reviewers: #okular, ngraham Reviewed By: ngraham Subscribers: pino, aacid, yurchor, ngraham, okular-devel Tags: #okular Maniphest Tasks: T8074 Differential Revision: https://phabricator.kde.org/D22064 M +62 -9 ui/annotationwidgets.cpp M +7 -1 ui/annotationwidgets.h M +25 -9 ui/guiutils.cpp M +8 -1 ui/guiutils.h M +1 -1 ui/pagepainter.cpp M +4 -4 ui/pageviewannotator.cpp https://invent.kde.org/kde/okular/commit/f15e8568a5330b6795b1dd5287a6a1530dc60476 *** Bug 416252 has been marked as a duplicate of this bug. *** *** Bug 435904 has been marked as a duplicate of this bug. *** *** Bug 447086 has been marked as a duplicate of this bug. *** I just came across this bug because I wondered why my signature (as a custom stamp) was not printed. I don't actually need custom stamps to be saved in the PDF in a portable way, but they need to be PRINTABLE. Currently, when printing a PDF document (even into another PDF), the custom stamps are missing. Other annotations and marks are kept. Is this a completely different issue or is this an aspect of this bug? If all you need is printing, then enabling 'force rasterization' in the print dialog will probably circumvent the problem for you. This has been fixed since Okular 21.12.0 assuming you run a moderately modern poppler, if you don't (e.g Neon users, you should either convince the Neon developers to stop using such an old poppler or find a new distribution to use). Thanks for the update. What is the minimum poppler version that we need? Poppler >= 21.10 *** Bug 451541 has been marked as a duplicate of this bug. *** "If all you need is printing, then enabling 'force rasterization' in the print dialog will probably circumvent the problem for you." Not working for me (21.08.1, poppler 21.06) @David Albert said it was fixed in 21.12 (with the appropriate Poppler version) . You have 21.08. @Albert Does this mean that Bug 275371, Bug 378186, and Bug 390293 should also be fixed? (In reply to Albert Astals Cid from comment #34) > This has been fixed since Okular 21.12.0 assuming you run a moderately > modern poppler, if you don't (e.g Neon users, you should either convince the > Neon developers to stop using such an old poppler or find a new distribution > to use). Here, with a bare Kubuntu 21.10 installetd I've poppler 21.06.1. I think it isn't a Neon problem. Simply poppler need at newer version. (In reply to dharman from comment #40) > I think it isn't a Neon problem. Simply poppler need at newer version. Neon doesn't want (or can't) provide a new poppler version, so it's a Neon problem. (In reply to Nate Graham from comment #39) > @Albert Does this mean that Bug 275371, Bug 378186, and Bug 390293 should > also be fixed? Both 378186 and 390293 are unrelated to this as far as i can see. Any chance getting this ported to 21.10 / Impish? because it's not showing for me (poppler 21.06.1-1) and it would be very handy... (In reply to davidblunkett from comment #43) > Any chance getting this ported to 21.10 / Impish? because it's not showing > for me (poppler 21.06.1-1) and it would be very handy... You're asking to the wrong people. Quite right - my mistake! |