Bug 267350

I just noticed the same issue. I had stored some filled out forms on an encrypted drive. I ran into a bug where the fields I entered didn't weren't being displayed after being saved (not even an empty field). I figured the file had been corrupted so I copied the original blank form over the filled out one. When I opened it all the information I had entered into the form was there despite the file having been overwritten. After looking around I found it had been written to .kde/share/apps/okular/docdata - on an unencrypted drive. This was quite startling to me and not what I expected. 

I can understand if there are limitations to the PDF format that prevent you from storing the data in the PDF file itself, however you should at least inform the user of where the data is being stored before writing it. Preferably, it should be stored in the same directory as the PDF as well.

Another limitation of doing it this way is that it appears impossible to have multiple copies of the same form filled out differently, even if saved in different directories. For example, I filled out my tax forms, and then created a new directory with the copied blank forms to do my girlfriend's taxes. However, when I opened them they had my value stored in them. 

The workaround was to rename the forms and then edit them, but it would match user expectations better if each copy of the form had it's own set of values.

Finally, I do think the priority on this bug should be higher as it relates to user privacy/security.

Agree with #2. I know the devs are aware of this because there are other issues regarding the opening files and having the form remain being filled out (intentional feature). However, unsure if they are aware of the security implications of this. Developers have any comment?

I ran into this problem too recently. In one department at my workplace, I set up a computer where employees can read PDFs and fill out PDF forms. There is one particular form that each of them has to fill out every two weeks. I discovered that I could avoid the problem of each person seeing (and having to delete) the details entered by the previous person by renaming the pdf file to a random temporary filename before opening it. Then I realized that by serving the pdf file from a local webserver, and having them open the pdf from a link in firefox, I would get the random temporary filename for free without having to script it. It is certainly a kludge, but it seems pretty usable for now :)

Besides form data also annotations were served in that extra docdata xml file.

For annotations this has been fixed recently: https://bugs.kde.org/show_bug.cgi?id=151614

I haven't had time yet to read the whole lengthy discussion there or to install the new version to see whether /how it affects form data.

Have just been bitten by this as well. The form I returned did not contain any form data.

A user should reasonably expect that form data is saved with the form, as this is what other pdf viewers do, and this is what the "File->Save As" option suggests it does. For a long form that has that has taken the user days to fill in, this is definitely an unexpected and unwelcome bug.

popper 0.22.1
okular 0.16.0 on kde 4.10

*** This bug has been confirmed by popular vote. ***

Marcus your problem has nothing to do with this bug, Warren is complaining about the data leakage into his own home folder.

What you are facing has nothing to do with this. So please open a separate bug about it.

To quote my original description:

Expected Results:
The user should be able to "Save as..." to a new file and just have a copy of the PDF with the filled in form data.

I believe this is exactly what Marcus was asking for.

Then you used a wrong subject and i decided to ignore this bug, "Save as..." should and does work here, please, i'm attaching a filled in (just the first two lines) with okular of the file you attached that opens fine in Adobe Reader and shows the data

Created attachment 78620 [details]
Filled in file

With "Hola" and "Pepe" in the first two lines

FWIW that filled in file was created with poppler 0.23 and can't be opened with 0.22 or older :-/ But that's a bug of those old versions, you can open it with Adobe Reader to see it working

It's the same bug, just a different title. Instead of saving the form data to the PDF, it gets saved externally.

Happy to hear it's solved in a newer version of poppler. 0.23 is the development branch of what will become 0.24?

Oh no, 0.22 works too, i just created it with 0.23 because it was what i had installed, let me attach one created with 0.22

Created attachment 78638 [details]
Filled file with poppler 0.22

The must be something different in our environments then. I'm using opensuse 12.3, okular 0.16.0 on KDE 4.10.0. Is there a compile-time option to enable this in poppler or okular?

I downloaded your second attachment and I can see the form data you have entered. I add my own form data and save, and when I close okular and reopen the same file, the text is there. But my text is saved externally. If I do the following, the form data reverts to your text

> mv morsa.pdf morsa1.pdf
> okular morsa1.pdf

and the form data I had saved is only visible in .kde4/share/apps/okular/docdata/107650.morsa.pdf.xml

Hello, Archlinux, kde 4.10.2, okular 0.16.2, I confirm I can open morsa.pdf and see the filled in fields(!), file elefante.pdf fails to even open. 

@Marcus: 
I confirm mv file and open keeps the filled in data. (Note difference between Save as.., and Save a copy)

Backends here: 

$ pacman -Qs poppler
local/poppler 0.22.2-1
    PDF rendering library based on xpdf 3.0
$ pacman -Qs ghostscript
local/ghostscript 9.07-1
    An interpreter for the PostScript language

ghostscript is mentioned in Okular as backend. 

It would be perfect if forms started to work! :)

Created attachment 78649 [details]
pdf with notes and drawing

Btw, if it helps, taking notes (drawing, stamps, highlighting etc) saves properly. Please verify painting2.pdf
Maybe the forms could be done in same way.

Marcus i can't help you here, i've proven it works as it has worked for a long time to be honest i should close this bug since it's basically a weird worded bug and people is just commenting on it with unrelated comments, if it doesn't work for you, you're doing something wrong or opensuse compiled something wrong, but not sure how can i help you debug it.

It seems that when I fill out a form with Okular (with poppler 0.22), it opens in Okular when I rename it or move it to another computer. But no matter what, it doesn't open in Adobe Reader.

(In reply to comment #20)
> It seems that when I fill out a form with Okular (with poppler 0.22), it
> opens in Okular when I rename it or move it to another computer. But no
> matter what, it doesn't open in Adobe Reader.

Please use "File -> Save as..." to save filled data with your PDF file.

I used save as, but I think it only happens with some PDF files. The tax form works perfectly, but I am attaching a file for which the filled out forms only appear in Okular, but not Adobe Reader.

Created attachment 80639 [details]
File for which forms only appear in Okular, not Adobe Reader

FoxIt Reader (under wine) shows filled forms.

It can be a bug in Adobe Reader (tested version XI (11.0.03), also under wine).

I am attaching the same form filled with Adobe.

Created attachment 80647 [details]
Same form filled out in Adobe

This form opens in both Okular and Adobe.

Created attachment 80666 [details]
XFA-warning

(In reply to comment #23)
> Created attachment 80639 [details]
> File for which forms only appear in Okular, not Adobe Reader
Yes, it's known issue. Basically that file uses a newer format to store form data, which is currently unsupported by poppler (and probably won't be for a long time, because it's very complex).
However, Okular from 4.10 with poppler 0.22 should warn you (see attached screen), doesn't it?

No I don't get that warning (using KDE 4.10.3), but I found a workaround, which is to print the file to a PDF, which opens with Adobe. And now I'm going to do a "sudo apt-get purge acroread acroread-bin". (When I installed it 2 days ago, it created a file located in / called C:\nppdf32Log\debuglog.txt)

This is obnoxious. My partner sent me a file with forms filled. I changed them. There's NO WAY to keep having editable forms and send her back the file with the changes!

I tried "save as" and still get the old data there. I had to do it as a print, which means she then manually updates in her program.

This situation is horrible.

(In reply to comment #29)
> This is obnoxious. My partner sent me a file with forms filled. I changed
> them. There's NO WAY to keep having editable forms and send her back the
> file with the changes!
> 
> I tried "save as" and still get the old data there. I had to do it as a
> print, which means she then manually updates in her program.
> 
> This situation is horrible.

This is not the way to report bugs. Please give developers some information about your system (name and version), version of Okular and version of Poppler libraries (can be determined with "pdftops -v" command in console or using your favorite package manager).

BTW, just works here (even for XFA documents if Foxit reader used).

(In reply to comment #30)
> (In reply to comment #29)
> > This is obnoxious. My partner sent me a file with forms filled. I changed
> > them. There's NO WAY to keep having editable forms and send her back the
> > file with the changes!
> > 
> > I tried "save as" and still get the old data there. I had to do it as a
> > print, which means she then manually updates in her program.
> > 
> > This situation is horrible.
> 
> This is not the way to report bugs. Please give developers some information
> about your system (name and version), version of Okular and version of
> Poppler libraries (can be determined with "pdftops -v" command in console or
> using your favorite package manager).
> 
> BTW, just works here (even for XFA documents if Foxit reader used).

Sorry, I just assumed this was the same as the original bug, i.e. an intentional and flawed design. I didn't think this was behaving differently from intended, I thought the issue was simply that the intention was a bad one.

If you intend it to actually work reasonably (which I'd hope), and it is something about my system, well: I'm on KXStudio, which is a derivative of Ubuntu. I'm using standard KDE 4.11.2 and Okular 0.17.2 and pdftops 0.18.4.

To clarify: if I open and change the forms myself, they persist on my machine. If I send the form as an e-mail to someone else, they lose my form changes. If I "save as" on my own machine, the new file reverts to the old form data and loses my changes even on my machine. Note that this is a case where the file I got from someone else started with some form data already, entered in some program other than Okular, and I was changing it.

(In reply to comment #31)
> (In reply to comment #30)
> > (In reply to comment #29)
> > > This is obnoxious. My partner sent me a file with forms filled. I changed
> > > them. There's NO WAY to keep having editable forms and send her back the
> > > file with the changes!
> > > 
> > > I tried "save as" and still get the old data there. I had to do it as a
> > > print, which means she then manually updates in her program.
> > > 
> > > This situation is horrible.
> > 
> > This is not the way to report bugs. Please give developers some information
> > about your system (name and version), version of Okular and version of
> > Poppler libraries (can be determined with "pdftops -v" command in console or
> > using your favorite package manager).
> > 
> > BTW, just works here (even for XFA documents if Foxit reader used).
> 
> Sorry, I just assumed this was the same as the original bug, i.e. an
> intentional and flawed design. I didn't think this was behaving differently
> from intended, I thought the issue was simply that the intention was a bad
> one.
> 
> If you intend it to actually work reasonably (which I'd hope), and it is
> something about my system, well: I'm on KXStudio, which is a derivative of
> Ubuntu. I'm using standard KDE 4.11.2 and Okular 0.17.2 and pdftops 0.18.4.
> 
> To clarify: if I open and change the forms myself, they persist on my
> machine. If I send the form as an e-mail to someone else, they lose my form
> changes. If I "save as" on my own machine, the new file reverts to the old
> form data and loses my changes even on my machine. Note that this is a case
> where the file I got from someone else started with some form data already,
> entered in some program other than Okular, and I was changing it.

It seems that poppler libraries in your system are too old (Okular itself is up-to-date). You need to install at least poppler-0.20 (or better 0.22) to have reliable forms and annotation editor which can save the data in PDF. Try to find some PPA with newer version (Debian Sid has 0.18.4, Ubuntu 13.04 has 0.20.5 and Ubuntu 13.10 has 0.24.1). Sorry.

Please take into account that some other applications (e.g. Inkscape) can be dependent on the old version of poppler, so the safest way is to build and install libraries on your own (it is not hard at all, so they do not break anything).

> It seems that poppler libraries in your system are too old (Okular itself is
> up-to-date). You need to install at least poppler-0.20 (or better 0.22) to
> have reliable forms and annotation editor which can save the data in PDF.
> Try to find some PPA with newer version (Debian Sid has 0.18.4, Ubuntu 13.04
> has 0.20.5 and Ubuntu 13.10 has 0.24.1). Sorry.
> 
> Please take into account that some other applications (e.g. Inkscape) can be
> dependent on the old version of poppler, so the safest way is to build and
> install libraries on your own (it is not hard at all, so they do not break
> anything).

Oh how strange, thank you. I will figure this out! Sorry to have made assumptions about things…

I've used "save a copy" accidentally, not realising that this was fundamentally different to "save as". In a lot of programs these both create a new file, but the former leaves you editing the original, while the latter allows to keep editing the newly created file.

Anyway, I've now realised that there is potentially a lot of private data (e.g. credit card information) stored in plain text on my computer. It seems that some of the information in ~/.kde/share/apps/okular/docdata/ is about the last-used zoom/position of documents? How can I find which of these documents contains form data?

*** Bug 343852 has been marked as a duplicate of this bug. ***

Real problem is that okular violates pdf specification: section "8.6.6 Forms Data Format" of PDF reference: http://partners.adobe.com/public/developer/en/pdf/PDFReference16.pdf

The standard way to store forms is to write the form data into the special section in the same pdf file.

Current way is prone to data loss and confusion, because users will send out the pdf assuming that form is there, when it isn't. Currently form data is implicitly attached to the current pdf file name, which is counter-intuitive.

I would put the high priority on this, because the major user visible function malfunctions due to misimplementation.

*** Bug 357741 has been marked as a duplicate of this bug. ***

I also stumbled about the fact that it seems to be impossible to have multiple instances of the same filled in form under the same file name and got the impression that "Save as..." does not work. After some experimenting I found out the problem with "Save as...":

There's two places where form data is stored: .kde/* and the pdf-file itself. Data is always saved to .kde/* (attached to filename?). If you use "Save as" it's also stored to the file. The problem is that this is not visible to the user and that .kde/* seems to have precedence. If you want to fill the same form twice you can easily run into problems:

Case a: Since form data is visible after closing, reopening, or moving the file, the user assumes that it's stored inside of the pdf. To fill a form a second time he copies the filled in form into a new location and changes the necessary data. Everything seems to work, but if he reopens the original file all it's data is changed to what he entered in the new place.

Case b: The user is aware that data is stored in the file only if he used "Save as". After filling the form the first time he uses "Save as". To fill the form a second time he copies the filled in form into a new location and changes the necessary data and uses "Save as" again. Now different data is saved in both versions of the file. But if he opens version 1 again, he sees the changes he did in version 2. The reason for this is that by changing version 2, the data in .kde/* is changed and this seems to have precedence when viewing version 1. If you delete th corresponding file in .kde/* you see what's stored in the file.

In both cases the user seems to have lost data. In a) this is true, in b) his data is just not visible.

Since I stumbled about this again: The following minor changes would increase usability of forms a lot since they make the behaviour of Okular more transparent.

a) Add a menu entry "save form data" that makes Okular store the data in the file itself.
b) When the user tries to close a filled in form without explicitly saving it: Warn the user that the form data is not stores in the file (but only cached somewhere else) and ask if she/he want's to save now.
c) Always give precedence to data saved in the file itself.

(In reply to Carsten Gräser from comment #40)
> Since I stumbled about this again: The following minor changes would
> increase usability of forms a lot since they make the behaviour of Okular
> more transparent.
> 
> a) Add a menu entry "save form data" that makes Okular store the data in the
> file itself.
> b) When the user tries to close a filled in form without explicitly saving
> it: Warn the user that the form data is not stores in the file (but only
> cached somewhere else) and ask if she/he want's to save now.
> c) Always give precedence to data saved in the file itself.

Carsten, the problem is that there is no code that saves to a file. There should be two choices:
* Saving to a pdf file itself
* Saving to a separate pdf file (according to the pdf spec)
But somebody for some unknown reason implemented writing it in the proprietary format into ~/.kde/share/apps/okular/docdata/ that makes okular unusable for filling forms. Nobody needs the function to save forms under ~/.kde

Yuri, this is not correct. Okular can in principle save the data to the pdf. But the interplay of this with saving to and loading from .kde/... is not very intuitive.

(In reply to Carsten Gräser from comment #42)
> Yuri, this is not correct. Okular can in principle save the data to the pdf.
> But the interplay of this with saving to and loading from .kde/... is not
> very intuitive.

What does it mean "can in principle"? Is the code saving it to the pdf there or not there? If it is there, can you send the link (line numbers)?

(In reply to Yuri from comment #43)
> (In reply to Carsten Gräser from comment #42)
> > Yuri, this is not correct. Okular can in principle save the data to the pdf.
> > But the interplay of this with saving to and loading from .kde/... is not
> > very intuitive.
> 
> What does it mean "can in principle"? Is the code saving it to the pdf there
> or not there? If it is there, can you send the link (line numbers)?
I gave a very detailed explanation of "in principle". just try out as described above.

(In reply to Carsten Gräser from comment #44)
> I gave a very detailed explanation of "in principle". just try out as
> described above.

I don't know what are you talking about. You don't make yourself clear.

@Yuri, Carsten was saying that the current Okular behavior *does* save data directly to the PDF *if* you use "save as" but otherwise not. The problem remaining is that it also saves to a hidden directory as well, and then conflicts and security issues can arise from that.

Ah, I see. I didn't realize the pdf form-saving code was even there. This makes the situation much better than I thought.

Thanks for clarifying this!

Somebody should just remove the code writing under ~/.kde4/share/apps/okular/

Today I received a PDF with forms filled in, that I had to forward as a normal PDF (using "save as" or print to PDF file). Some fields worked as expected. But some were missing in the resulting PDF.

After quite some time I tried editing every single form field, ending up with the same values as before, and then printing to to PDF. Then everything working, i.e. all form fields were saved to the PDF file.

I use Okular 0.25,  Platform 4.14.23

*** Bug 372488 has been marked as a duplicate of this bug. ***

Is there any use case that justifies storing form data in ~/.kde/share/apps/okular/docdata/ given that the same data can be stored in the pdf itself? 
It seems to me that there isn't.

(In reply to lutz.wrage from comment #50)
> Is there any use case that justifies storing form data in
> ~/.kde/share/apps/okular/docdata/ given that the same data can be stored in
> the pdf itself? 
> It seems to me that there isn't.

While I consider the current behavior undesirable, it does have its advantages.  In fact, it's the reason I decided to use okular for my taxes.  Here's the use case:

1. Generate some forms automatically (e.g., opentaxsolver computes my taxes and fills in government  forms).
2. Resulting forms require some manual tweaks (e.g., check boxes, fill in additional fields).
3. Discover forms need to be regenerated to correct a mistake.  Modify inputs and return to 1.

In this scenario, the work in 2 is lost if the results have been stored in the pdf, but is retained if the values are stored elsewhere, as okular currently does.  Even if the pdf in 2 is saved under a different name, so that the results are not literally lost, one must manually identify the changed information and reenter it.

There is another scenario in which the recreation of the information is less desirable.  If some of the manually entered information in 2 depends on the values from 1, e.g., you manually enter line 55 as a copy of line 32 but the automatically generated line 32 changes, then the previous manual data may be invalid.

I don't want my previous response to be taken as a vote for the status quo.  The behavior I would expect is:

1. If I don't hit save my work disappears.(*)  The current application does not have a save function (as distinct from save as), and I'm pretty sure that if I fill in a form, exit, and then open the form my old values will still be there.  Worse, if someone else using the same account opens the form, they will see my info.

2. If I do save (not just save as) my work will be saved with the file.

In this case I might not expect, but would be pleased if 
3. there were an option to save the form data to a separate file and restore it from a separate file.  I'd guess such a facility is consistent with the 1600 page XFA spec, though I can't say I know where :)

Because the current behavior violates these expectations, it is a security risk.   Someone's personal information may be exposed in ways unanticipated, and operations that usually assure security, like not saving a file or deleting it, will not work.  And operations that are expected to reveal info, like copying/mailing a pdf or operating on it with a different program, may instead conceal/disappear the information.

(*) Some usability experts argue that "work disappears if I don't save" is not the expectation of the lay user, and that our current model of "you must save to keep your work" is aggravating and unintuitive to them.  That may well be correct.  But unless the surrounding programs all start behaving this way, this behavior is undesirable.  An application that may be dealing with sensitive private information is not the place to pioneer new interface models.

(In reply to Ross Boylan from comment #52)
> I don't want my previous response to be taken as a vote for the status quo. 
> The behavior I would expect is:

Saving form data into the file is the behavior prescribed by pdf specification.
Leaving form files in any other place has so many disadvantages that it doesn't even make sense to list them.


It's amazing this isn't fixed for 6.5 years now.

Please, stop complaining, complaining doesn't do anyone any good, we know it needs improvement and we're working in a fix.

If you want to get it fixed earlier, either volunteer to help with the fix or hire someone to fix it faster.

> we know it needs improvement and we're working in a fix.

To be fair to the other commenters, we hadn't heard anything here for a few years, so I presumed that the devs were *not* working on a fix. Sometimes KDE bugs tend to drift around for a few years before being abruptly closed as "won't fix", so I can understand why people continue to complain.

Also, I was also unaware that you were even a dev until I mouse-overed your name and saw the @kde.org email address just then. I guess that's a fault of this website though; a visible dev tag similar to GitHub would be useful.

Definitely a good idea. Technically I'm a KDE developer too even though I don't have or use a @kde.org email address. Can you file a bug to that effect on https://bugs.kde.org/enter_bug.cgi?product=bugs.kde.org?

I'm not sure that highlighting the fact that someone is a KDE devleoper here is a good idea.
If I comment in a bug of a component which I never touched, I'm not sure why that comment should have some sort of special marking.

That said, we are off-topic here; the place to discuss this is the kde-community@ mailing list.

For people that have an idea how to compile and test stuff, please test https://phabricator.kde.org/D8642

This won't be anymore the case starting with the Okular that will be part of KDE Applications 17.12 (aka okular 1.3.0)

Hi,

OP here. I wanted to reach out and thank you for fixing this issue much more comprehensively than I imagined when I originally filled the issue.


Thanks,
wt