Bug 140006 - Create standard procedures to test KDEPrint (and all components it relies on) for regressions, and KDE applications' interaction with KDEPrint
Summary: Create standard procedures to test KDEPrint (and all components it relies on)...
Status: RESOLVED UNMAINTAINED
Alias: None
Product: kdeprint
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: unspecified Other
: NOR wishlist
Target Milestone: ---
Assignee: Kurt Pfeifle
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-01-13 05:43 UTC by Kurt Pfeifle
Modified: 2011-05-27 18:25 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Konqui window split into 4 parts; left: connected to localhost:23631 (running an SVN instance of CUPS), right: connected to localhost:631 (running distro-shipped CUPS 1.1.23) (229.02 KB, image/png)
2007-01-14 17:47 UTC, Kurt Pfeifle
Details
KWord file using a single TrueType font (10.54 KB, application/octet-stream)
2007-02-04 04:57 UTC, Kurt Pfeifle
Details
KWord file using a single TrueType font (10.56 KB, application/octet-stream)
2007-02-04 05:00 UTC, Kurt Pfeifle
Details
pdf file (10.54 KB, application/vnd.oasis.opendocument.text)
2007-02-04 05:02 UTC, Kurt Pfeifle
Details
first PDF generated from attachment 19530 (62.32 KB, application/pdf)
2007-02-04 05:08 UTC, Kurt Pfeifle
Details
second PDF generated from attachment 19530 (23.09 KB, application/pdf)
2007-02-04 05:10 UTC, Kurt Pfeifle
Details
first PDF generated from attachment 19530 (22.67 KB, application/pdf)
2007-02-04 06:57 UTC, Kurt Pfeifle
Details
second PDF generated from attachment 19530 (61.82 KB, application/pdf)
2007-02-04 06:58 UTC, Kurt Pfeifle
Details
first PDF generated from attachment 19530 (31.01 KB, application/pdf)
2007-02-04 14:03 UTC, Kurt Pfeifle
Details
second PDF generated from attachment 19530 (54.48 KB, application/pdf)
2007-02-04 14:03 UTC, Kurt Pfeifle
Details
kword-ghostscriptfonts-embedding-testdocument.odt (155.52 KB, application/vnd.oasis.opendocument.text)
2007-02-04 15:04 UTC, Kurt Pfeifle
Details
PDF version of attachment 19549 (284.68 KB, application/pdf)
2007-02-04 15:47 UTC, Kurt Pfeifle
Details
kword-ghostscriptfonts-embedding-testdocument.odt (156.79 KB, application/vnd.oasis.opendocument.text)
2007-02-04 16:50 UTC, Kurt Pfeifle
Details
PDF version of attachment 19549 (284.68 KB, application/pdf)
2007-02-04 18:11 UTC, Kurt Pfeifle
Details
PDF version of attachment 19549 (666.15 KB, application/pdf)
2007-02-04 18:19 UTC, Kurt Pfeifle
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kurt Pfeifle 2007-01-13 05:43:58 UTC
Version:            (using KDE KDE 3.5.5)
Installed from:    00
OS:                I Don't Know

Yesterday I received an e-Mail from a user who asked to keep his name and affiliation confidential. He is involved in preparations for a rather large Windows --> Linux/KDE migration, involving several thousand workstations, most of them in a managed KIOSK environment. They'll not be ready before 12 months from now for the first serious rollout, but they are already testing with KDE 3.5.3. Once they are ready, he hopes to be able to use KDE 3.5.5 or 3.5.6 (should it ever happen). They'll *not* using KDE 4.0, even if it were released this autumn, and it will be at least 3 years before they'd ever consider to use any 4.0.x version. And here is what he said about KDEPrint:

    "Printing is going to be pretty important to us. And KDE's printing 
     system has already proven to be designed with an architecture 
     underneath that will be able to meet most of our current needs. The 
     only thing is that there are a few corners where we find some more 
     polishing is asked for."

    "I understand that you do not intend to change any fundamental 
     things in KDEPrint after 3.5.5 is out. However, be aware that we 
     are able to build our own packages from Subversion, and we follow 
     any improvements that you guys are able to come up with. I want you 
     to understand that at least for _our_ users, improvements to the 
     post-3.5.5 branch will not be in vain, and we will make use of 
     everything you are willing to backport from your 4.0 improvements."

     [....]

    "Another thing, also about printing. What about adding a series of 
     'unit tests' to KDE subversion? I'm not exactly sure if unit tests 
     is the correct term. But it is surely a good idea to have a few 
     KDE and OpenOffice test files, as well as PDFs which can be used to 
     test standard print options. A lot of things could be tested with 
     scripts that send jobs to a virtual printer printing to file. Also, 
     it would be good to test everything against new CUPS versions 'in 
     the making'. Is there a possibility to make a CUPS test server run 
     alongside the standard server, but binding to a different port than 
     631? I saw you mentioning a possibility like this on the CUPS 
     mailing list some time ago."

Don't you too think this is a great suggestion? I do. And that's why I forward this into a bug report. From my past few days of sifting and ploughing through KDEPrint's bugzilla, I noticed that quite a few users are using KDEPrint for more than just occasional printouts, and they are aware of some of its most advanced features. They submit bug reports not just once or twice, but several times a year. These user would probably make good use of some test scripts and standard procedures to test KDEPrint code from Subversion on a regular basis.

I have some ideas in mind what I could contribute to start this off fairly soon. And as soon as a serious switch from PostScript to PDF as the standard spooling and print processing format in CUPS will kick in, this type of heavy testing is called for in any case.

Now.... I'd just hear from at least 3 people who have not yet contributed to KDEPrint (bug reporting activities don't count here) that they'd like to participate, and that they'll test my first script once I'm ready to release it.
Comment 1 Kurt Pfeifle 2007-01-13 05:45:48 UTC
Above, replace "KDE 3.5.5 or 3.5.6 (should it ever happen)" by "KDE 3.5.6 or 3.5.7 (should it ever happen)" 
                                       :-)
Comment 2 Sebastian Sauer 2007-01-13 12:53:24 UTC
and here is the first one who would like to participate at least with testing your first script or maybe even help to improve it :)
Comment 3 Sebastian Sauer 2007-01-14 12:44:43 UTC
Cause it seems the mail CC to 140006@bugs.kde.org didn't come through for whatever reason (btw, it never worked for me too and even CC_MAIL:xxxxxx@bugs.kde.org in a commit does not work here), I copy+paste the mail I got from Kurt here cause I guess it may useful for others to read it too;

--------------------

Well, what I have in mind is not so much a single script, but more a
complete set of scriptlets and utilities (as well as howtos and 
tips+tricks snippets and "best practices" articles).

Since I only "speak" Bash (and that not even completely), it will be
started on that level. Some of the better stuff may later even be
ported to other levels. Some of the ideas may sooner or later be
turned into C++/Qt code to go into KDEPrint proper.

The first one I'll tackle is my idea of a "CUPS Cloning"-script.
Hmmm, maybe it is not cloning so much... it will be grabbing the
current CUPS sources from their SVN, building them and start them 
as a separate instance (in user space) on a different port such as
23631. It will serve to test interoperability between CUPS-stable
and CUPS-SVN and can be used as counterpart to test current KDEPrint
code against. 

People will be able to test a lot of functionality without even
owning or having access to a real physical printer.

A few debugging tools will come with it: the ability to take a copy
of the print file on each level of the CUPS processing + filtering
stages: how it arrives, what it looks like after pstops has done
its work, etc. It will have a more sophisticated PDF-printing
backend than the simplistic KDEPrint print-to-file printer, so
that people can better look at some of the results (n-up printing,
image quaility, etc.)

The idea is to then extract from every bug report that is not just 
a simple mis-configuration a little reproducible test case & script 
that can first be used to help debug the problem, and by later be 
at turned into regression test.

Also, all major KDE applications and developers should get a set 
of tests to perform for their printing needs. I don't know how 
much of this can be automated, but I think a good part of it can
be... (see for example what I did with one bug report I processed 
last night: http://bugs.kde.org/show_bug.cgi?id=125150#c4 plus
attachements).

Take KWord for example: from the bugzilla entries (and my own ex-
perience with it) I can clearly see that there are re-curring 
problems seen by users (font rendering on screen, font represen-
tations on paper, font embedding, custome page sizes for export
to PDF and for printing to paper). These observations could be 
transformed into a set of test cases: "Here's the test file; these
are the fonts you need to have installed on your system; these
are the settings to be used to get optimal PDF export (printout)
and here's a script that you have to run to see the result; and
if it finds a problem, it will report it to you." Or: "To test a
new font, use these 3 template files (KWord, OpenOffice, Scribus);
activate the font for them; make screenshots and export to PDF;
compare the 3 results."

Similar test for KSpread, Krita, Karbon, $whatnot... I guess a
lot of these can make good use of Kross also.

Right now I'm working on the "Setup An SVN CUPS Server on port 
23631" script. And on the "Show Me A Better Matching Print Preview"
one (just a working "proof of concept").

I'll let you know any progress.
Comment 4 Sebastian Sauer 2007-01-14 13:06:00 UTC
So, basicly there are 2 cases;

1. Test KDEPrint itself which is independend of the applications that are using it. This is IM(H)O the most important part since if something here does not work all KDE applications are affected.
2. Concret content an application likes to print. Such tests need to be done for each major application. I would see here a minor priority since I don't really believe that e.g. printing in KWord could break such often. Also KWord 2.0 comes already with testcases that test internals like the text+layout stuff.

So, what I would suggest is, to take one application and implement tests for it to be sure things like layout, fonts, forms and whatever else e.g. a PDF is able to contain got tested. I see there 2 good candidates: KWord and KHTML. Both of them are rich "layout apps" and are able to produce complex enough content to test various cases that could break.
Why it should be quit possible to control both of them (dbus? kross? javascript? or just (k|q)unittest's?) an alternate may to write a small app that does nothing more then to produce some output that goes through KPrinter. The advantage of the later one would be, that such a "testapp" could then test explicit the cases and would be propably smaller + easier to maintain. The advantage to take an already existing app would be, that the app itself then has as result a good unittest for printing plus it may possible to reuse already existing stuff like the KHTML regression-suite or the KWord/OpenDoc unittests.
Comment 5 Kurt Pfeifle 2007-01-14 17:47:55 UTC
Created attachment 19280 [details]
Konqui window split into 4 parts; left: connected to localhost:23631 (running an SVN instance of CUPS), right: connected to localhost:631 (running distro-shipped CUPS 1.1.23)

Here's a screenshot. Konquueror window is split into 4 parts

 * left: connected to localhost:23631 (running an SVN instance of CUPS), 
   --> top: front page of current SVN version of CUPS (r6197)
   --> bottom: the "/printers" location with a few printers

 * right: connected to localhost:631 (running distro-shipped CUPS 1.1.23) 
   --> top: front page again (the distro-provided installation)
   --> bottom: the "/printers" location of this "old" instance
Comment 6 Kurt Pfeifle 2007-01-14 18:23:23 UTC
OK, so now I have my first additional CUPS instance running here.    :-)

I've implemented its creation by adding an "include Makefile.cupsclone" into the main Makefile. So you can compile CUPS as you would normally do.

But you have more options now: "make [tab] [tab]" now shows my additional build targets. You can run "make cups_clone", "make_printers_clone" and "make cups_clone_start". Heh... And it works pretty well once it is up.

make cups_clone copies all required files into a separate subdir-tree in the user's home, and symlinks for the rest which point to the build directory. So every time the build directory is updated, the running clone also has new bleeding edge binaries at its disposal (if the cupsd binary itself got rebuilt, it will of course only run if the old one is stopped or killed and the new one started up). If you wan to de-couple the new instance from the build directory (to safeguard yourself against b0rken updates), just copy the whole clone-tree to a third place, and use the --dereference ("follow symlinks") switch for the cp command.

The more I play with it, the more I'm convinced it could be a really useful tool for everyone who is interested to look a bit deeper into all the un-loved printing stuff (not just limited to KDEPrint).

It surely is not yet release-able for now. 

 1. The Bash code is horrible and uncommented. "Horrible" will probably never
    change, but some comments I can add for sure. 
 2. And it also needs at least a README which outlines the most important steps
    and procedures to follow and the envrionment to set up when someone
    seriously wants to test applications against a CUPS server that runs on a
    non-standard port such as 23631. 
 3. And last, I need to test the hack at least once on a virgin system. You 
    *always* find issues when you "install" your own software for the first
    time on a completely different system.

I cobbled this together in half a day. But of course, the initial "make cupsclone" had a few things missing which I only discovered when trying to run the new instance from its own clone tree. So I copied them over manually, and updated the Makefile accordingly -- but I may have missed stuff.

So this required cleansing and testing will still take some time. I also need to put it into some version control system (at least locally, for me), so I have it "organized" every time I can work on it. (I doubt there would be many people wanting to contribute on this particular thingie, but we'll see).
Comment 7 Kurt Pfeifle 2007-01-30 05:32:19 UTC
OK, a first beginning of the work intended to deal with this bug is now to be seen (and tested) here:

  http://developernew.kde.org/Projects/KDEPrint/Fonts
Comment 8 Kurt Pfeifle 2007-02-04 04:57:19 UTC
Created attachment 19528 [details]
KWord file using a single TrueType font

This KWord file uses only 1 font face, a TrueType font. It is 'Bitstream Vera
Serif' at different font sizes, colors and variants of bold, regular and
italic.
Comment 9 Kurt Pfeifle 2007-02-04 05:00:17 UTC
Created attachment 19529 [details]
KWord file using a single TrueType font

This KWord file uses only 1 font face, a TrueType font. It is 'Bitstream Vera
Serif' at different font sizes, colors and variants of bold, regular and
italic.
Comment 10 Kurt Pfeifle 2007-02-04 05:02:51 UTC
Created attachment 19530 [details]
pdf file

This KWord file uses only 1 font face, a TrueType font. It is 'Bitstream Vera
Serif' at different font sizes, colors and variants of bold, regular and
italic.
Comment 11 Kurt Pfeifle 2007-02-04 05:08:26 UTC
Created attachment 19531 [details]
first PDF generated from attachment 19530 [details]
Comment 12 Kurt Pfeifle 2007-02-04 05:10:43 UTC
Created attachment 19532 [details]
second PDF generated from attachment 19530 [details]
Comment 13 Kurt Pfeifle 2007-02-04 05:41:00 UTC
If www.kdedevelopers.org was up, I'd even blog about the riddle I've posed here.

I *think* I've found a general way to overcome some of the KWord/Qt weaknesses in handling printing and PDF-generating of documents that contain TrueType fonts....

But I need more testing, and I need at least 5 different testers running different versions of Linux, *BSD and/or KDE before I really publish my current solution. (This solution works for me as good as it possibly can, but it also has a few drawbacks and catches....)
Comment 14 Kurt Pfeifle 2007-02-04 06:57:42 UTC
Created attachment 19536 [details]
first PDF generated from attachment 19530 [details]
Comment 15 Kurt Pfeifle 2007-02-04 06:58:58 UTC
Created attachment 19537 [details]
second PDF generated from attachment 19530 [details]
Comment 16 Kurt Pfeifle 2007-02-04 14:03:34 UTC
Created attachment 19542 [details]
first PDF generated from attachment 19530 [details]

sorry for uploading the wrong times repeatedly (either I was an idiot, or
bugzilla's "obsolete this attachment, and take that new one instead"-feature
does not work as I imagined...)
Comment 17 Kurt Pfeifle 2007-02-04 14:03:39 UTC
Created attachment 19543 [details]
second PDF generated from attachment 19530 [details]

sorry for uploading the wrong times repeatedly (either I was an idiot, or
bugzilla's "obsolete this attachment, and take that new one instead"-feature
does not work as I imagined...)
Comment 18 Kurt Pfeifle 2007-02-04 15:04:32 UTC
Created attachment 19545 [details]
kword-ghostscriptfonts-embedding-testdocument.odt

.


This document was created with KWord. It uses 6 different font families (with
standard, italic, bold and bold+italic settings), all of which ship as part of
the Ghostscript font package. These fonts are PostScript fonts, mapping to the
respective original Adobe fonts (as listed in the file itself).

I want volunteers to test the following (and report back to me in email):

 * does your KWord find all these fonts on your system (if not, does it help to

   re-install the ghostscript font package)?
 * does your KWord recognize all the 'bold', 'italic', etc. typeface settings
in
   its menubar buttons (plese click into each of the respective strings)?
 * do you see the same "no-show" of features for URW Gothic and Nimbus Mono as
   I described on the bottom of the first page?
 * if you print this file to PDF via kprinter, do you see the same set of 
   fontnames embedded in the PDF (the "weird" name prefixes do not matter)?

(I also want to compare the results when loading this .odt file into OOoWriter
and Abiword, but I've not yet done so myself for lack of time. Any feedback on
these kind of tests I'd also appreciate very much!)

.
Comment 19 Kurt Pfeifle 2007-02-04 15:47:11 UTC
Created attachment 19548 [details]
PDF version of attachment 19549 [details]

.

Please use this PDF to check if your KWord (and maybe OOoWriter + Abiword) does
display it the same way on screen.

 * Does the alignment of the columns match? Or are the tabs messed up?
 * Does the page break happen at the same spot? Or is that messed up?

Thanks.

.
Comment 20 Kurt Pfeifle 2007-02-04 16:50:10 UTC
Created attachment 19549 [details]
kword-ghostscriptfonts-embedding-testdocument.odt

This version obsoletes the one from attachment 19545 [details]. (It is slightly modified
to make usage of tabs and spaces more consistent.)
Comment 21 Kurt Pfeifle 2007-02-04 16:52:09 UTC
Comment on attachment 19545 [details]
kword-ghostscriptfonts-embedding-testdocument.odt

This is now obsolete. Replaced by attachment 19549 [details].
Comment 22 Kurt Pfeifle 2007-02-04 18:11:19 UTC
Created attachment 19550 [details]
PDF version of attachment 19549 [details]

This time the *real* PDF (instead of PostScript)
Comment 23 Kurt Pfeifle 2007-02-04 18:19:46 UTC
Created attachment 19551 [details]
PDF version of attachment 19549 [details]

.


(Is it even possible to mistakenly upload *that* often PostScript files instead
of PDFs?!? !This time I did it in 2 separate steps: first obsoleted the old
file attachment, then confirmed its status, third opening a new tab for upload
the real PDF file...  *sigh*)
Comment 24 Dirk Mueller 2008-08-19 21:36:28 UTC
Comment on attachment 19530 [details]
pdf file

While this file says 
about itself in the first line that it is a PDF, in reality this one is the KWord soure file to create the two PDFs. The KWord file is using a single TrueType font only ('Bitstream Vera Serif') at different font sizes, colors and variants of bold, regular and italic.
Comment 25 John Layt 2011-05-27 18:25:44 UTC
KDEPrint is obsolete, unmaintained and will never be revived.  Closing all open bugs.