Bug 137436 - Adding support for cdrskin as an alternative to cdrecord/wodim
Summary: Adding support for cdrskin as an alternative to cdrecord/wodim
Status: CONFIRMED
Alias: None
Product: k3b
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Debian testing Linux
: NOR wishlist
Target Milestone: ---
Assignee: k3b developers
URL:
Keywords:
: 144305 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-11-16 03:48 UTC by Francois Marier
Modified: 2017-05-02 15:05 UTC (History)
12 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Adds support for cdrskin as a replacement for cdrecord (558 bytes, patch)
2006-11-16 03:49 UTC, Francois Marier
Details
cdrskin-burn-archlinux-iso-debugging-output1.txt (4.39 KB, text/plain)
2016-11-29 03:15 UTC, Leslie Zhai
Details
cdrskin-burn-archlinux-iso-debugging-output2.txt (4.42 KB, text/plain)
2016-11-30 03:04 UTC, Leslie Zhai
Details
growisofs-burn-archlinux-iso-debugging-output1.txt (1.03 KB, text/plain)
2016-11-30 03:05 UTC, Leslie Zhai
Details
cdrecord-burn-archlinux-iso-debugging-output1.txt (49.18 KB, text/plain)
2016-11-30 08:34 UTC, Leslie Zhai
Details
cdrskin-burn-archlinux-iso-debugging-output3.txt (4.20 KB, text/plain)
2016-12-01 02:41 UTC, Leslie Zhai
Details
cdrskin1.log (1.67 MB, text/x-log)
2016-12-01 02:43 UTC, Leslie Zhai
Details
cdrskin2.log (1.67 MB, text/x-log)
2016-12-01 02:48 UTC, Leslie Zhai
Details
Libburn_cdemu_host_status_test.patch (641 bytes, patch)
2016-12-02 02:18 UTC, Leslie Zhai
Details
cdrskin3.log (25.70 KB, text/x-log)
2016-12-02 02:19 UTC, Leslie Zhai
Details
cdrskin-burn-archlinux-iso-debugging-output4.txt (1.85 KB, text/plain)
2016-12-02 02:21 UTC, Leslie Zhai
Details
cdrecord.log.tar.bz2 (32.60 KB, application/x-bzip)
2016-12-02 02:22 UTC, Leslie Zhai
Details
cdrskin-burn-archlinux-iso-debugging-output5.txt (12.11 KB, text/plain)
2016-12-05 02:16 UTC, Leslie Zhai
Details
cdrskin-k3b1.log (222.30 KB, text/x-log)
2016-12-06 02:23 UTC, Leslie Zhai
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Francois Marier 2006-11-16 03:48:39 UTC
Version:           0.12.17 (using KDE KDE 3.5.5)
Installed from:    Debian testing/unstable Packages
OS:                Linux

cdrskin is a cdrecord-compatible program which uses libburn.  It would be nice to have support for it in K3b.

This was suggested by George Danchev <danchev@spnet.net> on the Debian bug tracker (http://bugs.debian.org/398398).
Comment 1 Francois Marier 2006-11-16 03:49:52 UTC
Created attachment 18578 [details]
Adds support for cdrskin as a replacement for cdrecord

Here is the patch I used to add support for cdrskin in the Debian package.
Comment 2 Sebastian Trueg 2006-11-16 10:20:07 UTC
Your patch might work but it is ugly. Even wodim is not integrated into K3b yet. If I do it I want to do it properly.
Is there any advantage of using cdrskin over cdrecord? Does it have the same interface?
Comment 3 Francois Marier 2006-11-16 14:14:05 UTC
Yes, I realize you would not want to apply this patch directly :)  (My goal was to add support for it changing as little code as possible and then revert my changes whenever K3b supports it natively.)

cdrskin seems to (mostly) have the same interface as cdrecord.  I'm not sure about the differences, but it seemed to work fine during the limited testing I did.

I'd say it's probably good to have support for it if libburn starts to catch on.
Comment 4 Sebastian Trueg 2006-11-16 15:41:00 UTC
On Thursday 16 November 2006 14:14, Francois Marier wrote:
> I'd say it's probably good to have support for it if libburn starts to
> catch on.


You are right but is libburn even actively developed. I think the last 
relaease is quite a while ago.
Comment 5 Mario Danic 2006-11-18 01:09:44 UTC
Hello Sebastian, Francois, 

I'm Mario Danic, one of the libburn (top-level name) project developer.
libburn's development actually stopped two years ago. If it was ever alive, but that's another thing to be discussed. Anyway, the project lives in another form on http://libburn.pykix.org which already release a 0.2.2 release with a lot of improvements. Soon to be released 0.2.6 looks rocking with support for -tao, multi sessions, a lot of bug improvements, etc. So let's sum up some differences cdrskin/libburn:

Cdrskin is mostly a compatibility layer that mimics cdrecord interface, while providing burning facilities on top of libburn. It's actually intended to be used for application as some provisional solutions until they can switch to using the library itself. It's also supposed to be used by apps probably at least until the libburn api become stable enough, which would mean 1.0 at least, unless developers are ready to adapt the changes. 

Now for libburn library. It's a ofcourse, a library which provides burning facility and is accompanied by a sister project libisofs for mangling ISO files.
For libisofs, a "GenIsoFs" will be written to mimic mkisofs once it reaches level we think is appropriate/mature enough. 

Kind regards,
Mario

Comment 6 Mario Danic 2006-11-18 01:14:01 UTC
In case I haven't made myself clear above, libburn is being actively developed and is progressing very fast.
Comment 7 Sebastian Trueg 2006-11-18 12:49:43 UTC
On Saturday 18 November 2006 01:09, Mario Danic wrote:
> I'm Mario Danic, one of the libburn (top-level name) project developer.
> libburn's development actually stopped two years ago. If it was ever alive,
> but that's another thing to be discussed. Anyway, the project lives in
> another form on http://libburn.pykix.org which already release a 0.2.2
> release with a lot of improvements. Soon to be released 0.2.6 looks rocking
> with support for -tao, multi sessions, a lot of bug improvements, etc. So
> let's sum up some differences cdrskin/libburn:
>
> Cdrskin is mostly a compatibility layer that mimics cdrecord interface,
> while providing burning facilities on top of libburn. It's actually
> intended to be used for application as some provisional solutions until
> they can switch to using the library itself. It's also supposed to be used
> by apps probably at least until the libburn api become stable enough, which
> would mean 1.0 at least, unless developers are ready to adapt the changes.
>
> Now for libburn library. It's a ofcourse, a library which provides burning
> facility and is accompanied by a sister project libisofs for mangling ISO
> files. For libisofs, a "GenIsoFs" will be written to mimic mkisofs once it
> reaches level we think is appropriate/mature enough.


sounds great. Please inform me once the lib is stable enough to include it in 
K3b to replace mkisofs.
Comment 8 Mario Danic 2006-11-29 00:28:23 UTC
Hello,

libisofs *IS* stable, but api *WILL* change for 1.0 I think. That being said, I'm sad to say that libisofs still lacks three to me at least, important features: HFS, HFS+, and El torito. As I'm investing a lot of work in libburn, I don't have much time for libisofs and I don't think I'll release without those features. Given our dev team is very small, dunno when it'll be ready with three above features.

Kind regards,
Mario
Comment 9 Sebastian Trueg 2007-04-16 17:39:55 UTC
*** Bug 144305 has been marked as a duplicate of this bug. ***
Comment 10 dE 2010-04-10 14:25:56 UTC
This is reality can be considered a bug. May burners fail to work under cdrtools and cdrkit. And there's just xfburn and brasero (which is a complete failure) which takes it's backend. It will be good if you give this a higher priority.
Comment 11 dE 2010-08-30 16:05:59 UTC
Some progress on this? Cause the other 2 programs suck big time...

Hard linking cdrskin to cdrtools exactly don't work as expected.
Comment 12 Kai Wb. 2011-03-25 09:12:46 UTC
There is a new Debian bug for this (the one referenced in comment #0 was closed): http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=519200

Apart from that there is a libburn 1.0.x now, which seems to indicate maturity (which was also the statement of the developers for the 1.0.0 release: "Mon Jan 17 2011 We go for release 1.0.0. This does not indicate a technological overhaul but shall emphasize the maturity of the software."). The new/current upstream project page can be found at http://libburnia-project.org/ and the whole libburnia project libraries should enable any program currently relying on external tools like cdrskin to write CDs/DVDs to rid themselves of this and use those libs instead.
Comment 13 dE 2011-03-25 11:15:10 UTC
I think cdrskin should be used as the backend, not the libraries... that actually reduces the chances of bugs. You know what happened to Brasero?
Comment 14 webdawg 2014-11-26 21:14:26 UTC
This is an 8 year old feature request and from my research cdrskin is the only way to burn BD-RE at full speed in Linux and even provides some method to use the actual cdrecord and such when its features are not present.

Is this ever going to happen?
Comment 15 Kevin Kofler 2016-11-05 01:58:58 UTC
Yes, in the long run, to work with something maintained, we will need K3b to work well with http://libburnia-project.org/ . Cdrecord has licensing issues preventing it from shipping in some distributions (https://fedoraproject.org/wiki/Forbidden_items#cdrtools), cdrkit/wodim and dvd+rw-tools/growisofs are both dead projects.

At the very least, its cdrskin tool should be supported similarly to wodim: check for it (I would check for "cdrskin", "cdrecord" and "wodim" in that order, or you end up picking up a "cdrecord" that is really wodim), then add hasFeature("cdrskin") checks where useful, i.e., cdrskin is probably preferable to growisofs for all burning tasks (both DVD and BluRay), and you need to check its support for various features. The proposed patch from comment #1 is far from sufficient. It only checks for the cdrskin binary (after wodim!), but it treats it as if it were cdrecord, so you end up with all the workarounds for cdrecord limitations.

But I think the best way would be to just use the libburn library and stop spawning external tools entirely.
Comment 16 Kevin Kofler 2016-11-05 13:58:22 UTC
There are actually 4 options to use libburnia:
* cdrskin (cdrecord-like CLI, needs minimal changes, but you should still detect it and disable cdrecord-specific workarounds for it, similarly to how wodim is handled),
* xorriso (native CLI, also replaces mkisofs/genisoimage),
* libburn (low-level, burning-only library),
* libisoburn (higher-level library, can do everything xorriso can do, there is even an API (xorriso.h) that directly mimics the xorriso CLI).
Comment 17 Kevin Kofler 2016-11-05 14:08:03 UTC
Of course, there are also options for standalone ISO generation:
* to go with cdrskin (drop-in replacement for cdrecord), there is a xorrisofs command (actually implemented by xorriso) that is a drop-in replacement for mkisofs,
* to go with libburn (low-level library), there is a libisofs library.
Comment 18 Leslie Zhai 2016-11-09 03:15:35 UTC
Hi Thomas,

I will add libburnia backend support W.I.P, and thanks again for your great help!  https://bugs.kde.org/show_bug.cgi?id=367639#c14

Regards,
Leslie Zhai - a KDE developer https://git.reviewboard.kde.org/users/lesliezhai/
Comment 19 Leslie Zhai 2016-11-09 07:04:52 UTC
Git commit 6177dcec4ca59f1781bbccc49670abd3126a3cae by Leslie Zhai.
Committed on 09/11/2016 at 06:55.
Pushed by lesliezhai into branch 'cdrskin'.

Prepare for migrating to libburnia

As Thomas Schmitt suggested in KDEBUG-367639 adapting to cdrskin/xorriso
because cdrkit/wodim and dvd+rw-tools/growisofs are both dead projects,
as Kevin Kofler described, only Linux distributions packages patch them for
fixing BUG...

So it is the time to migrate to cdrskin/xorriso, please give me more code
review and help, thanks a lot!

CCMAIL: scdbackup@gmx.net

M  +31   -0    libk3b/core/k3bdefaultexternalprograms.cpp
M  +20   -0    libk3b/core/k3bdefaultexternalprograms.h
M  +2    -0    libk3b/core/k3bglobals.cpp
M  +2    -1    libk3b/core/k3bglobals.h

http://commits.kde.org/k3b/6177dcec4ca59f1781bbccc49670abd3126a3cae
Comment 20 Thomas Schmitt 2016-11-09 07:57:42 UTC
Hi,

thank you for considering cdrskin and xorriso.

I have to point to some potential problems:

- libburn underneath both programs does on CD media only pure CD-ROM (data)
  or pure CD-DA (audio). Mixed mode, CD-XA, and other CD formats are not 
  supported.
  I think the missing features are covered by cdrdao.

- libisofs underneath does not produce UDF filesystems. This implies that 
  there is no way to produce a DVD video filesystem.
  For this use case, one will have to stay with mkisofs or genisoimage.


Have a nice day :)

Thomas
Comment 21 Leslie Zhai 2016-11-09 08:46:51 UTC
Hi Thomas,

> thank you for considering cdrskin and xorriso.

You are my mentor, you do not need to say thanks to students ;-P

> 
> I have to point to some potential problems:
> 
> - libburn underneath both programs does on CD media only pure CD-ROM (data)
>   or pure CD-DA (audio). Mixed mode, CD-XA, and other CD formats are not 
>   supported.
>   I think the missing features are covered by cdrdao.

https://bugs.kde.org/show_bug.cgi?id=367639#c14 you mentioned that lack of test cases and of the will to get into trouble with copyright holders of cloned CDs. so cdrdao might need to face the same issue!

> 
> - libisofs underneath does not produce UDF filesystems. This implies that 
>   there is no way to produce a DVD video filesystem.
>   For this use case, one will have to stay with mkisofs or genisoimage.

https://bugs.kde.org/show_bug.cgi?id=257602#c9 but how dvd+rw-tools achieved One or two picky Blu-ray players, a few standards-conformant commercial Blu-ray videos, a variety of addictive drugs to keep the programmer from running away...

@Kevin Kofler
I have no idea how other K3b or Brasero "users" test completely?! it is very poor test enviroment to me http://www.leetcode.cn/2016/08/k3b.html#test

Regards,
Leslie Zhai
Comment 22 Thomas Schmitt 2016-11-09 08:59:41 UTC
Hi,

Leslie Zhai wrote:
> > Mixed mode, CD-XA, and other CD formats are not supported.
> > I think the missing features are covered by cdrdao.

> you mentioned that lack of test cases and of the will to get into 
> trouble with copyright holders of cloned CDs.
> so cdrdao might need to face the same issue!

Might be. But afaik, the .cue file format does not forbid to mix
sector modes.
Indeed i lack of test cases for such CD layouts.

 
> > - libisofs underneath does not produce UDF filesystems. This implies that
> >   there is no way to produce a DVD video filesystem.

> but how dvd+rw-tools achieved

growisofs employs mkisofs or genisoimage for the job of producing the
filesystem. They have an option -dvd-video which implies UDF production.

Have a nice day :)

Thomas
Comment 23 Thomas Schmitt 2016-11-09 10:53:53 UTC
Hi,

to clarify the video topic:
libburn and thus cdrskin and xorriso can write a DVD or BD video image
to the medium. But libisofs and thus xorriso cannot produce that image.

It would be possible to teach libisofs support for DVD video. The info
is in ECMA-167, UDF specs for 1.02, and in genisoimage. 
There are two specialties involved: UDF itself and the peculiar directory
structure of DVD video. (For both we need the drugs to addict the programmer.)

With BD video it seems that nobody in the free software world offers yet
to produce the UDF 2.50 filesystem. The specs cost substantial money.
So one would need the BD examples and BD players for reverse engineering.
(The programmer is assumed to have been made weak-willed already.)


Have a nice day :)

Thomas
Comment 24 Kevin Kofler 2016-11-09 15:19:12 UTC
> I think the missing features are covered by cdrdao.

Unfortunately, cdrdao seems also dead upstream (last release in 2009). :-(

The burning tool situation is really sad.
Comment 25 Kevin Kofler 2016-11-09 15:36:07 UTC
Latest/last release of:
* cdrkit (wodim, genisoimage): 2010-10-17 (but things like the DVD writing code have not really been touched for longer than that; after 2007, mostly only genisoimage got fixes)
* dvd+rw-tools (growisofs): 2008-03-05
* cdrdao: 2009-10-05
Comment 26 Thomas Schmitt 2016-11-09 16:12:26 UTC
Hi,

the whole topic of CD layout beyond CD-ROM and CD-DA was already obscure
legacy when i began to learn about CD burning in 2006. For a while i
asked everybody i could find about realistic use cases. No success.
The specs might be in the non-public volumes of the
  https://en.wikipedia.org/wiki/Rainbow_Books

I could find in libburn-0.2 and public SCSI specs enough info about
CD-ROM and CD-DA including CD-TEXT to maintain or implement them, and
enough testers who confirmed that the results of libburn work for them.

The rumor that CD-ROM multi-session would need CD-XA seems to stem from
Microsoft's Joliet specs. Nobody cares or else one could not read Joliet
from multi-session DVD or BD media.

The two living projects (cdrtools by Joerg Schilling's cdrtools and
libburnia by Mario Danic and me) together cover nearly all burn cases
on CD, DVD, BD.

libburnia and cdrkit (wodim) together cover the same range.
Let wodim do the exotic CD stuff and cdrskin do the rest.
cdrskin has a fallback mode which decides on its own when to forward
the job to cdrecord or wodim:

  fallback_program=command
      Set  a command name to be executed if cdrskin encounters a known
      cdrecord option which it does not yet support.

cdrkit is fully dead but sources still exist in Debian:
  https://sources.debian.net/src/cdrkit/9:1.1.11-3/

--------------------------------------------------------------------

As for UDF, i really do not feel that libisofs should be in charge for it.
mkisofs can only do UDF 1.02 which is of technical interest only for
video DVD. No transitional users to see, which might want to access the
files via the ISO 9660 meta data.
So a pure UDF 1.02 producer seems more appropriate.
Or a copy of genisoimage ...


Have a nice day :)

Thomas
Comment 27 Dainius Masiliūnas 2016-11-09 17:46:46 UTC
For UDF, there's also udftools (mkudffs). It goes only up to 2.01, though.
Comment 28 Thomas Schmitt 2016-11-09 18:21:22 UTC
Hi,

afaik (hearsay from discussions about video UDF):
On Linux udftools have been superseded by UDF write support in the kernel.
Both are not suitable for DVD video because they cannot guarantee the
directory structure which is demanded.

The free DVD authoring tools seem to rely on mkisofs for the job.
On Ubuntu i see only dvdauthor and its frontends.
http://dvdauthor.sourceforge.net/ says
  "Things DVDAuthor does not do:
   Make a disc image or burn the disc: this is left to tools like mkisofs
   and dvd+rw-tools."

Have a nice day :)

Thomas
Comment 29 Leslie Zhai 2016-11-14 05:01:53 UTC
Git commit ad01f86ae10f8f213db23432586f4f110a8ec49b by Leslie Zhai.
Committed on 14/11/2016 at 04:51.
Pushed by lesliezhai into branch 'cdrskin'.

Add more testcase for K3b ExternalBinManager

Summary:
1. There are already binObject for cdrecord, growisofs, cdrdao, etc. so
I need to migrate cdrskin binObject;
2. There is already CdrdaoWriter, AudioCueFileWritingJob, etc. to handle
*.cue file format. so Let wodim do the exotic CD stuff and cdrskin do
the rest ;-)

M  +1    -0    src/CMakeLists.txt
M  +2    -0    src/k3bdebuggingoutputcache.cpp
M  +1    -1    src/k3bdirview.cpp
M  +8    -3    src/k3bjobprogressdialog.cpp
M  +15   -7    src/k3bthemedlabel.cpp
M  +185  -3    tests/CMakeLists.txt
M  +46   -3    tests/k3bexternalbinmanagertest.cpp
M  +5    -2    tests/k3bexternalbinmanagertest.h

http://commits.kde.org/k3b/ad01f86ae10f8f213db23432586f4f110a8ec49b
Comment 30 Leslie Zhai 2016-11-28 09:18:34 UTC
Git commit 88c0c90659b3dcde527788c3d79ce70fa9582b40 by Leslie Zhai.
Committed on 28/11/2016 at 09:10.
Pushed by lesliezhai into branch 'cdrskin'.

Very begining CdrskinProgram and CdrskinWriter

Summary:
It is skeleton of migrating to cdrskin, so it might be full of bugs, I
need to be faimilar with cdrskin command...

1. [UI] add cdrskin to the items of Writing app combo
2. [k3blib] add CdrskinProgram inherited from SimpleExternalProgram (not
including *.cue)
3. [k3blib] add CdrskinWriter (it might not suitable for cdrskin command)

Test plan:
burn , for example ArchLinux, iso

CCMAIL: scdbackup@gmx.net

M  +1    -0    libk3b/CMakeLists.txt
M  +53   -19   libk3b/core/k3bdefaultexternalprograms.cpp
M  +2    -13   libk3b/core/k3bdefaultexternalprograms.h
M  +1    -0    libk3b/jobs/k3biso9660imagewritingjob.cpp
M  +101  -0    libk3b/jobs/k3bmetawriter.cpp
M  +1    -0    libk3b/jobs/k3bmetawriter.h
M  +1    -0    libk3b/projects/CMakeLists.txt
A  +935  -0    libk3b/projects/k3bcdrskinwriter.cpp     [License: GPL (v2+)]
A  +105  -0    libk3b/projects/k3bcdrskinwriter.h     [License: GPL (v2+)]
M  +7    -0    src/k3bsystemproblemdialog.cpp
M  +2    -0    src/k3bwriterselectionwidget.cpp
M  +3    -0    tests/k3bexternalbinmanagertest.cpp

https://commits.kde.org/k3b/88c0c90659b3dcde527788c3d79ce70fa9582b40
Comment 31 Thomas Schmitt 2016-11-28 11:48:39 UTC
Hi,

Leslie Zhai wrote:
> I need to be faimilar with cdrskin command...

I will try to be of help as good as i can.


> +bool K3b::CdrskinProgram::scanFeatures(ExternalBin& bin) const
> ...
> +    fp << bin.path() << "write" << "-h";
> ...
> +    if (fp.execute() >= 0) {
> ...
>          if (output.contains("gracetime"))

I suppose this shall get the cdrskin help text by the equivalent of
  cdrskin -help 2>&1 | interpreter

Not sure how the cdrskin option "-help" is supposed to get in here.
I cannot find the corresponding gesture for cdrecord or wodim in
  https://cgit.kde.org/k3b.git/tree/libk3b/core/k3bdefaultexternalprograms.cpp?id=88c0c90659b3dcde527788c3d79ce70fa9582b40
but only one with cdrdao.

Run
  cdrdao write -h 2>&1 
indeed triggers a help text on stdout.

The corresponding run with cdrskin would be (like with wodim and cdrecord)

  cdrskin -help 2>&1

In general it seems better to derive the cdrskin wrapper code from
the one for wodim or cdrecord, rather than from cdrdao.

Option -xa1 is listed in the -help text and will not lead to an error if used.
Nevertheless it will not write XA Mode 2 but rather Mode 1 while stripping
8 surplus XA header bytes per sector from the data stream. The result is
then readable as CD-ROM data.

Additional cdrskin features are announced by option --help (with two dashes).
Those are not supported by cdrecord or wodim (although wodim knows --devices).


> +    if (bin.version() >= K3b::Version(2, 1, 1, "a29"))
> +        bin.addFeature( "blu-ray" );

I think this tries to determine cdrecord features from cdrecord version
numbers.

cdrskin says on option -version:

  cdrskin 1.4.6 : limited cdrecord compatibility wrapper for libburn
  Cdrecord 2.01a27 Emulation. Copyright (C) 2006-2016, see libburnia-project.org

So "blu-ray" would probably not be triggered.
(I will update the fake cdrecord version to a29. It exists on
request of K3B users who install their cdrskin as "/usr/bin/cdrecord".)

Blu-ray support was complete in cdrskin-0.6.2, 20 Feb 2009.

"burnfree" was supported from start (and is default).
"burnproof" is accepted as alias of "burnfree".

The other version number dependent features need to be checked what they
mean and whether cdrskin supports them.


> +bool K3b::MetaWriter::setupCdrskinJob()
> ...
> +                writer->addArgument( "-useinfo" );
> ...
> +                writer->addArgument( "-shorttrack" );

These options are not yet supported by cdrskin.
One will have to test the -help text before using them.


> +                if( image.isEmpty() ) {
> +                    // this is only supported by cdrecord versions >= 2.01a13
> +                    writer->addArgument( QFile::encodeName( d->infFileName( audioTrackCnt ) ) );

I assume this is a companion of "-useinfo". Needs to become conditional then.


> + d->process << "gracetime=2"; // 2 is the lowest allowed value (Joerg, why do you do this to us?)

I am Thomas and i allow gracetime=0. :))

Further cdrskin's gracetime default is 0.
Only with undersized tracks (< 600 KB) the gracetime is at least 15 seconds
in order to let a command line user abort the run.


> +    if( d->usedSpeed == 0 ) {
> +        // try to determine the writeSpeed

cdrskin accepts special speed values "0" = slowest, "-1" or "any" = fastest.
Default is fastest speed.


> + emit infoMessage( i18n( "Cdrskin version %1 does not support Blu-ray writing." ,d->cdrskinBinObject->version() ), MessageError );

Will this tell the cdrskin version (e.g. 1.4.6) or the pseudo cdreord one ?


> + // cdrskin only supports SAO for DVD
> + d->process << "-sao";

It supports -tao on all media except fast-blanked DVD-RW and DVD-R DL,
which both support only write type DAO.
On DVD+RW and alike, there is no difference between -sao and -tao.
On DVD-R, -tao chooses Incremental mode which is capable of -multi and
of appending new sessions to existing Incremental sessions.
On DVD+R, -sao emits a command RESERVE TRACK, whereas -tao does not.

One may omit both -tao and -sao in order to let cdrskin decide on base
of -multi, input source and Medium state which write type to use.


> +#warning Enable layer jump mode: add it to K3b::WritingMode and to the GUI

This mode is not planned to be implemented in cdrskin.
growisofs uses it and seems to fail more often than libburn.
So libburn for now lets the drive decide when to switch layers on
multi-layered media.


> +        else if( d->writingMode == K3b::WritingModeRaw ) {
> +            if( burnDevice()->supportsWritingMode( K3b::Device::WRITINGMODE_RAW_R96R ) )
> +            d->process << "-raw96r";

These write modes are not supported by cdrskin. They need to be made
contitional.


> +        // with cdrskin 1.11a02 burnproof was renamed to burnfree

Here you replaced "cdrecord" by "cdrskin" without checking the semantics.

Both names are supported by cdrskin since its first release.


> +    switch( d->formattingMode ) {
> +        case FormattingComplete:
> +            d->process << "blank=all";
> ...
> +            d->process << "blank=fast";

Aha. formattingMode means blanking mode. (I already wondered.)

cdrskin takes into respect that fast blanking of DVD-RW media yields
a medium state which is not suitable for -tao or -multi. I.e. only
DAO (-sao or -dao) is possible which means predictable track size
and only a single session.


> + // Cdrskin starts all error and warning messages with it's path
> + // With Debian's script it starts with cdrskin (or /usr/bin/cdrskin or whatever! I hate this script!)

Can this be a remnant rant about cdrecord and a Debian script ?

cdrskin has the habit to prepend its name to many error messages.
But not its path. Further there may be error messages with "libburn:"
prepended.

In general one will have to cross check the messages listed here with
the source of cdrskin, whether they appear there.


Have a nice day :)

Thomas
Comment 32 Thomas Schmitt 2016-11-28 12:02:05 UTC
Hi,

i see that i did not explain the blanking issue completely:

On DVD-RW, cdrskin "blank=fast" works like "blank=all".
Only "blank=deformat_sequential_quickest" will perform fast blanking
and thus restrict the medium to DAO for the next write run.

There is a MMC feature which is supposed to tell whether the drive
holds such a crippled medium. But not all drives tell correctly.
Some only confess when write type Incremental shall be selected.

So if multi-session on unformatted DVD-RW is an important goal, then
one should not use "blank=deformat_sequential_quickest".


Have a nice day :)

Thomas
Comment 33 Leslie Zhai 2016-11-29 03:11:55 UTC
Git commit 0cbea61ca4aa60ce3d6fb2dab67023d5ac8e1e4f by Leslie Zhai.
Committed on 29/11/2016 at 03:08.
Pushed by lesliezhai into branch 'cdrskin'.

Change features and arguments for CdrskinProgram and CdrskinWriter

Reviewers: Thomas Schmitt

Test plan:
burn ArchLinux iso
FAILED!
attachment: cdrskin-burn-archlinux-iso-debugging-output1.txt

CCMAIL: scdbackup@gmx.net

M  +29   -12   libk3b/core/k3bdefaultexternalprograms.cpp
M  +5    -27   libk3b/jobs/k3bmetawriter.cpp
M  +43   -34   libk3b/projects/k3bcdrskinwriter.cpp
M  +4    -1    src/rip/videodvd/k3bvideodvdrippingpreview.cpp

https://commits.kde.org/k3b/0cbea61ca4aa60ce3d6fb2dab67023d5ac8e1e4f
Comment 34 Leslie Zhai 2016-11-29 03:15:02 UTC
Created attachment 102515 [details]
cdrskin-burn-archlinux-iso-debugging-output1.txt

cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0 ~/virt.iso

k3b failed to burn ArchLinux iso with cdrskin Writing app, please have a look at the debugging output.
Comment 35 Thomas Schmitt 2016-11-29 09:11:08 UTC
Hi,

> https://bugs.kde.org/attachment.cgi?id=102515
> cdrskin: FAILURE : SCSI command 2Ah yielded host problem: 0x1 SG_ERR_DID_NO_CONNECT (Could not connect before timeout period)

This is a Linux SCSI command transport error emitted by iotcl(SG_IO).
Listed in http://www.tldp.org/HOWTO/SCSI-Generic-HOWTO/x291.html but
not much explained there.

In any case it is not the fault of cdrskin, libburn, or your cdrskin options.

If it was a real drive, i would advise to first check the cables and
to try another SATA or USB port. Another suspect would be the drive's
power supply.

The fact that cdrskin started burning without own error message is
an intermediate indication of programming success on your side.
The DVD emulator seems to have worked in the beginning, until it ran into
a problem with the Linux kernel.


> Starting to write CD/DVD at speed 18 in real TAO mode for single session.
> ...
> Track 01:   30 of  734 MB written (fifo 100%) [buf   0%]  42.5x.
> Track 01:   35 of  734 MB written (fifo  93%) [buf   0%]  189.6x.
> ...
> /usr/sbin/cdrskin -v gracetime=0 dev=/dev/sr1 speed=18 -sao -tao driveropts=burnfree -data -tsize=375808s -

Obviously the emulated drive does not honor the speed wishes issued
by libburn. (It is so fast that the watcher thread of cdrskin misses
several MB between its inquiries.)

---------------------------------------------------------------------------

For command line experiments with cdrskin i could offer libburn's emulated
drives.
cdrskin option

  --allow_emulated_drives

enables this feature. A (possibly not yet existing) data file can then be
used by an address option like

  dev=stdio:/tmp/my_pseudo_drive

This yields a behavior similar to DVD+RW media.

Disadvantage is that these pseudo drives will not show up as CD drives to
the kernel and thus will not be realistic examples when used from K3B.
(For anything but libburn the files are just plain data files. Like an
ISO image.)

Advantage is that no ioctl(SG_IO) is involved but just POSIX calls like
open(2), lseek(2), write(2). If the computer can reliable write files,
then it also can perform libburn pseudo-drive operations.

Have a nice day :)

Thomas
Comment 36 Thomas Schmitt 2016-11-29 11:34:17 UTC
Hi,

looking at
  https://cgit.kde.org/k3b.git/commit/?id=0cbea61ca4aa60ce3d6fb2dab67023d5ac8e1e4f
i still riddle about the version dependent feature detector in
  libk3b/core/k3bdefaultexternalprograms.cpp

Many of the version tests seem to still bear the cdrecord version numbers.
Like:
  if (bin.version() >= K3b::Version(1, 1, 1))
This needs to be translated to cdrskin numbers, if applicable at all.

-------------------------------------------------------------------------

You need to find out what the features mean in K3B.

For now i can only make statements about cdrskin aspects of the decision
making code:

"dvd-patch"
There is no "-dvd" in the "Cdrecord" pseudo version line or in the
real cdrskin version line.
So if this feature is applicable to cdrskin we need a different test for it.

"outdated"
Since cdrskin aims for (limited) compatibility with cdrecord 2.01,
i think you can safely remove this test.

"plain-atapi"
"hacked-atapi"
This may be about the "ATA:..." or "ATAPI:..." drive addresses.
Besides the fact that in Linux since about 2007 all IDE ATAPI drives have
SCSI addresses, cdrskin can handle drive addresses of form
"ATA:Bus,Lun,Target" and "ATAPI:Bus,Lun,Target", where the IDE drive
address is computed like
  printf("/dev/hd%c", 'a' + 2 * Bus + Lun);
E.g.: "ATA:1,1,0" = "/dev/hdc".
You may expect this feature unconditionally.

"short-track-raw"
No idea what that means. If "raw" is related to write modes like "-raw96",
then it is most probably not supported.

"audio-stdin"
Hard to say what this shall be.

"burnfree"
"burnproof"
No need to distinguish versions here. Both names are supposed to work.

"dvd"
DVD support was complete in cdrskin-0.4.4, 8 Mar 2008.

--------------------------------------------------------------------------

> + if (output.contains("-xamix") || bin.version() >= K3b::Version(1, 1, 1))
> bin.addFeature("xamix");

The version number seems to be cdrecord-ish. No cdrskin version supports
option "-xamix" yet.

--------------------------------------------------------------------------

> + if (image.isEmpty()) {
> + // TODO: this is a companion of "-useinfo". Needs to become conditional then.
> + writer->addArgument(QFile::encodeName(d->infFileName(audioTrackCnt)));

I think you should for now just throw an error in this case.
It looks like cdrecord gets .inf files instead of track sources.
No version of cdrskin can do this yet.

--------------------------------------------------------------------------

> + d->process << "gracetime=0"; // I am Thomas and i allow gracetime=0. :)

No, no. You are Leslie. :))
(Nevertheless, Thomas allows 0 gracetime.)

--------------------------------------------------------------------------

> + // TODO: so does need to check formatWritingSpeedFactor?
> + d->process << QString("speed=%1").arg(formatWritingSpeedFactor(d->usedSpeed, d->burnedMediaType, SpeedFormatInteger));

Interesting question.
cdrskin interprets speed numbers depending on the medium type:
CD = 153600 bytes/s, DVD = 1385000 bytes/s, BD = 4495625 bytes/s.

--------------------------------------------------------------------------

> + * Then what about raw16 and raw96p?

cdrskin supports only -audio and -data.
Options -xa1, -xa, -xa2, -mode2 do not lead to error but the payload is
nevertheless written as -data.

--------------------------------------------------------------------------

> + d->process << "blank=deformat_sequential_quickest";

This should only be applied to DVD-RW media, if ever.

It produces a crippled medium state which can only accept a single DAO
session. (On the other hand, full DVD-RW blanking lasts 20 to 40 minutes.)

--------------------------------------------------------------------------

Have a nice day :)

Thomas
Comment 37 Thomas Schmitt 2016-11-29 21:29:35 UTC
Correction: "ATA:1,1,0" is "/dev/hdd", not "/dev/hdc".
Comment 38 Leslie Zhai 2016-11-30 02:59:28 UTC
Git commit c17d307523a08c1e6042381bdbdfe1971fffc84a by Leslie Zhai.
Committed on 30/11/2016 at 02:52.
Pushed by lesliezhai into branch 'cdrskin'.

Change features and arguments for Cdrskin

Test plan1:
burn ArchLinux iso created by CDEmu with --allow_emulated_drives
FAILED! see debugging log attachment:
cdrskin-burn-archlinux-iso-debugging-output2.txt

Test plan2:
burn ArchLinux iso created by CDEmu with growisofs
SUCCESS! see debugging log attachment:
growisofs-burn-archlinux-iso-debugging-output1.txt

CCMAIL: scdbackup@gmx.net

M  +4    -24   libk3b/core/k3bdefaultexternalprograms.cpp
M  +1    -2    libk3b/jobs/k3bmetawriter.cpp
M  +23   -13   libk3b/projects/k3bcdrskinwriter.cpp

https://commits.kde.org/k3b/c17d307523a08c1e6042381bdbdfe1971fffc84a
Comment 39 Leslie Zhai 2016-11-30 03:04:26 UTC
Created attachment 102534 [details]
cdrskin-burn-archlinux-iso-debugging-output2.txt

Failed to burn ArchLinux iso created by CDEmu with --allow_emulated_drives
Comment 40 Leslie Zhai 2016-11-30 03:05:34 UTC
Created attachment 102535 [details]
growisofs-burn-archlinux-iso-debugging-output1.txt

But growisofs worked!
Comment 41 Thomas Schmitt 2016-11-30 08:18:01 UTC
Hi,

the growisofs log is very sparse. Are you sure it burned the whole session ?
If i do
  dd if=/dev/zero bs=1M count=734 | growisofs -Z /dev/null=/dev/fd/0
i get at least a final message
  builtin_dd: 375808*2KB out @ average 555.9x1352KBps


As for the cdrskin run: It still operated on the CDEmu drive because
it got the address "/dev/sr1" and not a path to a (possibly not yet existing)
regular file prepended by "stdio:".

It is interesting to see that both cdrskin burn attempts fail at 71 MB with 
the same Linux-to-drive connection problem. But currently i have no idea
how to find out what goes wrong down there.

When cdrskin gets the failure reply from ioctl(SG_IO) it is just sending
data by SCSI commands WRITE10 and inquiring the drive buffer fill by
SCSI commands READ BUFFER CAPACITY. One can hardly do anything wrong in
this moment. It is up to the drive and Linux to negociate when data can
be transmitted to the drive and when Linux has to wait because the drive
buffer is full. The job of libburn is to keep the timespan small between
the end of a WRITE10 command and the start of the next one.


Have a nice day :)

Thomas
Comment 42 Leslie Zhai 2016-11-30 08:34:18 UTC
Created attachment 102543 [details]
cdrecord-burn-archlinux-iso-debugging-output1.txt

Cdrecord worked too!

and cdrecord burned the whole session:
Track 01:    0 of  734 MB written.
Track 01:    1 of  734 MB written (fifo  81%) [buf   0%]  27.3x.
.
.
.
Track 01:  734 of  734 MB written (fifo 100%) [buf   0%]  80.9x.
Comment 43 Thomas Schmitt 2016-11-30 09:22:21 UTC
Hi,

about the newest commit
  https://cgit.kde.org/k3b.git/commit/?id=c17d307523a08c1e6042381bdbdfe1971fffc84a

> + bin.addFeature("plain-atapi");
> + bin.addFeature("hacked-atapi");

Did you find out what these features mean ?
(Reading libk3b/core/k3bglobals.h i get the suspicion they mean support
 for addresses "/dev/hdX" and "ATAPI:X,Y,Z" respectively. This would be
 supported by cdrskin. But i am not totally sure.)


> + bin.addFeature("burnfree");
> + bin.addFeature("burnproof");

I would decide for one of them. One never knows what other code piece
could be confused if two normally mutually exclusive features are present.


> + if (burnDevice()->supportsWritingMode(K3b::Device::WRITINGMODE_RAW_R96R) ||
> + burnDevice()->supportsWritingMode(K3b::Device::WRITINGMODE_RAW_R16) ||
> + burnDevice()->supportsWritingMode(K3b::Device::WRITINGMODE_RAW_R96P)) {
> +    d->process << "-data";

You may not do this.

The input data for those raw CD modes contain bytes which do not belong
to the payload but rather are CD sector metadata.
Those metadata formats which have only a few ornaments around data payload
get defaulted to -data by cdrskin. But the -raw* family expresses a different
wish of the user (normally one that is illegal in countries with strict
copyright enforcement).

You will have to refuse such wishes. Best would be not to offer these write
modes at all when cdrskin is to be used.


> + if (burnDevice()->description().contains("Virt"))
> + d->process << "--allow_emulated_drives";

This was a misunderstanding between us.
--allow_emulated_drives only lifts the ban on "stdio:/path" addresses. As
long as you use a "/dev/srN" address which leads to an optical drive, libburn
will use its MMC driver to operate it as optical drive, rather than as POSIX
file object.


> + if (d->burnedMediaType & Device::MEDIA_DVD_PLUS_RW)
> + d->process << "blank=deformat_sequential_quickest";

This is the wrong media type (DVD+RW). The special handling of "blank=fast"
applies to to DVD-RW, which has two media types in K3B:
MEDIA_DVD_RW_OVWR and MEDIA_DVD_RW_SEQ.

The type _OVWR indicates a formatted medium which needs no blanking unless
the user wants to deformat it. In that case "deformat_sequential_quickest"
and "deformat_sequential" brings it to type MEDIA_DVD_RW_SEQ.
But this is not necessarily desired when a user orders blanking for the
sole purpose of making the medium writable from scratch. This wish is
fulfilled by growisofs, cdrecord, cdrskin, and xorrecord without blanking
because one can write the medium in random access mode.
growisofs with option -M and program xorriso handle such media as appendable
if they contain an ISO 9660 filesystem.
So if you apply "deformat_sequential*" to MEDIA_DVD_RW_OVWR this would need
special permission by the user.

The type MEDIA_DVD_RW_SEQ behaves much like CD-RW, except the undesirable
consequences of fast blanking.

So i would exchange in above statement MEDIA_DVD_PLUS_RW by MEDIA_DVD_RW_SEQ
and leave MEDIA_DVD_RW_OVWR out of the game for now.


Have a nice day :)

Thomas
Comment 44 Thomas Schmitt 2016-11-30 09:33:59 UTC
Hi,

please run on the command line

  dd if=/dev/zero bs=2048 count=375808 | /usr/sbin/cdrskin -v -V dev=/dev/sr1 speed=18 -tao -data -tsize=375808s - 2>&1 | tee -i /tmp/cdrskin.log

This will yield in file /tmp/cdrskin.log a very verbous log of the SCSI
commands sent to the drive and of their replies from the drive. Maybe i
can learn something from the last commands before the failure.

If it now works by miracle, then replace
  2>&1 | tee -i /tmp/cdrskin.log
by
  >/tmp/cdrskin.log 2>&1
(The positions of "2>&1" are important.)


Have a nice day :)

Thomas
Comment 45 Thomas Schmitt 2016-11-30 09:35:36 UTC
Hi,

independently of the failure, there seems to be a bug in the composition
of the cdrskin command.

Both options -sao and -tao were given. -tao prevailed because it came
after -sao.
K3B should nevertheless know what it wants and only choose one of them.


Have a nice day :)

Thomas
Comment 46 Thomas Schmitt 2016-11-30 11:48:31 UTC
Hi,

i am now looking into the source code of growisofs (file transport.hxx).
There is no interpretation of
  sg_io_hdr_t.host_status
which in case of the cdrskin run bears the SG_ERR_DID_NO_CONNECT error
indicator.
So it could well be that growisofs just skips over this problem and
goes on with burning. 

cdrecord-2.01.01a77/libscg/scsi-linux-sg.c seems to consider the error
to be retryable. I.e. cdrecord would go on, too. But i assume it would
repeat the WRITE command, which is different from what growisofs would do.
cdrecord prints the sg_io_hdr_t.host_status only if scgp->debug is set.
There is a cdrecord option
  -d
which here triggers lots of messages of form
  host_status: 00 driver_status: 08

It would be interesting to see the output file /tmp/cdrecord.log of this run:

  dd if=/dev/zero bs=2048 count=375808 | /usr/bin/cdrecord -d -v -V gracetime=2 dev=/dev/sr1 speed=18 -sao driveropts=burnfree -data -tsize=375808s - >/tmp/cdrecord.log 2>&1

----------------------------------------------------------------------------

Please check whether the growisofs and cdrecord written pseudo media contain
an exact copy of the input data. (I currently doubt because i suspect that
cdrecord repeats the ERITE command whereas growisofs does not.)
Maybe you first have to create an ISO image and then burn it to media in a
second K3B run, so that you can compare image file and media content.

If so, then i will try to develop a changeset for libburn which lets it
ignore this (normally severe) error for your testing purposes.

----------------------------------------------------------------------------

Regrettably i cannot find a Debian package which would provide a command
"cdemu". Can you give me some hints how to install it on a Linux VM ?


Have a nice day :)

Thomas
Comment 47 Thomas Schmitt 2016-11-30 11:50:16 UTC
Correction: "ERITE" is a tyop. I meant "WRITE".
Comment 48 Leslie Zhai 2016-12-01 02:39:28 UTC
Git commit 5213bf69e5a0c1aa0d2115b2bbe851e7294bcbf0 by Leslie Zhai.
Committed on 01/12/2016 at 02:33.
Pushed by lesliezhai into branch 'cdrskin'.

Change features and arguments for Cdrskin

Test plan1:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. k3b burn ArchLinux iso
* FAILED: cdrskin-burn-archlinux-iso-debugging-output3.txt

Test plan2:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. dd if=/dev/zero bs=2048 count=375808 | /usr/sbin/cdrskin -v -V
dev=/dev/sr1 speed=18 -tao -data -tsize=375808s - 2>&1 | tee -i
/tmp/cdrskin1.log
* SUCCESS: cdrskin1.log

Test plan3:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. dd if=/dev/zero bs=2048 count=375808 | /usr/sbin/cdrskin -v -V
dev=/dev/sr1 speed=18 -tao -data -tsize=375808s - >/tmp/cdrskin2.log 2>&1
* SUCCESS: cdrskin2.log

Test plan4:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. dd if=/dev/zero bs=2048 count=375808 | /usr/bin/cdrecord -d -v -V
gracetime=2 dev=/dev/sr1 speed=18 -sao driveropts=burnfree -data
-tsize=375808s - >/tmp/cdrecord.log 2>&1
* SUCCESS: cdrecord.log

CCMAIL: scdbackup@gmx.net

M  +4    -1    libk3b/core/k3bdefaultexternalprograms.cpp
M  +5    -17   libk3b/projects/k3bcdrskinwriter.cpp
M  +1    -0    libk3bdevice/k3bdevice.cpp

https://commits.kde.org/k3b/5213bf69e5a0c1aa0d2115b2bbe851e7294bcbf0
Comment 49 Leslie Zhai 2016-12-01 02:41:04 UTC
Created attachment 102553 [details]
cdrskin-burn-archlinux-iso-debugging-output3.txt
Comment 50 Leslie Zhai 2016-12-01 02:43:34 UTC
Created attachment 102554 [details]
cdrskin1.log
Comment 51 Leslie Zhai 2016-12-01 02:48:40 UTC
Created attachment 102555 [details]
cdrskin2.log
Comment 52 Leslie Zhai 2016-12-01 03:08:36 UTC
cdrecord.log is tooo big! so

head cdrecord.log 
fs: 4194304 buflen: 4198400
cdrecord: shared memory segment attached at: 7FA161435000 size 4198400
buf: 7FA161435000 bufend: 7FA161836000, buflen: 4198400
buf: 7FA161435000 bufend: 7FA161836000, buflen: 4198400 (align 0)
scsidev: '/dev/sr1'
devname: '/dev/sr1'
scsibus: -2 target: -2 lun: -2
scg__open(/dev/sr1) -2,-2,-2
Warning: Open by 'devname' is unintentional and not supported.
l1: 0x6000001 l2: 0x7FFEB2A67180

tail cdrecord.log 
cmd finished after 0.000s timeout 200s

Executing 'prevent/allow medium removal' command on Bus 6 Target 1, Lun 0 timeout 200s
CDB:  1E 00 00 00 00 00
ioctl ret: 0
host_status: 00 driver_status: 00
cmd finished after 0.000s timeout 200s
cdrecord: fifo had 23488 puts and 23488 gets.
cdrecord: fifo was 0 times empty and 234 times full, min fill was 17%.
Set

PS: there are cdemu-client https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/cdemu-client and cdemu-daemon https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/cdemu-daemon
Comment 53 Thomas Schmitt 2016-12-01 09:20:12 UTC
Hi,

the reproducible success on the command line and the reproducible
failure underneath K3B is quite riddling.

Please test whether it still succeeds without option -V

  dd if=/dev/zero bs=2048 count=375808 | /usr/sbin/cdrskin -v dev=/dev/sr1 speed=18 -tao -data -tsize=375808s - >/tmp/cdrskin.log 2>&1

and please verify that cdrskin still fails underneath K3B.

I am still very interested in seeing the full cdrecord log file.
Please send it to scdbackup@gmx.net or put it somewhere for download.
The question is whether there are "host_status:" values which are not "00".

Also, i am still interested in an answer to the question whether the
pseudo-media burned by cdrecord and growisofs underneath K3B are
really flawless copies of the input data. (Plus some trailing padding.)


I will make a proposal how to build a cdrskin which ignores
SG_ERR_DID_NO_CONNECT when reported by the kernel, so that you can try
whether this works better underneath K3B.


Have a nice day :)

Thomas
Comment 54 Thomas Schmitt 2016-12-01 09:28:47 UTC
Hi,

the proposal to let libburn ignore SG_ERR_DID_NO_CONNECT is:

- Download the current cdrskin development tarball

   http://scdbackup.sourceforge.net/cdrskin-1.4.7.tar.gz

- Unpack in some directory of your choice and go to the unpacked top directory

   tar xzf cdrskin-1.4.7.tar.gz
   cd  cdrskin-1.4.7

- Modify the transport driver for Linux SG_IO:

----------------------------------------------------------------------------
--- libburn/sg-linux.c  2016-07-23 12:35:51.030269903 +0200
+++ .../libburn/sg-linux.c  2016-11-30 14:18:58.745724400 +0100
@@ -1906,6 +1906,13 @@ static int evaluate_transport_success(st
 
        /* See http://www.tldp.org/HOWTO/SCSI-Generic-HOWTO/x291.html */
 
+#define Libburn_cdemu_host_status_test yes
+#ifdef Libburn_cdemu_host_status_test
+       if (host_status == 0x01 &&
+           (driver_status & 0xf7) == Libburn_sg_driver_oK) {
+               /* SG_ERR_DID_NO_CONNECT */
+               {ret = 1; goto ex;}
+       }
+#endif
+
        switch(host_status) {
        case 0x00:
                host_problem =
----------------------------------------------------------------------------

  I understand that this would act like growisofs.
  Triggering a retry, like probably cdrecord does, needs some change with
  the caller of this function.

- Compile the modified libburn and create a statically linked cdrskin,
  which does not use your system-wide installed libburn.so:

    make
    cdrskin/compile_cdrskin.sh -g

- Check whether your cdrskin binary is willing to run

    ls $(pwd)/cdrskin/cdrskin
    cdrskin/cdrskin -version

  This should say

    /...absolute.path.../cdrskin/cdrskin
    cdrskin 1.4.7 : limited cdrecord compatibility wrapper for libburn
    ...
    Version timestamp :  2016.11.18.132335
    Build timestamp   :  ...

- Let K3B use that binary.
  Either by putting it where K3B found cdrskin before, or by letting
  K3B execute the binary by its absolute path.

  If the cdrskin version number 1.4.7 then appears in the K3B log, we
  know that the change is in effect.


Have a nice day :)

Thomas
Comment 55 Thomas Schmitt 2016-12-01 09:49:14 UTC
Hi,

comments about the newest changeset
  https://cgit.kde.org/k3b.git/commit/?id=5213bf69e5a0c1aa0d2115b2bbe851e7294bcbf0


> + d->process << "-tao"/* << "-sao"*/;

This hardcodes -tao. Why ?

I think with cdrecord, K3B has a preference for -sao.
K3B compensates the additional information needs of -sao on CD by giving the
track size with option tsize=. I am not sure whether it knows that -sao is
not usable together with -multi on DVD-R and unformatted DVD-RW.

My advise is to neither give -sao nor -tao explicitely but to let libburn
choose the write type according to other parameters and the medium state.


> + emit infoMessage(i18n("Writer does not support raw writing."), MessageWarning);
> + if (d->cdrskinBinObject->hasFeature("tao"))
> + d->process << "-tao";

Adding option -tao will not make the resulting data on medium better.
The input data for -raw* contain metadata. If you burn them as -data track,
then the reader will see it as garbage intermixed with the desired payload
data.

If the program gets to the case
   else if( d->writingMode == K3b::WritingModeRaw ) {
then you have no other choice but to refuse burning.
Actually it should be prevented in the user dialog that this write mode
gets selected at all.


Have a nice day :)

Thomas
Comment 56 Kevin Kofler 2016-12-01 10:12:23 UTC
> If the program gets to the case
>    else if( d->writingMode == K3b::WritingModeRaw ) {
> then you have no other choice but to refuse burning.
> Actually it should be prevented in the user dialog that this write mode
> gets selected at all.

Or the code should in that case select a burning tool that can do that. (cdrecord? wodim? cdrdao? Whatever supports it and is available on the user's system.) If none is found, then graying it out in the UI is the best option.
Comment 57 Thomas Schmitt 2016-12-01 11:57:53 UTC
Hi,

> Or the code should in that case select a burning tool that can do that. 
> (cdrecord? wodim? cdrdao?

That's well an option with cdrecord or wodim.

cdrdao is not as good a candidate because it does not understand the same 
options as the others. I assume it gets its sector formats by the TOC or CUE
file which controls its actions, not by a global K3B write mode.


cdrskin has an option
  fallback_program=...path...
which delegates the burn job to the given program path (e.g. /usr/bin/wodim)
if it detects options which it does not support, and if there are no
options which only cdrskin supports (e.g. drive_scsi_dev_family=).

I have to admit that i did not test this since years.


Are there K3B use cases known which would need -raw* writing ?


Have a nice day :)
Comment 58 Leslie Zhai 2016-12-02 02:17:11 UTC
Git commit a8074105e842babab18d7adcb290dd6eca3304bb by Leslie Zhai.
Committed on 02/12/2016 at 02:07.
Pushed by lesliezhai into branch 'cdrskin'.

Change arguments for Cdrskin

Use libburn git version 19a1b8e768e393ea0476c1a7224c58599f30628b
Bug fix: Option -dummy did not affect writing by direct_write_amount=

patch -Np1 -i Libburn_cdemu_host_status_test.patch

Test plan1:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. dd if=/dev/zero bs=2048 count=375808 | /usr/sbin/cdrskin -v dev=/dev/sr1
speed=18 -tao -data -tsize=375808s - >/tmp/cdrskin.log 2>&1
SUCCESS: cdrskin3.log

Test plan2:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. k3b burn ArchLinux iso
FAILED: cdrskin-burn-archlinux-iso-debugging-output4.txt

Test plan3:
1. cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0
~/virt.iso
2. dd if=/dev/zero bs=2048 count=375808 | /usr/bin/cdrecord -d -v -V
gracetime=2 dev=/dev/sr1 speed=18 -sao driveropts=burnfree -data
-tsize=375808s - >/tmp/cdrecord.log 2>&1
SUCCESS: cdrecord.log.tar.bz2

CCMAIL: scdbackup@gmx.net

M  +4    -3    libk3b/projects/k3bcdrskinwriter.cpp

https://commits.kde.org/k3b/a8074105e842babab18d7adcb290dd6eca3304bb
Comment 59 Leslie Zhai 2016-12-02 02:18:32 UTC
Created attachment 102566 [details]
Libburn_cdemu_host_status_test.patch
Comment 60 Leslie Zhai 2016-12-02 02:19:37 UTC
Created attachment 102567 [details]
cdrskin3.log

cdrskin v1.4.7 with Libburn_cdemu_host_status_test.patch
Comment 61 Leslie Zhai 2016-12-02 02:21:49 UTC
Created attachment 102568 [details]
cdrskin-burn-archlinux-iso-debugging-output4.txt

Hi Thomas,

Did you use cdemu create-blank --writer-id=WRITER-ISO --medium-type=dvd+r 0 ~/virt.iso

then choose ArchLinux iso for testcase?

Regards,
Leslie Zhai
Comment 62 Leslie Zhai 2016-12-02 02:22:59 UTC
Created attachment 102569 [details]
cdrecord.log.tar.bz2

cdrecord.log's tarball
Comment 63 Thomas Schmitt 2016-12-03 12:41:22 UTC
Hi,

sorry, i did not yet get to installing an Archlinux with CDEmu.
Real life and some other free software obligations ...

cdrskin3.log looks like a full success.
(It becomes more eye-friendly by:  sed -e 's/\r/\n/g' < cdrskin3.log | less )

Does it stem from a cdrskin run underneath K3B ?

If not, then please test whether this cdrskin survives a run under K3B.

--------------------------------------------------------------------------

Whatever, it looks like i'd need to install something similar to your
K3B development environment in order to reproduce the problem with CDEmu.

That's quite some hurdle.
So for now i have to ask you to make the crash experiments.

Interesting would be if you could temporarily hack your K3B so that
we get to see:

- Messages of (patched) cdrskin-1.4.7 under K3B with options -v -V .

- Messages of cdrecord under K3B with options -d -v -V.


cdrecord.log.tar.bz2 did not contain any host_status lines that were
not "00". But unpatched cdrskin succeeded in the same environment and
thus did not experience any other host_status than 0.

The resulting message list is indeed gigantic. I scanned for interesting
lines by
  cat cdrecord.log | grep 'host_status:' | grep -v 'host_status: 00'

--------------------------------------------------------------------------
cdrskin-burn-archlinux-iso-debugging-output4.txt

Hrmpf. CDEmu seems to hate the RESERVE TRACK command. This SCSI command
was triggered by libburn's automatic decision for the behavior of
option -sao because: The DVD+R was blank, no -multi was desired,
the input size was known in advance by option tsize=.

On a real DVD+R in a real drive this should well work.

For now i retract my objections against a hard coded option -tao.
If the user does not explicitely demand SAO (or DAO or Reserve Track),
then it is the safer bet with your kind of "optical drive".

So in
  https://cgit.kde.org/k3b.git/commit/?id=a8074105e842babab18d7adcb290dd6eca3304bb
you will have to revert the first change:

> - d->process << "-tao"/* << "-sao"*/;
> +#ifdef K3B_DEBUG
> + qDebug() << "DEBUG:" << __PRETTY_FUNCTION__ << "let libburn choose "
> + "the write type according to other parameters and the medium state.";
> +#endif

Does the K3B user get an opportunity to explicitely choose SAO/DAO ?
If so, then this should be taken into repsect here.

--------------------------------------------------------------------------
Other half of /commit/?id=a8074105e842babab18d7adcb290dd6eca3304bb :

>  emit infoMessage(i18n("Writer does not support raw writing."), MessageWarning);
> - if (d->cdrskinBinObject->hasFeature("tao"))
> - d->process << "-tao";

A warning is not enough. If the program gets to this point then the input
data will not be written to the CD medium in the way that is expected by
the producer of those input data. Readers will perceive the written medium
as CD-ROM (i.e. computer data) with garbage sprinkled over the payload.
Maybe the burn run will exceed the CD capacity.

The intention of the input producer is rather to influence parts of the
CD sectors which are not read by normal readers as payload but rather
tell the reading drive what kind of payload the CD sectors bear.
Further there are the "sub-channels" which bear a few bytes per sector
for various purposes.

Raw burning is a CD thing. DVD and BD have only sector format -data.
I assume raw is mainly used for cloning of CDs with unusual sector content.

--------------------------------------------------------------------------

Have a nice day :)

Thomas
Comment 64 Leslie Zhai 2016-12-05 02:11:27 UTC
Git commit 1d8b4309999df29f47b440d195eedca424760bb2 by Leslie Zhai.
Committed on 05/12/2016 at 02:03.
Pushed by lesliezhai into branch 'cdrskin'.

Change arguments for Cdrskin Writter

Summary:
- Show more verbose debugging log for K3B_DEBUG;
- Emit MessageError for not support raw writing;
- K3B user get *NO* opportunity via UI to explicitely choose SAO/DAO
- Cdrecord Writter debugging log is tooo big for generation so I just kill
the k3b process

attachment for testplan: cdrskin-burn-archlinux-iso-debugging-output5.txt

CCMAIL: scdbackup@gmx.net

M  +3    -1    libk3b/projects/k3bcdrecordwriter.cpp
M  +6    -3    libk3b/projects/k3bcdrskinwriter.cpp

https://commits.kde.org/k3b/1d8b4309999df29f47b440d195eedca424760bb2
Comment 65 Leslie Zhai 2016-12-05 02:16:57 UTC
Created attachment 102629 [details]
cdrskin-burn-archlinux-iso-debugging-output5.txt
Comment 66 Thomas Schmitt 2016-12-05 08:30:09 UTC
Hi,

> https://bugs.kde.org/attachment.cgi?id=102629
> cdrskin-burn-archlinux-iso-debugging-output5.txt

This failed due to the allergy of CDEmu towards SCSI command RESERVE TRACK.
As mentioned, this is caused by the lack of option -tao and the choice 
of libburn to handle CDEmu like a real drive which holds a DVD+R.

Please add option -tao and repeat the experiment.


Have a nice day :)

Thomas
Comment 67 Thomas Schmitt 2016-12-05 12:37:57 UTC
Hi,

for shorter logs you could try to burn only 100 MB to reduce the number
of logged SCSI commands by about 1/7.

---------------------------------------------------------------------------

We may also for now leave the CDEmu problems undiagnosed and concentrate on
getting the K3B and cdrskin aspects fixed.

The need to add option -tao for "DVD+R" media is a CDEmu problem, too.
I will examine the opportunities to recognize a CDEmu drive with libburn's
automatic write type delection. But that will in any case only be a hack
that is not in effect with currently released cdrskin versions.

Unconditional "-tao" is clumsy and might cause other problems or unfulfilled
expectations of the user.

So you too should consider to detect CDEmu by the Vendor and/or Model
id string of the drive. From the "Devices" lines in
  https://bugsfiles.kde.org/attachment.cgi?id=102515
like


  DEmu Virt. CD/DVD-ROM 1.10 (/dev/sr0, CD-R, CD-RW, CD-ROM, DVD-ROM, DVD-R, DVD+R) [DVD-ROM, DVD+R, CD-ROM, CD-R] [SAO, TAO, RAW, SAO/R96R, RAW/R16, RAW/R96R] [%7]

i expect that the Vendor is "CDEmu" and the Model is "Virt. CD/DVD-ROM".
(You may ask by a run of: cdrskin --devices)

I believe that the Drives lines are composed in
  src/k3bdebuggingoutputfile.cpp
by

  Q_FOREACH( K3b::Device::Device* dev, k3bcore->deviceManager()->allDevices() ) {
     addOutput( "Devices",
                QString( "%1 (%2, %3) [%5] [%6] [%7]" )
                .arg( dev->vendor() + ' ' + dev->description() + ...

So this would be the device object properties to inquire for adding "-tao"
in the case of Device->MediaType values MEDIA_DVD_PLUS_R or
MEDIA_DVD_PLUS_R_DL. (In CdrskinWriter the field is burnedMediaType, i guess.)

Have a nice day :)

Thomas
Comment 68 Leslie Zhai 2016-12-06 02:18:37 UTC
Git commit 702de8d57cebce3d6aefe74fa19f68d70650d0b4 by Leslie Zhai.
Committed on 06/12/2016 at 02:17.
Pushed by lesliezhai into branch 'cdrskin'.

Change arguments for Cdrskin Writter

FAILED: cdrskin-k3b1.log

CCMAIL: scdbackup@gmx.net

M  +6    -3    libk3b/projects/k3bcdrskinwriter.cpp

https://commits.kde.org/k3b/702de8d57cebce3d6aefe74fa19f68d70650d0b4
Comment 69 Leslie Zhai 2016-12-06 02:23:04 UTC
Created attachment 102637 [details]
cdrskin-k3b1.log
Comment 70 Thomas Schmitt 2016-12-06 13:37:22 UTC
Hi,

https://bugs.kde.org/attachment.cgi?id=102637
shows a failure around the same address (70+ MiB) as with cdrskin-1.4.6
but with a different reason. This time the ioctl(SG_IO) returned error
indication and errno says that the device was gone:

> cdrskin: ( Most recent system error: 19  'No such device' )
> --- SG_IO: return= -1 , errno= 19 , host_status= 0x0 , driver_status= 0x0

Maybe you find some messages about this event in your local system log.

------------------------------------------------------------------------

The only other indication of strangeness are diagnostic replies (Sense Data)
from the drive
> WRITE(10)
> 2a 00 00 00 88 40 00 00 10 00
> +++ sense data = 70 00 00 00 00 00 00 0A 00 00 00 00 00 15 00 00 00 00
> +++ key=0  asc=00h  ascq=15h

Key 0 is in SPC-3, table 27 described by the funny name "NO SENSE" and
"Indicates that there is no specific sense key information to be reported".
The error code combination ASC 0, ASCQ 15 is listed in SPC-3, table 28 as
"NO CURRENT AUDIO STATUS TO RETURN".
MMC-5, annex F lists ASC = 0 with ASCQ 00, 06, 16, or 17, but not 15.

------------------------------------------------------------------------

The statistical evidence is sparse. Hard to say whether it is just bad
luck that cdrskin gets hit or whether it teases the CDEmu code in some way.
It is not even clear whether the failures underneath K3B and the success
on the command line is of any significance.

(I once spent two days with trying to find out why libburn failed with
a particular drive and DVD-RW where growisofs succeeded. On the third day
growisofs started to fail and libburn succeeded. It was just a matter of
luck on a flaky combination of hardware.)

------------------------------------------------------------------------

For now i have no other proposal than to keep the CDEmu tests below 70 MB
of burn size and to later test larger sizes with a real DVD burner.

Have a nice day :)

Thomas
Comment 71 Leslie Zhai 2016-12-07 07:17:45 UTC
(In reply to Thomas Schmitt from comment #70)
> 
> (I once spent two days with trying to find out why libburn failed with
> a particular drive and DVD-RW where growisofs succeeded. On the third day
> growisofs started to fail and libburn succeeded. It was just a matter of
> luck on a flaky combination of hardware.)

Hi Thomas,

It is better to use Clang ThreadSanitizer to detect pthread related issue http://www.leetcode.cn/2016/11/analyzing-code-for-kde-qt-open-source-components.html#libburn I am analyzing code for K3b and other components via Clang Static Analyzer and Sanitizer ;-)

Regards,
Leslie Zhai
Comment 72 Thomas Schmitt 2016-12-08 11:58:42 UTC
Hi,

> It is better to use Clang ThreadSanitizer to detect pthread related issue

I currently see no indication that the problem with CDEmu is related to
pthreads. The logs report of refusal to perform RESERVE TRACK and of
connection problems between Linux kernel and CDEmu.
The problem indiations stem from the reply of the kernel to ioctl(SG_IO)
which it transmits in fields of the sg_io_hdr_t (as defined in file
/usr/include/scsi/sg.h), or as errno (as defined in /usr/include/errno.h
and its includelings) if the ioctl() call itself returns -1.

A real DVD burner that is not damaged accepts RESERVE TRACK.
If its cabling and the involved bus controllers are ok, it does not bail out
in the middle of burning.

The refusal of CDEmu on RESERVE TRACK probably stems from
  https://sourceforge.net/p/cdemu/code/ci/master/tree/cdemu-daemon/src/device-commands.c
function command_reserve_track in line 2199.
One would have to find possible reasons for the two refusals with
  cdemu_device_write_sense(self, ILLEGAL_REQUEST, COMMAND_SEQUENCE_ERROR);
in order to learn how to avoid them.

About the reply of sg_io_hdr_t.host_status == 1 and the errno == 19
i cannot say much. That's deeper inside the kernel's device driver
architecture than i ever had to climb down.

As said, i cannot exclude the possibility that some of libburn's activities
cause these problems with CDEmu. But i have no such experiences with my
own drives and got no such reports from other users of real drives.


Have a nice day :)

Thomas
Comment 73 Leslie Zhai 2016-12-09 01:37:37 UTC
(In reply to Thomas Schmitt from comment #72)
> Hi,
> 
> > It is better to use Clang ThreadSanitizer to detect pthread related issue
> 
> I currently see no indication that the problem with CDEmu is related to
> pthreads.

Never mind ;-) it might be Clang's issue ;-P such as:
https://llvm.org/bugs/show_bug.cgi?id=31304
https://llvm.org/bugs/show_bug.cgi?id=28331
https://llvm.org/bugs/show_bug.cgi?id=15836
https://llvm.org/bugs/show_bug.cgi?id=31017


> The refusal of CDEmu on RESERVE TRACK probably stems from
>  
> https://sourceforge.net/p/cdemu/code/ci/master/tree/cdemu-daemon/src/device-
> commands.c
> function command_reserve_track in line 2199.
> One would have to find possible reasons for the two refusals with
>   cdemu_device_write_sense(self, ILLEGAL_REQUEST, COMMAND_SEQUENCE_ERROR);
> in order to learn how to avoid them.
> 
> About the reply of sg_io_hdr_t.host_status == 1 and the errno == 19
> i cannot say much. That's deeper inside the kernel's device driver
> architecture than i ever had to climb down.
> 
> As said, i cannot exclude the possibility that some of libburn's activities
> cause these problems with CDEmu. But i have no such experiences with my
> own drives and got no such reports from other users of real drives.

I will report bug to CDEmu ;-)

Regards,
Leslie Zhai
Comment 74 Leslie Zhai 2017-05-02 14:49:17 UTC
Hi Thomas,

Rok had fixed it https://sourceforge.net/p/cdemu/bugs/101/
Let's try it :)

Regards,
Leslie Zhai
Comment 75 Thomas Schmitt 2017-05-02 15:05:17 UTC
Hi,

Leslie wrote:
> Rok had fixed it https://sourceforge.net/p/cdemu/bugs/101/

Oh. Now it is probably too tolerant. :))
(RESERVE TRACK may not be used with all media types. According to MMC-5
 it is related to CD, DVD+R, DVD-R, DVD+R DL, HD DVD, BD-R, and
 unformatted DVD-RW.
 So with DVD+RW, DVD-RAM, BD-RE, formatted CD-RW, formatted DVD-RW,
 and BD-R in mode RRM a real drive will probably throw an error.)

Whatever, the diagnosis confirms my suspicion that not libburn was to
blame in this case. Not this time.


Have a nice day :)

Thomas