Bug 257602

Summary: K3B cannot burn Blurays (or AVCHDs)
Product: [Applications] k3b Reporter: Dkottmair <dkottmair>
Component: generalAssignee: k3b developers <k3b>
Status: CONFIRMED ---    
Severity: wishlist CC: ashaduri, Bugs.kde.org, bugseforuns, caravena, dev.dliw, king_duckz, koutheir, ma9ick, mark.perino, scdbackup, simonandric5, trueg, zhaixiang
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: Unlisted Binaries   
OS: Linux   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description Dkottmair 2010-11-22 15:04:21 UTC
Version:           unspecified
OS:                Linux

Supposedly according to the changelog, K3B (and other OSS burning tools) can burn Blurays. But this is wrong, it's semantical trickery, since all they can do is burn data onto BD-R media with BD-Burners. You simply can NOT create an actual Bluray-Videodisc (or AVCHD for that matter!), because no OSS burning solution supports *creating* UDF 2.5 (or 2.6), which is a requirement for Bluray/AVCHD! mkudffs, which they all rely on, hasn't been updated for years, and it only supports UDF2.01 max! And the UDF 2.5 spec is from 2003, that's 7 years ago!

The commercial Nero 4 Linux can do it, but that's the only tool under Linux that can... I think it's simply just wrong if you can cut your HD-movies with Kdenlive or render them with Blender, can encode it BD/AVCHD-compatible with x264, but you cannot burn them so you can watch it on a Bluray player!

(okay, you also need TSmuxer for muxing into the Bluray/AVCHD filestructure, which also isn't OSS - but at least it's free, unlike Nero!)

Reproducible: Always

Steps to Reproduce:
Drag TSmuxer generated BDMV/CERTIFICATE into K3B, select UDF, press burn

Actual Results:  
A disk with UDF 2.01 that will not play in a Bluray player

Expected Results:  
A disk with UDF 2.5 that plays in a Bluray player

K3B also can't handle ISOs generated with UDF 2.5 in Nero, so you cannot even distribute ready-made Bluray/AVCHD-ISOs for burning!
Comment 1 angelgs 2011-12-29 00:30:39 UTC
*** This bug has been confirmed by popular vote. ***
Comment 2 mark.perino 2013-05-16 01:04:34 UTC
This is important because the only other software for burning UDF2.5+ is no longer available.
Comment 3 Bugs.kde.org 2015-08-29 08:50:01 UTC
Are there any news regarding this topic?
Comment 4 King_DuckZ 2016-01-05 16:10:18 UTC
I'm also interested in a fix for this bug. Is anybody going to look into it anytime soon? Any idea at all?
Comment 5 Leslie Zhai 2016-09-06 04:02:03 UTC
Hi, 

UDF v2.5.x is OK https://github.com/torvalds/linux/blob/master/fs/udf/osta_udf.h#L4
but I have *NO* idea about supporting v2.6.x https://en.wikipedia.org/wiki/Universal_Disk_Format

Perhaps we can deep into it, and try to implement for Linux kernel ;-)

Regards,
Leslie Zhai
Comment 6 dev.dliw 2016-09-06 08:13:54 UTC
AFAIK Linux can only *read* UDF version 2.5.x, writing is limited to UDF 2.01 and lower.
(https://en.wikipedia.org/wiki/Universal_Disk_Format#Compatibility)
Comment 7 Leslie Zhai 2016-09-06 08:21:48 UTC
I have not read linux/fs/udf carefully, but if so limited support, it is a bad news ;-(
Comment 8 Leslie Zhai 2016-09-06 08:32:59 UTC
*** Bug 344392 has been marked as a duplicate of this bug. ***
Comment 9 Thomas Schmitt 2016-09-06 10:56:36 UTC
The main problem is that google does not find any free tool to master
a UDF filesystem for Blu-ray. Burning the filesystem to media would
then be easy.
Aren't any Blu-ray players out there which play video files from other 
filesystems like ISO 9660 ? (If there are: Boycott the others.)

UDF specs are available for free as PDFs of ECMA-167 plus UDF-2.60.
But they are written in an unappealing way. So it would need some sincere
bribery to get this done. (One or two picky Blu-ray players, a few
standards-conformant commercial Blu-ray videos, a variety of addictive
drugs to keep the programmer from running away.)
It might also cost money to get official specs for Blu-ray video which
might name more constraints beyond UDF.
(I would not reject a contribution to libisofs. But integrating and testing
it will not be a picnic.)

--------------------------------------------------------------------

I doubt that https://bugs.kde.org/show_bug.cgi?id=344392 is a duplicate
of this bug report here.
Here we bemoan the lack of a Blu-ray video mastering tool.
344392 is about the suspicion that K3B insists in particular image formats
like ISO 9660 and rejects UDF.
Comment 10 Leslie Zhai 2016-09-07 01:56:07 UTC
Hi Dkottmair,

> Actual Results:  A disk with UDF 2.01 that will not play in a Bluray player

I do *NOT* have Blu-ray disc and players to reproduce the WRITE issue, and I have *NOT* carefully read about linux/fs/udf source code for current Linux kernel.

Hi Thomas,

> Burning the filesystem to media would then be easy.

As Dkottmair described, successfully burn the UDF to media, but is *NOT* v2.5 nor v2.6, that will not play in a Bluray player

> But they are written in an unappealing way. So it would need some sincere bribery to get this done.

So it is unfriendly to open source world?

> It might also cost money to get official specs for Blu-ray video which might name more constraints beyond UDF

my test enviroment is very very poor:
* plugable ASUS SDRW-08D2S-U https://pbs.twimg.com/media/CrT91GfVUAAyX9Z.jpg
* Audio CD and Video DVD for rippint test https://pbs.twimg.com/media/CrT91x6UAAA5Hmd.jpg
* CDEmu http://cdemu.sourceforge.net/about/libmirage/

then how to test Blu-ray and multisession for libburn project? by implementing some plugins for CDEmu?

Regards,
Leslie Zhai
Comment 11 Leslie Zhai 2017-06-12 01:53:52 UTC
*** Bug 381083 has been marked as a duplicate of this bug. ***
Comment 12 King_DuckZ 2018-02-03 16:38:38 UTC
Is there anything that can be done to help adding this feature? If there was some bounty to help towards buying whatever equipment developers need, I'd be happy to contribute to it.

As of now, running ImgBurn in Wine seems to be the only option, and unfortunately it's sketchy at best. Given the bluray price, a program that has a 50% success rate is not ideal.

I also found this https://github.com/pali/udftools and this http://devs.dhgirault.fr/article26.html hopefully it's useful stuff.
Comment 13 Thomas Schmitt 2018-02-03 18:29:06 UTC
Hi,

Dkottmair, 2010:
> K3B also can't handle ISOs generated with UDF 2.5 in Nero,
> so you cannot even distribute ready-made Bluray/AVCHD-ISOs for burning!

This should now be possible after Leslie Zhai's changes in the course
of our assessment of Bug 344392.

But some ornaments are still missing for IMAGE_RAW. See
  https://bugs.kde.org/show_bug.cgi?id=387765#c37
  https://bugs.kde.org/show_bug.cgi?id=387765#c38

King_DuckZ 2018:
> Is there anything that can be done to help adding this feature?

Motivationally: Ketracel-white or WWII Panzerschokolade.

Technically a standalone Blu-ray recorder/player and some Blu-ray movies
would help with getting valid examples and testing own results. UDF specs
are for free (ECMA-167 + UDF-2.50) but hope for success is only with
comprehensive examples.

Leslie generally could need a (USB connected ?) computer BD burner and
CD-RW, DVD-RW, DVD+RW, DVD-R, BD-R, BD-RE media for working on K3B bug
reports of all kind.
But that won't help much with the Blu-ray video authoring issue.

> https://github.com/pali/udftools

That's the UDF userland software about which the bug reporter complained
in 2010. Meanwhile quite obsoleted by the UDF read-write driver in the
kernel. (I think CD-RW formatting is still a valid job.)

> http://devs.dhgirault.fr/article26.html

5 years ago this would have been the man to bribe, drug, and equip with
examples and players. He roughly describes the plan to beef up mkisofs
clone genisoimage to produce UDF 2.50 like it does with UDF 1.02.

He seems to underestimate the other part of the job, namely to do for
Blu-ray what dvdauthor does for DVD. Compare
  https://en.wikipedia.org/wiki/Blu-ray#Data_format_standards
with
  https://en.wikipedia.org/wiki/DVD-Video#File_system
to get an impression of the differences.

Have a nice day :)

Thomas
Comment 14 King_DuckZ 2018-02-03 19:52:52 UTC
That sounds fair, if you guys have an account on librepay or bountyhunter let me know and I'll do what I can.

Speaking of bribing and drugging, is there anything at all we can do, or is it even useful, to ask the dude behind ImgBurn for help?

Last information I can give, the LG WH16NS40 is working very well for me. It's an external USB3 drive so it can be plugged and unplugged easily.
Comment 15 dev.dliw 2018-02-03 20:18:01 UTC
>He seems to underestimate the other part of the job, namely to do for
>Blu-ray what dvdauthor does for DVD.
If you want to create Bluray movie disks, this is a special case IMO.

The real problem is, that Linux still is unable to create UDF 2.50 or UDF 2.60 images:
https://github.com/torvalds/linux/blob/master/fs/udf/udf_sb.h#L10


This is also bad for plain Bluray data disks. AFAIK especially for BD-RE these higher UDF versions are essential for a long lifetime. The only option on Linux still is the Linux version of Nero (latest version from end of 2010).

If there's anyone who wants to tackle this, I'm very interested and also would support a bounty.
Comment 16 Thomas Schmitt 2018-02-03 20:42:44 UTC
Hi,

> That sounds fair, if you guys have an account on librepay or bountyhunter
> let me know and I'll do what I can.

I myself am not hyngry enough for UDF.
As said, the specification is public:
  http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-167.pdf
  http://www.osta.org/specs/pdf/udf250.pdf

What may be missing is exact info about
  https://en.wikipedia.org/wiki/Blu-ray#Data_format_standards
beyond the demand for UDF 2.50.

> ask the dude behind ImgBurn for help?

Best would be if you could talk him into releasing a Linux version.

> Last information I can give, the LG WH16NS40 is working very well for me.
> It's an external USB3 drive so it can be plugged and unplugged easily.

The K3B project could well need such a thing stationed at Leslie's home.

My own burner zoo is large enough and partly ill enough for the purpose
of burn backend development.

Have a nice day :)

Thomas
Comment 17 Thomas Schmitt 2018-02-03 20:58:36 UTC
Hi,

please excuse my clear language in this case:

dev.dliw@gmail.com wrote:
> The real problem is, that Linux still is unable to create UDF 2.50 or
> UDF 2.60 images:

When it comes to data storage on optical media, UDF has no practical
advantage over ISO 9660 with Rock Ridge, zisofs, and AAIP, except maybe
that you can circumvent the 25 year old ISO 9660 drivers of the BSD Unixes.

It is really only about DVD video and BD video. No other reason to use UDF,
unless you need to read large files by BSD Unix and don't want to extract
them by userland programs.

> This is also bad for plain Bluray data disks. AFAIK especially for BD-RE 
> these higher UDF versions are essential for a long lifetime.

Any use of BD-RE which is not mainly sequential will soon kill media and/or 
drive. The whole promise of Defect Management to replace overworked blocks 
simply does not yield good results in practice.
Keep read-write filesystem drivers away from BD-RE. Use a filesystem
image formatter and a burn program, like K3B does. If you do so, then it
does not matter to the medium what kind of filesystem the image contains.

Have a nice day :)

Thomas
Comment 18 King_DuckZ 2018-02-03 21:23:24 UTC
> The real problem is, that Linux still is unable to create UDF 2.50 or UDF 2.60 images

But is that relevant to Bluray burning? Because I'm literally writing an UDF 2.5 bluray disk as we speak, through ImgBurn/wine. Windows app or not, it's still a userland program running on kernel 4.4 (I guess wine is the actual process here).
Also, that project I linked earlier, udftools, they claim to have added udf 2.5 support to it.

For clarity, it's video blurays that I care about.

Btw, idk if I should blame wine, ImgBurn or my mediums but so far my success rate has been 1 out of 6 - I just counted the coasters in my bin.

> Best would be if you could talk him into releasing a Linux version.

I can see if I can get in touch and point him to this thread, but I know there have been several posts on his forum about making a linux port and he's always answered 'use wine'. I think even the Windows version is dead, last version I can see on his website is dated June 2013.
Comment 19 Thomas Schmitt 2018-02-03 21:51:23 UTC
Hi,

King_DuckZ wrote:
> Because I'm literally writing an UDF
> 2.5 bluray disk as we speak, through ImgBurn/wine. Windows app or not, it's
> still a userland program running on kernel 4.4

dev.dliw@gmail.com pointed to the filesystem driver of Linux for UDF.
ImgBurn on Wine most probably has its own UDF filesystem formatter, if it
is not using some MS-Windows system driver provided by Wine for that
purpose.

> udftools, they claim to have added udf 2.5 support to it.
> (https://github.com/pali/udftools)

There seems indeed to be life in there. Sorry for not looking
close enough on the first glimpse.

You should try to bribe pali so he helps you working towards a Linux
processing chain for Blu-ray video.

If you can show a description what input files to expect, what extra info
to inquire from the user, and how to process this up to a burnable image,
then you could ask K3B to give this procedure a nice user interface.

But with that processing chain alone, you would already be Linux heroes.

> Btw, idk if I should blame wine, ImgBurn or my mediums but so far my success
> rate has been 1 out of 6 - I just counted the coasters in my bin.

By what program on which operating system do you actually burn the BD medium ?
What are the symptoms of failure ?
Burn abort ? Not mountable afterwards ? Not playing in video player ?

Have a nice day :)

Thomas
Comment 20 King_DuckZ 2018-02-04 02:33:00 UTC
I can open an issue on their github, but just to be sure we're talking about the same thing: I already have a directory containing a BDMV subtree (and CERTIFICATE). I don't know if anything needs to be modified during the burning process, but as far as I'm concerned, I'm just asking to burn that directory onto a bluray that will work correctly on tv players. As things are now, I can already burn everything using K3B and UDF 2.01, but tv players will not recognise the bluray as a video one. Likely because of rule n. 1 here: http://www.blu-raydisc.com/en/Industry/Specifications/PublicSpecs.aspx

So as I see it as a user, K3B should just do the same thing as ImgBurn: you select the directories you want to burn, ImgBurn guesses from the tree structure that it's a video disk and asks if you want to use UDF 2.5 and make a video disk indeed. Then you press ok and wait for it to finish.

If you're talking about the actual creation of bluray videos from your own .mkv files for example, then I think that's going to be much more complicated. However, it's not what I'm asking for.

Either way, I found a mildly interesting forum thread here http://forum.doom9.org/archive/index.php/t-141361.html detailing the binary file formats (see frank's post). As I understand it, the burning program might have to edit some files before burning them, depending on the medium type. This is all new to me, but on another forum they refer to the same post and add this comment: "So depending on whether you choose "Remux to AVCHD (FAT32/PS3 compatability mode)" or Blu-Ray that headers will be corrected. Currently it's a 1/2 BD 1/2 AVCHD by tsMuxeR (see posting above with highlighting)".

So if that's the only thing the burning software has to do, and the rest is just a regular UDF filesystem that udftools can create, then all the components are there, or am I misunderstanding things?
Comment 21 Leslie Zhai 2018-02-04 05:54:44 UTC
(In reply to King_DuckZ from comment #12)
> Is there anything that can be done to help adding this feature? If there was
> some bounty to help towards buying whatever equipment developers need, I'd
> be happy to contribute to it.
> 
> As of now, running ImgBurn in Wine seems to be the only option, and
> unfortunately it's sketchy at best. Given the bluray price, a program that
> has a 50% success rate is not ideal.
> 
> I also found this https://github.com/pali/udftools and this
> http://devs.dhgirault.fr/article26.html hopefully it's useful stuff.

Hi King,

Thank you so much! But I argue that I don't need real equipment, CDEmu[1] is enough to reproduce general issues or debug for supporting general features, so I prefer K3B users build k3b from github[2] by themselves, and provide screenshot, debug log, or gdb back trace info.

Hi Thomas,

Thanks for your teaching! Yes, K3B is just the frontend, it is just like Compiler Technology, the MinddleEnd and BackEnd are much more important than FrontEnd, so please continue leading me to fix relative issues :)

[1] http://cdemu.sourceforge.net/
[2] https://github.com/kde/k3b

Regards,
Leslie Zhai
GCC developer https://github.com/loongson-community/gcc/
LLVM developer https://reviews.llvm.org/p/xiangzhai/
Comment 22 Thomas Schmitt 2018-02-04 08:42:20 UTC
Hi,

> I already have a directory containing a BDMV subtree (and CERTIFICATE).

I can only google for these terms to learn what they mean.
But in general this looks like a good start for exercising the last
steps of Blu-ray video production.
(When this part is achieved, there will arise the question how to make
 that subtree from own video files which are given in some common video
 codec's format.)

I have read rumors about DVD video that mkisofs -udf has to take care
not only to produce a valid UDF filesystem but also has to obey some
further layout prescriptions.
See in man genisoimage (or man mkisofs) the description of option -dvd-video.
It has two paragraphs. The first talks briefly of that layout, the second
mentions DVD-specific input files like VIDEO_TS. The equivalent of this
second part is hopefully covered by your BDMV tree.

But the first part will simply have to be tested. If it does not work
with a picky test player, then one will have to compare the byte-level
layout of a commercial Blu-ray video medium with the own UDF-2.50 result.

Burning to BD media is not a hard job then. Recent K3B should do. We know
from Bug 387765 that Archlinux already has it. (It warns of unknown image
format. After confirming, it burned a Nero made UDF-only image.)
If there are difficulties to get such a recent K3B, use the command line:

  growisofs -dvd-compat -Z /dev/sr0=image.udf
or
  cdrskin -v dev=/dev/sr0 fs=32m -eject image.udf

I'd use growisofs option -dvd-compat to get BD-R media closed. At lest
with DVD there are hints that appendable media won't work on all players.
(With BD-RE there is no closing on hardware level.)

For full nominal writing speed use

  cdrskin -v dev=/dev/sr0 fs=32m -eject stream_recording=on image.udf

This disables the immediate checkreding by Defect Management which slows
down writing by a factor 2 or 3.
On flaky media or drives, you paradoxically get better chances of success
if you do _not_ let the drive to this check-and-replace game.

> So as I see it as a user, K3B should just do the same thing as ImgBurn: you
> select the directories you want to burn, ImgBurn guesses from the tree
> structure that it's a video disk and asks if you want to use UDF 2.5 and
> make a video disk indeed. Then you press ok and wait for it to finish.

Well, that would rather be Leslie's desk. :))
He will probably need:
- The description of the command line program runs needed to produce
  the Blu-ray video disc image.
- Example input data (e.g. some commercial Blu-ray video discs)
- A computer Blu-ray burner.
- Testers who then verify that K3B indeed produces BDs which are
  playable on all Blu-ray video players.


Leslie wrote:
> But I argue that I don't need real equipment, CDEmu[1] is enough

I dare to contradict. A K3B developer should have a BD burner and
test media.

I as burn backend developer have 3 BD burners and 3 DVD burners.
Just to be able to distinguish individual burner flaws from general problems.
(It also helps to have slightly ill burners which challenge the program's
 error handling.)
A frontend programmer would only need 1 well working burner, although
a second one will help to check program behavior when multiple burners
are available in the system.

Have a nice day :)

Thomas
Comment 23 Thomas Schmitt 2018-02-04 10:07:02 UTC
Hi,

there is still the riddle why ImgBurn on Wine produces mostly failing
results but sometimes succeeds.

An the first glimpse this does not look much like a filesystem format
problem but more likely it is a problem between burner and medium,
or between ImgBurn, Wine, and Linux if you burn directly from ImgBurn.

So independently of the endeavor to produce BD video UDF on Linux,
King_DuckZ should try to let ImgBurn write to a file image instead
of the burner directly (if that is what is done currently).
Then use Linux command line burn programs to burn the medium 

  growisofs -dvd-compat -Z /dev/sr0=image.udf
or
  cdrskin -v dev=/dev/sr0 fs=32m -eject image.udf

and use Linux command "mount" to check whether the video files are 
readable afterwards. E.g. copy them to hard disk and watch for error 
messages.

If there are problems at burn time or if there are problems to mount the
BD and read the file, then i will try to diagnose them.
Send mail to: bug-xorriso@gnu.org
(growisofs is orphaned, cdrskin is made by me, as is xorriso.)

Please report any program messages of failed burn, failed "mount", or
failed read. With "mount" and "read", the output of command "dmesg"
will be of interest.
(Check for private info in there and remove it before posting publicly.)

Have a nice day :)

Thomas
Comment 24 King_DuckZ 2018-02-04 17:04:29 UTC
> there is still the riddle why ImgBurn on Wine produces mostly failing
> results but sometimes succeeds.
> 
> An the first glimpse this does not look much like a filesystem format
> problem but more likely it is a problem between burner and medium,
> or between ImgBurn, Wine, and Linux if you burn directly from ImgBurn.

The error ImgBurn gives me is something like "invalid address for write". Other users receiving that error were told it's because of the medium quality, however I doubt that 5 out of 6 50GB Panasonics are so bad that can't be written, that's more like a scam than just bad quality. Success rate with Verbatim is also 1 out of 2 tries, so I don't know what kind of blurays one would have to buy.

> So independently of the endeavor to produce BD video UDF on Linux,
> King_DuckZ should try to let ImgBurn write to a file image instead
> of the burner directly (if that is what is done currently).
> Then use Linux command line burn programs to burn the medium 
> 
>   growisofs -dvd-compat -Z /dev/sr0=image.udf
> or
>   cdrskin -v dev=/dev/sr0 fs=32m -eject image.udf

I followed your advice, I made a .iso with ImgBurn and burned it with your cdrskin command, on gentoo, kernel 4.12.12 with libburn-1.4.8-r1. The resulting bluray is unreadable on PS3 and on my computer, so another wasted disk. cdrskin output: http://paste.debian.net/1008790/

> and use Linux command "mount" to check whether the video files are 
> readable afterwards. E.g. copy them to hard disk and watch for error 
> messages.

I can mount the iso and navigate it just fine with mount -o loop,ro

> Please report any program messages of failed burn, failed "mount", or
> failed read. With "mount" and "read", the output of command "dmesg"
> will be of interest.
> (Check for private info in there and remove it before posting publicly.)

mount /dev/sr0 /mnt/cdrom/
mount: /mnt/cdrom: wrong fs type, bad option, bad superblock on /dev/sr0, missing codepage or helper program, or other error.

dmesg[ 6079.900117] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: (null)
[ 6599.454520] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: (null)
[ 6599.644744] UDF-fs: warning (device loop0): udf_load_vrs: No anchor found
[ 6599.644746] UDF-fs: Scanning with blocksize 512 failed
[ 6599.644834] UDF-fs: warning (device loop0): udf_load_vrs: No anchor found
[ 6599.644835] UDF-fs: Scanning with blocksize 1024 failed
[ 6599.644909] UDF-fs: INFO Mounting volume 'SS_LC_S1_D1', timestamp 2018/02/04 12:58 (1000)
Comment 25 King_DuckZ 2018-02-04 17:12:03 UTC
I forgot to specify I ran ImgBurn 2.5.8 with Wine 3.0 on Linux Mint, kernel 4.4. I don't have Windows so I can't test ImgBurn natively. Blurays successfully written with ImgBurn directly can be viewed on a PlayStation3 just fine.

ImgBurn also failed at writing 1 single layer DVD out of 2, Verbatim again. Same error message, same setup. I usually don't get failures when I do DVDs from K3B with the same LG drive and discs, so go figure, it might be a bug in wine or ImgBurn.

cdrskin --version
cdrskin 1.4.8 : limited cdrecord compatibility wrapper for libburn
Cdrecord 2.01a27 Emulation. Copyright (C) 2006-2016, see libburnia-project.org
System adapter    :  internal GNU/Linux SG_IO adapter sg-linux
libburn interface :  1.4.8
libburn in use    :  1.4.8
cdrskin version   :  1.4.8
Version timestamp :  2017.09.12.110001
Build timestamp   :  -none-given-
Comment 26 Thomas Schmitt 2018-02-04 18:58:14 UTC
Hi,

> The error ImgBurn gives me is something like "invalid address for write".
> Other users receiving that error were told it's because of the medium
> quality,

It's probably a miscoordination between burn program and drive.
There is SCSI error code
  5 21 02 INVALID ADDRESS FOR WRITE

Each SCSI WRITE command contains the block address where the write
operation shall start and the number of blocks which shall be written
with the payload bytes of the SCSI command.
If the drive replies "invalid address", then either one of the affected
blocks was out of the range of writable blocks on a BD-RE or a BD-R which
was formatted to "Pseudo Overwrite" (POW), or the start block address
was not the Next Writable address of a BD-R which was not formatted to POW.

> 50GB Panasonics

So probably double-layer BD-R. Unlike with DVD+R DL, we burn programmers
have no influence on the hopping between the layers. That's rather good,
as with DVD+R DL we had some problem reports with any of the possible
modes of operation.

My best guess would be that the passing-through of SCSI commands to the
burner is prone to disturbances or hick-ups.


> http://paste.debian.net/1008790/

No protests from the drive to see. It took nearly 50 GiB.

The only suspicious message is
> cdrfifo 0: on read: errno=22 , "Invalid argument"
This was when reading the input file into the ring buffer.
And it happened before writing began. But it did not prevent writing
... which i cannot reproduce yet.
All my tries with simulated errors abort early.

My apologies for the medium waste, anyway.

errno 22 is not a usual read error. man 2 read mentions O_DIRECT.
Maybe the distro configured libburn with non-default
  --enable-track-src-odirect
and your disk has physical blocks larger than 2048.
But Debian and therefore probably Mint has in the buildd log of libburn:
  disabled use of O_DIRECT with track input

If it's O_DIRECT, then a run with growisofs might encounter the same
problem. If i understand its source right, then it uses O_DIRECT
by default if Linux is younger than 2.5.

--------------------------------------------------------------------------

What was the exact cdrskin command you used and what do you get from
  ls -ld $path
with $path leading to the image file ?

--------------------------------------------------------------------------

What happens if you do (without risking a BD-R medium)

  cdrskin --allow_emulated_drives -sao -v dev=stdio:/dev/null fs=32m -eject $path

Does it report again
  cdrfifo 0: on read: errno=22 , "Invalid argument"

If so, then we need to build cdrskin from source with surely O_DIRECT
disabled.

--------------------------------------------------------------------------

What does a hex editor say about your burned BD-R ?
Does it show any non-zero bytes after maybe some first MBs of zeros ?

--------------------------------------------------------------------------

Have a nice day :)

Thomas
Comment 27 Thomas Schmitt 2018-02-04 19:28:53 UTC
Hi,

grrr. The waste of your BD-R is to blame on libburn's automatic decision
to handle this burn under the SAO/DAO model: Known track size, as much
write preparations in advance as possible.
After the preparations are done, it makes no sense to abort the burn.
With CD media the burner would get stuck. With BD-R the track would
already be reserved but not written.

The medium whould have been saved by the explicit cdrskin option -tao.
In this case no writing would have happened if the error already occurs
at the first read attempt on the input file.
Even better would have been option -waiti with data comming from stdin:
  cdrskin -v dev=/dev/sr0 fs=32m -eject -waiti - <image.udf
which would not have touched the drive at all, if no data were available.

Sorry again. I should have proposed the most rugged variant.

My question about the hex editor is obsolete now.
I see in the cdrskin log the message that no input bytes were read:
  Track 01: Total bytes read/written: 0/49199448064 (24023168 sectors).

But the reason for the error 22 on read(2) has still to be found.

Have a nice day :)

Thomas
Comment 28 Thomas Schmitt 2018-02-04 19:42:48 UTC
Hi,

if the O_DIRECT theory is correct, then this should work without errno 22

  cdrskin --allow_emulated_drives -v dev=stdio:/dev/null fs=32m -eject \
          -waiti - <$path

and report as many read bytes as written bytes

  Track 01: Total bytes read/written: 49199448064/49199448064 (24023168 sectors).

If this works as expected, then you could risk another BD-R medium and do

  cdrskin -v dev=/dev/sr0 fs=32m -eject -waiti - <$path

In this case i need to know from where you got libburn.so.
It might be necessary that the upstream programmer talks to the distro
packager.

Have a nice day :)

Thomas
Comment 29 Thomas Schmitt 2018-02-05 09:31:23 UTC
Hi,

i can reproduce the cdrskin problem by configuring libburn with option
  --enable-track-src-odirect

The bug is a regression from end of 2009 by release 0.7.4, when cdrskin
forgot that it must prepare special buffer memory by mmap(2) when libburn
is configured to do O_DIRECT.

Many thanks to King_DuckZ for reporting it.
But i still need to know from where the cdrskin binary stems and, if it's
not the standalone version, from where libburn.so (aka libburn4.so) stems.

The proposed workaround really avoids the problem here:

  cdrskin -v dev=/dev/sr0 fs=32m -eject -waiti - <image.udf

(Tested cowardly with a BD-RE medium. I still owe you a BD-R medium.)

--------------------------------------------------------------------------

growisofs does not show the problem, because its memory preparations suffice.
(Reading articles about O_DIRECT raises concerns about whether these
 preparations will always succeed.)

xorriso does not show the problem because it uses the built-in fifo of
libburn, which did not yet exist when cdrskin was founded in 2006.
This fifo knows how to prepare its buffer for O_DIRECT.

Nevertheless, the situation is tricky. libburn and its applications could
by several ways get out of sync in this aspect.
So now i am pondering whether to completely drop the opportunity in libburn
to use O_DIRECT.

Have a nice day :)

Thomas
Comment 30 King_DuckZ 2018-02-11 18:36:32 UTC
> In this case i need to know from where you got libburn.so.
> It might be necessary that the upstream programmer talks to the distro packager.

My distro is Gentoo, info about the package:

$ emerge -pv dev-libs/libburn

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] dev-libs/libburn-1.4.8-r1::gentoo  USE="track-src-odirect -debug -static-libs" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB


> What does a hex editor say about your burned BD-R ?
> Does it show any non-zero bytes after maybe some first MBs of zeros ?

# head -c 3145728 /dev/sr0 | hexdump -C                                                                                                                                        
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|                                                                                                               
*                                                                                                                                                                                            
00300000

I can post the failed bluray to a physical address, if it's of any help.
Comment 31 Thomas Schmitt 2018-02-11 19:05:24 UTC
Hi,

> [ebuild   R    ] dev-libs/libburn-1.4.8-r1::gentoo  USE="track-src-odirect -debug -static-libs" 0 KiB

Here we have the trigger which activates my mistake of 2009.
I wonder why nobody noticed the problem on Gentoo.

Regrettably packages.gentoo.org yields error 500 today.
But i will later try to talk the package maintainer out of option
"track-src-odirect" and to find out since when cdrskin is broken on Gentoo.

> # head -c 3145728 /dev/sr0 | hexdump

Yes it's clear meanwhile that only zeros were written. Usage model "SAO":
Once started, do not stop before the planned end of the burn is reached.

I still ponder how to close this pitfall. There are pros and cons 
to weigh about the model of SAO, which cannot be aborted early, and TAO 
which can be aborted until the first WRITE command was sent to the drive.

--------------------------------------------------------------------

As said, for me the following works even if the program was built
with configuration option --enable-track-src-odirect :

  cdrskin -v dev=/dev/sr0 fs=32m -eject -waiti - <image.udf

Have a nice day :)

Thomas
Comment 32 Thomas Schmitt 2018-02-12 21:20:51 UTC
Hi,

i got feedback from Daniel Pielmeier, the Gentoo maintainer of libburn.
He fulfilled my wish to remove the problematic configuration option by:
  https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=55b96c727287728374a11de48028112417dbded5
Many thanks to him for working around my bug.

On my question
> > What would King_DuckZ have to do to get it on his machine ?
he answered
> He just needs to wait a few hours until the change propagates to the
> mirrors. After syncing with "emaint --sync" he should get the changes 
> by calling "emerge libburn".

Have a nice day :)

Thomas
Comment 33 King_DuckZ 2023-11-06 10:15:40 UTC
Any hope to see some progress on this?
Comment 34 mrmikee 2023-11-06 10:18:47 UTC
NOTICE: I do not monitor this account, this is for junk mail only. Soon I will delete this account.  Seller beware!!!