In certain earlier postings, I wrote about the possibility of burning Blu-rays using the UDF File System, but using the GUI-application ‘K3b’, under Linux. One big problem with this approach was, that the version of UDF being applied was only 1.02, and that it was part of an ISO9660 / UDF bridge-format, which the earliest ISO9660-capable devices are able to read because of the ISO9660 backwards-compatibility, but which would present some of the error-correction capabilities that UDF is supposed to offer, to more-modern devices, via UDF 1.02 .
The issue with this would be, that UDF 1.02 may still not be robust enough, in its error-correction, and that we wish to burn UDF 2.01 File Systems, using open-source software, under Linux.
(Edit 10/22/2017 :
Actually, my recent findings seemed to suggest, that if we use ‘cdrecord’ to burn a Data-Disk Project, the UDF version it applies may already exceed that standard. )
This posting will describe how such a File System can be written using a script, which I have tested myself, but which does not offer any type of GUI at all.
First of all, in order for this to work, our Linux computer needs a reasonably recent Kernel version, because what the script will do is create an empty UDF File System, and then mount that as a loop-device, so that the same script can batch-copy the contents of the Present Working Directory, (the PWD, as it’s called under Linux,) to this File-System, after which the FS is unmounted, and burned onto a Disk.
If our Kernel-version is not recent, then to mount and/or to batch-copy may fail, and for no other reason. I’ve tested this to work using Kernel version ‘4.4.0-30-generic’, which is by far not the standard Kernel-version that my Debian / Jessie repositories would offer.
Another prerequisite is, the package called:
Which we would use to create the empty File System, but which we won’t be using to do anything other than that. ‘udftools’ can be installed using our standard Debian package-manager, and offers support up to UDF 2.01 as its maximum. This package also contains the commands for growing a UDF File System, ‘wrudf’, but because I did not trust the package-version of UDF being offered, I chose not to use it. ‘wrudf’ is supposed to work somewhat like the ‘growisofs’ command would work, within a GUI-application such as K3b, but K3b does not recognize it out-of-the-box. In fact, I do not envy anybody, who needs to use ‘wrudf’ to grow their UDF File System.
This is the script which I have tested:
#!/bin/bash if [ ! -e "/dev/sr1" ] ; then exit 1 fi if [ ! -x "/usr/bin/mkudffs" ] ; then exit 1 fi if [[ $UID -ne 0 ]] ; then echo "This script must be run as root" 1>&2 exit 1 fi rm -f /tmp/image.udf echo "Creating Sparse File..." truncate -s 24G /tmp/image.udf echo "Formatting File System..." mkudffs --media-type=dvdram --spartable=2 --vid="BD_$1" /tmp/image.udf || \ exit 1 echo "Mounting File System..." mkdir -p /media/udfimage || exit 1 mount -t udf -o loop,rw /tmp/image.udf /media/udfimage rm -rf /media/udfimage/lost+found chown -R root:root /media/udfimage echo "Writing to File System. This may take some time..." cp -rf ./* /media/udfimage echo "Unmounting File System..." umount /media/udfimage echo "Burning File System. This may take some time..." unset SUDO_COMMAND export GENISOIMAGE=/usr/bin/genisoimage growisofs -dvd-compat -Z /dev/sr1=/tmp/image.udf echo "Cleaning Up..." rm -f /tmp/image.udf eject echo "Done."
There are a few observations about this script, which anybody would wish to know about, who might want to use it. First of all, this script will expect one command-line argument, which is going to be prefixed with the string ‘BD_’ and then applied as the volume-ID of the file-system to be created. This will appear on computers, and on some playback-devices, as the name of whatever disk we have inserted. The exact naming is not critical, just as the exact naming for ISO9660-based Volumes may not be critical. It’s a formality which should be taken care of.
Secondly, this script expects to be run while the PWD is whatever directory we wish at the root of our created File System, and expects to be run as root. This poses an obvious security gap, as anybody could use this script in order to burn a copy of a folder, which he or she never had permission to read, if he could get root privileges with this script.
I have done my best to allow this script to be run using ‘sudo’, if it was placed in my system-directory:
But, I have never tested whether it can be made to run using ‘sudo’ in this way. It might not, because it consists of a shell-script, which might only cause the script-interpreter to be elevated to ‘root’, but not all the commands within the script! My readers may test this as they wish, but since I’m the only real user of my own computers, I’ve always just felt comfortable to make myself ‘root’, and then just to run this script.
In the interest of allowing this script to be run via ‘sudo’, I have set the variables ‘SUDO_COMMAND’ and ‘GENISOIMAGE’ as would be appropriate. This has to do with the behavior of ‘growisofs’ to use an external helper, in order actually to grow certain types of File Systems, and an unscrupulous user could run this as ‘sudo’, and could set this external command to be anything he wanted it to be. It would allow an unscrupulous user to execute an arbitrary program as root, if my script did not take care to re-set this variable to the correct path.
The reader may be reassured to know, that by default, if one of his scripted commands throws an error, the execution of the script will continue after that command, as if nothing had gone wrong. This discovery actually cost me several blank disks, until I had sprinkled a few error-checking commands into it.
Hence, the output which the user sees could be:
Error: ‘/tmp/image.udf’ Is A Directory
After which Bash will try to keep running the rest of the script… And then, the next important question becomes, whether the ‘mkudffs’ command can eventually still execute successfully, which I did bracket, If it cannot store the File-System Image it creates at that location…
I also figured, that I was more likely to uninstall the package ‘udftools’ by accident, than I was, eventually to downgrade the kernel, on a computer which I had already established, could mount a UDF 2.01 File System correctly. But, if the reader has such fears, he can also just add:
|| exit 1
To the mount command…
I suppose another observation I should add about how this script will behave, is that it will create an empty, 24GB File-System Image right off the bat, and start to build the data within. A possible nuisance, because this would represent the total amount of raw space that a device would hold – unformatted – and the entire 24GB will be written to the Blu-ray disk every time the script is used, regardless of how little data we might actually want written to the disk. I chose 24GB because the true amount of raw space on a single-layered Blu-ray is actually ~24.8GiB, and the format does not allow for fractional values to be put on the command-line. So for our purposes, we only have 24.0GiB of raw, unformatted space to work from.
I suppose I could have sat down with a calculator, to determine that 24.7GiB ~= 25292MiB …
(Updated 10/24/2017 : )
And my script assumes that the Blu-ray burner will be an external device, connected via USB 3, and referred to as ‘/dev/sr1′. The default burner built-in to any computer, would first be ‘/dev/sr0′, and the reader may adjust this according to his own needs.
There could be a reason, why this might not be used to burn Blu-ray movies per se:
- It is not only important that all the files and sub-folders be present and accounted for, but also, especially at high data-rates, that the sequence with which blocks are written to the File-System Image, be the same, as the sequence with which the media-stream is to be played back !
Because this script uses the ‘cp’ command actually to fill the File-System, the order in which blocks are written to the FS Image, could be as random, as the order with which the kernel would batch-copy files and sub-folders to an actual device. I.e., the kernel will batch-copy to the mounted Image-File, as if it was actually a device, and in whatever unpredictable order the kernel otherwise proceeds – on the assumption that doing so will just accelerate the speed with which the Copy Command can work.
So this is good to archive data, but may cause problems for an actual Movie-Blu-ray. What is written to the Image File in a certain order, will get written to the Blu-ray, in the same order.
Finally, it should be observed that as my script is written, the ‘gowisofs’ command has no idea what it is writing to the optical disk, because ‘gowisofs’ does not ‘understand’ UDF 2.01 File Systems. My use of this command has as advantage, that it controls the writing process correctly, and gives a progress-report although in text-mode, but when given a File-System Image which it does not recognize, it actually uses the utility ‘dd’ as a back-end for writing the actual File-System, instead of using what ‘GENISOIMAGE’ recommends. Only, in some hypothetical way, this script could get ‘tricked’ into using the correct helper-command, in which case this variable needs to be set correctly.
Now, this exercise can be taken further, in that we may assume we have a pre-mastered File-System Image, which ‘K3b’ will not open, because we may have a UDF File System, which has been exported by some application, but which may simply have been given the File-Name Suffix ‘.ISO’ anyway, in spite of not being an ISO9660-compatible Image. In such a case, our application may actually have written blocks into this Image, in the order they are to be played back as a stream, and then, just to burn the File-System Image, will preserve this ordering, as far as that may exist in the Image File.
That application could be ‘tsMuxer’, if given the option to export to a ‘Blu-ray .ISO File’ …
To deal with such a scenario, I have both written and tested the following script:
#!/bin/bash if [ ! -e "/dev/sr1" ] ; then exit 1 fi echo "Burning File System. This may take some time..." unset SUDO_COMMAND export GENISOIMAGE=/usr/bin/genisoimage growisofs -dvd-compat -Z /dev/sr1=$1 eject
As with the other script, an argument is expected on the command-line, but this time, that argument specifies the exact name of either the .ISO-File, or the .UDF-File, to be burned, must not contain any spaces, and the volume-ID will follow as pre-mastered within that file. Again, if ‘growisofs’ does not recognize what it sees, it will just default to using ‘dd’ as its back-end. And again, ‘growisofs’ will show us a progress dialog, and control the process, regardless.
One big plus to this script, as opposed to the earlier script, is that it can be given a File System Image which is small – maybe 4GB small – and will only take the time to write as much data as the File-System Image contains, thereby perhaps hastening our ordeal.
And finally, this second script does not need to be run as ‘root’, as long as Debian-users belong to the ‘cdrom’ group, which will give each user direct access to ‘/dev/sr1′, as that device-file is created, because an external burner has been connected, and has been recognized by the kernel.
BTW: The version of this script which I actually used myself, did not have the ‘eject’ command at the end. However, I have found myself in the situation often, on Linux machines, that a disk-write had finished, and that I simply had to type ‘eject’ myself, in a terminal-window, as a regular user, to eject an optical disk.
This command in itself will run into trouble, if we have more than one optical disk inserted at once, because it’s ambiguous then, which disk we want ejected.
Further, even though this seems obvious to me, I should put in writing, that the UDF Version that will be burned, does not depend on this latter script, but rather, on whichever application we chose, to master the .ISO-File / .UDF-File.
- ‘tsMuxerGUI’ has as advantage, being Open-Source and Free. Yet, I have no idea which UDF Version it puts, when we select ‘Blu-ray ISO’. And, AFAIK, this application does not give the user any ability to specify a Disk Menu, which many home-movie-authors take for granted.
- As an alternative, we could use “DVDFab“, a paid-for application which is GUI-based, run it under ‘Wine’, but tell it to author our Blu-ray to an .ISO-File, just so that we can later burn this .ISO-File using the above script (externally to ‘Wine’) …
In either case, If our authoring application exports a v2.50 UDF / ISO -File, then we may end up with a file that our Linux-computer cannot itself play, because
UDF support under Linux, still stops at v2.01 . In such a case however, we have achieved the maximum, at satisfying standalone playback-devices.
(Edit 10/22/2017 : )
According to certain sources, up-to-date Linux kernels are at least capable of mounting a UDF v2.50 File System in read-only mode.
Furthermore, If I use the media-playing application “VLC” to play a commercially-produced Blu-ray, that Blu-ray may already have been authored using UDF v2.50. But I did run into considerable difficulty in getting the VLC playback-application to play the Blu-rays, which I created according to these instructions, even though both my test-disks play fine on a Sony Blu-ray player.
I have added an observation, about previewing Blu-rays which we’ve created using the methods described here, on the same computer we used to create the disks.
(Edit 10/23/2017 : )
I actually have a second version of this script, which on one of my computers is named:
The main way in which this script differs from the first, is that it sets the size of the File System to 4G instead of to 24G . And the second way in which it differs from the first script, is in that the commands are commented out, to Clean Up at the end, and to remove ‘image.udf’, just in case that UDF-Image-File would be needed for documentation purposes.
This second script was useful to me, when I wanted to create a Data-DVD, to archive data, but was not satisfied with the version I had obtained using ‘K3b’ – i.e., I was not satisfied with the versions which I could get 100% using a GUI. And so the following little test seems meaningful, that compares the two disk-images that I had created, to archive the same data, for the same project:
dirk@Klystron:~/Documents/Time-Capsule Project$ file -s Omega-TC.iso Omega-TC.iso: UDF filesystem data (version 1.5) 'Omega-TC' dirk@Klystron:~/Documents/Time-Capsule Project$ file -s image.udf image.udf: data dirk@Klystron:~/Documents/Time-Capsule Project$
What this test reveals, is that when given a valid UDF v2.01 File-System Image, the ‘file -s’ command simply outputs ‘data’. Yet, I was able to mount that Data-DVD just fine, and navigate its contents using the ‘Dolphin’ file-browser – a GUI-based program.
Well the ‘file -s’ command also just outputs ‘data’, when run on the type of ‘Blu-ray ISO File’, that we obtain from ‘tsMuxerGUI’, and that plays fine on a standalone playback device, for Blu-rays.
(Edit 10/24/2017 : )
My explanation for this shortcoming of the ‘file’ command would have to be, that although up-to-date Linux kernels are capable of mounting UDF 2.01 File-Systems, user-space programs largely still fail to support this standard.