One project which I had half-installed on my laptop named ‘Klystron’ some time ago, but which was not working, was software that would make the task straightforward to carry out, to burn Blu-ray movies from a Linux computer. Because I finally wanted to get that working, I spent time on this in-depth today.
One fact which many people understand, but few people know how to manage, is that Blu-ray movies are not supposed to be burned using the ISO9660 File-System, nor, with the .ISO-Files typically associated with that FS, that store its images. Technically, Blu-ray movies are supposed to be burned using a File-System called ‘UDF’, and version 2.50 of that preferably. Under certain circumstances, v2.01 of UDF may have to suffice, since Linux support for v2.50 is still lagging.
(Edit 10/19/2017 :
I suppose that one detail which I should have included all along, is that I was doing this on a Debian / Jessie system, which is also known as Debian 8. This is also known as the ‘oldstable’ version of Debian now, just because of technological progress. Therefore, Debian 8 was unable to do certain things, which Debian 9 can do out-of-the-box.
I will explain below, why The patch which I described in this posting, should not be undertaken on a Debian / Stretch system, which is also known as Debian 9, and which would be the up-to-date version of Debian as I’m writing this. )
I will spare the reader a lengthy account of what did not work. In order for this to work, I needed to have the Debian Multimedia Repository installed in my /etc/apt/sources.list , which should be straightforward for other people to duplicate. And my main purpose in having this repository, was to get the package ‘tsmuxergui’, version 2.6.11 . In addition, I was working with ‘K3b’ , v2.0.2 .
‘tsmuxergui’ is a GUI-front-end for ‘tsMuxer’, which is a program that can be used to set up Chapters and other playback details, as well as the 1920×1080, H.264-compressed video files of course, that are supposed to make up the program on the final Blu-ray.
As my burner, I used the external ‘Pioneer BDR-XD05′, that connects via USB 3.
There is one additional component which I needed, before K3b was willing and able to burn the UDF File System required, which it is not able to do out-of-the-box:
According to its authors, the versions of cdrecord that have been placed in the standard repositories belongs to ‘cdrkit’, not ‘cdrtools’, and cdrkit fails to provide the back-end, which K3b would need to burn UDF. Yet, to try to perform a binary install of the out-of-tree version, would have been very dangerous to my system. So what I did, and what I would urge other people to do, is to use one of the source-code (tarballs) from above.
First, if the reader has ‘wodim’ installed from the package-manager, I would recommend uninstalling that, just to make sure that package-version-binaries are not overwritten by the out-of-tree versions. Then, I used the above source-code, to custom-compile ‘cdrtools’.
(Edit 10/19/2017 :
If the reader is using Debian / Stretch. aka Debian 9, then the above precaution should fail. The reason for this is the fact, that under Debian / Stretch, K3b depends on ‘wodim’. Therefore, telling the package-manager to uninstall ‘wodim’, will most probably also cause the uninstallation of ‘K3b’.
Now, I can think of several reasons for which Debian Maintainers might have made K3b to depend on ‘wodim’ in Debian Stretch. One possibility, is that they might just not like this patch being applied.
But it seems to be confirmed to me – according to this test – that under Debian / Stretch, K3b is able to burn UDF / Blu-ray File-Systems out-of-the-box. And what that would mean is that to apply this patch is neither required nor desired. )
The nice feature about this version, is that it does not even install itself to ‘/usr/local/bin’. Instead, it installs its binaries to ‘/opt/schily/bin’ , when we finally give the command ‘make install’ with root privileges, so that the ultimate risk of messing up the system is small.
(Updated 10/23/2017 : )
So once we have the required software installed, the work-flow to burning a Movie-Blu-ray is as follows:
- Verify that K3b 2.0.2 recognizes the directory with the special ‘cdrtools’, and that it offers their use as a potential default, by checking “Settings” -> “Configure K3b” -> “Programs”. By default, it will already have Schily’s directory listed in its search-path, which I find convenient, because that way, if I’m to share my laptop with other users, those users will not have to do any special configuring to get their K3b to work.
- Use ‘tsMuxer’ to create a folder of sub-folders, that will include ‘BDMV’ and ‘CERTIFICATES’, that will contain our movie-project.
- With our Blu-ray burner connected, fire up K3b, and select ‘New Data-Project’. Name the volume as desired.
- Add the sub-folders named above to the root folder of our project.
- In the settings within K3b, after we click to open the ‘Burn’ dialog, choose the UDF File-System explicitly in the ‘File-System’ tab, as well as to go to the final tab, and to make sure we do not burn a Multi-Session Disk. Leave the back-end to ‘Auto’.
One detail in this undertaking which had me frustrated, was that K3b would generally display that it was starting a new Blu-ray disk, using ISO9660, even if it was using the package-provided ‘cdrecord’ as its Writing App. I can finally tell that it is using UDF, through its progress dialog now being changed slightly, and through it automatically telling me that it is burning using ‘CDRecord‘, and not, using ‘GrowISOFS’. As the naming might suggest, ‘GrowISOFS’ will only grow an ISO FS, and in my configuration, is still located in the directory ‘/usr/bin’, where the package-manager left it.
After that, I was able not only to play my movie on my dedicated Blu-ray player, but also to mount this disc, and then to do the following:
- In a command-console, as user, navigate to the ‘/dev’ directory, and give the command:
- ‘df -T sr1′
Where ‘/dev/sr1′ is the name of the external burner. As long as the disc is mounted, my command-line tells me that it is of a UDF File-System Type.
(Edit 10/23/2017 :
One fact which I have learned, after setting up this version of ‘cdrtools’ to work with ‘K3b’, is that specifically when used to burn a Video-DVD, ‘mkisofs’ builds a hybrid file system,
that starts out as an ISO File System, but to which – in this scenario – only UDF content is added.
What this means in practice, is that the UDF Version compliance will not be very strict. There might be a version-number embedded (1.02) , but customized behavior when adding files. Therefore, since I have already written shell-scripts that use the package ‘udftools’, and that are capable of growing and then burning strict version 2.01 file systems, those may continue to have some advantage to me in the future.
It can just be more fun sometimes, to use the GUI.
This also means that the ‘CDRecord’ utility in question is only capable of adding files, to the file system which the bundled ‘mkisofs’ has created.
More recently I have been able to pinpoint, that when this configuration is used to burn a Data Project, which has been set to Type UDF, ‘cdrecord’ will burn a satisfactory v1.50 File System. )
I do know that the movie plays on my Blu-ray playback-device.
There is another way to use ‘tsMuxer’, by telling it to output what it calls ‘a Blu-ray ISO File’. This also works, but not with K3b. The resulting file-name will have the .ISO extension, but will actually be a .UDF File-System Image, which K3b does not know how to open. I was able to burn that so-called .ISO File, by using a shell-script. And its resulting Blu-ray also played fine, on my playback-device.
Note: Another detail which has slowed down my own pursuit of a solution, is that If we simply tell K3b via the GUI, to burn a DVD, but to burn it using UDF, then K3b just ignores this setting. In that situation, K3b decides for us that when it comes to the older-format DVDs, ISO9660 is the standard to be used. The majority of DVDs are burned with ISO.
There’s some possibility however, that if I simply select the Writing App to be ‘cdrecord’ now, instead of ‘Auto’, I may be able to override that…
When I decided to burn a Data-DVD using UDF ( v2.01 ) several weeks ago, again I needed to use a special shell-script…
(Edit 10/23/2017 :
One observation I’ve made about this type of project in practice, through tests, is that a Video-Blu-ray disk can result, which our own Linux computer’s customary applications can no longer play back, but which does play properly on a Standalone Playback-Device – i.e. which still plays fine on my Sony.
Because any serious author of Blu-ray content would want to preview the disks he created, on the Linux computer he used to create them, I would draw the reader’s attention to This latest observation, on how this can be achieved. )
I believe that the use of UDF for Blu-ray disks also needs to be seen in a differing context. This file-system was once meant to be used for DVD-RAMs. These were optical storage disks which acted as ‘an extended hard drive’ for a computer – back in the days when optical discs still tended to offer greater capacity than (magnetic) hard drives. We could just use those as an external HD, also in a time when we didn’t have USB-keys.
What this meant was that the DVD-RAM would be accessed, written to and read, in turn, by numerous computers. For this reason it was important that these disks has a sophisticated file system.
Under Linux, users were not expected to burn them per se with special software, but also, like Windows users, just to insert them into compatible DVD-drives, and access them at random. It was the responsibility of the kernel to be able to mount them. However, when we typically bought them from the store, they weren’t formatted. So some Linux users were advised at the time, only to use K3b to format them.
! Better advice would be to use ‘mkudffs’ from the command-line.
When UDF gets used to store video information statically, one important fact changes: Suddenly, they will be formatted, and written to within a short space of time, by a small group of connected programs – all running on one computer. After that, other computers are expected to read them, but no longer to modify their contents. UDF has been downgraded, to a file system for write-once, read-mostly storage.
For that reason, the strict compliance with a specific standard, should not be as important as it once was. The fact that ‘CDRecord’ is suddenly only able to write data once, to an FS that was formatted with ‘mkisofs’, when both programs belong to the same package: ‘CDRTools’, is not a big problem. The only real issue that comes next, is whether every computer / media-player / gaming station, can read those disks as intended.
At that point it also becomes easy for the programmers of the burning-software just to say, ‘We only use a subset of the features. But what we use works.’ If these were DVD-RAMs, then the other computer could suddenly start to use features, which the present computer neglected to implement. The other computer could write changes to the DVD-RAM, which the present computer stated it had the version-number for, but which maybe it cannot read. )
(Edit 10/13/2017 : )
I had a nagging doubt, about whether the present configuration would still allow me to burn legacy ISO9660-File-System disks. And so I carried out another test.
In theory, what the reconfigured K3b-setup does, is to use ‘mkisofs’ every time, in order to create the initial, empty File-System, regardless of whether we want an ISO or a UDF File-System grown. Then, depending on how we set up the Burning Preferences, either ‘cdrecord’ or ‘growisofs’ will be used to populate that FS with our data, where ‘growisofs’ has remained as the package-version on my system. In my case, the GUI will actually choose ‘cdrecord’ by default.
What could go wrong here, is that ‘mkisofs’ could always create a hybrid file-system, ignoring my other settings, and then to give the command:
- ‘df -T /dev/sr0′
Would always indicate a UDF-type file-system, because that Linux command will always state the maximum of the two capabilities, even if my intent had been to grow ISO content only.
So what my latest test consisted of, was to use the same configuration of K3b that my posting describes, but to burn a Data-DVD, and to set the Burning Preferences to “Linux + Windows” (i.e., not UDF) , and to set the back-end to ‘growisofs’. The result was, that once I mounted the DVD+R that I just burned, and gave the above command, it revealed that the File-System-type was truly ‘iso9660′, not ‘udf’ (even though it had been created with ‘mkisofs’.
And so while it is possible, that ‘cdrecord’ as a back-end will always grow a UDF File-System using its own, internal preferences, we can actually override that by specifying that we want our FS grown with ‘growisofs’ instead, which will never grow a UDF…
What this means, is that I did not, after all, give any reader of this blog poor advice.