## Getting Pulseaudio to schedule real-time threads under Debian / Stretch and beyond.

On the computer which I name ‘Phosphene’, I have Debian 9 / Stretch installed, and, going the popular route, the Debian maintainers have decided that ‘Pulseaudio’ should be the main sound server, while in certain cases, it can be made to coexist with the ‘Jack’ / ‘Jack v2′ daemon, the latter for more demanding needs. In my case, starting Jack suspends Pulse.

In addition, Debian systems I’m familiar with run ‘pulseaudio’ as user, and not as root.

But, the way my Pulse daemon is configured, includes a ‘loopback’ module, that sends audio that has been input and sampled from the analog input of my (“Creative X-Fi”) sound card, back to its analog output, so that I can listen to an old-fashioned-but-modern radio receiver, over my speakers, without having to connect anything special to do so.

The problem that I was experiencing was, buffer underruns, and sound drop-outs, due to the way this default sound server was operating the sound card. And, surely enough, it had something to do with a by-now famous issue, of allowing Pulse to schedule certain threads to run with real-time priority. This can be enabled in the config file ‘/etc/pulse/daemon.conf‘, but fails to become actual when just enabled so.

Trying to find an answer to why this attempt was failing, sent me on a trip to many Web-sites and BB-postings, most of which had as disadvantage, the fact that with Pulse, there is an old way and a new way to accomplish this, and the fact that most of the BB-postings assumed the old way, being dated to the year 2016 and earlier.

Rest assured, under Debian 9 / Stretch, authorizing Pulse to schedule some threads with RT priority, is managed according to ‘the new way’. Just to recap how ‘the old way’ worked:

The file ‘/usr/bin/pulseaudio‘ needed to be set with ‘SETUID root’, and when run this way, if the sound server detected that the real user (not the effective user) belonged to the group ‘pulse-rt‘, would schedule the threads it needed to run with RT, and would then drop privileges, so that according to the process-monitor ‘top’, it would never even give a hint that anything had in fact been scheduled with RT.

This is how ‘the new way’ is supposed to work:

If the user needs for Pulse specifically to run with RT, he must actually install the package ‘rtkit‘, which runs as a system user, and which sets certain threads to have RT priority in a curated way, that’s meant to be ‘safer’, than it still is for any one user to belong to a group, that simply allows him or her to run arbitrary processes with RT. In fact, ‘the old way’ was also meant as a similar safeguard.

I had installed the specified package, rebooted my system – with Pulse given its new startup parameters, and found alas, that the file ‘/var/log/syslog‘, which was benefiting from an elevated logging level set by me, revealed numerous messages that I did not understand.

Trying to figure out what was happening resulted in more than one day’s frustration, and then finally, to some peace of mind…

(Updated 4/16/2020, 13h15 … )

## How Low-Latency CODECs can have a Time-Delay on PCs and Mobile Devices.

aptX is a CODEC, which uses two stages of “Linear Filters”, which are also known as “Convolutions”. And aptX gets used by some of the Bluetooth Headphones, that have it as a special feature, to be able to play HiFi music.

If we could just assume for the moment that each of the filters used by aptX is a 6-tap filter, meaning a filter with 6 coefficients, which is a realistic assumption, it would seem that ‘low latency’ is implied.

aptX subdivides the uncompressed spectrum into 4 sub-bands, about 5 kHz wide each, and each of which has been converted into a parallel 11.025 kHz stream of samples, for further processing. At first glance, one would assume that the latency of such a filter is the amount of time it takes, for an input sample of sound to make it past 6 coefficients then. This would mean that the latency of one filter stage is less than 1 millisecond. And so the next question which a casual observer might ask would be, ‘Why then, is there a noticeable time-delay when I listen to my Bluetooth Media?’

In this case, this time-delay has nothing to do with the Wavelets used, or the low-pass and band-pass filters themselves. When we listen to a stream on a PC, a Laptop, or a consumer Mobile Device such as a smart-phone, there are stages involved, which most users do not see, and which have nothing to do with these individual 6-tap filters.

aptX is actually implemented on the hardware side, within the Bluetooth chip-set of the source of the stream. It does not even rely on the CPU for the processing.

But what happens to any audio streamed on a consumer PC / Laptop / Mobile Device, is that first a user-space application needs to transfer the audio into kernel space, that next a kernel module needs to transfer the stream, essentially, to the hardware interface, and that then, firmware for the chip-set allows the latter to compress the stream on its way out via the Bluetooth antenna.

In consumer computing, every time an audio stream needs to be transferred from one process to another, there is a buffer that stores a minimum interval of sound. And the number of buffered samples can be higher than what we imagine, because software specialists try to make up for possible multi-tasking here, that could cause the stream to skip or drop, because the actual CPU has been called to do some background processing, outside of playing back the audio stream. Such a condition is called a “Buffer Underflow” when it happens. So the delay of each of these buffers is commonly kept high, so that all the audio we are hearing, has been delayed as a unit, and so that the CPU can still perform additional tasks, without necessarily causing the audio to skip.

Such buffering does not just happen once, in consumer electronics, but several times.

The situation is different, when aptX is built-in to professional equipment, that gets used in concerts etc.. Here, the chips are not embedded in all-purpose computing devices, but rather into dedicated devices. And here, the buffering has essentially been eliminated, so that the technology can be used live.

Dirk