My personal answer, to whether Hyper-Threading works under Linux.

I have been exploring this subject, through a series of experiments written in Python, and through what I learned when I was studying the subject of System Hardware, at Concordia University.

When a person uses a Windows computer, this O/S provides all the details of scheduling processes and threads. And arguably, it does well. But when a person is using Linux, the kernel makes all the required information available, but does not take care of optimizing how threads are scheduled, specifically. It becomes the responsibility of the application, or any other user-space program, to optimize how it will take up threads, using CPU affinity, or using low-level C functions that instruct the CPU to replace a single line in the L1 cache

In the special case when a person is writing scripts in Python, because this is an interpreted language, the program which is actually running, is the Python interpreter. How well the scheduling of threads works in that case, depends on how well this Python interpreter has been coded to do so. In addition, how well certain Python modules have been coded, has a strong effect on how efficiently they schedule threads. It just so happens that I’ve been lucky, in that the Python versions I get from the Debian repositories, happen to be programmed very well. By other people.

Dirk

 

Solving the recent Firefox ESR problem, with the Expired Extension Signing Certificate.

One of the problems that befell (almost all) Firefox users, midnight from May 3 to May 4, 2019, was, that many extensions which we had installed had suddenly become deactivated because an Intermediate Certificate used to sign those extensions, had simply expired. Apparently, somebody overlooked that status of the certificate in question.

To remedy this situation as quickly as possible, Mozilla offered to publish a “Study”, which is a kind of project that Firefox users can subscribe to, and the purpose of which usually is, to allow Mozilla to conduct experiments on users’ browsers. This study had as purpose however, just to install a renewed certificate.

One important piece of advice is, ‘If you are still experiencing this issue, Do not try to uninstall and reinstall the extensions. Doing so would delete all the data stored with the extensions, while simply to have them disabled, and then to have them re-enabled, will allow most extensions to keep their stored data!’

Therefore, to receive this fix, what Firefox users were advised to do, was to go into ‘Tools -> Preferences -> Privacy & Security’, and there, to Enable Studies. The problem which I experienced myself was, that this advice was not narrowed enough, for Linux users with ‘Firefox ESR’. First of all, the Linux versions of Firefox have their Preferences sub-menu under ‘Edit’, not under ‘Tools’, but that was the least significant problem. What was worse was that, under my Linux distribution, the option we were advised to check was greyed out, and could not be checked:

Screenshot_20190505_075343

(Edit 5/10/2019, 22h10 : )

Because as of this time, under Debian / Jessie as well as under Debian / Stretch, the official fix for this problem has been pushed through the package repositories, it’s no longer advisable by any means to apply the workaround described here. However, the update under Debian / Stretch was a bit slow in coming, for which reason this workaround served me well.

(End of Edit.)

However, I was able to get this feature of Firefox ESR to work anyway. And what follows is how I did that…

(Updated 5/06/2019, 16h00 … )

Continue reading Solving the recent Firefox ESR problem, with the Expired Extension Signing Certificate.

Understanding ‘Meltdown’ and ‘Spectre’ in Layman’s Terms

One of the pieces of news which many people have heard recently, but which few people fully understand, is that in Intel chip-sets in particular, but also to a lesser degree in AMD-chip-sets, and even with some ARM (Android) chip-sets, a vulnerability has been discovered by researchers, which comes in two flavors: ‘Meltdown’ and ‘Spectre’. What do these vulnerabilities do?

Well, modern CPUs have a feature which enables them to execute multiple CPU-instructions concurrently. I learned about how this works, when I was taking a System Hardware course some time ago. What happens is meant to make up for the fact that to execute one CISC-Chip instruction, typically takes up considerably more than 1 clock-cycle. So what a CISC-Chip CPU does, is to start execution on instruction 1, but during the very next clock-cycle, to fetch the opcode belonging to instruction 2 already. Instruction 1 is at that point in the 2nd clock-cycle of its own execution. And one clock-cycle later, Opcode 3 gets fetched by the CPU, while instruction 2 is in the 2nd clock-cycle, and instruction 1 is in the 3rd clock-cycle – if their is any – of each of their execution.

This pushes the CISC-Chip CPUs closer to the ideal goal of executing 1 instruction per clock-cycle, even though that ideal is never fully reached. But, because CPU-instructions contain branches, where a condition is tested first, and where, in a roundabout way, if the non-default outcome of this test happens to be true, the program ‘branches off’, to another part within the same program, according to the true logic of the CPU-instructions. The behavior of the CPU under those conditions has also been made more-concurrent, than a first-glance appraisal of the logic might permit.

When a modern CISC-Chip CPU reaches a branching instruction in the program, it will continue to fetch opcodes, and to execute the instructions which immediately follow the conditional test, according to the default assumption of what the outcome of the conditional test is likely to be. But if the test brings about a non-default logical result, which will cause the program to continue in some completely different part within its code, the work which has already been done on the partially-executed instructions is to be discarded, in a way that is not supposed to affect the logical outcome, because program flow will continue at the new address within its code. At that moment, the execution of code no longer benefits from concurrency.

This concurrent execution, of the instructions that immediately follow a conditional test, is called “Speculative Execution”.

The problem is, that Complex Instruction-Set CPUs, are in fact extremely complex in their logic, as well as the fact that their logic has been burned as such, into the transistors – into the hardware – of the CPU itself, and even the highly-skilled Engineers who design CPUs, are not perfect. So we’ve been astounded by how reliably and faithfully actual, physical CPUs execute their intricate logic, apparently without error. But now, for the first time in a long time, an error has been discovered, and it seems to take place across a wide span of CPU-types.

This error has to do with the fact that modern CPUs are multi-featured, and that in addition to having concurrent execution, they also possess Protected Memory, as well as Virtual Memory. Apparently, cleverly-crafted code can exploit Speculative Execution, together with how Virtual Memory works, in order in fact to bypass the feature which is known as Protected Memory.

It is not the assumption of modern computers, that even when a program has been made to run on your computer – or your smart-phone – It would just be allowed ‘to do anything’. Instead, Protected Memory is a feature that blocks user-space programs from accessing memory that does not belong to them. It’s part of the security framework actually built-in to the hardware, that makes up the CPU.

More importantly, user-space programs are never supposed to be able to access kernel memory.

(Updated 01/11/2018 : )

Continue reading Understanding ‘Meltdown’ and ‘Spectre’ in Layman’s Terms