An Aspect to Hugetlbfs, which Many Sites Omit

I was recently troubleshooting the question of Huge Pages under Linux, which are pages of virtual memory with a size of 2MB each, instead of 4KB. And I was interested in the question of statically-allocated ones, even though modern, 64-bit Linux systems also offer Transparent Huge Pages. There was a gap in what was posted online, concerning the possibility or need to mount an actual ‘hugetlbfs’ file system, according to whether specific programs use one.

Under systems such as my own, as soon as the system boots with hugetlbfs enabled – i.e. a non-zero number of them – the kernel automatically creates a mount-point at ‘/dev/hugepages’. This mount-point is created without any administrator ‘fstab’ entry, and belongs to user and group ‘root’. According to some needs, such a mount-point would better be created as belonging to a specific group, and as having the option ‘mode=1770′ set. And so before checking the default behavior of my kernel, I also created a suitable mount-point at ‘/mnt/hugepages’. The question remained in my mind, of whether any way exists to give the automatic mount-point at ‘/dev/hugepages’ my custom options. And the answer seems to be No.

Here’s what the online documentation forgets to mention:

If you have a program which requires such a mount-point, there will be a line in its config file, that states where it’s located. Any files created in such a mount-point will then consume statically-allocated huge pages of RAM.

Because MySQL is not an example of a program that requires this, its config file also requires no line, to tell it which mount-point to use. MySQL uses memory-allocation functions directly, to ask the kernel for huge pages.

I suppose that it does no real harm, if there is more than one hugetlbfs mounted at any time, as long as the unneeded one is not wasting any pages, let’s say as long as absolutely no files are being created in the unused mount-point. And then if we need to, we can in fact give our custom hugetlbfs mount-point whatever properties we think it should have, via the mount options or the fstab.

Because I didn’t need the one I had created, I simply got rid of it again. Besides which, the fact that one is automatically created at ‘/dev/hugepages’ these days, suggests that future programs that need it, will already be configured to look for it there. And then it would also make sense, if those programs were able to deal with the fact that that one belongs to user and group ‘root’.

Dirk

 

Print Friendly, PDF & Email

There has been a Dist-Upgrade on my Server.

This server is hosted on a Debian / Jessie (Linux) computer which I own and run myself. Under Debian – Linux systems, the most thorough kind of update which can be carried out is called a ‘dist-upgrade’ or a ‘d-u’ for short. Just this evening, I saw that suddenly there were 93 software packages, which all did need an upgrade, and saw, that I could not just leave this type of upgrade to the usual, automated services. Therefore, I decided to administer the 93 package-upgrades given, via a dist-upgrade command. This can be stressful, or exciting, or both, because it can give a Linux user an improvement, or it can in some cases actually cripple our systems. I’m glad to say that this Linux box I name ‘Phoenix’ did not get crippled. It’s still fully bootable.

But due to this procedure, the Web-server was also down, from 20h15 through until 20h40 or so. I see that my blog is still here though, after the d-u .

I think that most software updates can be fun and games. But this particular upgrade also chose to include my graphics driver, which I was particularly fussy about. The past version of the graphics driver on this box was extremely stable, and I was trying to avoid doing any sort of upgrade to it, but now doing so was the only way to keep my box compatible with future upgrades.

It has sometimes happened to me, that the screen might just freeze – even though it’s a Linux computer – due to stability problems with other graphics drivers – especially with the ‘mesa’ driver, which tries to software-render an OpenGL equivalent. But what has been most stable for me in recent months, was the ‘GLX’ driver, which does full hardware, OpenGL rendering as it’s supposed to, and which under modern Linux systems is even capable of a ‘TDR’ equivalent, a Timeout Detection and Recovery, which will restart a crashed GPU without harming the active session.

If in the near future I find that my screen does freeze, or that there are TDR issues, a sinking feeling will go through my heart, because that would signal that a completely stable graphics driver has been replaced unnecessarily, with an unstable one. And in the act of doing so, all my package-management scripts even recompiled the DKMS kernel module for the graphics driver in question, because that is the correct way to install it.

Oh Yes, I see that the Apache Web-server software, which my machine hosts, has been given an upgrade as well. But as I see it, this was the least likely set of packages, for the maintainers to have botched. So it’s my full assumption that Web-server activity will continue without error.

Dirk

 

Print Friendly, PDF & Email

I discovered an IPv6-related malfunction.

During past weeks, I had noticed that sometimes my browser’s home page, which is the Google Site I’m logged in to, would simply fail to display as I started my browser, or would fail to display even if I tried to click on a link that leads to it. Curiously, although this could have been a traffic-filtering issue from the content-provider, it was not affecting the same browser, logged in to the same account, from my other computers. At first I suspected that this could have been due to memory fragmentation, and perhaps a poor attempt by the browser on this one machine to allocate large segments of memory at once, which it would have had to be keeping silent at the same time. But it turns out, that the cause of this malfunction was something different.

What gave this away was that almost, only Google sites were affected, and no other large, complex sites. E.g., calling up my own blog’s pages would never fail.

It so happens that I have a Teredo Client running, which gives me access to IPv6 addresses. More to the point, it gives the IPv6 world contingent access to my site. But my machine has been running as a dual-stack machine for some time, which means that it can work internally with IPv6 addresses as well as IPv4, even though my ISP isn’t IPv6 -ready yet.

Well apparently what was happening, was that Google specifically has ‘true’ IPv6 addresses for most of its sites (unlike my lower-grade, ‘fake’ Teredo address). Most other Web-sites which I’ll be clicking on, don’t. It seems that my browser was preferring to retrieve all Google pages via IPv6, which meant that every time I sat down in front of it, it was also using the Teredo proxy. This put an undue load on the proxy, who in turn started dropping most packets routed to Google by my host. Which in turn was leading to a blank page.

Also, every time I reboot my computer, I get assigned a new, virtual IPv6 address. This was the true reason, fw the malfunction would seem to go away directly after a reboot, but them simply set in again.

Since I use “Iceweasel”, which is the Linux equivalent to “FireFox”, my solution was to open a new tab, and to type ‘about:config’ into the URL bar, at which point the typical warning popped up, which I always dismiss with an affirmative answer. And then in the Search Bar, I typed “IPv6″. There was only one setting that matched: “network.dns.disableIPv6″. This setting defaults to ‘false’. I double-clicked the setting, which toggled it to ‘true’, and restarted Iceweasel. And Tada – Problem solved!

It suits me fine that I have IPv6 enabled for connectivity, but not for browsing. Nor for harassing my IPv6-providing Teredo server.

Dirk

 

Print Friendly, PDF & Email

Android App Permissions Dialog

Most Android users are at least vaguely aware, that every time we install or update an app, we’re shown a dialog with a list of permissions the app is requesting on our mobile device. We can either Allow or Deny this request.

What people should be aware of as well, is that by default, Android did not allow us to Accept or Decline each permission on its own. We were shown the whole list, and would then have to either Accept or Decline the entire list, and in the latter case, the app would not install, or the update would not take place.

This was a rather powerless feature, because when we declined an update, Google Play would just come back within short order, and offer the same update again. Also, there was no way to opt out of updating for one specific app. So we would then either be obliged to accept the update at a later time, or to uninstall the app.

This was the status-quo up to and including “Android Lollipop”. The Android version that came after Lollipop, and which is the current version, is called “Marshmallow”. And the main, key improvement which Marshmallow offers, is control by the user, for each individual permission the app is asking for. With Marshmallow, the user is no longer obliged either to accept the entire list of permissions or to reject it. He can grant or deny any specific permission, and then still install the update, which gets rid of the messages for that update.

One reason fw this is important, is the possibility of a “Privilege Escalation”, which is also a known form of cyber-attack. Privilege Escalation means, that an already-installed app can ask for progressively more permissions during each update, which users often don’t pay strict attention to, so that after several updates, the app has a dangerous collection of them on our device.

Granted, most of the time the apps need a large number of permissions for innocuous purposes, or maybe because they’re just not programmed well enough, to work without those. But the potential exists for too liberal a set of permissions eventually to compromise our privacy online, or even our online security.

This is why, regardless of whether we have Marshmallow or not, we should in fact be examining the requested permissions each time, before we simply grant them.

Having said that, I don’t have Android Marshmallow yet. This is secondhand information, from a friend of mine who is in the know, and who has Marshmallow on at least one of his devices.

Dirk

 

Print Friendly, PDF & Email