Multitasking

During this posting from some time ago, I wrote at length about the subject of interrupt prioritization. I wanted to demonstrate that I really did study System Software, and that therefore, I know something which I can pass along, about the subject of multitasking, which is a different subject.

The perspective from which I appreciate multitasking, is the perspective of the device-driver. According to what I was taught – and it is a bit dated – a device-driver had a top half and a bottom half. The top half was invoked by user-space processes, in a way that required a system-call, while the bottom half was invoked by interrupt requests, from the hardware. This latter detail did not changed if instead of using purely interrupt-driven I/O, the hardware and driver used DMA.

A system-call is also known as a software-interrupt.

The assumption which is made is that a user-space process can request a resource, thus invoking the top half of the device driver, but that the resource is recognized by the device driver as being busy, and that processes on the CPU run much faster than the I/O device. What the device-driver will do is change the state of the process which invoked it from ‘running’ to ‘blocked’, and it will make an entry in a table it holds, which identifies which process had requested the resource, as well as device-specific information, regarding what exactly the process asked for. Then, the top half of the device-driver will send whatever requests to the device that are needed, to initiate the I/O operation, and to ensure that at some future point in time, the resource and piece of data which were asked for, will become available. The top half of the device-driver then exits, and the process which invoked it will typically not be in a ‘running’ state anymore, unless for some reason the exact item being requested was immediately available. So as far as the top half is concerned, what usually needs to happen next is that some other process, which is in the ‘ready’ state, needs to be made ‘running’ by the kernel.

The bottom half of the device driver responds to the interrupt request from the device, which has signaled that something has become available, and looks up in the table belonging to the device driver, which process asked for that item, out of several possible processes which may be waiting on the same device. The bottom half then changes the state of the process in question from ‘blocked’ to ‘ready’, so that whenever a ‘ready’ process is about to be made ‘running’, the previously-blocked process will have become a contender.

The O/S kernel is then free to schedule the ‘ready’ process in question, making it ‘running’.

Now, aside from the fact that processes can be ‘running’, ‘ready’, or ‘blocked’, a modern O/S has a differentiation, between ‘active’ and ‘suspended’, that applies to ‘ready’ and ‘blocked’ processes. My System Software course did not go into great detail about this, because in order to understand why this is needed, one also needs to understand that there exists virtual memory, and that processes can be swapped out. The System Software course I took, did not cover virtual memory, but I have read about this subject privately, after taking the course.

The kernel could have several reasons to suspend a process, and the programmer should expect that his process can be suspended at any time. But one main reason why it would be suspended in practice, would be so that it can be swapped out. Only the kernel can make a ‘suspended’ process ‘active’ again.

Continue reading Multitasking

Caveat in using Visual Studio Express 2015

On my Windows 7 computer ‘Mithral’, I recently installed and started using “Visual Studio Express 2015″, which has Update 3.1, and am happy with it overall.

However, there is a type of malfunction which can take place, and apparently did. This IDE detected that I have 8 cores, and decided as a default that it would try to build as many as 8 targets concurrently, to maximize its speed. It did not detect that my 8 cores are threaded as 4, which is of minor significance. I changed the setting to 6 simultaneous builds, which means to me, that for each of the 6 targets, only one source file should be in the process of being compiled at once.

A type of condition seems to be possible, in which for an unknown reason, the application starts to build an extreme number of targets, which on my system meant maybe 15-20 targets. In the Task Manager, I saw up to 20 ‘cl.exe’ processes running at once. This was what led to an ‘OOM’ (‘Out-Of-memory’) condition, and required me to force the application to close from the Task Manager, as the IDE window had also stopped responding.

After that, I was not sure of the status of my build, so that I Cleaned the build, and also rebooted my machine, because the worst thing to my mind, would have been a not-clean shutdown. It seems that the actual reboot may not have been necessary.

After the reboot, I was able to build my Solution again, with no ill effects.

I am not sure what triggers this error, but do remember that on that occasion, I had Minimized the IDE application window and Restored it again. The whole thing gave me a bigger scare than was necessary, since no long-term corruption resulted.

Dirk

 

I have been test-driving Visual Studio Express 2015.

One of the projects which I attempted today on the Windows 7 desktop computer I name ‘Mithral’, was to compile OGRE 1.10 . This is an unstable build of OGRE, and it would be helpful for me to know whether this instability comes more, from the software, or from the weaker graphics card on my Linux laptop ‘Klystron’, which I have already had to switch to OGRE 1.9 .

My initial attempt to compile OGRE 1.10 failed in a foreboding way: The rendering window would open, and then be black for a second, and then give way to the nondescript Windows Error box, telling me that the program had crashed. There were no traces of error messages in the log to explain why. This is called “a silent crash”. Hypothetically, it could point to a borked compiler setup.

So what I did next was to download an OGRE 1.9 SDK, which had been entirely pre-compiled by OGRE devs. But then I knew this had been a waste of time, because it nowhere proves that my compiler can actually compile. And yes, that SDK was unstable on my stronger graphics card.

I have come to learn something. Even having a Microsoft compiler does not guarantee that I will be able to compile a DirectX rendering engine. The main reason for this, is the fact that DirectX 9 is almost deprecated. The up-to-date Microsoft SDK no longer includes libraries and header files, which legacy DirectX applications linked against – including OGRE. This means that the OGRE SDK can offer DirectX 11 support, not Dx 9, and its DirectX 11 support is unstable out-of-the-box. This is ultimately a fault of the software.

What I did next was to compile OGRE 1.9 . When I was setting up the CMake parameters to do so, I realized that when I had been setting up CMake for 1.10 , I had made mistakes that could have led to severe code-linking issues. Specifically, under Windows, it is tedious how we need to link to each core-dependency one-by-one. Under Linux or MinGW these can all get picked up in an automated batch. But with MSVC, it is not so easy.

Compiling OGRE 1.9 with the OpenGL2 and the OpenGL3+ rendering engines was a success, and so finally proved that my new compiler can produce moving, 3D images. Unfortunately though, 1.9 was code that still used the deprecated way of linking to the Windows SDK for Dx 9 and 11 , so that I could not build the DirectX 11 engine after all.

I found that just with OGRE 1.9.0 , and OpenGL2, I was able to get a larger set of animations to run, from the Sample Browser, than I was on my laptop. This proves, that much of the trouble I was having with ‘Klystron’, or before that, ‘Maverick’, were in fact hardware issues.

The Iso-Surface Demo works along a different principle than I had anticipated. It is one of those applications, which use a Fragment Shader, which renders to a Vertex Buffer, set up to look like a Pixel Buffer. The Pixel format output has been cleverly engineered also to correspond to a vertex attribute structure, thus achieving what was once known as ‘a poor mans Geometry Shader’.

The Iso-Surface Demo is supposed to work, even with the OpenGL2 rendering engine. Only, on my laptop, there is no support for Render To Vertex Buffer, aka ‘R2VB’.

With OGRE 1.9 , the OpenGL3+ rendering engine remains unstable as heck, unusable.

There is an issue with how VS 2015 ultimately works. Since ‘Mithral’ possesses 8 cores, threaded as 4, VS will ultimately try to build up to 8 targets at once. This pushes performance to the max, but at the expense of stability. Today I was pushing this compiler for hours and hours – and I later learned that it truly maxes out all 8 cores.

I found a setting to correct this for the future. Given 8 cores, I would like a maximum of 6 compile targets worked on concurrently. This is just, so that the system will have a full CPU left, to work on other tasks, should things go wonky. Because by the end of the day, things did go wonky – for whatever reason.

Dirk

(Edit 09/16/2016 : ) Another disadvantage, If we have an 8-core CPU, and If our compiler wants to build 8 targets concurrently for that reason, is the fact that 1 source file being compiled at a time, for each target, can consume an unpredictable amount of RAM. If the amount of RAM on the system is not taken into consideration, an ‘OOM’ (‘Out Of Memory’) condition can arise, because of the arbitrary 8 jobs running at once.

And I think that last night, such an OOM condition did arise, because I was installing tons of software… I have 8GB of RAM on ‘Mithral’. I performed numerous defragmentations as well, and, because many programs do have memory leaks, everything last night may have led up to an OOM condition by the end of the day.

I also installed Boost 1.61, and Boost 1.59, where before I only had Boost 1.47. And Boost 1.61 may in fact be necessary for OGRE 1.10 to compile, as another reason why my first attempt to compile that had failed.