## Photogrammetry Acknowledgement

In this earlier posting, I described a form of “photogrammetry” in which an arbitrary, coarse base-geometry is assumed as a starting point, and from which micropolygons are spawned, in order to approximate a more-detailed final geometry.

I must acknowledge that within this field, a domain also exists, which is not like that, and in which the computer tries to guess at a random, arbitrary geometry. Of course, this is a much more difficult form of the subject, and I do not know much about how it is intended to work.

I do know that aside from the fact that swatches of pixels need to be matched from one 2D photo to the next, one challenge which impedes this, is the fact that parts of the (yet-unknown) mesh will occlude each other to some camera-positions but not others, in ways that computers are poor at predicting. To deal with that requires such complex fields as “Constraint Satisfaction Programming” – aka ‘Logic Programming’, etc..

(Edit 01/05/2017 : Also, if we can assume that a 2D grid of pixel-swatches is being tagged for exact matching, and that only horizontal parallax is to be measured, the problem of entire rows of rectangles that all have the same signature can be cumbersome to code for, where only the end-points change position from one photo to the next… And then their signature can end, to be replaced by another, after which, on the same row, the first set of signatures can simply resume.

Further, If we knew that this approach was being used, Then we could safely infer that the number of mesh-units we derive, will also correspond to the number of rectangles, which each photo has been subdivided in to, not the number of pixels. )

If that was to succeed, I suppose it could again form a starting-point, for the micropolygon-based approach I was describing.

I do know of at least one consumer-grade product, which uses micropolygons.

## Modern Photogrammetry

Modern Photogrammetry makes use of a Geometry Shader – i.e.  Shader which starts with a coarse grid in 3D, and which interpolates a fine grid of microplygons, again in 3D.

The principle goes, that a first-order, approximate 3D model provides per-vertex “normal vector” – i.e. vectors that always stand out at right angles from the 3D model’s surface in an exact way, in 3D – and that a Geometry Shader actually renders many interpolated points, to several virtual camera positions. And these virtual camera positions correspond in 3D, to the assumed positions from which real cameras photographed the subject.

The Geometry Shader displaces each of these points, but only along their interpolated normal vector, derived from the coarse grid, until the position which those points render to, take light-values from the real photos, that correlate to the closest extent. I.e. the premise is that at some exact position along the normal vector, a point generated by a Geometry Shader will have positions on all the real camera-views, at which all the real, 2D cameras photographed the same light-value. Finding that point is a 1-dimensional process, because it only takes place along the normal vector, and can thus be achieved with successive approximation.

(Edit 01/10/2017 : To make this easier to visualize. If the original geometry was just a rectangle, then all the normal vectors would be parallel. Then, if we subdivided this rectangle finely enough, and projected each micropolygon some variable distance along that vector, There would be no reason to say that there exists some point in the volume in front of the rectangle, which would not eventually be crossed. At a point corresponding to a 3D surface, all the cameras viewing the volume should in principle have observed the same light-value.

Now, if the normal-vectors are not parallel, then these paths will be more dense in some parts of the volume, and less dense in others. But then the assumption becomes, that their density should never actually reach zero, so that finer subdivision of the original geometry can also counteract this to some extent.

But there can exist many 3D surfaces, which would occupy more than one point along the projected path of one micropolygon – such as a simple sphere in front of an initial rectangle. Many paths would enter the sphere at one distance, and exit it again at another. There could exist a whole, complex scene in front of the rectangle. In those cases, starting with a coarse mesh which approximates the real geometry in 3D, is more of a help than a hindrance, because then, optimally, again there is only one distance of projection of each micropolygon, that will correspond to the exact geometry. )

Now one observation which some people might make, is that the initial, coarse grid might be inaccurate to begin with. But surprisingly, this type of error cancels out. This is because each microploygon-point will have been displaced from the coarse grid enough, that the coarse grid will finally no longer be recognizable from the positions of micropolygons. And the way the micropolygons are displaced is also such, that they never cross paths – since their paths as such are interpolated normal vectors – and so no Mathematical contradictions can result.

To whatever extent geometric occlusion has been explained by the initial, coarse model.

Granted, If the initial model was partially concave, then projecting all the points along their normal vector will eventually cause their paths to cross. But then this also defines the extent, at which the system no longer works.

But, According to what I just wrote, even the lighting needs to be consistent between one set of 2D photos, so that any match between their light-values actually has the same meaning. And really, it’s preferable to have about 6 such photos…

Yet, there are some people who would argue, that superior Statistical Methods could still find the optimal correlations in 1-dimensional light-values, between a higher number of actual photos…

One main limitation to providing photogrammetry in practice, is the fact that the person doing it may have the strongest graphics card available, but that he eventually needs to export his data to users who do not. So in one way it works for public consumption, the actual photogrammetry will get done on a remote server – perhaps a GPU farm, but then simplified data can actually get downloaded onto our tablets or phones, which the mere GPU of that tablet or phone is powerful enough to render.

But the GPU of the tablet or phone is itself not powerful enough, to do the actual successive approximation of the micropolygon-points.

I suppose, that Hollywood might not have that latter limitation. As far as they are concerned, all their CGI specialists could all have the most powerful GPUs, all the time…

Dirk

P.S. There exists a numerical approach, which simplifies computing Statistical Variance in such a way, that Variance can effectively be computed between ‘an infinite number of sample-points’, at a computational cost which is ‘only proportional to the number of sample-points’. And the equation is not so complicated.
 s = Mean(X2) - ( Mean(X) )2 
(Next)

## An Example of How WebGL is supposed to Work

In This Earlier Posting, I described how I had recompiled the Atomic Game Engine, to act as a platform running on my Linux laptop ‘Klystron’, from which among other things, I should be able to create games, and export those into ‘WebGL Format’.

WebGL is an ‘HTML5′ specification, which not all browsers support fully, but which essentially allow for hardware-accelerated 3D graphics to run on the browser, as propagated from a Web-server. It has the same gist that the now-deprecated ‘VRML’ used to have, with the exception of being in binary – partially – and being more powerful.

Atomic Game Engine comes with numerous Demos, that are designed to help users like me, learn how to use their system for creating games, and to test their platform.

The following is not a game I created, instead being a simple game delivered through the installation of Atomic Game Engine to my laptop. So this is something which the developers of Atomic Game Engine created, but which I am able to deploy as well, just because I installed Atomic Game Engine. And it may not work on every browser. I hear that Firefox supports WebGL particularly well. And, it is slow to load, because after everything has been said and done, WebGL is still a less efficient platform for 3D content, than actual ‘OpenGL’, or ‘DirectX’ applications would be.

If you load this on some browsers, they will display a message box, stating that ‘A Script is Taking Very Long To Run – Do You Wish To Continue?’ Because we know that this script is supposed to contain a 3D game, presumably we would allow it to keep running. But even when we do, we need to have a fairly strong graphics chip-set, in order for this to render properly. It renders correctly on my laptop ‘Klystron’ – over a network – but not on my server-box ‘Phoenix’, even though both have the same browser version, presumably because of the weak GPU on ‘Phoenix’. And building Atomic Game Engine, did not require me to install any special modules to my system, as a plausible answer to why that laptop is able to play the animation.

On ‘Phoenix’, this content eventually loads, but only displays a black window. And that is correct, even though I am the publisher and code-maintainer for this little project.

Link To An Atomic Game Engine Sample

Enjoy.

Dirk

## The laptop ‘Klystron’ suspends to RAM half-decently.

One subject which I have written a lot about, was that as soon as I close the lid of the laptop I name ‘Klystron’, it seems to lose its WiFi signal, and that that can get in the way of comfortable use, because to close the lid also helps shield the keyboard of dust etc..

This Linux laptop boots decently fast, and yet is still a hassle to reboot very often. And so I needed to come up with a different way of solving my problem, on a practical level. My solution for now, is to tell the laptop to Suspend To RAM, as soon as I close the lid. That way, the WiFi signal is gone more properly, and when the laptop resumes its session, the scripts that govern this behavior also re-initialize the WiFi chipset and its status on my LAN. This causes less confusion with running Samba servers etc., on my other computers.

There is a bit of terminology, which I do not think that the whole population understands, but which I think that people are simply using differently from how it was used in my past.

It used to be, that under Linux, we had ‘Suspend To RAM’ and ‘Suspend To Disk’. In the Windows world, these terms corresponded to ‘Standby’ and ‘Hibernate’ respectively. Well in the terminology today, they stand for ‘Sleep’ and ‘Hibernate’, borrowing those terms from mobile devices.

There are two types of Suspend working in any case.

In past days of Linux, we could not cause a laptop just to Hibernate. We needed to install special packages and modify the Grand Unified Bootloader, before we could even Suspend To Disk. Suspending To RAM used to be less reliable. Well one development with modern Linux which I welcome, among many, is the fact that Sleep and Hibernate should, in most cases, work out-of-the-box.

I just tried Sleep mode tonight, and it works 90%.

One oddity: When we Resume, on this laptop, the message is displayed on the screen numerous times, of a CPU Error. But after a few seconds of CPU errors, apparently the session is restored without corruption. Given that I have 300 (+) processes, I cannot be 100% sure that the Restore is perfectly without corruption. But I am reasonably sure, with one exception:

The second oddity is of greater relevance. After Waking Up, the clock of the laptop seems to be displaced 2 days and a certain number of hours into the future. This bug has been observed on some other devices, and I needed to add a script to the configuration files as a workaround, which simply sets the system clock back that many days and that many hours, after Waking. Thankfully, I believe that doing so, was as much of a workaround as was needed.

One side-effect of not having done so, before being aware of the problem, was that the ‘KNotify’ alarms for the next two days have also all sounded, so that it will take another two days, before personal organizer – PIM – notifications may sound for me again.

The fact that numerous CPU errors are displayed bothers me not. What that means, is that the way the CPU goes to sleep, and then wakes up, involves power-cycling in ways that do not guarantee the integrity of data throughout. But it would seem that good programming of the kernel does provide data integrity, with the exception of the system clock issue.

But the fact that the hardware is a bit testy when using the Linux version of Sleep, also suggests that maybe this is also the kind of laptop that powers down its VRAM. It is a good thing then, that I disabled the advanced compositing effects, that involve vertex arrays.

There is a side-note on the desktop cube animation I wanted to make.

In general, when raster-rendering a complex scene with models, each model is defined by a vertex array, an index array, one or more texture images etc., and the vertex array stores the model geometry statically, as relative to the coordinate-origin of the model. Then, a model-view-projection matrix is applied – or just a rotation matrix for the normal vectors – to position it with respect to the screen. Moving the models is then a question of the CPU updating the model-view matrix only.

Well when a desktop cube animation is the only model in the scene, as part of compositing, I think that the way in which this is managed differs slightly. I think that what happens here, is that instead of the cube having vertex coordinates of +/- 1 all the time, the model-view matrix is kept as an identity matrix.

Instead, the actual vertex data is rewritten to the vertex array, to reposition the vertices with respect to the view.

Why is this significant? Well, if it was true that Suspending the session To RAM also cut power to the VRAM, it would be useful to know, which types of data stored therein will seem corrupted after a resume, and which will not.

Technically, texture images can also get garbled. But if all it takes is one frame cycle for texture images to get refreshed, the net result is that the displayed desktop will look normal again, by the time the user unlocks it.

Similarly, if the vertex array of the only model is being rewritten by the CPU, doing so will also rewrite the header information in the vertex array, that tells the GPU how many vertices there are, as well as rewriting the normal vectors, as when they are a part of any normal vertex animation, etc.. So anything resulting from the vertex array should still not look corrupted.

But one element which generally does not get rewritten, is the index array. The index array states in its header information, whether the array is a point list, a line list, a triangle list, a line strip, a triangle strip… It then states how many primitives exist, for the GPU to draw. And then it states sets of elements, each of which is a vertex number.

The only theoretical reason fw the CPU would rewrite that, would be if the topology of the model was to change, which is as good as never in practice. And so, if the VRAM gets garbled, what was stored in the index array would get lost – and not refreshed.

And this can lead to the view, of numerous nonsensical triangles on the screen, which many of us have learned to associated with a GPU crash.

Dirk