The current state of Augmented Reality

In spite of the subject line of this posting, my state of knowledge on that topic may be somewhat incomplete. I apologize to the reader if this is so.

A decade or more ago, Augmented Reality started to make its way to mobile devices, mainly smart-phones, and it consisted of an application that would record the scene which the forward-facing camera was viewing, and would perform some amount of image recognition, or of decoding of humanly imperceptible optical codes, and the app would then overlay CG content over that camera video, to give an enhanced experience to the user on the phone’s display, such as, to highlight certain geographical features of tourist sites, or, to highlight certain commercial products which the user was also seen as a potential customer of…

I think that one phenomenon which has taken place with AR has been, that that market was not lucrative, so that AR apps which did exactly those things have become less frequent. However, some such apps still exist, such as the Android app named ‘ROAR‘. This app does not require that regular users create accounts, but does require that content designers do so. Yet, content designers for this app have expressed pessimism in the question of, whether to create content of this form will pay off in increased profits, because of the simple fact that a customer would need to have this exact app installed, and be running it, either, when visiting a certain store, or, when pointing the app at a product which has already been purchased, say, in order to obtain instructions on how to set up the product. One might say that there is less of a will to invest as much money as was done decades ago, into content creation.

But while this assessment sounds rather bleak, there now exist some newer forms of AR, that have led to more apps. In one new form of AR, the user points his phone at an inside room, and places virtual furniture into it, in order to preview what the best arrangements of furniture would be, before actually spending the money and committing to buy said furniture.

The main reason I don’t want to link directly to such apps is the fact, that they sometimes belong to one specific furniture company, and I don’t want to play favours.

Additionally, AR apps now exist, which do exactly one thing: To act as rulers, i.e., to give a measuration of the distance between two points in 3D, when the 3D scene has been video-recorded in 2D. This would be one example. I can’t really tell how accurate such a ruler app finally is. I’m only documenting that it exists. It seems to have averaged more than 4 stars in user satisfaction reviews.

Another type of AR which exists now, takes the form of “Google Lens“. This was once integrated into ‘Google Photos’, but has been made a separate app. It acts as a search engine, but in order to use it, instead of typing in a search, the user points his phone-cam at a scene or object.

But then, there is also a form of AR, the only purpose of which seems to be, to allow the user to start with a photo, and to create an animation from it himself, that has some sort of fictitious- or fantasy- aspect, and which is simply supposed to look interesting. The resulting animation can then be posted on social media, to impress friends. In one case, the result is an animation in which parts of the photo end up seemingly to move, while in another case, random objects which the user uploaded to a Web-site, are placed ‘in front of’ an actual scene, just like virtual furniture was, except that no intention ever existed, to place those objects physically.

Therefore, in some forms, AR still exists. I suppose that yet another, big context in which it exists is, gaming.

Dirk

 

Some realizations about Digital Signal Processing

One of the realizations which I’ve just come across recently, about digital signal processing, is that apparently, when up-sampling a digital stream twofold, just for the purpose of playing it back, simply to perform a linear interpolation, to turn a 44.1kHz stream into an 88.2kHz, or a 48kHz stream into a 96kHz, does less damage to the sound quality, than I had previously thought. And one reason I think this is the factual realization that to do so, really achieves the same thing that applying a (low-pass) Haar Wavelet would achieve, after each original sample had been doubled. After all, I had already said, that Humans would have a hard time being able to hear that this has been done.

But then, given such an assumption, I think I’ve also come into more realizations, of where I was having trouble understanding what exactly Digital Signal Processors do. It might be Mathematically true to say, that a convolution can be applied to a stream after it has been up-sampled, but, depending on how many elements the convolution is supposed to have, whether or not a single DSP chip is supposed to decode both stereo channels or only one, and whether that DSP chip is also supposed to perform other steps associated with playing back the audio, such as, to decode whatever compression Bluetooth 4 or Bluetooth 5 have put on the stream, it may turn out that realistic Digital Signal Processing chips just don’t have enough MIPS – Millions of Instructions Per Second – to do all that.

Now, I do know that DSP chips exist that have more MIPS, but then those chips may also measure 2cm x 2cm, and may require much of the circuit-board they are to be soldered in to. Those types of chips are unlikely to be built-in to a mid-price-range set of (Stereo) Bluetooth Headphones, that have an equalization function.

But what I can then speculate further is that some combination of alterations of these ideas should work.

For example, the convolution that is to be computed could be computed on the stream before it has been up-sampled, and it could then be up-sampled ‘cheaply’, using the linear interpolation. The way I had it before, the half-used virtual equalizer bands would also accomplish a kind of brick-wall filter, whereas, to perform the virtual equalizer function on the stream before up-sampling would make use of almost all the bands, and doing it that way would halve the amount of MIPS that a DSP chip needs to possess. Doing it that way would also halve the frequency linearly separating the bands, which would have created issues at the low end of the audible spectrum.

Alternatively, implementing a digital 9- or 10-band equalizer, with the
bands spaced an octave apart, could be achieved after up-sampling, instead of before up-sampling, but again, much more cheaply in terms of computational power required.

Dirk

Overheated Circuitry

One of the things which I do frequently, is ‘walk around’, or, ‘use public transit’, with my disposable earphones plugged in to my Samsung Galaxy S9 Smart-Phone, and listening to music. These earphones are clearly not the ones, which had the AKG seal of approval, and which shipped with the phone. But this week-end marks the second heat-wave this Summer, when outside daytime temperatures exceeded 31⁰C, with direct sunlight and not a cloud in the sky. And under those conditions, the battery of my phone starts to hit a temperature of 42⁰. One of the facts which I know is, that Lithium-Ion batteries like the one in my phone do not tolerate temperatures exceeding 41⁰C.

A peculiar behaviour which has set in for the second time, during this second heat-wave of the season, is that the music I was listening to would either back-space to the beginning of the song, or skip ahead one song, or just stop. So, a catastrophic sort of explanation I could think of would be, that the entire phone, with its battery, is finally just having a meltdown. But, a second possibility exists, that merely the chip in the earphone-cord could be malfunctioning. After all, the little pod in the earphone-cord has one button and a mike, and it’s actually cheaper to mass-produce the chip that makes it work, than it would be to mass-produce other sorts of discrete components. One cheap chip could just be malfunctioning in the extreme heat, and not the entire, complex circuitry of the phone. (:1)

The earphones cost me about $15, while the phone is much more expensive than that.

But even if it was true, that only the little remote-control in the earphone-cord was malfunctioning, this can lead to impractical situations, because just random patterns, of unreal button-press-combinations, could also send the software of my phone into a confused state, and even so, if the circuitry in the smart-phone never malfunctioned. This behaviour could get misinterpreted by the security apps of the phone, let’s say, as though somebody had ripped the earphone-cord off my head, and thrown all my possessions around.

All that was really happening was that my music was no longer playing, as I was walking home normally, in the heat, with my overheated electronics. And when I got home, my actual phone never displayed any signs of having malfunctioned.

(Updated 8/17/2019, 17h50 … )

Continue reading Overheated Circuitry

Trying to bridge the gap to mobile-friendly reading of typeset equations, using EPUB3?

One of the sad facts about this blog is, that it’s not very mobile-friendly. The actual WordPress Theme that I use is very mobile-friendly, but I have the habit of inserting links into postings, that open typeset Math, in the form of PDF Files. And the real problem with those PDF Files is, the fact that when people try to view them on, say, smart-phones, the Letter-Sized page format forces them to pinch-zoom the document, and then to drag it around on their phone, not getting a good view of the overall document.

And so eventually I’m going to have to look for a better solution. One solution that works, is just to output a garbled PDF-File. But something better is in order.

A solution that works in principle, is to export my LaTeX -typeset Math to EPUB3-format, with MathML. But, the other EPUB and/or MOBI formats just don’t work. But the main downside after all that work for me is, the fact that although there are many ebook-readers for Android, there are only very few that can do everything which EPUB3 is supposed to be able to do, including MathML. Instead, the format is better-suited for distributing prose.

One ebook-reader that does support EPUB3 fully, is called “Infinity Reader“. But if I did publish my Math using EPUB3 format, then I’d be doing the uncomfortable deed, of practically requiring that my readers install this ebook-reader on their smart-phones, for which they’d next need to pay a small in-app purchase, just to get rid of the ads. I’d be betraying all those people who, like me, prefer open-source software. For many years, some version of ‘FBReader’ has remained sufficient for most users.

Thus, if readers get to read This Typeset Math, just because they installed that one ebook-reader, then the experience could end up becoming very disappointing for them. And, I don’t get any kick-back from ImeonSoft, for having encouraged this.

I suppose that this cloud has a silver lining. There does exist a Desktop-based / Laptop-based ebook-reader, which is capable of displaying all these EPUB3 ebooks, and which is as free as one could wish for: The Calibre Ebook Manager. When users install this either under Linux or under Windows, they will also be able to view the sample document I created and linked to above.

(Updated 1/6/2019, 6h00 … )

Continue reading Trying to bridge the gap to mobile-friendly reading of typeset equations, using EPUB3?