May 27, 2008

four quick notes about j2me

one: Obfuscation must be handled with care.
If you choose to obfuscate your midlet, you gain two things: the final size will be reduced, and the contents will be hard to disassemble.
But. If you turn on obfuscation in NetBeans, it will be used for your local debug builds as well. Due to that, you will lose debugging information, notably filenames and line numbers in stack traces.

two: Version strings must be handled with care.
My phone (SE K610i) refused to install a midlet with version "0.2.991", or anything longer than 6 characters. A different phone (Siemens CX65) installed a midlet with version "0.2.99sync1", which then mysteriously crashed with a NullPointerException that wasn't there with version "0.2.99".
This is bad magic. If anyone can explain it, please do.

three: From what i gathered, there is no way to list files contained in a JAR at runtime.
If you want to do that, create a text index at compile time and include it in a known location.

and four: Some older phones, even though MIDP 2.0, don't call Canvas.sizeChanged event.
Or maybe they do, but not (citing the spec) "at least once before the Displayable is shown for the first time." [here] Or maybe they even do that, but don't specify the proper size.
So don't rely on it.

May 22, 2008

openWIG status report

Version 0.2.99 was released sometime this night, as opposed to what I wrote last week.
It has pretty navigation and crude but functional reading from filesystem. Everything important is on the project page, follow the link on your right.
...oh, and did I mention that we can now decode the GWC format? Well, we can. Feel free to load your memory card with cartridges and head outside.

good news about j2me's I/O

Well yes, it is horrible, but not nearly as horrible as I originally thought.
Once again, I misinterpreted a part of the spec. You do need to reopen the InputStream if you want to seek() back, that's true, but. If you keep a FileConnection object for the file in question, you can call getInputStream() on it as many times as you want - the security prompt is shown only for first access, then the phone is supposed to remember that you already have permission to do that (and it appears that many (if not most) phones actually adhere to this).
I assume that this holds true for other functions on a FileConnection as well, but I didn't test that yet.

Oh, and that RMS caching idea I had was slow and stupid.

May 14, 2008

openWIG status report

Just a quickie: navigation is almost done, filesystem access is not.
But, since i had to employ quite a bunch of clever tricks (let's call them that, at least until they all fail spectacularly), i need guinea pigs to test them on as many phones as guineapigly possible.
There will be a highly experimental version 0.2.99, as a precursor to actual 0.3.0 (which, according to my roadmap, should have at least partially usable filesystem access). I will probably release it sometime this night.

j2ME has no seek() method

let me stress this: J2ME doesn't have seek() method.
How gay is that? (pardon my french)

File I/O in Java has been traditionally too generic to be of any use (as are most parts of its standard library), but there was always a way to do what you need without excessive hassle.

Not true in J2ME.
Imagine that you are trying to read data from a file in a certain format. The file contains three distinct JPEG images. First one starts at offset 500 and has 1000 bytes. Second one starts at offset 1501 and has 1000 bytes as well, and the third one starts at offset 2502 and has also 1000 bytes, not that it matters here. Let's pretend that you know all this beforehand.
When you open the file, you get only an InputStream. If you want to read the second JPEG, you call stream.skip(1501), which, hopefully, brings you to the start of the data. However, unlike traditional seek(), this means that you are actually reading the first 1501 bytes and silently discarding them.
Let's just hope that it's optimized to do a seek() under the hood.
There are basically three options what to do next. In order of increasing stupidity:
  1. read() 1000 bytes into a byte array and work with that. If you happen to be using an API function expecting an InputStream, construct a ByteArrayInputStream over your array and use that.
  2. Implement your own LimitedLengthInputStream, which allows API functions to read up to 1000 bytes from the original stream, while catching all the crucial calls like close() in order to preserve the stream for further use.
  3. Hand over the original stream as-is and hope that it will "just work" with all the trailing garbage in the stream. Added value of this approach is that it effectively kills the stream and you need to open a new one.
All was fine up to this point, now let's say that you want to return to the first image and read that. How do yo go about it?
The answer, as is common to Java, is surprisingly simple: you don't.
That's right. There is no reliable* way to go back in an InputStream. And, in J2ME, you have nothing but InputStreams. Bad for you, go open a new one.

* Well yes, there is this mark()/reset() function pair. In theory, you mark() a position in a stream and then reset() when you want to return to it. In practice, my phone said "mark/reset not supported".
(This was on a resource stream, maybe it will work on file streams, but somehow i doubt it. Plus, if it failed on one phone, it will fail on others.)

Oh, and about opening a new stream - did I mention that if your midlet isn't certified, each attempt to getInputStream() gives a pop-up warning to the user? As does basically any other operation on the filesystem?
That has enough material for a medium sized rant on its own.

So, for now, my solution is to walk through the file in one pass and save the individual files from it to a RecordStore. Then retrieve them at will.

UPDATE:
The situation with pop-ups is not nearly as horrible as it appeared at first. See my new article on the subject.

Dilution of Precision

a.k.a. DOP, a.k.a. PDOP, HDOP and VDOP. And, in some very special cases, TDOP, but i haven't seen that one and have no idea what's it good for.
What they are and how to make heads (but not tails) of them.

In today's GPS applications, you will see those values under "precision". Plus, if you happen to be writing a GPS application, you will see them in NMEA messages. When people compare their GPS receivers, they compare precision by DOP (at least I and my friend did).
But unless the application is pretty clever, it won't tell you the actual position error, only the DOP values. And, unless you as a programer are pretty clever, you can't read the actual position error from the GPS receiver! Which is slightly worse.

So. As some of you probably already noticed, DOP stands for Dilution of Precision. HDOP is Horizontal DOP (that is latitude and longitude), VDOP is Vertical (altitude). PDOP is Positional, which is an overall measure containing both horizontal and vertical, and TDOP is Time DOP, which doesn't seem to be good for anything useful.
In various sources you can find that DOP values are dimensionless and they express "geometric strength" of satellite configuration. And that lower is better.
Which is all nice, but doesn't tell you anything about what the hell those numbers mean.

So here goes: DOP values are coefficients. That is the most important thing to know about them. To obtain the position error range in some meaningful units, e.g. metres, you multiply the GPS receiver's precision by the DOP value.

That's all there is to it. Just take a DOP, multiply it by a known value, and there you go, now you have a meaningful number. Piece of cake, really. Why couldn't they tell you earlier, right?

Alright, i confess, that's not all. This is where the real fun begins.

For one, there seems to be no way of determining your GPS unit's precision. It certainly won't tell you through NMEA. So you can either read the full-of-propaganda manual, or take an educated guess.
My educated guess, based on GPS system properties and quality of today's receivers, is that average precision is 5 metres. In fact, my favourite GPS app, TrekBuddy, shows HDOP x 5 m as an error range. And for openWIG I'm going to do the same.

And for two, while some receivers (such as my NaviLock BT-451 with uBlox Antaris4 chip) exclude signal strength from DOP (that is technically correct approach, since DOP is geometric strength), others apparently count it in (technically wrong, but way more useful).
Either that, or SiRF starIII chip's geometric capabilities are way below Antaris's. What's a poor guy like me to think, eh?

Anyway, now you know what a DOP is. Have fun dealing with it, I certainly will.

May 7, 2008

openWIG status report

In short: the fun is over, now begins the hard work.

In longer...
Well.
Version 0.2.6, released some two weeks ago, doesn't have it all, but it has most of it: rather reliable zone detection and navigation, support for all basic Wherigo constructs (including Timers) and their events, images (preloaded in the jar file), PlayAnywhere features...

There are still quite a few loose ends in the Lua library, and i'm looking forward to tying them up. But unfortunately, there is a number of things whose priority is much higher.

One, navigation screen.
What we have now is something along the lines of "239,2m northwest". Which is nice, but not really precise. What we need is a graphical display with an arrow pointing towards the destination. Perhaps with some additional info - distance, target/current coordinates, speed, signal strength...
HandyGeocaching has it all, and i'll probably borrow most of the code. But to be honest, i dread the moment when i'll have to open the HG project again. Don't get me wrong, there's nothing with the code ... it's just that it's a Netbeans 5.5 project with visual designers, and it really really dislikes Netbeans 6. Or vice versa. Anyway, it's so ugly it's not even funny.
I'm actually thinking of writing my own navi display from scratch, just to avoid this :e/

Two, preferences and persistence.
This is where it starts to get funny - the midlet reached a level of complexity where some kind of preferences screen is necessary.

  • GPS mode - Manual, Bluetooth, Built-in, Serial port.

    The latter two are not done and i'll again have to dive into HG to fetch them. But that doesn't matter, bluetooth alone is enough fun. Last connection info should be persistent. And Manual mode has a few differences of its own.

  • Navi screen refresh intervals.

    Turns out that some phones of a certain vendor who shall remain nameless (cough cough Nokia cough) (and possibly some others, but that is yet to be seen) take a screen update as an instruction to scroll up. Which is all fun and games until somebody loses an eye, i mean, until you notice that the navigation info is on the bottom. And since the refresh interval is 1 second (synchronized to game engine ticks and presumed GPS ticks), unlucky owners of the affected phones can't really see the navi info.

    If the screen was to refresh at, say, 10 seconds, their direction would not be so precise, but at least it would be readable.

  • Last opened cartridge, information about cartridge directory, logs directory etc.

    All those things that suddenly appear when you load and save files to the filesystem.


Which brings us to...
Three, filesystem access.
Yesterday I checked in a class that can load things from gwc cartridge. It is useful by itself, no doubt about that - at least it will enable people to save their cartridges into the jar without any external tools (apart from a zip, obviously, but everyone has zip today). But to make it really interesting, you have to be able to open things from a memory card.
That means filesystem classes plus browser gui plus persistence (see above) and a whole lot of other minor changes. Such as...

Four, brand new main-screen UI.
Because the current one doesn't really expect to have more than one cartridge. We need file browser (see above), cartridge selector with preview (splash screen, description etc.), Preferences screen (see above) and ability to close cartridge and open a new one.

And the best part is that this all needs to be done in one step, otherwise it won't make any sense.
Well, the navi screen i can release separately. But now i have gwc reading, not navigation. And gwc reading doesn't make sense without filesystem. Et cetera et cetera.