Wednesday, April 27, 2016

Rejsekortscanner Pro

Forleden frigav jeg version 2.0 af Rejsekortscanner, der repræsenterer en væsentlig stabilisering af fortolkningen af rejsekortet samt forbedringer i brugeroplevelsen. Men alt dette blev faktisk sat i gang p.g.a. folk der skrev til mig, at de ønskede en version uden reklamer - en sådan version er nu tilgængelig!



Versionen uden reklamer har jeg valgt blot at kalde Pro, fordi noget skulle den jo hedde der adskilte den fra gratis versionen. App'en henvender sig til folk der bevidst hellere smider en skilling istedet for at være udsat for et reklamebanner. Jeg hader selv reklamer, hvorfor jeg har været tæt på helt at droppe reklamebanneret. Men selv om man ikke ligefrem bliver rig af at have en niche app der er installeret hos 17.000 brugere, så får man trods alt betalt lidt mobilregning mv. og der ligger stadig rigtig mange timers arbejde bag Rejsekortscanner.

Spørgsmål og svar

  • Er 25kr ikke lige dyrt nok?
    Næææ det synes jeg egentlig ikke. Momsen udgør 5kr, Google skal have 6kr og skattefar skal have 7kr - så jeg tjener omkring 6kr netto, hvilket er nok til en Mars bar!

  • Hvorfor har det taget så lang tid?
    Udover at dette er foregået i min fritid, så har jeg også følt at kvaliteten skulle væsentligt i vejret når man begynder at tage betaling for noget. Der er også brugt meget energi på at få en masse gode test eksempler op. Jeg har ikke resourcer der tilnærmelsesvis kommer tæt på hvad Rejsekort f.eks. selv kunne stille med, så jeg er dybt afhængig af de fejlrapporter brugere indrapporterer som typisk ender med at indgå som tests. Sidst men ikke mindre, jeg er nok lidt af en perfektionist - kompromiser kan være nødvendige, men jeg bryder mig ikke nødvendigt om dem.

  • Er den eneste forskel ved Pro versionen, at der ikke er reklamer i den?
    Ja, men dette betyder dog også, at Pro versionen fylder mindre, er knapt så ressourcetung og derfor også starter hurtigere.

  • Hvorfor kan app'en ikke installeres på min enhed?
    Det skyldes desværre, at app'en kræver en enhed med understøttelse for en bestemt type NFC ved navn Mifare Classic. Det er kun NFC chips fra firmaet NXP der har denne support og det svinger meget hvilke fabrikanter og modeller der bruger disse. Derfor forsøger jeg at styre hvad Google Play tilbyder, men det er lidt af en umulig kamp da nye enheder hele tiden kommer til.

  • Hvilken enhed skal jeg bruge for at kunne benytte app'en?
    Generelt så kan mange Sony modeller (Z1, Z3, Z5) læse rejsekortet, ligesom en del fra HTC (M7, M8, M9) men det er kun lidt ældre modeller fra Samsung der med sikkerhed kan bruges (S3 og S5). Huawei (P7 og P8) har også enheder på markedet der kan benyttes. Det samme gør sig gældende for LG (G3 og G4). Google's egne Nexus 6P og Nexus 5X skulle også være et sikkert valg. Men hvis du står og skal til at købe ny mobil og ønsker at kunne følge med på dit rejsekort, er det sikreste simpelthen at prøve app'en i butikken før du køber - det har jeg selv gjort et par gange.

  • Kan jeg teste app'en selv om den ikke er tilgængelig via Google Play?
    Ja det kan du godt, men det kræver du slår "Ukendte kilder" til inde under sikkerhedsindstillingerne på din enhed. Dernæst kan du installere den rå APK fil uden om Google Play, som du kan downloade her. Jeg hører meget gerne om dine erfaringer så jeg kan opdatere listen på Google Play - email mig på casper+rejsekortscanner@bangbits.com.

  • Hvad er fremtidsplanerne for app'en?
    Jeg kunne rigtig godt tænke mig at kunne vise rabatniveauer (rabattrin) samt serviceniveauer (standard, DSB 1' mv.) til brugeren så det er nok det næste jeg vil kigge på.



Thursday, April 21, 2016

Rejsekortscanner 2.0

Jeg har i længere tid arbejdet på en betalingsversion af Rejsekortscanner, til dem der efterspørger at komme af med reklamerne. Men når man lancerer noget man tager betaling for, vil kunder naturligt forvente høj kvalitet og jeg følte ikke helt at den gamle version kunne leve op til dette.

Dette betød, at jeg i større grad skulle benytte mig af den dokumentation jeg har fået adgang til fra Rejsekort A/S, ligesom jeg havde brug for en stor pulje af tests for at sikre imod regressioner - for jeg har stadig begrænset adgang til den forretningslogik der ligger til grund for rejsekortet (hvad er den maksimale rejsetid, hvad er den maksimale transit tid, hvordan hånderes vintertid osv.).

Derfor besluttede jeg mig for, at både den gratis (med reklamer i) samt kommende betalingsversion, skulle baseres på samme kode og derfor begyndte arbejdet først og fremmest på en version 2.0 af den gratis udgave. Det har taget længere tid end ønsket, men det var dét der skulle til. App'en er altså blevet skrevet helt om fra bunden af, med et utal af forbedringer. Her er et par af dem:
  • Afkodningen af data på rejsekortet er langt mere troværdig da jeg har haft adgang til den officielle dokumentation fra Rejsekort A/A
  • Over 600 unit tests bliver brugt til at sikre imod introduktioner af nye fejl (regressioner)
  • App'en har fået et friskt nyt design, fordelt på skærmene "Status", "Hændelser" samt "Detaljer"
  • På trods af, at app'en henter meget data ud, er den blevet hurtigere til at kommunikere med rejsekortet, typisk 1/3 sekund
  • App'en vi nu fortælle dig om du har glemt at checke ud hvis det er længere end 12 timer siden du er checket ind
  • Sommer/vintertid-problematik håndteret som det skal - dette har altid være problematisk i den gamle version
  • Understøttelse af Dansk og Engelsk sprog - før kunne kun benyttes Dansk
  • Minimeret programstørrelse til blot 1.9Mb - på trods af nye features og bedre grafik


Et af de få minuser jeg bør nævne er, at jeg har været nødt at fjerne supporten for gamle Android 2.3 (API 10) enheder således at minimumskrevet nu er Android 4.03 (API 15). Der var mindre end 0.1% af mine brugere der benyttede sig af så gamle enheder, at det ganske enkelt ikke længere gav mening at bruge energi på at holde app'en kompatibel.

Betalingsversionen kommer meget snart, jeg skal lige have testet den nye kode via gratis-versionen først. Herunder kan du se nogle eksempler på skærme fra den nye app.

Status skærmen for et kort der er checket ind


Status skærmen for et kort der er checket ud


Status skærmen for et kort der ikke er checked ud i tide


Status skærmen for et kort der er blevet blokeret p.g.a. misbrug og lignende


"Hændelser" skærmen hvor de seneste rejser kan aflæses


"Detaljer" skærmen hvor information om kortnr, type, status, saldo, optankningsbeløb mv. kan aflæses




Find app'en i Google Play, eller hent den hér via Dropbox. Pro versionen uden reklamer skulle også lande på Google Play i løbet af nogle dage.

Tuesday, April 19, 2016

Beware of SQLite and the Turkish Locale

Today I came across a truly puzzling issue on Android. An otherwise tried and tested application crashed consistently when running on a device using Turkish as the current locale.

App crashing on a Turkish device

The problem occurs in a large app developed for a customer, inside a proprietary binary component, so no direct debugging was possible. All I had available was a vague stack trace showing the root problem to be a NullPointerException from trying to parse an integer:

Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Integer.intValue()' on a null object reference

Not much to go on, the goal was now to figure out just what could cause such a problem just from running on a device using Turkish Locale.



Having spent some time on Google, searching for other people's trouble with the Turkish Locale, it became clear that there are issues with lowercase Strings in Turkish. I started working on a small isolated app which would aid in narrowing down the exact issue. It wasn't long until the suspicion I had turned out to be spot on.

SQLite with Turkish Locale

The problem was caused by relying on the device Locale for representing SQL Strings and the fact that the SQLite parser must be converting the SQL literals to uppercase. That becomes a problem, because in Turkish the lower case "i" becomes "İ" in uppercase. Inside SQLite, the parser is therefore seeing "İNSERT" rather than "INSERT" which is of course not valid SQL. This is actually specified in the Android documentation for the Locale class and the subject was also covered by none other than Jeff Atwood some years ago.

To reproduce the problem, create a new project in Android Studio with an empty activity. Place the following XML for the layout:

<?xml version="1.0" encoding="utf-8"?> 
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" 
    android:layout_width="match_parent" 
    android:layout_height="match_parent"> 
 
    <Button 
        android:layout_width="300dp" 
        android:layout_height="48dp" 
        android:text="SQL insert in lowercase" 
        android:gravity="center_horizontal" 
        android:onClick="insert"/> 
 
</RelativeLayout> 


...and add the following Java code to the activity:
public class MainActivity extends AppCompatActivity { 
 
    private final static String DML_CREATE = "CREATE TABLE IF NOT EXISTS demotable(Name VARCHAR);"; 
    private final static String SQL_INSERT = "insert into demotable values('John Doe');"; 
 
@Override
protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(DML_CREATE); } setContentView(R.layout.activity_main); }
public void insert(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT); } } }

Although being perfectly valid Java and SQL, the code above will crash consistently when running on a device with a Turkish Locale!

Caused by: android.database.sqlite.SQLiteException: near "İNSERT": syntax error (code 1): , while compiling: İNSERT İNTO DEMOTABLE VALUES('JOHN DOE');

The stack trace makes it blatantly obvious what happens. If only I had such a nice stack trace originally I could've saved a few hours, but such is life when you depend on proprietary closed source components. :(

Solutions

There are two typical proposals for how to solve the issue. This can either be done by always converting SQL Strings to upper-case:
 
 
public void insertUppercase(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT.toUpperCase()); } }

...or (better in my opinion) make sure SQL Strings are always interpreted using the root locale by converting to lower-case while providing the neutral root locale:
 
 
public void insertLowercaseEnglish(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT.toLowerCase(Locale.ROOT)); } }

Although I have not tested it, it's also likely that a solution could be to externalize the SQL as a simpler character set (ASCII) rendering the the issue moot. However, most developers will probably just embed SQL in Java unicode Strings. Feel free to comment on this if you have other solutions.

Lesson learned

Something tells me this is a very common bug out there, the intricate details of "i" in the Turkish Locale are probably only known to Turks. The only reason why more Java developers don't run into this, is probably because they typically operate and control the host runtime. However, when dealing with an app, the environment is not as neatly encapsulated and testable so this problem bubbles up to the surface and hits you hard.

The lesson learned must be to never rely on the default Locale, not even for something as innocent as Java Strings. While we are able to remedy the issue when using SQL as an embedded DSL, one can easily imagine the same bug sneaking in countless other ways. For example, annotations come to mind, where you do not have the luxury of being able to manipulate the String before its sent to be parsed by some annotation processor!

Monday, April 4, 2016

Getting the reference address of a variable on Android

In Java, we've always had our share of undocumented fun through the sun.misc.Unsafe class. You can allocate memory, access data fast without range checks and - what this post is about - get the memory address of some object.

Motivation

So why would you do this in a memory managed language like Java? Well, it's arguably rare you have an actual need, however it can come it handy at times. For instance, I needed it when debugging some weird behavior on Android. I had a strong suspicion that the (insanely complex) Android life-cycle and associated serialization aspects were the cause of me seeing an object instance mutate (I.e. change its hash value). Unfortunately Android Studio doesn't support putting a watch on a variable. What I really needed was a way to document identity equality, just as we can document value equality with the hashCode() mechanism.

Android specifics

It's not hard to find examples of the use of sun.misc.Unsafe on the Internet. Using it on Android rather than the JVM however, means we're operating under some fairly unique constraints. For once thing, you can not import the sun.misc.Unsafe class at all! Secondly, while Android does include an implementation of sun.misc.Unsafe at runtime, it differs substantially from the official unofficial (if you know what I mean) implementation we know from the Oracle JVM. However, by finding the relevant source code at the AOSP, and some reflection magic, we have enough ammo to attack the problem. The end result is the following UnsafeUtils class.

import java.lang.reflect.Field;
import java.lang.reflect.Method;

/**
 * Utility class for working with the undocumented java.misc.Unsafe class
 * on Android.
 */

public class UnsafeUtils {

    private static Object getInstance(){

        try {
            Class clazz = Class.forName("sun.misc.Unsafe");
            Field theUnsafe = clazz.getDeclaredField("THE_ONE");
            theUnsafe.setAccessible(true);
            return theUnsafe.get(null);

        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
        catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }

    public static long addressOf(Object o) {
        final Object[] array = new Object[] {o};

        final Object unsafe = getInstance();
        try {
            final Method arrayBaseOffsetMethod = unsafe.getClass().getMethod("arrayBaseOffset", Class.class);
            int arrayBaseOffset = (int)arrayBaseOffsetMethod.invoke(unsafe, Object[].class);
            final Method getIntMethod = unsafe.getClass().getMethod("getInt", Object.classlong.class);
            return (int)getIntMethod.invoke(unsafe, array, arrayBaseOffset);
        } catch (Exception e) {
            e.printStackTrace();
        }
        return 0;
    }
}


Using UnsafeUtils

Using the utility class is just a matter of getting an instance and calling the addressOf(...) method with the target object as an argument.


        String n = "java";
        System.out.println("n=\"java\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = "rocks";
        System.out.println("n=\"rocks\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = "java";
        System.out.println("n=\"java\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = new String("java");
        System.out.println("'n=new String(\"java\"): {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");


The code above will output something along the line of the following - keep in mind, the memory addresses listed will be different with each run.

n="java": {hashCode=3254818,addressOf=323913376}
n="rocks": {hashCode=108686766,addressOf=323913568}
n="java": {hashCode=3254818,addressOf=323913376}
n=new String("java"): {hashCode=3254818,addressOf=323913856}



Notice how changing our code from referencing a String with "java" to "rocks", makes it point to a new address. Changing it to point to "java" again reuses the existing constant in the pool due to a the Java compiler being smart about it (it treats Strings somewhat unique). Making a new reference using the "new" operator, bypasses this compiler optimization and allocates a new instance on the heap.

If you play around a bit, you will also start to notice the size of references and you can also easily distinguish the difference between objects created on the stack vs. the heap.

Disclaimer

Take this for what it is, a helper utility for understanding what's going on and for debugging purposes. The code may be buggy, may not work on all versions of Android/Dalvik/ART and is definitely not production safe. Although the code is adapted for my use, it's very unoriginal - so assume a Public Domain license. Do with it what you want, just don't hold me accountable. :)


Tuesday, January 5, 2016

The letdown by Air Canada

This is the personal account of me and my family's recent troublesome journey and treatment by Air Canada, when traveling home to Denmark after having spent christmas in Montreal, Canada 2015.


Toronto Pearson airport, my daughter sleeping in a corner while my wife is watching over her.

Let me clarify that we are not unaccustomed to international traveling, having crossed the Atlantic at least 80 times over the previous 15 years. I’ve personally experienced my share of cancelled flights, delays and hotel stays etc., some of which I have even blogged about before. However, what I describe in the following takes the price as the worst travel experience ever, not least because it has to do with my infant daughter. I may get a detail or two wrong because of the stressful circumstances, but it has been written down a day after arriving home and cross-checked by my wife so can assumed to be quite accurate. My goal with this is first and foremost, to remember why Air Canada is never going to have me as a customer again and secondary as a base for filing an official complaint when I have recovered some energy.

You can read the full account in this Google Docs document