Monday, September 19, 2016

Best-practice Android logging

A blog post about Android logging? Must be a slow news day you are probably already thinking, but hear me out on this one. This is a quick tour around my personal best-practices and you may likely not agree with all of them - if so, let me know in the comments. ;)

Android Studio and LogCat

On Android, logging has always been an integral part of the development experience, first through the standalone LogCat utility and since through the Android Studio integration. The build-in LogCat now even offers advanced filtering capabilities based on logging level, regular expressions for package names, logging tag and the actual log messages. However, all this power is really only useful if you play along nicely at the source level as well, or you will drown in log messages without really knowing how to specify what you want to see. This is a common problem as the Android OS and any libraries you may use will spit out an obscene amount of logging, especially at the verbose level.

Use correct log levels

This isn't really Android specific since it's a problem all over the Java space. Many projects don't classify log severity correctly and some can't even be bothered to use logging at all and pipes stuff to System.out and System.err. While there are a LOT of logging frameworks for Java and they differ in functionality, most of them agree on the logging levels. On Android, the levels are:
  • ERROR – Don't use; if something this severe has happened, it is the result of an exceptional state rendering the application context invalid! The application should crash gracefully and collect relevant info for submission as a crash report to Google, CrashLytics, TPA etc.
  • WARN – Something bad happened, but the application is still able to recover and continue executing albeit perhaps in a limited fashion. A developer can filter on the WARN level in order to learn about application states which are candidates for being handled better.
  • INFO – Important business process that changes the state of the application significantly (database update, external system request, network connectivity change etc.) has completed. A developer can filter on the INFO level in order to get the big picture of what is going on at the task and flow level, which a non-developer should also be able to understand.
  • DEBUG – Detailed developer insight into core decisions, states and events.
  • VERBOSE – Don't use; very detailed trace information, intended only for temporary development insight (debugging and troubleshooting), should not risk being left in code and potentially be checked into Git (you are using Git right?). Instead you'll want to use a logging breakpoint as explained below.
I think one of the problems with logging being used wrong it that it takes too much energy deciding on the appropriate level. By reducing the levels down to just 3, it becomes somewhat simpler to decide which one to use. Use only WARN, INFO and DEBUG and use them accordingly; WARN for things that should not happen but is recoverable, INFO for major changes and DEBUG for minor changes.

Logging breakpoint

I mentioned in the above section, that VERBOSE should not be used. This is because the stuff you'd put at the VERBOSE level is so detailed that it doesn't belong in source code (logging statements riddled everywhere makes code harder to read). Instead, Android Studio offers a powerful hybrid mechanism between debugging and logging - basically it allows you to set breakpoints and rather than have these suspend program execution as usually happens, they merely log an expression to LogCat.

This makes so-called logging breakpoints a perfect fit what you would otherwise have put in verbose logging statements at the source level. You can see this feature demoed here.

LogCat Filtering

As an alternative to viewing the complete system log, the "Show only selected application" context filter is not too hard to understand and use, but even this filter will result in a lot of irrelevant garbage from app and OS libraries being used from the app process. I would like to be able to recommend using custom logging filter configurations, but unfortunately I find them too unreliable to trust in practice. For instance, it seems like (with the present version of Android Studio anyway) there is no way to whitelist debug and verbose messages (anything more detailed than info messages) and that's just a no-go for me. However, it's Ok as we can filter at a line level instead and accomplish much the same. Unless I'm hunting for something specific, I always use a black-list approach, since you never want to exclude relevant messages you do not know about in advance! This means entering a rather long and complex regular expression, but it only has to be done once. The following filter clause is a common one I use, but it will vary a lot depending on the amount of noise being generated on the specific device you are using:
This way it becomes a joy to see what's going on and simply jump up or down in the severity filter depending on how fine grained you want to follow along.

Tweak the LogCat settings

To the left of the LogCat window is a vertical toolbar which you may also want to utilize a bit more - it took me a while before I started taking full advantage. Press the cog wheel to go to Settings to configure the LogCat headers and remove the checkbox in "Show date and time" as well as "Show process and thread IDs". This will give you 30 characters more on each line for actual valuable info. If you need to time something, there are better ways to do so using profiling tools like i.e. systrace. Sometimes I also remove the package name since the truly interesting part remains the TAG and associated message. While you are in the LogCat toolbar, familiarize yourself with the other options. Soft wraps can be very handy when you'd rather not miss something in a large message - typically serialized data. The restart button is also going to be needed from time to time, as adb or its integration with the device, is notoriously unstable and can play tricks on you.

Even with filtering in place, sometimes you just have groups of related log info you'd rather be collapsed somehow. This is easy to do by right-clicking on the line in question (typically first line of what you want hidden in the fold) and writing an expression to match what will get folded. It's also here you can shorten excessive stack-traces and hide irrelevant internal calls.

The above screenshot shows 3 similar lines for each Firebase remote config property loaded. The screenshot below shows how the 3 related yet different lines are now folded up under the parent "Successfully obtained Firebase remote config" log line.

Last but not least, the color scheme used by Android Studio for LogCat can be improved upon a bit by making level-indication a bit more obvious (this becomes even more important when you are running a dark theme). I like to make all levels different in color and style so its easy to identify.

Log application aspects rather than class names

While the vanilla approach to writing logging statements on Android is to define a TAG constant with the same name as the class, this is a stupid practice in my opinion. Sure it is easy to adhere to (no thinking required) and can work well for small projects with only a few activities or views. This however, does not scale to projects with 50+ views and/or apps with complex life-cycles - it is simply too difficult to troubleshoot a problem and sooner rather than later you'd have to resort to a slow and tedious debugging session. So what to do instead then?
During execution of an app, one is typically interested in certain course-grained aspects of an app, and probably not so much which file logic lives in. For this reason, I like to define global logging tags based on app aspects. It doesn't matter too much what you choose for these aspect names, but make them generic and/or something that makes sense to your app. I typically always have the following:

     * Logging tag used for common UI lifecycle events
    public static final String LOG_UI = "UI";

     * Logging tag used for any kind of network I/O communication
    public static final String LOG_NET = "NET";

     * Logging tag used for storage; local files, preferences and databases
    public static final String LOG_DATA = "DATA";

     * Logging tag used for business logic and app related things not
     * already covered by the other log tags
    public static final String LOG_APP = "APP";

Reading a log now becomes super easy and at the INFO level, even non-technical people should be able to follow along if need be:

Always log major UI flow events

Apps are, by definition, highly interactive and UI heavy. It thus makes a whole lot of sense, to trace UI aspects accordingly. The nice thing about UI events, is that it's surprisingly simple to get some broad standardized logging up and running which you won't have to worry about again when adding more views in the future.
To easily log UI lifecycle events for any and all activities, simply provide a custom application class and in its onCreate() method, register for lifecycle callbacks:

    public void onCreate() {
        Log.i(LOG_APP, "Creating Application");

        registerActivityLifecycleCallbacks(new ActivityLifecycleCallbacks() {
            public void onActivityCreated(Activity activity, Bundle savedInstanceState) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " created");

            public void onActivityStarted(Activity activity) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " started");

            public void onActivityResumed(Activity activity) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " resumed");

            public void onActivityPaused(Activity activity) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " paused");

            public void onActivityStopped(Activity activity) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " stopped");

            public void onActivitySaveInstanceState(Activity activity, Bundle outState) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " saved");

            public void onActivityDestroyed(Activity activity) {
                Log.i(LOG_UI, activity.getClass().getSimpleName() + " destroyed");

Oh and while you are in there, do yourself the favor and enable StrictMode for your debugging builds so that you bad dynamic behaviors of your app are logged.

    if (BuildConfig.DEBUG) {

Modern apps are often revolving around just one activity but many fragments instead, so it's equally if not more important (due to their insane complex lifecycle) to be able to log fragment events. It can be done in a similar way, by overriding onAttachFragment() in a base class for your activities:

    public void onAttachFragment(Fragment fragment){
        Log.i(LOG_UI, "Fragment " + fragment.getClass().getSimpleName() + " attached to " + this.getClass().getSimpleName()  );

Now you have an easy way to follow the UI flow of your application without which should help interpret the context of surrounding log messages. I find these major UI events to be important enough to deserve an "info" level classification. In fact, I also add various other UI/INFO level logging lines for menus etc, as in the following:

    public boolean onNavigationItemSelected(MenuItem menuItem) {
        Log.i(LOG_UI, "Option '" + menuItem.toString() + "' selected");

Logging templates

Android Studio is based on one of the most powerful IDE's available in modern programming so it also comes with handy templates. The logging templates available out of the box are context aware and will save you much time once you get used to them.

Simply enter the template characters and expand with the tab key. The templates "logm", "logr" and "logi" remains my favorites.

Don't log in production!

Logging is for development purposes - there are plenty of other tools for analytics and crash-reporting, so respect these separation of concerns and avoid compiling logging into your app. Only error, warning and info logs are acted upon at runtime, but verbose and debug levels are still compiled in. Regardless, there simply is no reason to keep any kind of logging for a production app - it's just unnecessary noise. I like to use ProGuard to strip the logging, which is a simple matter of adding the following to your ProGuard file:

-assumenosideeffects class android.util.Log {
    public static boolean isLoggable(java.lang.String, int);
    public static int e(...);
    public static int w(...);
    public static int i(...);
    public static int d(...);
    public static int v(...);

Note that if you are using other logging frameworks or custom wrappers, you may also want to target these explicitly to remove the bad weed "at the root" so to speak.

In conclusion

The IDE and associated toolbox is our friend, and we do best to get to know it well since much time and frustration can be saved on that account. Hopefully there's a trick or two in the above you take away from this blog post, but let me just end with showing a matrix I use conceptually when narrowing down a problem in an app based on the proposed logging levels and application aspects above.

The first matrix demonstrates filter settings when just wanting to following along and learn how the UI is put together when i.e. getting to know a new application. The second matrix is how you would think (and thus configure the logging) when wanting to screen an app quickly after a test suite has run, whether something exceptional happened anywhere. The last example shows how one would configure the filter when looking for issues in some app logic doing network communication.

Wednesday, May 25, 2016

Rejsekort, NFC og smartphone kompatibilitet

Det sker ikke så sjældent, at jeg får en mail på noget lignende følgende form:
Min telefon har da NFC, hvorfor kan jeg så ikke downloade din app?
Hvorfor sker ikke noget når jeg scanner kortet med din app?
Jeg savner et sted hvor man kan se om hvilke telefoner der er kompatible med din app (Rejsekortscanner). Jeg kan finde nogle spredte informationer skrevet af dig. Nogle er 2 år gamle på din blog. Hvorfor vedligeholder du ikke bare en liste?
Det er jo gode spørgsmål som jeg har svaret på rigtig mange gange de seneste par år. Det siger sig selv, at som udvikler er jeg naturligvis interesseret i, at gøre det så nemt og smertefrit for potentielle brugere som muligt - men Rejsekort og NFC er og bliver bare et kompliceret emne. Herunder har jeg forsøgt at bryde emnet ned i let-fordøjlige bidder.
  • Hvad er NFC?
    NFC (Near Field Communication) udspringer af envejskommunikation RFID (Radio Frequency IDentification), og tillader kontaktløs tovejskommunikation med et smartcard. Det er altså muligt, over en meget kort afstand, at kommunikere elektronisk med en chip på f.eks. et adgangskort, Rejsekort eller Dankort. Dette sker på frekvensen 13.56MHz og med en båndbredde på 106 kbit/s. Det var firmaerne NXP (tidligere Philips) samt Sony, der pionerene NFC i sin tid og i dag er især NXP stadig en stor spiller inden for NFC, men Samsung, Broadcom mv. er også kommet til. NFC teknologien er nu (delvist) standardiseret af NFCForum under ISO/IEC 14443A. Men sådan har det ikke altid været!

  • Hvad er Mifare?
    Før NFC blev hvermandseje og indbygget i de fleste mobiltelefoner, var teknologien mere eksklusiv og blev primært brugt til elektroniske adgangskort, billetter mv. Til dette var der behov for et såkaldt krypteret "sikkert element", altså et område på kortet man ikke bare sådan uden videre kan læse og skrive til - af hensyn til sikkerhed og svindel. En af disse teknologier fra NXP hedder Mifare og det er dén teknologi rejsekortet også benytter. Selv om NXP var med til at standardisere NFC, så var de smarte nok til at sørge for en udvidelsesmekanisme, således at de stadig kunne bibeholde deres egen overbygning, i form af Mifare - uden at dette stred direkte imod standarden. Ligesom NXP har Mifare som overbygning, har Sony iøvrigt deres Felicia teknologi.

  • Burde alle NCF telefoner ikke kunne læse rejsekortet?
    I teorien jo, for protokol og frekvens er helt den samme og andre producenter end NXP kan stadig godt få lov til at kommunikere med Mifare typen, mod at betale en licens til NXP og implementere det i deres driver. Problemet er bare, at det gør ingen af de andre store spillere som Sony, Samsung og Broadcom. Istedet læner de sig, måske fornuftigt nok, op af den åbne standard - uden at tænke på evt. propriotære udvidelser. Det er derfor en smartphone skal have en NFC chip fra NXP, for overhovedet at have mulighed for at kunne læse rejsekortet!

  • Hvorfor vælger Rejsekort A/S at benytte sig af Mifare?
    Jo det gør de, fordi det i mange år har været de-facto standard inden for elektronisk rejsehjemmel og fordi teknologien er fra før mobiletelefoner med NFC blev hvermandseje. Rejsekort A/S har også haft krav om en pæn portion hukommelse på kortet, p.g.a. det danske system er et nationalt asynkront system, så der var ikke rigtig så mange alternativer. De har også haft krav til sikkerhed, p.g.a. risikoen for svindel med data på kortet. I praksis er der dog ikke megen sikkerhed, da krypteringen blev knækket tilbage i 2007 ligesom nøgler ligger spredt rundt på internettet og i diverse apps. Sikkerheden af systemet må derfor formodes primært at blive håndteret via overvågning af backend systemerne. Rejsekort A/S er sidenhen overgået fra Mifare Classic til Mifare Plus der er sikret bedre mod hacking. Her i 2016 findes der åbne alternativer ligesom, eftersom den fysiske sikkerhed er ikke-eksisterende, man burde kunne formattere Mifare kortet til kun at bruge standardiserede NDEF records - så ville alle NFC telefoner i princippet kunne aflæse kortet, og så ville krypteringen blot ligge i software.

  • Findes der ikke en liste over telefoner et sted?
    Der er faktisk ikke et sted på nettet med troværdig information omkring en telefon har Mifare support eller ej, det bedste der findes er en liste på shopnfc men denne har ofte fejl og mangler. Jeg forsøgte faktisk også i starten at vedligeholde en liste, men det skulle hurtigt vise sig at være en uholdbar praksis. Der kommer nye telefoner, samt nye versioner af eksisterende telefoner til, og det er ingen nem sag at finde ud af, om den har en NFC chip fra NXP eller ej - nogen gange kan man finde ud af dette på ifixit, men her lander kun de største og mest kendte telefoner. Den eneste sikre metode er derfor simpelthen at gå ned i din lokale butik, og prøve at installere og afprøve f.eks. Rejsekortscanner.

  • Hvorfor virker Google Play filtreringen ikke altid?
    Prøv f.eks. at se det screenshot herunder, fra Google Play. Som du måske kan se, er der 432 manuelt udelukkede enheder alene fra Samsung! Dette er altså telefoner med NFC, men som jeg (af den ene eller anden årsag) ikke mener har en NFC chip fra NXP og derfor ikke vil kunne bruge min app Rejsekortscanner. Bemærk også, at der er flere forskellige S6, S6 Edge og S6+ modeller - så man KAN faktisk slet ikke skrive "Virker ikke med S6" for det kommer an på hvilken af de 15 versioner det drejer sig om. Samsung S4 Mini (som min kone har) har aldrig kunne læse Mifare, men seneste model kan sjovt nok godt, så nu har jeg specifikt åbnet for denne. Sidst men ikke mindst, ser jeg også en del rapporter fra brugere der har enheder som Google Play simpelthen ikke kender endnu (f.eks. PHICOMM PASSION).

Det bedste jeg p.t. kan gøre, er at medsende så meget jeg kan omkring en brugers telefon, når de indsender en fejlrapport i app'en. Dette er nogengange nok til, at jeg kan identificere den inde i Google Play og åbne på eller lukke ned for den. Så når der kommer nye modeller til (og versioner af modeller), har disse jo som udgangspunkt lov til at installere Rejsekortscanner. Kun når jeg bliver bekendt med modeller/versioner der giver problemer (typisk brugere der sender vrede mails eller giver én stjerne som bedømmelse) skynder jeg mig ind på Google Play for at tilføje i filtret. Det er dog ikke altid jeg når det i god tid (jeg har job, familie osv. som så mange andre) og så er jeg da lidt ked af at få hivet gennemsnittet ned på ratingen af app'en når det nu ikke er min skyld, men sådan er det bare.

Sidst men ikke mindst, er der nogen gange driver problemer på telefoner på bestemte versioner af Android. Eksemplerne på dette er:
  • Visse Samsung telefoner med Android 5, returnerer ved en fejl 17 bytes for en Mifare blok hvor den kun skal returnere 16
  • På visse HTC One kørende med Android 5 har er NfcA null i TechExtras og skal trækkes ud via et hack
  • På Sony Xperia Z3 findes der 2 NfcA med forskellige SAK værker i TechExtras, som skal patches med et hack
  • Nexus X5 som ser ud til at virke 1/10 gange om som derfor er blokeret indtil jeg kan finde en løsning

På grund af desse problemer, forsøger jeg nu at logge (via Google Analytics) når folk bruger Rejsekortscanner og de IKKE kan læse kortet p.g.a. manglende Mifare. Dette samler jeg op i noget statistik og snart vil jeg formentlig kunne bruge dette til at lukke ned noget hurtigere. Det betyder at jeg på sigt nok VIL være i stand til at lave en form for liste, men den vil være meget stor, særdeles teknisk og vil aldrig være så brugervenlig at hr og fru Danmark bare lige kan se "S5, jo den er på listen".

Håber det var et klart nok svar - i en desværre noget uklar NFC/Mifare verden.

Monday, May 2, 2016

Android: Remove a dependency from a build variation

Android's build variations (flavor and buildType) feature is an awesome way of maintaining several cuts from the same source code. I've used it both for white-label branding solutions and for Free vs. Pro versions. However, sometimes you have different dependencies between the versions where you will usually have to live with the fact, that all variations of your app gets the superset of dependencies. There are ways around this however, and I used it to shave over 400Kb off from a Free variation to a Pro variation.

The problem

Java is a static language, so you can't just go ahead and rely on late-bound linking and have Proguard slice and dice. It's unclear to me, whether the Android variations mechanism extends to Java source code, but I don't think it does. It's also unclear to me, whether an SPI approach could be used, letting the build class-path inject different implementations of some API for use at compile-time. What IS clear to me however, is that we can avoid pulling in a dependency for a flavor its not needed for, which means we will have to use reflection for the flavor that actually needs the stuff.

Removing AdMob from the Pro variation

It doesn't really matter what this app does, the important aspect is that there's a Pro flavor which does not contain the Google Ads from the Free version. So the first thing we're going to need, is to tell Gradle that the AdMob dependency is only for the free flavor. With Gradle, this is as easy as prepending the flavor name in front of the scope, such that "compile" becomes "freeCompile":

Now the dependency is not included for the Pro version. The next thing we need to do, is fix the problem of the static linking with the Ads dependency. The culprit code looks like the following:

protected void installAd(){ final AdView adView = (AdView)findViewById(; // Only the "free" versions of layouts has an AdView if(adView != null){ final AdRequest adRequest = new AdRequest.Builder().build(); adView.loadAd(adRequest); } }

The above works great in the Free version, but since we modified the Gradle build script, the Pro version no longer compiles! The second thing we then need, is a way to only use the Google Ads for the Free flavor, and do so through reflection in order to not require compile-time linking.

protected void installAd(){ final View adViewInstance = findViewById(; // Only the "free" versions of layouts has an AdView if(adViewInstance != null){ try { final Class builderClass = Class.forName("$Builder"); final Object builderInstance = builderClass.newInstance(); final Method buildMethod = builderClass.getDeclaredMethod ("build"); final Object adRequest = buildMethod.invoke(builderInstance); final Class adViewClass = Class.forName(""); final Method loadAdMethod = adViewClass.getDeclaredMethod("loadAd", adRequest.getClass()); loadAdMethod.invoke(adViewInstance, adRequest); } catch (Exception e) { Log.w(TAG, "Ads could not be loaded due to: " + e.getMessage()); } } }

Note that in the code examples above there's actually no build flavor check to be seen. For this, I rely on the flavor mechanism again. Only the layout from the Free flavor has the view. If this view can not be found, as is the case for the Pro flavor, I skip over doing anything.

The third and final step is easy to forget. The powerful post-processing tool Proguard will remove all unused classes in an attempt to trim down the size of our shippable production build. However, we now rely on reflection in the code, which Proguard's static analysis of the source code won't be able to tell. In stead, we need to instruct Proguard NOT to optimize away the specific code that we rely on. If we don't do this, the reflection magic will have no effect and in my case that means no ads will load but it's also not uncommon to see the app simply crash! Edit your and include the appropriate directives:

-keepclassmembers class* { 
   public *; 

And while I'm in there, I remove all that logging which serves no purpose in a production build anyway. I think it's safe to assume that no end user is sitting with adb and logcat!

-assumenosideeffects class android.util.Log { 
    public static boolean isLoggable(java.lang.String, int); 
    public static int v(...); 
    public static int i(...); 
    public static int w(...); 
    public static int d(...); 
    public static int e(...); 

There's a fourth and somewhat optional step to mention. The code that you removed, may also let you remove permissions from your application manifest. This is the case for me with the Google Ads, which requires "android.permission.INTERNET" and "android.permission.ACCESS_NETWORK_STATE". My Pro flavor no longer needs these, and no app should request more permissions than it needs, so lets remove this. Once again, the power of the Android flavor mechanism comes to the rescue. Previously I had just one AndroidManifest.xml located in /src/main folder. We can leave this one as the "master" and remove the AdMob specific stuff, which is not applicable to the Pro flavor. Then add a new AndroidManifest.xml under /src/free/ and include the stuff specific to the Free version, namely the requires permissions which will be merged into one.

<?xml version="1.0" encoding="utf-8"?> 
<manifest xmlns:android="" 
    <!-- This is needed for Google AdMob --> 
    <uses-permission android:name="android.permission.INTERNET" /> 
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/> 

That's it! The Pro flavor is now trimmed down to about 400Kb lighter. This results in a smaller download, speedier load-time and lower memory pressure. Do you know of a better way then please let me know though the comments!

Wednesday, April 27, 2016

Rejsekortscanner Pro

Forleden frigav jeg version 2.0 af Rejsekortscanner, der repræsenterer en væsentlig stabilisering af fortolkningen af rejsekortet samt forbedringer i brugeroplevelsen. Men alt dette blev faktisk sat i gang p.g.a. folk der skrev til mig, at de ønskede en version uden reklamer - en sådan version er nu tilgængelig!

Versionen uden reklamer har jeg valgt blot at kalde Pro, fordi noget skulle den jo hedde der adskilte den fra gratis versionen. App'en henvender sig til folk der bevidst hellere smider en skilling istedet for at være udsat for et reklamebanner. Jeg hader selv reklamer, hvorfor jeg har været tæt på helt at droppe reklamebanneret. Men selv om man ikke ligefrem bliver rig af at have en niche app der er installeret hos 17.000 brugere, så får man trods alt betalt lidt mobilregning mv. og der ligger stadig rigtig mange timers arbejde bag Rejsekortscanner.

Spørgsmål og svar

  • Er 25kr ikke lige dyrt nok?
    Næææ det synes jeg egentlig ikke. Momsen udgør 5kr, Google skal have 6kr og skattefar skal have 6-7kr - så jeg tjener også omkring 6-7kr netto, hvilket er nok til en Mars bar!

  • Hvorfor har det taget så lang tid?
    Udover at dette er foregået i min fritid, så har jeg også følt at kvaliteten skulle væsentligt i vejret når man begynder at tage betaling for noget. Der er også brugt meget energi på at få en masse gode test eksempler op. Jeg har ikke resourcer der tilnærmelsesvis kommer tæt på hvad Rejsekort f.eks. selv kunne stille med, så jeg er dybt afhængig af de fejlrapporter brugere indrapporterer som typisk ender med at indgå som tests. Sidst men ikke mindre, jeg er nok lidt af en perfektionist - kompromiser kan være nødvendige, men jeg bryder mig ikke nødvendigt om dem.

  • Er den eneste forskel ved Pro versionen, at der ikke er reklamer i den?
    Ja, men dette betyder dog også, at Pro versionen fylder mindre, er knapt så ressourcetung og derfor også starter hurtigere. Derudover er det sandsynligt, at jeg vil betragte Pro versionen som mere stabil og "røgteste" nye features i den gratis version.

  • Hvorfor kan app'en ikke installeres på min enhed?
    Det skyldes desværre, at app'en kræver en enhed med understøttelse for en bestemt type NFC ved navn Mifare Classic. Det er kun NFC chips fra firmaet NXP der har denne support og det svinger meget hvilke fabrikanter og modeller der bruger disse. Derfor forsøger jeg at styre hvad Google Play tilbyder, men det er lidt af en umulig kamp da nye enheder hele tiden kommer til.

  • Hvilken enhed skal jeg bruge for at kunne benytte app'en?
    Generelt så kan mange Sony modeller (Z1, Z3, Z5) læse rejsekortet, ligesom en del fra HTC (M7, M8, M9) men det er kun lidt ældre modeller fra Samsung der med sikkerhed kan bruges (S3 og S5) samt nyeste S4 Mini. Huawei (P7 og P8) har også enheder på markedet der kan benyttes. Det samme gør sig gældende for LG (G3 og G4). Google's egne Nexus 6P og Nexus 5X skulle også være et sikkert valg. Men hvis du står og skal til at købe ny mobil og ønsker at kunne følge med på dit rejsekort, er det sikreste simpelthen at prøve app'en i butikken før du køber - det har jeg selv gjort et par gange.

  • Kan jeg teste app'en selv om den ikke er tilgængelig via Google Play?
    Ja det kan du godt, men det kræver du slår "Ukendte kilder" til inde under sikkerhedsindstillingerne på din enhed. Dernæst kan du installere den rå APK fil uden om Google Play, som du kan downloade her. Jeg hører meget gerne om dine erfaringer så jeg kan opdatere listen på Google Play - email mig på

  • Hvad er fremtidsplanerne for app'en?
    Jeg kunne rigtig godt tænke mig at kunne vise rabatniveauer (rabattrin) samt serviceniveauer (standard, DSB 1' mv.) til brugeren så det er nok det næste jeg vil kigge på.

Thursday, April 21, 2016

Rejsekortscanner 2.0

Jeg har i længere tid arbejdet på en betalingsversion af Rejsekortscanner, til dem der efterspørger at komme af med reklamerne. Men når man lancerer noget man tager betaling for, vil kunder naturligt forvente høj kvalitet og jeg følte ikke helt at den gamle version kunne leve op til dette.

Dette betød, at jeg i større grad skulle benytte mig af den dokumentation jeg har fået adgang til fra Rejsekort A/S, ligesom jeg havde brug for en stor pulje af tests for at sikre imod regressioner - for jeg har stadig begrænset adgang til den forretningslogik der ligger til grund for rejsekortet (hvad er den maksimale rejsetid, hvad er den maksimale transit tid, hvordan hånderes vintertid osv.).

Derfor besluttede jeg mig for, at både den gratis (med reklamer i) samt kommende betalingsversion, skulle baseres på samme kode og derfor begyndte arbejdet først og fremmest på en version 2.0 af den gratis udgave. Det har taget længere tid end ønsket, men det var dét der skulle til. App'en er altså blevet skrevet helt om fra bunden af, med et utal af forbedringer. Her er et par af dem:
  • Afkodningen af data på rejsekortet er langt mere troværdig da jeg har haft adgang til den officielle dokumentation fra Rejsekort A/A
  • Over 600 unit tests bliver brugt til at sikre imod introduktioner af nye fejl (regressioner)
  • App'en har fået et friskt nyt design, fordelt på skærmene "Status", "Hændelser" samt "Detaljer"
  • På trods af, at app'en henter meget data ud, er den blevet hurtigere til at kommunikere med rejsekortet, typisk 1/3 sekund
  • App'en vi nu fortælle dig om du har glemt at checke ud hvis det er længere end 12 timer siden du er checket ind
  • Sommer/vintertid-problematik håndteret som det skal - dette har altid være problematisk i den gamle version
  • Understøttelse af Dansk og Engelsk sprog - før kunne kun benyttes Dansk
  • Minimeret programstørrelse til blot 1.9Mb - på trods af nye features og bedre grafik

Et af de få minuser jeg bør nævne er, at jeg har været nødt at fjerne supporten for gamle Android 2.3 (API 10) enheder således at minimumskrevet nu er Android 4.03 (API 15). Der var mindre end 0.1% af mine brugere der benyttede sig af så gamle enheder, at det ganske enkelt ikke længere gav mening at bruge energi på at holde app'en kompatibel.

Betalingsversionen kommer meget snart, jeg skal lige have testet den nye kode via gratis-versionen først. Herunder kan du se nogle eksempler på skærme fra den nye app.

Status skærmen for et kort der er checket ind

Status skærmen for et kort der er checket ud

Status skærmen for et kort der ikke er checked ud i tide

Status skærmen for et kort der er blevet blokeret p.g.a. misbrug og lignende

"Hændelser" skærmen hvor de seneste rejser kan aflæses

"Detaljer" skærmen hvor information om kortnr, type, status, saldo, optankningsbeløb mv. kan aflæses

Find app'en i Google Play, eller hent den hér via Dropbox. Pro versionen uden reklamer skulle også lande på Google Play i løbet af nogle dage.

Tuesday, April 19, 2016

Beware of SQLite and the Turkish Locale

Today I came across a truly puzzling issue on Android. An otherwise tried and tested application crashed consistently when running on a device using Turkish as the current locale.

App crashing on a Turkish device

The problem occurs in a large app developed for a customer, inside a proprietary binary component, so no direct debugging was possible. All I had available was a vague stack trace showing the root problem to be a NullPointerException from trying to parse an integer:

Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'int java.lang.Integer.intValue()' on a null object reference

Not much to go on, the goal was now to figure out just what could cause such a problem just from running on a device using Turkish Locale.

Having spent some time on Google, searching for other people's trouble with the Turkish Locale, it became clear that there are issues with lowercase Strings in Turkish. I started working on a small isolated app which would aid in narrowing down the exact issue. It wasn't long until the suspicion I had turned out to be spot on.

SQLite with Turkish Locale

The problem was caused by relying on the device Locale for representing SQL Strings and the fact that the SQLite parser must be converting the SQL literals to uppercase. That becomes a problem, because in Turkish the lower case "i" becomes "İ" in uppercase. Inside SQLite, the parser is therefore seeing "İNSERT" rather than "INSERT" which is of course not valid SQL. This is actually specified in the Android documentation for the Locale class and the subject was also covered by none other than Jeff Atwood some years ago.

To reproduce the problem, create a new project in Android Studio with an empty activity. Place the following XML for the layout:

<?xml version="1.0" encoding="utf-8"?> 
<RelativeLayout xmlns:android="" 
        android:text="SQL insert in lowercase" 

...and add the following Java code to the activity:
public class MainActivity extends AppCompatActivity { 
    private final static String DML_CREATE = "CREATE TABLE IF NOT EXISTS demotable(Name VARCHAR);"; 
    private final static String SQL_INSERT = "insert into demotable values('John Doe');"; 
protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(DML_CREATE); } setContentView(R.layout.activity_main); }
public void insert(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT); } } }

Although being perfectly valid Java and SQL, the code above will crash consistently when running on a device with a Turkish Locale!

Caused by: android.database.sqlite.SQLiteException: near "İNSERT": syntax error (code 1): , while compiling: İNSERT İNTO DEMOTABLE VALUES('JOHN DOE');

The stack trace makes it blatantly obvious what happens. If only I had such a nice stack trace originally I could've saved a few hours, but such is life when you depend on proprietary closed source components. :(


There are two typical proposals for how to solve the issue. This can either be done by always converting SQL Strings to upper-case:
public void insertUppercase(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT.toUpperCase()); } }

...or (better in my opinion) make sure SQL Strings are always interpreted using the root locale by converting to lower-case while providing the neutral root locale:
public void insertLowercaseEnglish(View view){ try(SQLiteDatabase demoDatabase = openOrCreateDatabase("demotable",MODE_PRIVATE,null)){ demoDatabase.execSQL(SQL_INSERT.toLowerCase(Locale.ROOT)); } }

Although I have not tested it, it's also likely that a solution could be to externalize the SQL as a simpler character set (ASCII) rendering the the issue moot. However, most developers will probably just embed SQL in Java unicode Strings. Feel free to comment on this if you have other solutions.

Lesson learned

Something tells me this is a very common bug out there, the intricate details of "i" in the Turkish Locale are probably only known to Turks. The only reason why more Java developers don't run into this, is probably because they typically operate and control the host runtime. However, when dealing with an app, the environment is not as neatly encapsulated and testable so this problem bubbles up to the surface and hits you hard.

The lesson learned must be to never rely on the default Locale, not even for something as innocent as Java Strings. While we are able to remedy the issue when using SQL as an embedded DSL, one can easily imagine the same bug sneaking in countless other ways. For example, annotations come to mind, where you do not have the luxury of being able to manipulate the String before its sent to be parsed by some annotation processor!

Monday, April 4, 2016

Getting the reference address of a variable on Android

In Java, we've always had our share of undocumented fun through the sun.misc.Unsafe class. You can allocate memory, access data fast without range checks and - what this post is about - get the memory address of some object.


So why would you do this in a memory managed language like Java? Well, it's arguably rare you have an actual need, however it can come it handy at times. For instance, I needed it when debugging some weird behavior on Android. I had a strong suspicion that the (insanely complex) Android life-cycle and associated serialization aspects were the cause of me seeing an object instance mutate (I.e. change its hash value). Unfortunately Android Studio doesn't support putting a watch on a variable. What I really needed was a way to document identity equality, just as we can document value equality with the hashCode() mechanism.

Android specifics

It's not hard to find examples of the use of sun.misc.Unsafe on the Internet. Using it on Android rather than the JVM however, means we're operating under some fairly unique constraints. For once thing, you can not import the sun.misc.Unsafe class at all! Secondly, while Android does include an implementation of sun.misc.Unsafe at runtime, it differs substantially from the official unofficial (if you know what I mean) implementation we know from the Oracle JVM. However, by finding the relevant source code at the AOSP, and some reflection magic, we have enough ammo to attack the problem. The end result is the following UnsafeUtils class.

import java.lang.reflect.Field;
import java.lang.reflect.Method;

 * Utility class for working with the undocumented java.misc.Unsafe class
 * on Android.

public class UnsafeUtils {

    private static Object getInstance(){

        try {
            Class clazz = Class.forName("sun.misc.Unsafe");
            Field theUnsafe = clazz.getDeclaredField("THE_ONE");
            return theUnsafe.get(null);

        } catch (ClassNotFoundException e) {
        catch (Exception e) {
        return null;

    public static long addressOf(Object o) {
        final Object[] array = new Object[] {o};

        final Object unsafe = getInstance();
        try {
            final Method arrayBaseOffsetMethod = unsafe.getClass().getMethod("arrayBaseOffset", Class.class);
            int arrayBaseOffset = (int)arrayBaseOffsetMethod.invoke(unsafe, Object[].class);
            final Method getIntMethod = unsafe.getClass().getMethod("getInt", Object.classlong.class);
            return (int)getIntMethod.invoke(unsafe, array, arrayBaseOffset);
        } catch (Exception e) {
        return 0;

Using UnsafeUtils

Using the utility class is just a matter of getting an instance and calling the addressOf(...) method with the target object as an argument.

        String n = "java";
        System.out.println("n=\"java\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = "rocks";
        System.out.println("n=\"rocks\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = "java";
        System.out.println("n=\"java\": {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");
        n = new String("java");
        System.out.println("'n=new String(\"java\"): {hashCode=" + n.hashCode() + ",addressOf=" + addressOf(n) + "}");

The code above will output something along the line of the following - keep in mind, the memory addresses listed will be different with each run.

n="java": {hashCode=3254818,addressOf=323913376}
n="rocks": {hashCode=108686766,addressOf=323913568}
n="java": {hashCode=3254818,addressOf=323913376}
n=new String("java"): {hashCode=3254818,addressOf=323913856}

Notice how changing our code from referencing a String with "java" to "rocks", makes it point to a new address. Changing it to point to "java" again reuses the existing constant in the pool due to a the Java compiler being smart about it (Strings are somewhat unique). Making a new reference using the "new" operator, bypasses this compiler optimization and allocates a new instance on the heap.

If you play around a bit, you will also start to notice the size of references and you can also easily distinguish the difference between objects created on the stack vs. the heap.


Take this for what it is, a helper utility for understanding what's going on and for debugging purposes. The code may be buggy, may not work on all versions of Android/Dalvik/ART and is definitely not production safe. Although the code is adapted for my use, it's very unoriginal - so assume a Public Domain license. Do with it what you want, just don't hold me accountable. :)

Tuesday, January 5, 2016

The letdown by Air Canada

This is the personal account of me and my family's recent troublesome journey and treatment by Air Canada, when traveling home to Denmark after having spent christmas in Montreal, Canada 2015.

Toronto Pearson airport, my daughter sleeping in a corner while my wife is watching over her.

Let me clarify that we are not unaccustomed to international traveling, having crossed the Atlantic at least 80 times over the previous 15 years. I’ve personally experienced my share of cancelled flights, delays and hotel stays etc., some of which I have even blogged about before. However, what I describe in the following takes the price as the worst travel experience ever, not least because it has to do with my infant daughter. I may get a detail or two wrong because of the stressful circumstances, but it has been written down a day after arriving home and cross-checked by my wife so can assumed to be quite accurate. My goal with this is first and foremost, to remember why Air Canada is never going to have me as a customer again and secondary as a base for filing an official complaint when I have recovered some energy.

You can read the full account in this Google Docs document