Monday, August 10, 2015

Duck typing in Java

Duck typing in Java

According to wikipedia, duck typing is when "an object's suitability is determined by the presence of certain methods and properties (with appropriate meaning), rather than the actual type of the object". Duck-typing thus is almost the definition of a dynamic language like Ruby, Python, Groovy etc. Unlike a hybrid like i.e. C# 4 (thanks to it's dynamic modifier), Java's type-system does not allow duck-typing - it's an object oriented paradigm where polymorphism is meant as a static modelling mechanism of a type hierarchy. However, the dynamic proxy feature introduced with Java 1.3, does allow us to emulate duck typing. First a disclaimer though, I am far from the first to blog about the subject, even Brian Goetz (now Java's language architect) blogged about it back in 2005.

Dynamic Proxy

It was back in 2000 that Sun introduced the Dynamic Proxy functionality to JDK 1.3. As the name implies, it caters to the well known Proxy pattern and it does so in a dynamic fashion. In short, a dynamic proxy makes it possible to create an instance of some interface dynamically at run-time. For many years, it has been the underlying work-horse of more advanced and exotic functionalities in Java frameworks for doing AOP, ORM, remoting, access control etc. Using a dynamic proxy can cause some controversy due to the dynamic nature, which is far from Java's traditional static type system. So right off the bat, let's simply write a small utility class (embedded DSL) that makes it easy to use the Java dynamic proxy feature:

public final class DuckType {

    private final Object source;
    
    private DuckType(final Object source) {
        this.source = source;
    }

    public static DuckType on(Object source){
        return new DuckType(source);
    }

    private class DuckTypeProxy implements java.lang.reflect.InvocationHandler {
        @Override
        public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
            final Method delegate = findMethodBySignature(method);
            return (delegate == null) ? null : delegate.invoke(DuckType.this.source, args);
        }
    }

    public <T> T as(Class<T> clazz) {
        
        AssertHaveCompatibleMethodSignatures(clazz);

        return asLenient(clazz);
    }

    public <T> T asLenient(Class<T> clazz) {
        return generateProxy(clazz);
    }

    private void AssertHaveCompatibleMethodSignatures(Class clazz) {
        for (java.lang.reflect.Method method : clazz.getMethods()) {
            if (findMethodBySignature(method) == null) {
                throw new ClassCastException("Not possible to ducktype " 
                        + source.getClass().getSimpleName() 
                        + " as " + clazz.getSimpleName() 
                        + " due to missing method signature " + method.toString() 
                        + ". If a No-Operation behavior is preferred, consider "
                + "calling asLenient(..) instead!");
            }
        }
    }

    @SuppressWarnings("unchecked")
    private <T> T generateProxy(Class<T> iface) {
        return (T) Proxy.newProxyInstance(iface.getClassLoader(), new Class[]{iface}, new DuckTypeProxy());
    }

    private Method findMethodBySignature(Method methodToMatch) {
        try {
            return source.getClass().getMethod(methodToMatch.getName(), methodToMatch.getParameterTypes());
        } catch (NoSuchMethodException e) {
            return null;
        }
    }
}

The above utility allows us to treat any object as an instance of some interface, even if it does not implement given interface. What will happen instead, is that the methods will dispatch dynamically and thus will lookup methods at run-time and forego any checking at compile-time. This has some interesting implications which can be useful in a mocking scenario, for decorating or for proxying.

Proxying

The majority of use cases, which also lays name to the API itself, is to act as a proxy - completely detaching types. A proxy is just an object that stands in place of another. It's not uncommon in a multi-layered architecture, to have many versions of some type, say a Customer class. In order to isolate each layer and avoid leaky abstractions, best practice is to wrap and copy data as they enter and exit each layer (the same goes for exceptions of course).


This practice not only becomes tedious to type, it also incurs a considerable repetition overhead on the code base which code quality tools (SonarQube etc.) and code quality organizations (SIG, BlackDuck etc.) will flag as suspicious. So while the layered design is nicely decoupled to only caring about one layer (and possible the layers immediately next to), it doesn't come for free.

The solution often seen in Java, is to add a vertical interface layer (you'll typically recognize these as *-api projects) for a cross-cutting abstraction layer, allowing some common super-type to be known throughout the system. Then, by referring only to the interface when throwing data around, nothing layer-specific escapes and no manual copying of data from one structure to another, needs to happen.


The problem with this approach, is that you better be damn sure you got it right to begin with (and we all known how great Ivory Tower designs work, right?), because changing interfaces later on is, by definition, a breaking change that requires updating all downstream implementations. So while the tight coupling does remove duplication, it does so at a cost.

This is where a dynamic proxy can come into play. Some languages are of course dynamic in nature, while a few modern languages introduced a hybrid approach, allowing certain corners of an application to be coded with less type checking from the compiler. Java does not offer such a cop-out natively, but we can however use dynamic proxies to achieve much of the same thing.

public class ProxyTest {

    interface Duck {
        public String speak();
    };

    class Cat{
        public String speak(){
            return "Miau";
        }
    };

    public static void main(String[] args) {
        new ProxyTest();
    }
    
    public ProxyTest(){
        Duck catTreatedAsDuck = DuckType.on(new Cat()).as(Duck.class);
        System.out.println(catTreatedAsDuck.speak());
    }    
}

Which will output the following to the console:

run:
Miau
BUILD SUCCESSFUL (total time: 0 seconds)

So you'll notice that calling speak() on the catTreatedAsDuck object, which is an instance of the Duck object, will invoke the speak() on the underlying Cat object - even if Cat really doesn't implement the Duck interface!

This example may seem a bit silly, but it demonstrates how to essentially "inject" an interface, and that can be very useful when you have to work with legacy code, auto-generated code or multi-layer architectures where you do NOT have a commonly shared contract/interface. I have used this approach before on production systems, where lots of compiler-compiler auto-generated types had to be treated by a lot of similar methods. Proxying these objects behind an interface, allowed me to remove a lot of redundant code in favor of DRY. The cost of doing this, is a bit of dispatch speed, but realistically speaking most won't even be able to measure any difference unless they do a huge amount of calls. Another cost, is that of loosing type-safety - there is no help from the compiler, so just as is the case with dynamic languages, having integration-tests becomes absolutely paramount!

Remote proxy

Another type of proxy which should also be mentioned, is the remote kind, where you use it as a communication mechanism out-of-process and possibly over a network. In 2007 I wrote a small framework called HttpRMI which is a super simple way of calling a remote method over HTTP, using Java's vanilla serialization mechanism underneath (note: today I would not use this, there are better alternatives like i.e. Hessian) or Spring Remoting, but it's implemented exactly the same way as the proxy above, except it's made to implement two separate client-server parts.

The server part is made out by DynamicProxyFactory and the client part by HttpRmiServlet. With these little helpers, we can use the Dynamic Proxy as a remoting mechanism as demonstrated below:

public interface SampleContract{
    public String getHello ();
}


public class SampleServlet extends HttpRmiServlet implements SampleContract{
    public String getHello(){
        return "Hello World";
    }
}


public class SampleClient{
    public SampleClient(){
        SampleContract contract = DynamicProxyFactory.create(SampleContract.class, "http://localhost:8080/SampleServer/SampleServlet");
        System.out.println( contract.getHello() );
    }
}


run:
Hello World
BUILD SUCCESSFUL (total time: 0 seconds)


So the SampleContract on the client is obviously not the same SampleContract as is running on the server, but for all practical purposes, there's no way of knowing this on the client-side. This is generally a sanctioned way of using the dynamic proxy unlike what I showed earlier.

Dynamic dispatch can be ok

The dynamic proxy can of course be combined with other design patterns and also for stubs, although with modern dependency injection and mocking frameworks (that handles more than just interfaces), it isn't used much for this. Using dynamic dispatch in a static Java environment, is by definition a bit controversial. I know of at least one code quality audit organization that opposes using a dynamic proxy because it's "complicated" (I'm looking at you SIG), even after showing how much redundant code can be removed from the code base.

My view on this is less black-n-white and I tend to appreciate "static when you can, dynamic when you must", to quote Anders Hejlsberg, chief architect of C#. Having just been on a large enterprise project using Grails and Groovy, where bits and pieces can blow up anytime when you hit "Run", I definitely favor static modelling and compile-time checking from compiler and IDE. However, some aspects of an application can indeed benefit from dynamic dispatch and I'll argue that layer boundaries within an application are a good candidate for this. Whether you consume a web-service, parse an XML file, talk to a database etc., you need to have sufficient testing in place between the layers anyway. There is also a good chance that, if abstractions have been broken down accordingly, the interface is made out of larger but fewer calls rather than many smaller ones. In other words, there shouldn't be any observable run-time cost associated with doing dynamic dispatch between layer boundaries so it remains more of a theoretical cost than a real one.

What about you, agree or disagree with such a hybrid approach? Let me know in the comments why! :)

Friday, May 8, 2015

AVR ATtiny presentation with examples

Where I work, we hold bi-monthy gatherings where we present and/or debate technologies and tools. As a closet electronics engineer, I thought it could be interesting to drag my components to the office and let me colleagues, have some fun with writing programs targeted for 64 bytes of RAM - quite an anticlimax to our 8GB RAM workstations and 128GB RAM servers!



I put together a Back to Basics presentation about the popular Atmel ATtiny/ATmega microcontrollers, and put the accompanying code up on Github. These are the same family of chips being used by the popular Arduino stack, but I prefer to play with the bare metal using the GCC compiler, there's just something cool about seeing your C program being able to run 10 years on a couple of AA batteries due to the extreme effeciency! While nothing spectacular about it, protocol dictates that I mention this to be released under the Creative Commons license.

Monday, November 17, 2014

ER301 NFC/Mifare driver in pure C#

A YouTube viewer found my old demonstration of ER301 anti-collision and asked if I could make the source code available. So I cleaned it up a bit and wrote this small blog entry about it.

When I started tinkering with the unified Danish transit card about a year ago, it was using a cheap Ehuoyan ER301 NFC/Mifare reader which I got from ebay. This was the first time I played with NFC stuff even if I would since move on to other and better devices. The cool thing about the ER301 however, apart from the price, is that it's relatively easy to interface with. This is because it's using a plain virtualized serial port with native (cp2102) driver available in Linux and even Windows.




No software came with the ER301 reader but there was a download link at the ebay site. The software was very basic, and all examples revolved around a driver DLL for which the source code sadly was not provided. With the documentation in hand and tracing the traffic going over the serial interface, I decided to implement it in pure C# so I would not depend on Windows and static architecture libraries.

Although originally developed with Mono/Monodevelop, the current sources revolves around a Visual Studio solution, with the driver as one project and a simple console application as another. Note that the driver is not complete yet and it caters only to the needs I had at the time (complete dump of data), but it should be fairly easy to extend it with the granularity you might need, since most of the core API have already been implemented. It supports Mifare Classic S50 (1K) and S70 (4K). It can easily be made to support other types as well as long as it's supported by the ER301 hardware.



ER301.NET Console app dumping data from a Mifare Classic S70/4K

The source code is available on Github under GPL2 and most of the interesting stuff takes place in ER301.cs. I am looking into releasing a (WinForms) client application as well, but I need to clear it with a customer of some derivative work first.

Sunday, November 9, 2014

Øredev 2014 - The good and the bad

Last year I went to the 2013 GOTO conference in Aarhus, Denmark. This year, I got the opportunity to visit the 2014 Øredev conference in Malmø, Sweden. The following is just a short reflection of my experiences and they may of course be entirely different than those of other attendees.





The whole setup of Øredev is similar to GOTO and with 1300 attendees I believe they are about the same in size. Being in Sweden, albeit right across the water from Copenhagen, the crowd was predominantly Swedish with perhaps 20-30% of Danes. The speakers were slightly less known than those at GOTO, although a few speakers from Thoughtworks, Oracle and Microsoft stood out.

The good
Øredev feels quite a bit more intimate and cozy than GOTO did. It could be due to the constrained space and the lighting, but the pillow sacs, live jazz and plenty of popcorn, chips, candyfloss and beer sure didn't derail that perception.



Wifi coverage was generally good and there were plenty of sessions to attend as a whole. With decent food and no shortage of coffee, cake, snacks and candy, there was no risk of running out of energy anytime soon. At some point they did run out of the local no-name cola brand and finally gave us real coca-cola soft-drinks. If there's ever an event where you should not skimp on brand name cola, it's probably a conference for developers! :)

The venue also had more of a geek festival feel to it than GOTO had, with its 3D printing, classic console platforms, arcade machines, Occulus Rift etc. There were of course still the typical product pitch stands but it didn't feel quite as dominating.



Unlike at GOTO, where lunch takes place in another building at a specified time, lunch took place in the middle of the venue and was spread out over 1½ hour during talks. This meant that people would not all have to stand in line at the same time nor would it feel quite as chaotic. Also, lunch was served from 3 different locations in the halls, contributing further to the feeling of being able to breathe.

It's also very impressive that the Øredev video's are available some 24h after their capture, something other conferences could definitely take a lesson from. This means it's not quite as annoying to have gone into the wrong talk, since you can always catch up on the one you missed. I remember waiting several months for some of the GOTO videos to come out and I have since forgotten about those I missed seeing.

Last but not least, because of Sweden's very restrictive alcohol policies, you can drink a lot more beer before feeling it. A standard Tuborg with 4.6% alcohol is watered down to 3.1%. Not sure whether to characterize this as either good or bad, but it was an interesting observation none the less.

The bad
Interestingly, and unlike at GOTO, the organizers had not chosen to create an app for the event. This was unfortunate, since the supplied programs were not at all detailed enough to make you decide between conflicting talks - 8 simultaneous tracks are hard to choose from, when all you have to go by is a condensed headline.



In stead, you had to rely on the website which didn't work much of the time due to an unresponsive server and also suffered from navigation issues (on Android anyway). It's a good example of how you can scale so much better by taking advantage of a richer client (read: app) with its own state.

Staying with the app theme, it would also have been nice if rating of sessions was possible from your smartphone - sure there were the occasional iPad with a green-yellow-red star on, but it was only for some sessions at some doors. The GOTO app rating was buggy but at least they had put a bit more thought into it.

Finally, before leaving the app aspect, I also missed being able to ask questions during sessions for immediate followup at the end of the talk. This was something I took great advantage of at GOTO, in order to engage the talkers with my own thoughts and issues. At Øredev only a few of the speakers had set aside time for questions and I think that's a real shame. I got to ask 3 questions at Øredev whereas at GOTO I got to ask 11 questions and also opened up an email discussion about micro-services with Sam Newman from Thoughworks in the following weeks.

The seating at Øredev was perhaps a bit worse than at GOTO, with even the one large theater being devoid of armrests. It's a minor detail for sure, but after spending a day sitting down on tight hard chairs in jam-packed rooms, it would be nice with a bit of cushioning.

The worst experience at Øredev was definitely the interview with Nile Rogers. He was scheduled as a special keynote thursday night about "Passion and collaboration". Now granted, I never heard of the man and I have no special relationship with black hipsters from the 70's, but I still dare to say the interview missed its intended purpose. While Nile was certainly an experience and clearly commanded a guitar in the brief moments we were allowed to hear that, at many occasions I was left with "WTF is this?". At no time did the interviewer succeed in bringing either passion or collaboration into the picture, but seemed more interested in celebrating Sweden for its musical roots and making statements confusing poor Niles. Most of my colleagues had left midway in, I left after 1½ hour where the interviewer had apparently only addressed 4 out of a total 10 questions! If you'd like to see for yourself if it was really that bad, the interview is available here but I'd encourage you to spend your spare time on some of the other video's.





In conclusion
It was a very nice conference experience. In particular I'd like to highlight the cozy venue, widespread plentiful content and the fact that the logistics just simply worked. I do think its a pity, that a large amount of the attendee fee went to pay first class airfare, bodyguard, interviewer etc. for Nile Rogers and his wife and entourage. These conferences should not only be available to large corporations with enough money on their hand and I get the impression that the fee would've been lowered by 1000kr if they didn't have to cater to spoiled famous musicians.

I probably got more professionally out of my visit to GOTO simply because of the inherent ability to engage with the speaker through the questions. If Øredev would adjust these things, it would be a slam dunk which conference to go to.

For the mortal developers not engaged by large corporations, don't forget there's a very approachable and affordable conference in Copenhagen nov. 14 called Driving IT. Having attended IDA events before, I'd love to attend this one as well, but I think 2 conferences within one month is probably pushing the limit with my employer.


Saturday, October 18, 2014

TopIt - Window utility for Windows

It's been many years since Linux became my #1 OS of choice, for a variety of reasons. However, I changed job recently and one unfortunate side effect of that, is that I'm now having to spend my work hours on Windows again. Apart from the lackluster command-line, bloated environment, inhospitable development experience etc. etc. one of the annoying aspects, is how desktop and window management basically haven't changed since Windows 95 - for instance, it's still up to the individual application in question, whether it wants to allow the user to pin down a window, promoting its z-order to always be on top. This blog entry is about my attempt at bringing a bit more power to window management as seen from a Linux user.

Gnome Linux desktop
Talking about "the Linux desktop" is a bit of a misnomer. There are of course countless of XOrg window managers. All the recent KDE and Gnome versions I've come across, supports being able to pin a window on top (promote z-order), move a window to a specific workspace/desktop and control a windows transparency/opacity.


In Cinnamon, the Mint fork of Gnome Shell, z-order behavior is build right in!


Crap-ware nightmare 
There have never been a shortage of 3'rd part tools available for Windows to empower users, the problem is... finding them and knowing the flowers from the bad weed. Traditionally there's lots of spy-ware, share-ware, trial-ware in the Windows Eco-system.

I set out with a relatively modest goal, to find a simple application that would allow me to pin a window as the top-level window - something I use all the time for small TODO lists, calculator, chat applications etc.

One application turns up in Google searches as the de-facto solution to the problem, namely Window On Top, which also got a favorable mention on LifeHacker. I downloaded the application, installed it and all seemed fine and dandy... until I was met by a nag screen.


Bait-n-switch in action, Window On Top appears to be trial-ware!

Although neither the authors website, nor LifeHacker, has any mentioning of it being trial-ware, the application costs a whopping $20 - a ridiculous amount for something so simple! A similar utility called MooO which also got a mentioning on LifeHacker, is apparently infested with tool-bars and who knows what other crap.

Realizing full well how I may sound like a grumpy old fart with a long beard and entirely too much time on my hands, these crap-ware experiences made me want to try to solve this problem for myself.

Introducing TopIt
It's been some 10-15 years since I played around with the Windows API's. Back in the Visual Basic and MFC days, if you had to do something just remotely interesting, you had to call C/C++ functions from the core DLL's, so I had a pretty good idea as to how to go about this.

It took a few mornings and evenings of tinkering (when the wife and baby are sleeping), but I now have something entirely functional. In fact, it does more than just toggle a window on/off top - I guess you could argue that it already suffers from feature creep.


TopIt manipulating opacity/transparancy of a third-part window

These features are to be considered stable and tested:
  • <Alt>+<Z> Toggles the "always on top" Z-order lock of a window
  • <Alt>+<X> Completely hides the window temporarily (until combo hit again)
  • <Alt>+<C> Toggles the re-sizable/movable border of a window on/off
  • <Alt>+<Num Plus> Increases opacity (removes transparency) of a window
  • <Alt>+<Num Minus> Decreases opacity (adds transparency) of a window
The following features are tested but somewhat experimental:
  • <Alt>+<Num 5> Tries to expand window to take as much desktop space as available
  • <Alt>+<Num 2> Selects bottom edge of window for manipulation
  • <Alt>+<Num 8> Selects top edge of window for manipulation
  • <Alt>+<Num 4> Selects left edge of window for manipulation
  • <Alt>+<Num 6> Selects right edge of window for manipulation
These features are highly experimental and may or may not work:
  • <Alt>+<Num 7> Selects  top-left edges of window for manipulation
  • <Alt>+<Num 9> Selects  top-right edges of window for manipulation
  • <Alt>+<Num 1> Selects  lower-left edges of window for manipulation
  • <Alt>+<Num 3> Selects  lower-right edges of window for manipulation

You can get TopIt by downloading the raw x86/x64 executable or by cloning my reposatory on GitHub and building it yourself from source. The latter is preferable, as its the bona-fide approach to ensuring what you install on your computer. The memory footprint is absolutely minuscule, requiring just about 800KB of RAM.

Note that if you download the binary executables, you will have to launch it automatically when Windows boots. You can do so by including it in the Windows Start Menu "Startup" folder.

Since starting out on this project, I have since become aware of the utility called PowerMenu which looks very close to what I wanted originally, apparently without being crap-ware. The application is donor-ware, so should be safe to download and you'll likely want to use it rather than TopIt, since it is older and thus better tested. However, my efforts are not entirely wasted, as I plan to expand on TopIt further to get me some more power-features at a minimum overhead. Lot's of ideas come to mind:
  • Application screenshot (invoking "Save as..." dialog)
  • Emulation of multiple workspaces/desktops
  • Dockable and snapping windows
  • Menu item injection
  • Move applications to/from systray
  • Programmable hotkeys
Should you have any cool ideas which would boost your productivity on Windows, you are of course more than welcome to drop me a line about it or fork the code on GitHub and add it yourself. :)