Thursday, August 20, 2009

Tweaking javac leniency

While a great fan of type-safety and static checking, I also like a certain amount of leniency from compilers. Sun's Java compiler is a good example of one that goes overboard in some anal, misunderstood attempt at servicing the developer.
In the following I will explain why I think so and how to fix it by modifying the publicly available source code for the compiler. Note that although I have done something similar before, I am no compiler expert and never will be. The modifications shown in this post are mere tweaks and I will stay far away from anything below semantic analysis.


"Exception is never thrown"

Have you ever had a good coding session, with design, implementation and refactoring happening at an immense speed as you try to perfect the internals of a piece of code? All of a sudden you are distracted because a try block no longer contains any methods that declares a checked exception. So now, in order to satisfy the compiler, you'll have to focus on removing these alternate, albeit dead and harmless paths. Indeed, you will have to carefully uncomment everything but the body of the try block.

As a concrete but contrived example, imagine something like this:



Settings settings = null;

try {
settings = Settings.loadFile("settings.conf");
settingsObservers.fireChangeEvent();

} catch (IOException ex) {
// Logging...
}
finally{
// Cleanup...
}




Now then imagine you need to test something real quick, and thus prefers to just new up the Settings() object or something similar, so you comment out the line where it loads from a file:



Settings settings = new Settings();

try {
// settings = Settings.loadFile("settings.conf");
settingsObservers.fireChangeEvent();

} catch (IOException ex) {
// Logging...
}
finally{
// Cleanup...
}




That won't work of course, because of the paranoid rules surrounding checked exceptions. Instead, we'll see the compiler complain:


casper@workstation:~$ javac ExceptionNeverThrown.java
ExceptionNeverThrown.java:13: exception java.io.IOException is never thrown in body of corresponding try statement
} catch (IOException ex) {
1 error



So in order to have it run, you meticulously comment out everything related to the try-catch-finally block:



Settings settings = new Settings();

//try {
// settings = Settings.loadFile("settings.conf");
settingsObservers.fireChangeEvent();
//} catch (IOException ex) {
// // Logging...
//}
//finally{
// // Cleanup...
//}



That's just stupid and gets even worse if you have any kind of propagating hierarchy in place! Static checking should be an assistance rather than an annoyance, this is clearly a case of the static dial being placed a tad too high.


Catching checked exceptions without a throw

We can fix this by getting our hands on the OpenJDK source, in my case I opted for the easy-to-use Kijaro sandbox, which Stephen Colebourne set up to encurrage this kind of hacking. Kijaro includes instructions on how to build javac for both Windows and Linux. If you want to try this kind of hacking yourself, you are going to need a checkout of this sandbox or a similar OpenJDK branch. Alternatively, if all you want to do is play with the tweaks I'll demonstrate here, you may just get a copy of my modified javac.

The semantic analysis parts of OpenJDK is largely contained in the package com.sun.tools.javac.comp, for our purpose we're going to need Flow.java hosting a bunch of data-flow analysis methods that's responsible for raising error conditions surrounding the use of checked exceptions. The compiler walks the AST of the source code through a double-dispatch mechanism (visitor pattern) that calls methods in Flow.java with the current AST node as argument. This means all we have to do is locate the proper callback and modify it according to our need. Down around line 951 you should see the following:



public void visitTry(JCTry tree) {
...

if (chk.subset(exc, caughtInTry)) {
log.error(l.head.pos(),
"except.already.caught", exc);
} else if (!chk.isUnchecked(l.head.pos(), exc) &&
exc.tsym != syms.throwableType.tsym &&
exc.tsym != syms.exceptionType.tsym &&
!chk.intersects(exc, thrownInTry)) {
log.error(l.head.pos(),
"except.never.thrown.in.try", exc);
}

...
}



Evidently, here's some logic that looks like it's logging an error if a checked exception is not being thrown (catch-list does not intersect with thrown-in-try-list). Let's change the line from logging an error, to logging a warning:




log.warning(l.head.pos(), "except.never.thrown.in.try", exc);




That's actually all that's needed in the compiler itself. However, note the obvious reference to a resource key "except.never.thrown.in.try". This is part of a reference to an entry in the file com.sun.tools.javac.resources.compiler.properties. If you open this you'll notice the following line:



compiler.err.except.never.thrown.in.try=\
exception {0} is never thrown in body of corresponding try statement



The key does not match the one from the log statement completely, as it is prepended with "compiler.err.". Since we changed the logging from an error to being a warning, javac will search in vain for an entry with the key "compiler.warn.except.never.thrown.in.try". As we can not simply fix this by changing the key reference in Flow.java, we are going to modify the existing, or add a new entry to compiler.properties:



compiler.warn.except.never.thrown.in.try=\
exception {0} is never thrown in body of corresponding try statement




Now compile Kijaro/OpenJDK and watch what happens when you use your new javac build to compile our previous Settings sample:


casper@workstation:~$ tweakedjavac ExceptionNeverThrown.java
casper@workstation:~$



We have successfully modified the compiler to make our life a little easier. There are a bunch of similar tweaks one could make, all in the same easy fashion as explained above. For instance, I have converted checked exceptions from being an error to being a warning (hint: errorUncaught on line 298), which means they don't get in the way of rapid development while at the same time not really losing any of the benefits - production code should compile without warnings anyway.
Likewise, I have made it so that the unreachable statement error is now also just a warning (hint: scanStat on line 493), thus allowing me to short-circuit a method or similar with a return statement without having me temporarily comment out the remaining code. To demonstrate all of this in one go, take a look at the following sample code:



import java.io.IOException;

public class TweakedJavaCTest{
public static void main(String... args){
// Test of "checked exception not caught" (throws InterruptedException)
Thread.sleep(100);
System.out.println("After sleep...");

// Test of "checked exception not thrown"
try{
System.out.println("Inside try...");
}catch(IOException e){
}

// Test of "unreachable statement"
return;
System.out.println("This will never be executed!");
}
}



With a stock javac, you won't get very far:


casper@workstation:~$ javac TweakedJavaCTest.java
TweakedJavaCTest.java:12: exception java.io.IOException is never thrown in body of corresponding try statement
}catch(IOException e){
^
TweakedJavaCTest.java:17: unreachable statement
System.out.println("This will never be executed!");
^
TweakedJavaCTest.java:6: unreported exception java.lang.InterruptedException; must be caught or declared to be thrown
Thread.sleep(100);
^
3 errors



In fact, we have no artifact to run. With the tweaked compiler it's an entirely different matter however:


casper@workstation:~$ tweakedjavac TweakedJavaCTest.java
TweakedJavaCTest.java:12: warning: exception java.io.IOException is never thrown in body of corresponding try statement
}catch(IOException e){
^
TweakedJavaCTest.java:17: warning: unreachable statement
System.out.println("This will never be executed!");
^
TweakedJavaCTest.java:6: warning: unreported exception java.lang.InterruptedException; must be caught or declared to be thrown
Thread.sleep(100);
^
3 warnings



Since we've reduced the previous errors to now being just warnings, we'll get an actual build which we can run:


casper@workstation:~$ java TweakedJavaCTest
After sleep...
Inside try...
casper@workstation:~$



Voila, pretty easy eh? If you want to play with this javac build you can grab it here. To compile a Java source file with this javac build, you need to use it this way:


casper@workstation:~$ java -Xbootclasspath/p:tweakedjavac.jar -ea:com.sun.tools -jar tweakedjavac.jar TweakedJavaCTest.java



In conclusion

I'm actually surprised at how easy it was to make these small tweaks. While some people clap their hand at checked exceptions, I think this more lenient version is how the Java compiler should have behaved from day one. The obvious drawback is that you are required to build and distribute your own javac which won't sit very well in many organizations, even if you could still use it as a less-hassle development tool for yourself. The other issue is that of IDE support, although it should be relatively easy* to plug this javac into NetBeans, other IDE's rely entirely on their own parser.

It would be nice if tweaks like these would be considered for the official JDK7, since it doesn't actually break backwards compatibility. However, Sun is an extremely conservative company and has not shown interest in fixing or evolving the compiler over the last couple of years.
Perhaps the way forward is an alternative approach, which does much the same, but without requiring a modified compiler. Reinier Zwitserloot from the Lombok project is dabbling on such an approach which you might want to check out.


* I did give it a quick try, packing up a tools.jar and placed in the JAVA_HOME/lib folder and making sure NetBeans were using this platform for my project. However it did not work as expected. While I was able to build inside NetBeans, the syntax highlighting did not pick up on my modifications.


Update

I have since added this to Kijaro under the branch leniency (you'll need a java.net login).

Thursday, August 6, 2009

Android book roundup

While it may no longer be so much for reference purposes as in the past, I still really like to have good books at my disposal as a software developer. The DPI of a book still beats that of any screen and I can bring a book to the restroom without raising eyebrows from my girlfriend. In the following I will come with a brief but hopefully useful review of the 3 Android books I have in my library.

O'Reilly's Android Application Development


The book covers Android 1.1 stuff and is thus already a little dated. The content and organization is a odd and lackluster I find, for instance there's a chapter on signing and publishing your application which comes before chapters on basic views and widgets. It also doesn't come with an ebook/PDF so there's really not a whole lot going for this book, you are probably better of looking elsewhere.





Manning's Unlocking Android


Also covers 1.1 stuff, but better content and definitely better organized than the O'Reilly book. It comes with ebook/PDF as well as source code. It does make certain assumptions regarding the skills of the reader (for instance how the Eclipse IDE works, as examples are Ant based and needs to be imported) but I reckon that most readers would be quite satisfied with this book. It has foreword by none other than Java Posse's chief editor Dick Wall.




Commonware's The Busy Coder's Guide to Android Development


This is the most recent published book in this mini roundup. I only have the ebook/PDF version so I am not 100% sure whether the dead tree version covers 1.5/Cupcake as well. You actually get 3 books, incl. one on advanced topics such as reading sensory and camera data. It works by 1 year subscription basis and without DRM, so you are likely to also get 1.6 and 2.0 coverage by simply downloading the books at a later point in time when they have been revised. The material itself feels very to-the-point, yet it's actually the only book I have come across that provides a thorough introduction to lists - an essential part of any Android application.

In conclusion

It's my humble opinion that all the above books favors XML layout a bit too much, as it makes examples harder to read/type - and lets face it, it's limited how complex a layout will be on such a small screen anyway. This appears to be more or less dictated by the architecture of Android so not much to do about that. If you have developed in JSF vs. Wicket/GWT then you know what I'm talking about. :)

If you only want one book on the subject, "The busy coders guide to Android Development" would be the one to go for. The content, organization and presentation is just unmatched by any of the others, though "Unlocking Android" was by no means a bad book. However, "Android Application Development" was a disappointment.


Update
Since writing this initial entry, Romain Guy from the Android team have informed me on this thread, that XML is favored in order to have multiple resources set up declaratively such that Android can automatically select the best match depending on hardware and environment. That makes a lot of sense of course, even if I still think this is an aspect better introduced to the reader after having tried some basic UI building with Java in order to even out the initial learning curve. After all, Java provides beginners with familiarity, type-safety and code completion. Although I've talked to other developers who share this opinion (slides from a Danish JUG meeting), yours may of course vary.