Polyglot Window: Chris Hansen's Blog

Switching to IntelliJ IDEA

2012-01-14T12:29:00.000-08:00

IntelliJ IDEA 11 came out recently, and after some frustrations with Eclipse I decided to give the Community Edition a try. To my surprise, I've decided to stay with it, and I thought it was a good opportunity to dust off the old blog to explain why. Maybe it will be useful to others who have been considering trying IDEA for a while but aren't sure if it will be worth their time.

Eclipse

First off, let me give Eclipse some credit. I've been more than satisfied with it over the past 6 or 7 years. I haven't had any major issues with Eclipse, but it's been a bit like death by a thousand stings. Most of the pain comes from a few poor quality plugins or good plugins not playing nicely with each other.

Eclipse's incremental compilation is incredible. It's probably the biggest feature that differentiates Eclipse from IDEA, both pro and con. The pro is immediate feedback. By the time you're ready to run your tests, Eclipse has done most of the heavy lifting and is ready to launch. However, that means the heavy lifting happens while you're editing or otherwise interacting with the UI, which can make the IDE feel less responsive at times. The JDT can also affect some plugins, which I'll get into later.

Enter IDEA

Later I'll list a bunch of small features that don't seem like much individually, but combined make a big difference. But first, the big differences.

Responsiveness
The biggest difference I've noticed with IDEA is its responsiveness in regard to user interaction. It feels lightweight and snappy compared to Eclipse. Best of all, there's immediate code completion with no keyboard combination to press and no waiting. This is huge. There have been several times I've cursed out Eclipse for having to wait a few seconds after hitting Ctrl Space. Especially in Scala code (more on that later).

Maven Integration

Surprisingly, IDEA has better Maven integration than Eclipse, despite Sonatype's great work on m2e. This is most likely an indirect result of Eclipse's incremental compilation. Because of this killer feature, Eclipse cannot delegate the build to Maven. That means m2e has to integrate the whole Maven build lifecycle into the JDT. This is called "project build lifecycle mapping" and the result is more configuration and plugin bloat. What's worse, if you're missing a lifecycle mapping it's an error by default: "Plugin execution not covered by lifecycle configuration". This was the straw that broke the camel's back for me.

Even the pom file editor is better in IDEA. For example, it will autocomplete Maven groupIds, artifactIds and versions for artifacts you don't yet depend on, which has saved me from going outside of the IDE (especially versions). Also, it has built-in completion for Maven plugin configuration parameters in the pom file which I've had to go to the web to look up in the past. "Navigate to Managing Dependency" lets you navigate from a dependency to its corresponding dependencyManagement section, even if its in a parent pom. Plus it has an "Introduce Property" pom file refactoring which is a time-saver.

Scala Integration
Scala integration in Eclipse is improving at a fast pace and soon may surpass that of IDEA, but for now I give IDEA the slight edge. It makes sense because Scala is in the same boat as Maven in Eclipse of having to integrate with the JDT. With Scala this is much more of a problem, though. The sheer amount of work that the Scala compiler has to do means incremental compilation of Scala code significantly affects the performance of Eclipse.

For the most part, Scala "just works" in IDEA, with most of the same features Java has plus the Scala-specific ones you'd expect. It even has a "Convert Java file to Scala" refactoring which is also known as the Ugly Scala Code Generator (TM), but it might save you some time if you're converting a project over. However, I have had some rare phantom errors in IDEA, meaning code marked red in the editor that compiles just fine. Also, since editor parsing is separate from compilation, these don't go away when you do a build or run tests like they might in Eclipse.

The Rest
This is the list of minor improvements which I've found useful. To keep it brief, I won't go into too much detail on each one, so I apologize if these are unclear. Many of these are examples of IntelliJ living up to its billing as an intelligent IDE because they require a deep understanding of your code.

Ability to search for actions (Cmd Shift A). I cannot overstate how valuable this has been in my transition. You can look up any action (refactoring, searching, etc.) and execute it while keeping your hands on the keyboard. It also displays keyboard shortcuts so eventually you won't have to look it up again.
You can limit the scope of any search (text search, class hierarchy, etc.) to only production code or only test code.
Highlights classes as being unused when they are only referenced by tests.
Colors tabs differently based on the nature of the class. For example, test classes are green.
Colors annotations differently to distinguish them in imports.
Rename refactoring prompts to rename constructor and setter parameters in addition to getters and setters.
Console output from each test is self-contained, but you can get to the full console output if you need it.
Cmd click navigation from a string literal to a file with the same name in the project.
"Go To Test Subject" (Cmd Shift T) from a test class navigates to the class under test.
Optionally warns you when you're about to commit code with warnings/errors, and can create a separate changelist for you with only these changes if you choose not to commit.
"Shelve commits" lets you put aside your current changes so you can work on the current revision (or a different one) if you need to.
Autocomplete within string literals.
When you rename a class, it prompts you to rename similar variable names where the class is being used.
Extract method also finds duplications with different method arguments. I used to manually "pre-refactor" to get this same behavior in Eclipse by first extracting local variables for the intended arguments.
Smart step-into when debugging so that you can step into a specific method on a line of code which calls many methods.
If you're editing a class file and add a reference to a class name that can't be resolved, there's a quick fix to "Add Maven Dependency".
Introduce parameter can move local variable initialization code from within a method to the method's callers.

And here's some features that look useful, but I haven't tried yet.

Analyze dependencies.
Analyze backward dependencies.
Refactor -> replace inheritance with delegation.

Conclusion

It's mostly a bunch of little things that I like better about IDEA. Any one of them wouldn't be enough to make the switch, but combined make me feel more productive even after a few short weeks.

I know it's just a matter of time until I buy the Ultimate Edition, and I'm ok with that. You win this round, JetBrains. ;)

ARM Blocks in Scala: Revisited

2009-03-26T19:42:00.000-07:00

Over a year ago, when I was new to Scala, I tried implementing Automatic Resource Management. I was not pleased with the result, so I ended up recommending ManagedResource from Scalax. After I came across a couple of links to my original post, I decided to point out a different approach that I have seen which seems to work well and appeals to me, personally.

Next, I'll try to figure out where I went wrong originally, but those who are only interested in the new approach can skip the next section.

Unnecessary Complication
One of my original goals was to allow multiple resources per block. This ended up making the implementation much more complicated, especially with regard to exception handling. Also, in my first approach I included an exception handling mechanism which turns out to be unnecessary if we're only dealing with one resource at a time. The biggest problem with my first attempt, however, was that the resource had to be initialized outside of the ARM block, which, when multiple resources are initialized, can lead to the very resource leaks which we are trying to avoid. It was a good effort, however, if I do say so myself, and helped me to learn the language a bit.

A Simpler Approach
Lucky for me, Scala's first expert, Martin Odersky, also thought ARM was a good example of the flexibility of the language. I don't know when he started using ARM in his talks, but the first time I saw it was at JavaOne 2008, and then again at the Scala Lift Off event the following weekend. I will not re-post Odersky's implementation here without his permission (not that I think he would object), but I recommend that you take a look at slide 21 of his FOSDEM 2009 presentation (which was hopefully posted with his permission).

Odersky's approach is much more elegant. It only allows a single resource to be managed per block, which is actually all you need since ARM blocks can be nested to support multiple resources. Exceptions are not caught so that the caller can handle them properly. Simplicity. This ends up being very similar to the approach that C# has taken, so he chose to name the method the same as the C# "using" keyword.

Here's how you use it:


//print a file, line by line
using(new BufferedReader(new FileReader("test.txt"))) {
  reader => {
    println(reader.readLine)
  }
}

//copy a file, line by line
using(new BufferedReader(new FileReader("test.txt"))) {
  reader => {
    using(new BufferedWriter(new FileWriter("test_copy.txt"))) {
      writer => {
        var line = reader.readLine
        while (line != null) {
          writer.write(line)
          writer.newLine
          line = reader.readLine
        }
      }
    }
  }
}

I actually like this approach better than the ManagedResource from Scalax because the syntax feels more natural; it feels like you are using a language feature. I always thought ManagedResource was a bit strange because it felt like an abuse of the for-comprehension, but that's just my opinion.

Since I hadn't seen anyone point out Martin Odersky's "using" implementation, I thought I would post about it. I think ARM is great and the fact that it can be implemented as a library in Scala is phenomenal.

If we're lucky, Java SE 7 will have ARM thanks to Project Coin, but it's getting more and more obvious these days that Java is not progressing as a language. Now if only I could do my everyday coding in Scala....

Introducing Pojomatic

2009-01-07T21:09:00.000-08:00

New year, new project
I'd like to start off the new year by introducing a project I've been working on for a while now along with Ian Robertson, a colleague of mine at Overstock. The project itself is actually not new at all, considering we began working on it around April of last year and it's based on something similar we've been using internally at Overstock since the early days of Java 1.5. The project is called Pojomatic, and what's new is that it's open source (Apache 2.0 license) and there's a release candidate available (1.0-RC1) on Sourceforge or the Maven central repository.

What does it do?
Pojomatic is a Java library which provides automatic and configurable implementations of the hashCode() equals(Object) and toString() methods inherited from java.lang.Object using annotations. POJOs (Plain Old Java Objects) + automatic implementations of common methods = Pojomatic.

This is a useful because it is generally a good idea to override hashCode() equals(Object) and sometimes toString(). One could manually implement these methods, but that is time-consuming and prone to error (e.g. forgetting to check for null everywhere). Instead, one could have an IDE generate implementations of these methods for you. As with a lot of generated code, this can be like slapping your code with an ugly stick. Besides, I'd often add fields and/or methods to the class later and forget to re-generate new implementations, which may have no effect or may lead to very subtle, hard to detect bugs (e.g. two objects are equal when they shouldn't be => two different people mistaken for the same person => money is deposited to the wrong account => one person is happy, while you are not because you have to work late to track down and fix the bug).

How do I use it?
The easiest way to use Pojomatic is to put one annotation (@AutoProperty) on your class and delegate the desired method(s) to the corresponding static methods in Pojomatic. For example:

@AutoProperty
public class Person {
  private final String firstName;
  private final String lastName;
  private final int age;

  public Person(String firstName, String lastName, int age) {
    this.firstName = firstName;
    this.lastName = lastName;
    this.age = age;
  }

  public String getFirstName() {
    return this.firstName;
  }

  public String getLastName() {
    return this.lastName;
  }

  public int getAge() {
    return this.age;
  }

  @Override public int hashCode() {
    return Pojomatic.hashCode(this);
  }

  @Override public String toString() {
    return Pojomatic.toString(this);
  }

  @Override public boolean equals(Object o) {
    return Pojomatic.equals(this, o);
  }
}

By default, @AutoProperty tells Pojomatic to automatically detect all of your fields and use them in all of the Pojomatic.* methods. We'll see the different options later, but in the above example, all of the fields will be used for hashCode() equals(Object) and toString() as shown here:

public static void main(String[] args) {
  Person johnDoe = new Person("John", "Doe", 32);
  System.out.println(johnDoe.hashCode());
  System.out.println(johnDoe.equals(new Person("John", "Doe", 32)));
  System.out.println(johnDoe.toString());
}

Outputs:

-2068529904
Person{firstName: {John}, lastName: {Doe}, age: {32}}
true

Using all fields for each of hashCode() equals(Object) and toString() is usually not advised, however, so @AutoProperty can be configured to include automatically detected properties in any (valid) combination of these methods (including something in hashCode() without including it in equals(Object) violates the contract of hashCode(), so this is not allowed). Additionally, properties can be configured individually via the @Property annotation. When both annotations are present, @Property is used since it applies only to the one property. Using @Property gives you complete control and makes @AutoProperty optional. A common practice is to have all properties included in equals(Object) and toString(), while only one or two key properties are included in hashCode() like so:

@AutoProperty(policy=DefaultPojomaticPolicy.EQUALS_TO_STRING)
public class Book {

  @Property(policy=PojomaticPolicy.ALL)
  private final String isbn;

  private final String title;

  public Book(String isbn, String title) {
    super();
    this.isbn = isbn;
    this.title = title;
  }

  public String getIsbn() {
    return isbn;
  }

  public String getTitle() {
    return title;
  }

  @Override public int hashCode() {
    return Pojomatic.hashCode(this);
  }

  @Override public String toString() {
    return Pojomatic.toString(this);
  }

  @Override public boolean equals(Object o) {
    return Pojomatic.equals(this, o);
  }
}

Both @AutoProperty and @Property can use accessor methods (getters) instead of fields in situations where using accessors is more desirable or when a SecurityManager prevents Pojomatic from accessing private fields through reflection.

Briefly, there is also a feature which will let you customize the String representation of each property. For example, this would be useful if one of your properties contains sensitive data such as an account number, credit card number or social security number (see AccountNumberFormatter). Pojomatic provides the ability for you to define your own formatter implementations as well.

Feedback
Pojomatic has been a lot of fun to work on, and I hope you will find it useful. I'm confident that after trying it out, you will not want to go back to handcrafting equals methods or using ugly IDE-generated code instead. Any feedback is appreciated, as well as feature requests, so what do you think?

Learning Wicket

2008-10-27T22:14:00.000-07:00

I'm always on the lookout for different ways of doing the things I'm familiar with. Once in a while, I learn a better way of doing things. Even if I don't, I find that it helps me realize what I like and dislike about the techniques that I'm used to.

Take creating web applications for example. I'm familiar with JSF, both with and without Seam. Overall, I would say that I agree with the ideas behind JSF, but in practice it is not easy to be productive in developing a medium to large sized web app. It seems like some things that should be very easy to do are difficult, but maybe that's just me. Seam adds a bit more complexity, but helps overall. To the JSF world of managed beans, Seam adds ways to manage the rest of your objects called scopes. This is a good idea on paper, but feels a bit like using global variables (albeit with managed lifecycles).

So I decided to give Wicket a try. What stood out to me right away is the simplicity of it. There's very little configuration, and no XML other than web.xml (which makes me realize how much I dislike having stuff like page navigation in an XML file). Each page template corresponds to one Java class. Also, there is no EL or OGNL so your markup is sure to have no logic whatsoever, which is great for those of us who prefer to code in Java. That's the bottom-line benefit that I see with Wicket: everything happens in Java code. Which means you get all of the benefits of writing Java code in an IDE from debugging to refactoring, etc. I haven't tried it, but I've read that it's easy to use alternative JVM languages such as Scala and Groovy instead, if you are so inclined.

Wicket is very friendly to the web designer because it does not invade the markup, so the designer can see what they are getting without firing up a web container. For now, I haven't seen any Wicket components which look good "out of the box" (translation: you probably need a web designer). However, there is no reason why there couldn't be such component libraries for Wicket. That will come with increased adoption, if it happens. After all, creating a component library in Wicket means putting your classes and markup in a JAR file, so it couldn't be easier.

Some minor gripes I have with Wicket are that the markup has to stay in sync with your Java class (but all web frameworks I have used also have this problem), it can sometimes add junk to your URLs after a form submit (maybe there's a way around that). Also, there's quite a bit of casting when using version 1.3, but they are adding generics to version 1.4 which should minimize that. Overall, I'm very impressed.

Resources:

Tapestry and Wicket Compared

Wicket home for useful documentation, examples, etc.

JavaWorld Series on Wicket by Nathan Hamblen

Ideal command prompt

2008-03-12T21:46:00.000-07:00

In setting up my new laptop, I've been experimenting with different command prompt settings in bash. Here is what I've settled on for now (click for a more clear image). It's simple, yet gives me all of the information I need in a format that is easy for the eye to parse. That's because the text color of the full path is inverted from your other colors.

If you are interested in using/tweaking it:
export PS1="\u:\[\e[0;07m\w\e[m\]> "

What's your ideal PS1? Here's a how-to article. It's written with Linux in mind, but the information applies to pretty much any environment where bash is present (OS X, Unix, Cygwin, etc.).

EDIT: My original post did not wrap the color code in \[ and \] which caused the first line wrap to overwrite the current line. Fixed!

Rantings of a recent Mac convert

2008-03-11T23:22:00.000-07:00

I ordered a new MacBook Pro (Penryn w/ multi-touch) which arrived last Monday (which probably explains my lack of posting since then). Although I do have a bit of experience with OS X and actually do my programming at work on a Mac mini (I'm in the minority here), this is my first purchase of any Apple product (which means I don't own an iPod - *gasp*).

The Desktop OS Wars

As far as OSes are concerned, I don't mind Windows, but I find that it gets in my way sometimes. That is fine for most people because they know the workarounds (e.g. reboot, re-install every 6 months...). To me, this is a waste of time since other OSes don't have this problem. This observation is mostly from XP and previous versions, but based upon what I've heard about Vista, I don't expected much better. Also, the lack of a robust terminal/shell (with things like adequate scripting) is completely inexcusable when the competitors run bash and the like.

Linux is a rock-solid OS. It dominates the server environments, and rightly so. The problem is the desktop user experience. The average computer user cannot be expected to survive here, and even the most technically savvy users often find themselves searching through forum posts for hours on how to overcome some hurdle or another. Driver support is non-existent for the most part. The installation process is flawed for the vast majority of distros (one exception is Ubuntu and the like). Also, while package management is great, there are often packages that are missing and/or outdated, so it loses much of its utility. Granted, Linux in general (and Ubuntu in particular) are making great strides in this area, but the progress is too little, too late.

That brings me to OS X. I think Apple's really gotten it right. Take something solid (BSD - though modified with a very different kernel) and put a nice UI on top of it, and that's exactly what they've done. I have a terminal session (bash) open 90% of the time when I'm programming, which is indispensable. I never payed much attention to aliasing or font rendering before, but I can't help but notice how much better everything looks. Some of that has to do with the top-notch (read: expensive) displays that Apple uses, but at work I've used the same (non-Apple) display on Windows, Linux and OS X and there is a clear difference.

Final Thoughts

I have to say, I am very impressed with my MBP so far. It does everything I want it to, without any of the quirks that are present in other hardware/OS configurations. I think this is due to the relatively small number of available hardware configurations for Macs. This puts Apple in the unique position of being able to test every possible hardware configuration - they know it works. I've heard from other Mac users that "everything works", but I don't think I really believed them or understood the implications of that statement. Now I get to experience it firsthand, and it's refreshing.

Anyhow, I hope to get back to posting more Scala stuff soon, and maybe something about Java for a change.

ARM Blocks in Scala, Part 3: The Concession

2008-02-23T10:31:00.000-08:00

Update: Here's a better approach to Automatic Resource Management in Scala.

After a couple of attempts (part 1, part 2) at implementing Automatic Resource Management in Scala, I've decided not to "reinvent the wheel" here. I will defer instead to the implementation found in Scalax. I did not know about their ManagedResource class before my first post (many thanks to the commenters who pointed me there), and if I had known about it I may not have made the attempt. I'm glad I did, however, because it gave me a chance to improve my Scala skills. I'm not "there" yet, though, as I still find myself writing Java-like code in Scala. When I catch myself doing so, I am usually able to refactor it into the Scala style, which ends up being more compact and elegant. Come to think of it, that is why I recommend ManagedResource over my approach. I would say mine is the Java-like approach, while ManagedResource is more elegant and more consistent with Scala style.

Let's look at the same example from my first two posts using ManagedResource:


  def createReader = ManagedResource(new BufferedReader(new FileReader("test.txt")))
  def createWriter = ManagedResource(new BufferedWriter(new FileWriter("test_copy.txt")))
  
  //copy a file, line by line
  for(reader <- createReader; writer <- createWriter) {
      var line = reader.readLine
      while (line != null) {
        writer.write(line)
        writer.newLine
        line = reader.readLine
      }
  }

Why is ManagedResource better?
It may be clear from the example, but let's discuss what it is that makes the usage of ManagedResource cleaner than the previous approach. ManagedResource uses for-comprehensions, and that alone solves many of the problems I encountered. For example, I had the problem of being able to define and initialize a resource and still being able to reference it inside of the block of code. For that reason, I had to define an initialization function (in part 2), but "for" takes care of this nicely: for(a <- ManagedResource(new SomeResource())) .... It also takes care of the many-resources problem elegantly, without using varargs: for(a <- createA; b <- createB; ...). In short, "for" seems like the right tool for the job.

That being said, I think ManagedResource does have some room for improvement. For example, consider the following code segment:


  def createReader = ManagedResource(new BufferedReader(new FileReader("test.txt")))
  def createWriter = ManagedResource(new BufferedWriter(new FileWriter("test_copy.txt")))

 try {
    //copy a file, line by line
    for(reader <- createReader; writer <- createWriter) {
      try {
        var line = reader.readLine
        while (line != null) {
          writer.write(line)
          writer.newLine
          line = reader.readLine
        }
      } catch {
        case e: IOException => println("Exception thrown while copying: " + e.getMessage)
      }
    }
  } catch {
    case e: IOException => println("Exception thrown upon open or close: " + e.getMessage)
  }

As you can see, we have total control inside of the block of code, but if an exception occurs while initializing or disposing we have no way of knowing which of the two steps was the culprit. This is probably not an issue most of the time, but I could see a possible need for handling exceptions in initialization differently from exceptions in disposal. Maybe this can be improved (it would have to be non-intrusive for the more common, general case), or maybe some level of control must be sacrificed.

Also, at the time of writing, ManagedResource exposes methods for opening (initializing) and closing (disposing) a resource: "unsafeOpen" and "unsafeClose". These cannot be called from within the block of a for-comprehension, but I see no need to make them public, "protected[control]" should be the maximum visibility - if that. Making them public is a mistake because it allows for the same type of resource leaks we set out to quash. In fact, anyone who is calling these methods externally requires more control over where and when resources are initialized and disposed, and should not be using ManagedResource to begin with. If there is a legitimate reason for exposing these methods, I would like to see it.

Overall, however, I have to congratulate Scalax (Jamie Webb in particular) for getting it right. Scalax is still in a very early stage, so maybe the concerns addressed here will be addressed by the time it is ready for release. These concerns are relatively minor anyway, so I recommend using ManagedResource as is.

ARM Blocks in Scala, Part 2

2008-02-15T20:58:00.000-08:00

Update: Here's a better approach to Automatic Resource Management in Scala.

I didn't plan on writing a follow-up to my first post on ARM Blocks in Scala, but I think I will be able to improve upon some things. I don't plan on any future posts on this topic, but I will not hesitate to do another one if sufficient progress can being made.

Here's where we left off with Arm:


object Arm {
  type CloseType = { def close() }

  def manage(resources: CloseType*)(block: => Unit)
  (implicit exceptionHandler: (Exception) => Unit) = {
    try {
      block
    } finally {
      resources.foreach( resource => {
        try {
          resource.close()
        } catch {
          case e: Exception =>
            try {
              exceptionHandler(e)
            } catch {
              case fatal: Throwable => fatal.printStackTrace() //last resort
            }
        }
      })
    }
  }
}

Motivation

I chose to write a follow-up because of a couple of important omissions that I made in my previous post. The first and foremost is that of resource initialization (as "helium" pointed out). The problem is that resource initialization happens outside of the scope of the Arm.manage method. This violates the principle motivation for ARM Blocks by forcing the developer to correctly handle resource disposal in the case that one or more resources is initialized successfully and others are not.
For example:


val reader = new BufferedReader(new FileReader("test.txt"))
val writer = new BufferedWriter(new FileWriter("test_copy.txt"))

If reader initializes successfully and writer does not, reader needs to be closed. This is exactly the type of problem we were trying to avoid with ARM Blocks in the first place. I didn't have a solution for this problem at the time, but I forgot to mention it in my post - a mistake on my part.

Secondly, I didn't mention how an exception within the block of code itself should/would be handled. This means that one would have to wrap the entire call to Arm.manage in a try/catch block in order to handle any exceptions that may occur inside the block itself. Not ideal.

In this post, I hope to address these two concerns. The solution I present here is by no means the best solution, and I am confident it can be improved upon as well.

For starters, let's take care of the first issue, which is the easier of the two:


object Arm {
  type CloseType = { def close() }

  def manage(resources: CloseType*)(block: => Unit)
  (implicit exceptionHandler: (Exception) => Unit) = {
    try {
      try {
        block
      } catch {
        case e: Exception => exceptionHandler(e)
      }
    } finally {
      resources.foreach( resource => {
        try {
          if (resource != null)
            resource.close()
        } catch {
          case e: Exception =>
            try {
              exceptionHandler(e)
            } catch {
              case fatal: Throwable => fatal.printStackTrace() //last resort
            }
        }
      })
    }
  }
}

Here we reuse the existing exception handler function. This is more convenient that declaring two different exception handlers, but has the issue of knowing if the exception being handled occurred upon closing or in the block itself. We will correct this after we address the second issue.

The issue of initialization is a more complicated one. The approach taken here is to define an initialization function, much like our exception handling function. It should be implicit as well in the case that resources have already been initialized and no initialization is necessary. Here's an attempt:


object Arm {
  implicit def defaultInitializer() = {}
  implicit def defaultExceptionHandler(e: Exception) = {}

  type CloseType = { def close() }

  def manage(resources: (() => CloseType)*)(block: => Unit)
  (implicit initializer: () => Unit, exceptionHandler: (Exception) => Unit) = {
    try {
      try {
        initializer()
        block
      } catch {
        case e: Exception => exceptionHandler(e)
      }
    } finally {
      resources.foreach( resource => {
        try {
          val value = resource()
          if (value != null)
            value.close()
        } catch {
          case e: Exception =>
            try {
              exceptionHandler(e)
            } catch {
              case fatal: Throwable => fatal.printStackTrace() //last resort
            }
        }
      })
    }
  }
}

Here we define a default initializer as well as a default exception handler. Defining the exception handler is not really necessary because of the implicit identity function in Scala's Predef, but defining the default initializer may be helpful. In Arm.manage, the initialization and exception handling functions are included in the list of implicit parameters. Only one such list is allowed in the Scala language and it is required to be the last list of params. Also, if we are to use an initialization function, the resources themselves cannot be passed as arguments because it will be call-by-value (i.e. in Arm.manage, they will remain uninitialized). Instead, we use call-by-name and accept an arbitrary number of functions that return resources - remember that methods are functions in Scala. This is a necessary change, but adds a bit of clutter when it comes to usage. We will save an example until later because of a deficiency in this implementation.

In this version of Arm.manage, when an exception occurs and our handler is called, we have no idea if the exception happened in initialization, in the block of code, or when trying to close the resources.

Let's address this issue:


object Arm {
  abstract class ManagementException(val cause: Exception) extends Exception(cause) {
    override def getCause() = cause
  }
  class InitializationException(cause: Exception) extends ManagementException(cause)
  class ExecutionException(cause: Exception) extends ManagementException(cause)
  class ClosingException(cause: Exception) extends ManagementException(cause)

  implicit def defaultInitializer() = {}
  implicit def defaultExceptionHandler(e: ManagementException) = {}

  type CloseType = { def close() }

  def manage(resources: (() => CloseType)*)(block: => Unit)
  (implicit initializer: () => Unit, exceptionHandler: (ManagementException) => Unit) = {
    try {
      var executeBlock = true

      try {
        initializer() //initialize the resources
      } catch {
        //forward exceptions to the handler
        case e: Exception =>
          executeBlock = false //do not continue if initialization fails
          exceptionHandler(new InitializationException(e))
      }

      if (executeBlock) {
        try {
          block // execute the block
        } catch {
          //forward exceptions to the handler
          case e: Exception => exceptionHandler(new ExecutionException(e))
        }
      }
    } finally {
      //close all of the resources properly
      resources.foreach( resource => {
        try {
          val value = resource()
          if (value != null)
            value.close()
        } catch {
          case e: Exception =>
            try {
              exceptionHandler(new ClosingException(e))
            } catch {
              case fatal: Throwable => fatal.printStackTrace() //last resort
            }
        }
      })
    }
  }
}

So now, our exception handler signature takes a ManagementException, which has the original exception as its cause - accessable by e.cause or e.getCause(). We know, based on the type of ManagementException, where it is coming from. Instead, we could define three different exception handling methods, but I think it's easier to use if we only have to define (at most) one exception handler.

Let's see an example:


var reader: BufferedReader = null
var writer: BufferedWriter = null

def getReader() = reader
def getWriter() = writer

def initResources() = {
  reader = new BufferedReader(new FileReader("test.txt"))
  writer = new BufferedWriter(new FileWriter("test_copy.txt"))
}

def handle(e: ManagementException) = {
  e match {
    case ie: InitializationException =>
      println("Failed to initialize: " + ie.getCause())
    case ee: ExecutionException =>
      println("Could not copy files: " + ee.getCause())
    case ce: ClosingException =>
      println("Failed to close a resource: " + ce.getCause())
  }
}

//copy a file, line by line
manage(getReader, getWriter) {
  var line = reader.readLine
  while (line != null) {
    writer.write(line)
    writer.newLine
    line = reader.readLine
  }
} (initResources, handle)

This one passes the methods for initialization and exception handling as explicit parameters.

Here's the same example with a different approach:


var reader: BufferedReader = null
var writer: BufferedWriter = null

implicit def initResources() = {
  reader = new BufferedReader(new FileReader("test.txt"))
  writer = new BufferedWriter(new FileWriter("test_copy.txt"))
}

implicit def handle(e: ManagementException) = {
  e match {
    case ie: InitializationException =>
      println("Failed to initialize: " + ie.getCause())
    case ee: ExecutionException =>
      println("Could not copy files: " + ee.getCause())
    case ce: ClosingException =>
      println("Failed to close a resource: " + ce.getCause())
  }
}

//copy a file, line by line
manage(() => reader, () => writer) {
  var line = reader.readLine
  while (line != null) {
    writer.write(line)
    writer.newLine
    line = reader.readLine
  }
}

This one uses implicit defs for our initialization and exception handling functions. For this to work, we cannot import the implicit defs of these functions from Arm because they would conflict. I should point out that if you provide one function explicitly, you have to provide the other explicitly as well. Also, this example defines getters for reader and writer "inline".

Room for Improvement

The new syntax is not nearly as easy to read as the original. This concession was necessary, though, for the sake of completeness. The problem of an exception thrown from the handler still remains from the first post, as well.

A couple of readers pointed me toward scalax, which provides a ManagedResource class which provides the same functionality using for-comprehensions. I took a look at it and it looks pretty well done, but I haven't yet had an opportunity to use it so I can't comment on how it works in practice. I would be interested to see an example of this which manages more than one resource.

ARM Blocks in Scala

2008-02-12T18:28:00.000-08:00

Update: Here's a better approach to Automatic Resource Management in Scala.

Currently, there is a proposal by Joshua Bloch for the addition of Automatic Resource Management Blocks (ARM Blocks) to Java, possibly as soon as version 7. The idea is that Java should take the responsibility of correctly disposing of resources out of the hands of the developer, much like it did with memory management in its initial design. Personally, I think it is a great idea that wouldn't add too much baggage to the language.

After seeing a couple of implementations of this concept in Scala (here and here), I decided to give it a try myself. I think you'll see that Scala is a very flexible language that already facilitates things like ARM Blocks without explicit language support - mostly thanks to functional programming constructs like closures. In this post, I will assume you have some basic to intermediate knowledge of Scala, including a basic knowledge of its function syntax: "(ArgType1, ArgType2, ...) => ReturnType".

We will construct an Arm object (singleton) which will provide a usable syntax for an ARM Block and offer correct resource disposal.

Here is a first pass:


object Arm {
  type CloseType = { def close() }

  def manage(resource: CloseType)(block: => Unit) = {
    try {
      block
    } finally {
      resource.close()
    }
  }
}

...and an example:


import Arm.manage

//read in a file
val reader = new BufferedReader(new FileReader("test.txt"))
manage(reader) {
  var line: String = reader.readLine
  while (line != null) {
    println(line)
    line = reader.readLine
  }
}

Arm.manage takes a resource to manage, and a block of code (no-arg function that does not return anything) to execute. The only restriction on the resource is that it has a close() method that does not return anything (implicit return type of Unit). The definition of CloseType is just for readability. This is a form of statically checked "duck typing". Notice, in the example, that the block of code passed into Arm.manage looks seamless - as if manage was a Scala keyword. The only extra bit of clutter that is necessary is the "import Arm.manage" (or Arm._ if you prefer), but I can live with that.

This is simple and does exactly what we need, but it is limited to only one resource. We can do better:


object Arm {
  type CloseType = { def close() }

  def manage(resources: CloseType*)(block: => Unit) = {
    try {
      block
    } finally {
      resources.foreach( resource => {
        try {
          resource.close()
        } catch {
          case e: Exception => e.printStackTrace()
        }
      })
    }
  }
}

Here we use Scala's syntax for varargs: CloseType*. Passing zero parameters is allowed here because resources.foreach() does not execute if resources is empty, but I could see an argument for requiring at least one param. This version of Arm.manage solves the problem of managing several resources, but introduces a new problem - what to do when the call to close() throws an exception. These difficulties are mentioned in the ARM proposal, with no solution settled upon. I propose using an optional callback, which is easy enough to do in Scala (and a variant of that would probably work in Java too, now that I think about it):


object Arm {
  type CloseType = { def close() }

  def manage(resources: CloseType*)(block: => Unit)(implicit exceptionHandler: (Exception) => Unit) = {
    try {
      block
    } finally {
      resources.foreach( resource => {
        try {
          resource.close()
        } catch {
          case e: Exception =>
            try {
              exceptionHandler(e)
            } catch {
              case fatal: Throwable => fatal.printStackTrace() //last resort
            }
        }
      })
    }
  }
}

...with an example:


val reader = new BufferedReader(new FileReader("test.txt"))
val writer = new BufferedWriter(new FileWriter("test_copy.txt"))

//copy a file, with exception handling
manage(reader, writer) {
  var line = reader.readLine
  while (line != null) {
    writer.write(line)
    writer.newLine
    line = reader.readLine
  }
} {e => //handle it}

...and another:


val reader = new BufferedReader(new FileReader("test.txt"))
val writer = new BufferedWriter(new FileWriter("test_copy.txt"))

//copy a file, no exception handler
manage(reader, writer) {
  var line = reader.readLine
  while (line != null) {
    writer.write(line)
    writer.newLine
    line = reader.readLine
  }
}

Here is Arm in its first non-beta version. The exception handler is declared as "implicit", which means that it is not required - as in the second example. The callback has to take an Exception as a parameter, however, which is not as specific as one could hope for. In Scala, this does not present too much of a problem because of pattern matching. The real problem is what to do when the exception handler throws an exception and we need to keep closing the resources. I don't have a good answer for that one, but the "give up and just print a stack trace" approach is not unprecedented. If you don't believe me, just run the following Java code (I think I saw this concept in a Java Puzzler once):


Thread currentThread = Thread.currentThread();
currentThread.getThreadGroup().uncaughtException(currentThread, 
  new RuntimeException("Oh no!"));
System.out.println("Moving on");

Furthermore, we have no way to associate an exception with the object which was the source of the exception. There is no doubt in my mind that Arm.manage in this form can be improved upon, but it satisfies our goals. It even gives us control over what to do when an exception is thrown during disposal (within limitations).

One alternative to this implementation would be to declare an overridden Arm.manage method for each type of resource that one would like to manage (e.g. java.io.Closeable, java.sql.Statement, java.sql.Connection, etc.). This gives the benefit of dealing with a more specific type of exception in the callback method, and the possibility of resources that have a method other than close() to dispose them. This comes at the cost of a bit of clutter and rigidity since it does not support an arbitrary object with a close() method. It is definitely worth considering, but in the end I think the flexibility of the "duck typing" approach wins out here.

Another technique that can be used with Arm.manage is implicit type conversion (aka views). This could be useful if you would like Arm.manage to call a method other than close(). Of course it won't call a method other than close(), but you can achieve the same effect. For example:


  class Disposable {
    def dispose(): Boolean = {
      println("Disposing")
      return true
    }
  }

  abstract class AbstractCloseType {
    def close()
  }

  implicit def disposable2closeable(disposable: Disposable): CloseType = {
    new AbstractCloseType() {
      def close() = {
        disposable.dispose()
      }
    }
  }

  val disposable = new Disposable()
  manage(disposable) {
    //do something
  }

I hope I've shown some of the power and flexibility of Scala. Many of you already know this and it's is why you're reading posts about Scala. To the others - this is just scratching the surface of Scala's potential, I believe. Scala is malleable enough to let you do things like creating DSLs or to write simple code that is virtually indistinguishable from Java. All while not having to give up on static typing or the JVM (or CLR for that matter).

I look forward to writing more code in Scala. I don't know if it will be the "Next Big Thing(TM)", but it is certainly refreshing - and that doesn't hurt when it comes to success.

Update: See the follow-up.

Welcome

2008-02-11T23:42:00.000-08:00

It's not much to look at as of yet, but welcome my blog anyway. My name is Chris W. Hansen, and I'm a computer programmer by trade. Others would demand the title "Software Developer" or "Software Engineer", and they are fine titles, but I feel that programmer is a term that sufficiently describes what it is I actually do. Besides, whenever I tell someone I'm a Software Engineer, they inevitably say something like, "Oh... so what is it exactly that you do?"

In future posts, I plan to discuss various aspects of computers, technology and programming, including programming languages, techniques and computer science theory (you know, the "fun stuff"). Right now, I'm in the process of learning an exciting (to me) language called Scala, so I imagine many of my posts (at least initially) will be on that topic.

I'm relatively new to blogging in general, so any tips or advice as I go along are welcome.

Before moving on to my next post (when it arrives), you may want to brush up on Scala. Here are a few links to get started (in no particular order).

Scala Introduction - the basics, straight from the horse's mouth

Scala for Java Refugees - this is a very informative read, and I strongly recommend the whole (6-part) series

First Steps to Scala - written by the language designers themselves