Saturday, November 26, 2011

Group Programming and Issue-based Project Management

I spent the last couple weeks doing some group-based programming on a small WattDepot project. The project was to build a command-line interface that lets a user poll the energy data collected from one of the dorms on the University of Hawai'i--Manoa campus. Different commands allow the user to see the current power consumption of a dorm tower or lounge, to rank the different dorm towers by energy consumed over a given time period, to see the energy use of a tower or lounge for a given day, or to see the total energy consumed by a tower or lounge since a given date.

I haven't done any group programming in about 10 years, but I found the return to it quite enjoyable. I find it motivating to know that someone else depends upon and is eagerly awaiting my changes. I found myself prioritizing this project over my other projects to make sure I wasn't holding someone else up.

Our group consisted of only two programmers. We used Google Project Hosting's issue-tracking system to manage the project. We met occasionally to map out the course of the project, and we would create new issues for tasks that needed to be done. Then one of use would accept responsibility for the task and work on it. Once it was done, the other person would look over it and mark the issue as verified. Since the different issue pages allow for threaded discussion, I found this is a good way to track the progress of the project. It was also very easy to link from source code commit logs to the relevant issues and also back from an issue to those commits that solved it.

We happened to worked in shifts, and so we never had to deal with a code merge. I'm a little disappointed I haven't had a chance to experience this yet.

We also used continuous integration. That means we had a Jenkins server setup that independently verified that each commit to the Google repository still compiled successfully and passed all tests and QA checks. This was a comfort. After a commit, I'd always wait a few minutes to make sure I didn't receive an email from Jenkins. When I didn't, I felt more confident signing off for the night.

Our project was plagued with external problems, though. I had to make a emergency 5-day trip back to the mainland for a funeral during the middle of the project. My partner, Jeff, broke his leg around the same time. Besides being on painkillers after that, he wasn't able to drive to campus. So we only had one initial face-to-face meeting, but we compensated with a couple phone meetings and nearly-daily overview emails in addition to the issue-tracking threads. Near the end of the project, the WattDepot server went down, which halted production and then broke all of our test cases. This meant that our build appeared broken for about 3 days while we waited for the new server to generate enough data for us to work with.

All this planning and coordination certainly did come with an overhead. I'd say this project took 2 to 3 times longer than if we'd just done the project independently. But, like I said, I found the group atmosphere quite motivating, and it probably helped to have a couple pairs of eyes look over all the code.

Tuesday, November 8, 2011

WattDepot Katas

So I spent part of this week and all of this evening getting up to speed with WattDepot by completing a few practice katas. Each kata took me about an hour, though for different reasons.

The first couple katas were simple requests from a WattDepot server.
Given the sample code from the WattDepot client tutorial, this required only a big copy-and-paste and the modification of a couple lines.

These still took a couple hours, though, because I did the first couple katas in class on a day that we tried pair-programming. I think this might be a good practice once both programmers are comfortable with the API and understand the problem at hand. As it was, we spent most of the time setting up our Eclipse projects and then reading the assignment and the WattDepot documentation over each other's shoulders. I also switched groups halfway through, and we were also required to swap laptops periodically. This means the second pair I was in ended up starting over again at the beginning of one of the katas on a different machine.

Lulled into a false sense of security that these katas would all be as easy as the first couple, I returned to the project late this afternoon. I first took some time to install the FindBugs plugin in Eclipse. (I've been gradually installing plugins for tools that I have so far been using Ant to run. It's generally tedious and slow to run Ant compared to a plugin. But it's been a slow transition since I like to spend an hour or so reading the documentation and playing around with each plugin before I install the next one.)

After cleaning up what code I threw together in the last few minutes of class to see it work, I finished up kata #2. That kata and the ones after it all require sorting a list of key-value pairs by value. This stumped me a for a little while because this should be very easy... but it's not. I surfed a few 1/2 to 1-page posts of code that sort a Map by value. There were a few nice one-liners, but they required Google Collections and I didn't want to mess with an extra library.

In the end, this is my solution to sorting a key-value list by value. It's conceptually a "two-liner", but all the generics and the anonymous class make this look horrible:

  Map<String, Long> map = ...  //the map to be sorted by value

  //dump map into a list of key-value pairs...
  List<Map.Entry<String, Long>> entryList = 
      new ArrayList<Map.Entry<String, Long>>(map.entrySet());
  //...and sort it with a comparator that sorts based on values
  Collections.sort(entryList, new Comparator<Map.Entry<String, Long>>() {
      public int compare(Map.Entry<String, Long> left, 
                         Map.Entry<String, Long> right) {
        return left.getValue().compareTo(right.getValue());

  //now print the list in value order
  for (Map.Entry<String, Long> entry : entryList) {
    System.out.print(entry.getKey() + ": " + entry.getValue());

This solution also assumes you don't need map-like access to the sorted version, but just need to run through it for a simple task like printing.

The other time-consumer was learning how to use java.util.GregorianCalendar and
javax.xml.datatype.XMLGregorianCalendar. (There's also org.wattdepot.util.tstamp.Tstamp,
though I didn't find this to be as useful as I'd initially hoped.) This was another example
of something that should be easy but isn't. For example: "Give me the date of the most recent
Monday". It seems we have to start with the current time, for which I chose to use a GregorianCalendar object. Then, after a few lines spent modifying that time to the desired relative Monday date,
I had to translate it to an XMLGregorianCalendar. This was fairly long and messy with the various required
factory objects and exception handlers. I still like Java a lot, but I must admit that detractors
have a point about how much grinding tedious boilerplate you have to wade through sometimes.

I also need to take a minute at some point to figure out how to control Eclipse's insertion of try/catches. The quick default just wraps the one line in a try block, which requires so much reformatting that the feature is hardly worth using in most cases.

In the end, I got all the katas done. This was the first time I've written an application that deals with a web connection behind the scenes. It is pretty slow and variable (read: flaky) at times due to network access delays and potential connection time-outs. For my last kata--which took a couple minutes to gather all the data it needed--I added a little "status indicator" by printing a series of dots.

I spot-tested my katas as I went. However, if they were production code, I think I'd like to test them a little bit more to make sure I'm really pulling the right data and that all my calendar manipulations worked correctly. But since they are mainly katas intended to practice solving the problem, I think they went quite well.

Tuesday, November 1, 2011

Energy Consumption in Hawaii

I'll soon be working with WattDepot, so this week I've been learning a bit about energy consumption in Hawaii as background.

First, I had to review bit of high school physics regarding the difference between energy and power. Power is rate at which work is done or energy is converted over time. Power is measured in watts. One watt is equal to one joule/second. So a 60W lightbulb burns 60 joules/second.

Energy is... well, energy--the stuff that makes things move, go, change, or happen. It is force applied over a distance. When related to household electricity, it is usually measured in kilowatt-hours, or kWh. One kWh is equal to 3.6 megajoules.

I also learned that Hawaii is in a pretty unique situation when it some to exploring alternate/green energy sources. Our energy on Oahu costs about 30 cents per kWh, which is 3x what it costs on the mainland. (And on the neighbor islands it costs 6x mainland prices!) This is because we rely predominantly (70+%) on shipped oil, whereas the mainland can also rely on nearby coal and natural gas sources. We have a number of small, discrete electrical grids here, rather than the massive interconnected grid that spans much of the mainland. This means we can't buy, sell, or shift extra energy around to neighboring states (or even neighboring islands) as demand shifts. This is fairly inefficient.

But, on the bright side, this expensive energy means that alternative energy sources may be more attractive here in Hawaii than they would be on the mainland. The Hawaii Clean Energy Initiative has also mandated that 70% of Hawaii's energy come from clean sources by 2030. To our benefit, we have nearly every kind of potential renewable energy source here: wind, solar, waves and tidal motion, hydrothermal, and geothermal sources.

Data about current energy use and how this usage will be affected by alternative sources will be essential. This information can also provide feedback to end-users to help them reduce their consumption, which is also an important goal. There is currently some interesting research going on at the University of Hawaii regarding smart sensors and tracking current use... which brings us back to WattDepot. It is currently being used to track energy use in some of the UH dorms.

I'll be learning more about the specifics of WattDepot this week. Check back again soon for more info!