HabitMaster: Alpha release

HabitMaster is now live in the wild. You can see the new one-page HabitMaster project site that links to everything else you might want to see, including a running Heroku instance.

Overall, this deadline has been met with limited success. I'm calling this an alpha release because the HabitMaster achievements (in other words, the actual gamification parts!) are still unimplemented. It means HabitMaster is just a basic habit tracker right now, and still a little rough around the edges at that.

This has mostly been due to just slow progress. According to my time tracker, I averaged around 8 hours a week on ICS691 projects for most the semester. I stepped that up to averaging around 12 hours a week for the past month, but I still don't appear to have much to show for it. Yet it seems all of my implementation projects go this way: I continue to underestimate just how much code is required to do things.

I spent most of my time implementing back-end details, such as streak detection given a list of activities and a particular schedule type. I was excited to use a Python generator (basically a fancy iterator) to good effect there.

I also came up with a workable solution to handling polymorphism in database objects. Specifically, my habits link to a Schedule object, but there are two subtypes of Schedule: DaysOfWeekSchedule and IntervalSchedule. Grabbing habit.schedule would give me a row from the Schedule table, which was fairly useless. I ended up adding a cast() method to Schedule that would resolve that instance to the proper subtype for me.

Django's unittest framework was very handy. I didn't get into the UI unit tests that actually let you test requests and responses, though. I would like to explore that more. I'm still impressed with Django and its documentation.

I picked up more HTML5 tags, which has been enjoyable. I dig all the semantic tags they have now. Twitter Bootstrap and CSS design in general was a time drain, though. There was a lot of fidgeting required to get everything laid out right. I'm still a lousy graphical designer. CSS still trips me up on things that seem like they should be easy, such as define a DIV of a given relative width and then align a couple items to the left and one or two to the right within that DIV. And then have them all play nicely as the window resizes. I will say that Bootstrap helped compared to trying to do this from scratch in CSS, though.

GitHub (and to a lesser extent Heroku) has been an increasing pleasure to use. I put GitHub Pages and GitHub's wiki, issues, and milestone features to work on this project. I code alone so often that I've developed some lazy git practices, though. The main one is that my commits are based on work-session rather than issue. It doesn't affect anyone else, but it means I often commit things half-done or in a broken state. That is going to come back to haunt me if I ever have to rollback to a previous verison. I also have a lot yet to learn about merging and pulling with git. I ran into a bit of that while I was editing on Windows, Linux, and through the GitHub web interface today.

As you can probably tell, I enjoyed all the technical learning this project has afforded me.

Regarding the gamification aspects, I have this lesson to share: It takes a lot of time. Any gamification mechanics will be in addition to implementing the underlying service you want to provide. The gamification needs to be well-thought out, too. If possible, leveraging an existing framework could help cut down on that time... though it means you'll need to spend the time learning the framework and then being bound by its limitations.

The other thing that HabitMaster has clarified for me is the difference between gamification and a serious game. I see HabitMaster as an example of the former but not of the latter. To me, gamification means being inspired in your design by the fun and rewarding engagement we find in games--even when you're designing something that is clearly not a game. Because it's not a game, you can't just paste game mechanics onto your service. You need to think about the experience you're going for... which brings us to deeper concepts like mastery, autonomy, meaning, flow, engagement, reward, and the like. A serious game, on the other hand, is when you take something that is clearly a game and try to graft non-game goals or outcomes on to it.

HabitMaster will likely be dormant for a little while as I finish up some other pressing projects, but I do hope to resume development when I get some time. I'll also be starting a new teaching job in July. I wonder if I might bring some gamification to bear on course design there.

Django on Heroku: Revisited

Last week, I spent quite a while getting a simple "Hello World" Django app in place on Heroku. This week, I wanted to see if I could repeat the process by posting a small working demo project. Things went much more smoothly this time around.

I used git to grab a local clone of the project. I used virtualenv to setup a virtual environment and pip to load that environment with the project's requirements.

I created a local PostgreSQL database so I could test things locally. I remembered to set the DATABASE_URL environmental variable and to sync Django with the new database. I debated on whether I needed to run python manage.py collectstatic in order to move the website's static files (such as scripts and stylesheets), but the site ran fine locally as-is without it.

Since everything seemed to work fine locally, it was time to push to Heroku. I double-checked that .gitignore was in place with venv and *.pyc in it. When I ran git add . I realized that I'd done all of this testing without touching a single project file. That's a neat design.

I created a new heroku app and used git to push the files to heroku. I initially forgot to sync the database on Heroku's end, but the resulting error message was obvious and it only took a minute to correct. To make sure everything really worked, I used Django's default admin interface to add some database contents (a poll with 4 possible answers). Everything seems to be running fine.

Given the light expected traffic, I didn't bother to switch out the default Django server for gunicorn.

All of this took me about 1 hour, compared to last week's 20 hours. Now that I've got this down, maybe Heroku isn't so bad after all!

Django on Heroku: A Reader's Companion

I recently setup a fresh VirtualBox Debian Linux system. Within that context, I then walked through the Heroku tutorial on Getting Started with Django on Heroku. I ran into a number of errors along the way. Most of them took a few minutes (or an hour) of surfing to find a solution. If you've found this page, perhaps you hit one of those same error messages, and so my solutions will save you a little time.

I'll follow the sections of the current Django on Heroku tutorial here and just add the errors and solutions I encountered.

Prerequisites

There are a bunch of different Python installers out there: pip, distribute, setuptools, easy_install, etc. You'll need both pip and virtualenv... but virtualenv is listed specifically as a prereq at the start of the tutorial. According to the virtualenv page, the preferred way to get virtualenv is to use pip... but the best way to get pip is to use virtualenv. To resolve this, I decided to install pip globally... but pip requires distribute. Of course, the easiest way to get distribute is to use pip to install it (haha!)... but there is also this option where you use curl (or wget) to download an installation script. Once you have distribute, you can do something similar for pip.

Start a Django app inside a Virtualenv

It turns out the virtual environment needs a few things installed locally too...

Error: pg_config executable not found.

You need the postgresql package installed. (Use apt-get install postgresql to install it.)

Error: You need to install postgresql-server-dev-X.Y for building a server-side
extension or libpq-dev for building a client-side application.

Okay, so you also need the postgresql-server-dev-9.1 package (currently).

In file included from psycopg/psycopgmodule.c:27:0:
./psycopg/psycopg.h:30:20: fatal error: Python.h: No such file or directory
compilation terminated.

And the python-dev package too.

Hooray! At this point, I could get a Python app created in the virtualenv and run it. Things seemed downhill for me from here until...

Deploy to Heroku

Typing heroku create gave me:

!    Heroku client internal error.
getaddrinfo: Name or service not known (SocketError)

This error was coming from the excon gem. heroku login gave me the same error. Turns out I had a DNS problem. I entered 8.8.8.8 (Google's public DNS server) into /etc/resolve.conf and tried it again. Worked.

Then git push heroku master lead to:

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

I needed to generate a public key with: ssh-keygen -t rsa By default, this puts the key into ~/.ssh/id_rsa.pub. You then need to register it with Heroku: heroku keys:add ~/.ssh/id_rsa.pub

At this point, I could execute the basic Django app both locally (with foreman start) and on Heroku.

Although my app wasn't actually using the database, I figured I'd better try syncing with it anyway...

Syncing the database

I wanted to run it locally first, so I needed to setup a local postgreSQL database. (The tips on the assignment page were very helpful here.) Also, you need to be postgre user in order to execute the psql command that starts the PostgreSQL console. (It took me a while to even track that command down.) So:

root@tinkVr:~# su postgres
postgres@tinkVr:/root$ psql
could not change directory to "/root"
psql (9.1.8)
Type "help" for help.

postgres=# CREATE USER django WITH CREATEDB PASSWORD 'djangoapp';
CREATE ROLE
postgres=# CREATE DATABASE a17 OWNER django;
CREATE DATABASE
postgres=# \q

You'll probably want to change the lowercase details (user, password, and database names) to something else. Note that you do not need to update your Django project's settings.py with these details because dj_database_url will pull these from from the DATABASE_URL environment variable. (If you log into Heroku's web interface and check your database settings, you can see what these values will actually be in the Heroku environment.) However, to run locally, you do need to set this variable appropriately. In my case, this translates to:

export DATABASE_URL=postgres://django:djangoapp@localhost/a17

typed at the command prompt.

At this point, I could successfully run python manage.py syncdb locally. However, heroku run python manage.py syncdb lead to a timeout. This happened even when just trying to open a simple shell on Heroku with heroku run bash. Heroku does offer some troubeshooting on this. It turns out port 5000 was being blocked by my ISP (the wired network in the Computer Science department on campus). I switched to the public UH wireless and I was able to connect.

The rest of tutorial went smoothly for me. (pip installs gunicorn for you.) Hopefully it goes smoothly for you too!

Next up: Fleshing out the Django app to actually do something...

Group Programming and Issue-based Project Management

I spent the last couple weeks doing some group-based programming on a small WattDepot project. The project was to build a command-line interface that lets a user poll the energy data collected from one of the dorms on the University of Hawai'i--Manoa campus. Different commands allow the user to see the current power consumption of a dorm tower or lounge, to rank the different dorm towers by energy consumed over a given time period, to see the energy use of a tower or lounge for a given day, or to see the total energy consumed by a tower or lounge since a given date.

I haven't done any group programming in about 10 years, but I found the return to it quite enjoyable. I find it motivating to know that someone else depends upon and is eagerly awaiting my changes. I found myself prioritizing this project over my other projects to make sure I wasn't holding someone else up.

Our group consisted of only two programmers. We used Google Project Hosting's issue-tracking system to manage the project. We met occasionally to map out the course of the project, and we would create new issues for tasks that needed to be done. Then one of use would accept responsibility for the task and work on it. Once it was done, the other person would look over it and mark the issue as verified. Since the different issue pages allow for threaded discussion, I found this is a good way to track the progress of the project. It was also very easy to link from source code commit logs to the relevant issues and also back from an issue to those commits that solved it.

We happened to worked in shifts, and so we never had to deal with a code merge. I'm a little disappointed I haven't had a chance to experience this yet.

We also used continuous integration. That means we had a Jenkins server setup that independently verified that each commit to the Google repository still compiled successfully and passed all tests and QA checks. This was a comfort. After a commit, I'd always wait a few minutes to make sure I didn't receive an email from Jenkins. When I didn't, I felt more confident signing off for the night.

Our project was plagued with external problems, though. I had to make a emergency 5-day trip back to the mainland for a funeral during the middle of the project. My partner, Jeff, broke his leg around the same time. Besides being on painkillers after that, he wasn't able to drive to campus. So we only had one initial face-to-face meeting, but we compensated with a couple phone meetings and nearly-daily overview emails in addition to the issue-tracking threads. Near the end of the project, the WattDepot server went down, which halted production and then broke all of our test cases. This meant that our build appeared broken for about 3 days while we waited for the new server to generate enough data for us to work with.

All this planning and coordination certainly did come with an overhead. I'd say this project took 2 to 3 times longer than if we'd just done the project independently. But, like I said, I found the group atmosphere quite motivating, and it probably helped to have a couple pairs of eyes look over all the code.

Project Hosting: Hosted for the Very First Time

This week added another tool to my programming toolbelt: project hosting and software configuration management.

When working on my dissertation project, I used Subversion (SVN) locally to keep track of different versions of the project's source code. This meant I didn't have to worry about making a big implementation mistake because I could always rollback to a previous verion. Since I was the only programmer, I only committed weekly. The commit notes made for a good log of each week's work.

But this week, I setup a test project at Google Project Hosting for my BusyWeek robocode robot.

Setup went pretty smoothly. Google provides a pretty standard and simple interface to all their services, so nothing was too surprising there.

The built-in wiki and bug-tracking tools are pretty nice. (I haven't had a chance to explore the bug-tracking yet, though.) The wiki markup can be annoying--but isn't that true of every wiki system? Personally, I found the requirement to indent every * list bullet by two spaces rather annoying. And I couldn't seem get my numbered list to continue if I included a code block within one of the points. While all the various wiki markups are supposedly easier, I find I prefer HTML in these cases because then I have clear and explicit control. But this was all pretty minor stuff. The Preview button was a lot of help here, so that I didn't fill the revision history with minor edits.

It's also a little strange that editing a wiki changes the revision number on your software. However, I suppose the contents of the wiki and the contents of the code base are highly related, so this approach probably provides a clearer project history.

I did get side-tracked in selecting a license, though. I still haven't found a simple license that I really like, especially among the OSI licenses. Since my Ant build files and one of my test files is based on a DaCruzer robot project, I figured I had better use the same license. (This is fine, since I like the Apache 2.0 License.) However, neither my project or DaCruzer actually includes any of that license information in the source code itself. I should probably look into that at some point.

Anyway, it's exciting to actually have a project hosted somewhere now! Fun stuff.