Kevin’s Blog

My team and I have just finished our final HCI project.

Unlike most course assignments, this one has a dramatic back story. When the project was announced, I started throwing around the idea of making a multitouch display. I lined up sponsors and some great deals on some hardware. Unfortunately, after I had gathered the team and made the project proposal, it was immediately shot down by the professor for being “too difficult.”

After the original project idea met its untimely demise, we decided to move on. Eric took the initiative to go to Professor Steven Feiner’s office hours and came up with the idea of having a “pile” interface, where images would be stacked on top of and physically interact with each other.

On Tue, 27 Nov 2007 at 4:40pm, ______@columbia.edu wrote:

Project Name: Polarscape

Our plan is to support a radial graph incorporating polar coordinates, in which the distance from the origin acts as one axis, and the angle around the origin acts as the other.

On the y axis, the axis mapped to the radius of the radial graph, we plan to place images based on the popularity of their tags within the image collection. The more popular an image?s tags are within the collection, the closer to the center the object will be placed.

We also give the user the option of displaying temporal data along that axis in terms of newest images go in the center and oldest images are displayed in the outer rings.

We also give the user the option of displaying the images at a given angle proportional to their time of year so pictures in winter will all be grouped together in a quarter of the pile.

We plan to group images in a “pile” in which images are thrown on top of others, as the navigator moves the cursor over a pile, images in the currently relevant local area will be brought to the top of the pile and will be spread out using a custom layout algorithm to give the navigator a better view of the area around the location of interest. .

In order to give the user the ability to define and axis and place tags along it, we have two ideas, either we use the pile idea and allow the user to put tags at a given radius from the origin and place images in rings that given distance, or build a pile per tag and place the piles along a linear axis. Suggestions would be appreciated.

Time went on, we occasionally met and talked about the “pile,” and before we knew it, we only had the weekend to complete the project from scratch. On Friday, we met, but instead of working, we somehow ended up listening to Indian popular music from yesteryear and arguing whether or not we should even follow our own project proposal. On Saturday, I sent out emails and scoured the campus, but my teammates were nowhere to be found. On Sunday, I was at the library at 4 AM because I had accidentally burned a Teflon pan in my apartment in a futile attempt to boil water, and was waiting for the noxious fumes to dissipate.

I decided to check my email. I noticed that none of my team members had responded to emails I sent out Saturday about the project. I panicked, checked the SVN repository, and was relieved to discover that Tejas had submitted something late Saturday night.

I updated my local repository, and to my dismay, discovered that the basic framework Eric had made earlier no longer compiled. NetBeans’s automagically generated, non-Mac compatible GUI Builder code had polluted our fledgling implementation.

At that point, I decided something drastic needed to be done. There was no way we were going to fulfill our project proposal with some form elements pasted together in a Swing interface. I sent out an email telling my team where I was in case they wanted to meet, and I started to code. At 7:21 AM, I had the Picasa API interface set up (thanks to Google’s Java gdata client and Scott from Google) and a psuedo-database layer to locally store the pictures and information fetched from gdata queries. By 8:40 AM, I had tag, month-of-the-year, and hour-of-the-day searches working. By 10:48 AM, I had started working on the UI, and had thumbnail caching working.

At this point, I started to get somewhat hungry. I took a break and went across the street to M2M. I tried to order my usual Kimchi Udon, but they politely informed me that lunch did not start until noon. I decided to start coding in their dining area while I waited. Before lunch, I had figured out how to tell a Java app to go full screen, and started working on displaying and interacting with images.

After lunch, Eric met me at the CS Lounge, and I started coding the interfaces between what was going to be the view/controller and the database. I got keyboard input to work, and then started echoing the typed letters to the screen in a large serif font — without relying on a swing widget. I had query submission working. Eric continued to work on the original framework.

At 2:17 PM, Tejas responded to one of my earlier emails about having a meeting.

Hi lets meet up tonite…around 12am ? Me and Divya are currently working from a different State (NJ). We can try to integrate our code tonite…

A different state?

It turned out they were worried more about filling the generic assignment requirements and had gone ahead and created their own project to meet them, but they planned to ignore the original project proposal. Tejas is probably the highest scoring student in our user interface design class. So, I trusted that he would be able to meet the generic project requirements to the letter. I felt relieved that we now had the basics covered, but still uneasy, since I knew he was using NetBeans’s automatic GUI builder, which would make the project nearly impossible to merge.

Just after our midnight teleconference, something strange happened to Eric’s aging Dell, and he lost all of the work since his last commit - which meant all he had left was the code to render a 3d-looking electric-blue ring in the middle of the screen.

We decided that Eric should continue writing the required documentation while I continued coding, since I was more familiar with the rapidly mutating code base. We integrated his ring-drawing code and then started creating the information spaces.

Creating the Seasons / Day and Night visualizations was quite straight forward. The cyclical nature of the data fit well with our radial graphing requirement. Brightness was also simple, since it was merely a one-dimensional value from 0 to 255, which I mapped to a spiral.

When we arrived at sorting the images by the average of their three color components, we hit a mental block. How do you visualize 3D data on a 2D plane and have it makes sense to normal people? While Eric thought about that question, I worked on the “custom view” requirement. About this time, Tejas came back to New York to remind us that PolarScape was doomed and that his splinter project was our savior.

We decided to split for a little while. I continued to work on the application, and Eric attended his vegetarian cooking class, promising to bring me back something from Roti Roll afterwards. I was going to implement a physics engine for pushing things around the pile, get panning and zooming working, and make the custom view. When he returned two hours later, I had wasted two hours working on the hardest part, the physics engine, only to get stumped by the constant appearance of strange Java2D artifacts. I told Eric that physics probably wasn’t going to happen before the deadline, which loomed just a few hours away, and we dropped the feature.

The food was delicious.

Implementing panning and zooming took about five minutes. Eric fell asleep.

I woke him up and he took over coding — creating the view for the color breakdown — while I thought about how to design the custom view. After a few moments, he came up with the idea of replacing the blue ring with a ring generated by iterating through the color space of integers. The result looked like a 3D rendered ring of structured rust — it didn’t turn out too well. So, instead of generating the color information space background, we decided to make three primary-colored Gaussian blobs in Photoshop. Eric created a simple affinity expression to render the pictures near their respective dominant average primary color values. I added calculations for primary color image statistics to the database, and the color view was complete.

At about this time, less than two hours before we had to turn in the project, Eric kept saying, “We’re DOOOOoooomed!.” His unwavering confidence in the team’s ability to meet the deadline was an extraordinarily effective motivational tool.

To implement the custom view, I simply made it such that the user would specify a tag, place the tag by clicking somewhere on the display, and we would fetch the appropriately tagged pictures from the database. For several minutes, I was at a loss as to why no images were appearing. It turns out that very few people on Picasa actually tag their images. Thinking quickly, I decided to try to get the search keywords instead, but the gdata API only returned “null” when asked for keywords associated with every piece of media I tried. As a simple workaround, I had our tag table automatically tag photos with their respective queries as they were downloaded off the web. This turned out to work quite well. Finally, I came up with a simple equation to draw a radial distance indicator for each tag — a circle with a circumference that intersects the point where the tag is placed along with a radius intersecting at the same user-specified point. For a while I tried to get the tag to draw a drop shadow, but then I gave up because there just wasn’t enough time to waste one something that could be hand-waved during the presentation.

We were almost finished. I handed off my zipped “dist” directory to Tejas, who packed both submissions into the same final turn-in. It took a couple tries to get right, but we eventually got everything in on time. Some parts were turned in seperately, but in the end, I think the course staff figured out all the submissions were from the same team.

For our presentation on Wednesday, Divya worked her “positive person” magic and spun a story that made us look good (2 projects instead of 1! Twice the Value!). After we presented our projects to the class, someone actually came to the front of the lecture room afterwards and told me the PolarScape interface “rocked!” That felt good. Then someone came up to me after class and said the interface was really awesome. That felt good too. I relayed the good news to my team and we all laughed and shook hands. I guess we weren’t doomed after all.

You can download PolarScape under the Apache 2.0 license

The program is super hackish, but it has a neat story and might be useful to others.