Saturday, September 1, 2012

Rediscovering Vrouw Maria

Summary

Even though I run my own freelancing company, I don't object to the old-fashioned salary work either. I've been working at the New Media department of my old school Aalto University, at a semi-regular basis. The jobs there use Unity as well. This one, in particular, a project between the Aalto University and the Finnish Maritime Museum. It's a digital museum installation through which the visitors can explore the wreck of a ship that sank in the Gulf of Finland in the 18th century. The installation is viewed in stereoscopic 3D and controlled with Kinect.

The Team (Medialab side)

  • Lily Diaz-Kommonen acted as project lead and designer.
  • Markku Reunanen participated in project management and design.
  • Ferhat Sen acted as designer and contributed with his expertise with the Kinect controller.
  • Mikko Hovi built the ship models.
  • Svetlana Maras created our immersive soundscape.
  • Heikki Sillanpää provided us with GUI design and related graphics.
  • Minna Piirainen participated in design and user testing.
  • I brought everything together in Unity, participated in design (especially the gesture interaction part), and implemented the gesture interaction.
  • Lots of other people from both Medialab and the Maritime Museum participated as well. I've only mentioned the most central Medialab people to keep the list short.

Technologies I Used

  • Unity 3D engine and development tool
  • C# for scripting
  • Kinect with OpenNI and NITE libraries

Working with Kinect

The most interesting part of this project was, without question, the creation of the gesture based interface with Kinect. It was surprisingly straightforward to get data out of Kinect and use it (when it wasn't outright malfunctioning for some obscure reason) but designing the right gestures was a challenge. Not only did we need a system that allowed the user to do all the necessary things, such as swimming in all three dimensions, but it also had to be natural and obvious for people who had never used Kinect - or even a computer - before.

The first control metaphor we tried was swimming. I came up with the (very common, I heard) idea that moving your hands far away from your body would push you to the appropriate direction in the water, while moving them close to your body would not. It worked quite well when I used it or when my teammates used it after a bit of an explanation but when we tried it with test users... it went less well.

There are, we discovered, a lot of different ways to swim. Some people would do something like we intended while some would just kind of flap their arms with very small motions. Some would scoop the water like a dog, and those are only the variations I remember right now. And this was with new media students who had instructions on how to use the system. So after discovering that you can't assume people to share your specific idea of an abstract concept such as swimming, especially when brought into a not so natural context, we decided to do something simpler.

We went through a few variations but eventually ended up with a pretty plain pointing metaphor. Pointing forward moves you forward, pointing to the sides turns you, and pointing up and down moves you up and down respectively. The trickiest part was moving down, as resting your hand by your side is also a very natural gesture for just stopping and resting for a while. We ended up requiring that you point forward and down to go downward.

Time will tell if gesture based interaction and interactive exploration are the thing of the future for museum installations or just a curiosity and failed experiment. Whichever it is, however, it has taught me a lot about usability.

By the way, I mentioned below that there are few moments as nice as getting your game to run smoothly on a mobile device. Well, one of them is the moment when your installation is first made available to the public - and they love it.

No comments:

Post a Comment