Talking iOS Apps

Summary

The most long-term client relationship I've had was with Yann le Coroller from Tayasui. When the number of titles is concerned, most of my published work also comes from this collaboration. The apps in question aren't exactly games but more like toys. Each app has a character (usually a cute one), that responds to touching in various ways, and most importantly, repeats back whatever the user says. Each also has some kind of extra functionality.

List of Titles

• The Mouth
• Talking Carla (small contributions)
• Talking Carl iPad
• Pat the Talking Cat
• Talking Air Penguin
• Mr Poo

Technologies Used

• Unity Pro, iOS addon
• iPhone, iPad
• prime31 plugins: inApp Purchases, Social Networking, Game Center
• 2D Toolkit plugin (GUI)
• Megafiers plugin
• Playmaker FMS plugin (finite state machines)
• Unity Asset Server

iTunes Connect

I've already discussed the quirks of mobile devices in an earlier entry, but this was the first time I got familiar with Apple's publishing and sandbox testing system: iTunes Connect.

Strangely enough, publishing an app turned out to be fairly streamlined, as long as you make sure your application information and info.plist contents are correct. However, they seem to have gone the extra mile to make testing network-based features, such as inApp Purchases, as awkward as possible. I'm still not sure why I needed to create a fake app, publish it and reject it just to test whether you can indeed go through the puchasing sequence in your app.

Playmaker FSM

We used many plugins to speed things up, and one of them was Playmaker FSM, that allows building finite state machines. Basically, it lets you create program logic by connecting boxes with lines. You then attach scripts to the boxes to determine what the program does while it's in that state.

I found this system kind of interesting. It brings a nice, intuitive high-level structure to the program flow: each FSM is always in one state and one state only. You'd usually stay in one state for multiple frames, as opposed to the usual Unity program flow where the Update method and the like are called every frame for every object and the logic branches there, only to return to the Update method next frame. Also, if you wrote scripts for Playmaker, you'd have to think in terms of activities, not actors. You'd consider how the activity would start, what would be done every frame, and on what conditions would it end and what would be done next. These activities often ended up quite modular and reusable, though not always: sometimes I had to write a script for a simple equation used only once.

I'm not convinced it's going to replace conventional scripting, though: my eventual impression was that it suits situations where there isn't a lot happening simultaneously, and where the sequence of tasks doesn't branch too much. Otherwise the FSMs tended to become bloated. It's also hard to avoid copypasting similar states just because you need them in two different parts of the tree. However, the approach lets non-programmers participate more directly in creating program logic, so it definately has its place in projects like this.

Rediscovering Vrouw Maria

Summary

Even though I run my own freelancing company, I don't object to the old-fashioned salary work either. I've been working at the New Media department of my old school Aalto University, at a semi-regular basis. The jobs there use Unity as well. This one, in particular, a project between the Aalto University and the Finnish Maritime Museum. It's a digital museum installation through which the visitors can explore the wreck of a ship that sank in the Gulf of Finland in the 18th century. The installation is viewed in stereoscopic 3D and controlled with Kinect.

The Team (Medialab side)

• Lily Diaz-Kommonen acted as project lead and designer.
• Markku Reunanen participated in project management and design.
• Ferhat Sen acted as designer and contributed with his expertise with the Kinect controller.
• Mikko Hovi built the ship models.
• Svetlana Maras created our immersive soundscape.
• Heikki Sillanpää provided us with GUI design and related graphics.
• Minna Piirainen participated in design and user testing.
• I brought everything together in Unity, participated in design (especially the gesture interaction part), and implemented the gesture interaction.
• Lots of other people from both Medialab and the Maritime Museum participated as well. I've only mentioned the most central Medialab people to keep the list short.

Technologies I Used

• Unity 3D engine and development tool
• C# for scripting
• Kinect with OpenNI and NITE libraries

Working with Kinect

The most interesting part of this project was, without question, the creation of the gesture based interface with Kinect. It was surprisingly straightforward to get data out of Kinect and use it (when it wasn't outright malfunctioning for some obscure reason) but designing the right gestures was a challenge. Not only did we need a system that allowed the user to do all the necessary things, such as swimming in all three dimensions, but it also had to be natural and obvious for people who had never used Kinect - or even a computer - before.

The first control metaphor we tried was swimming. I came up with the (very common, I heard) idea that moving your hands far away from your body would push you to the appropriate direction in the water, while moving them close to your body would not. It worked quite well when I used it or when my teammates used it after a bit of an explanation but when we tried it with test users... it went less well.

There are, we discovered, a lot of different ways to swim. Some people would do something like we intended while some would just kind of flap their arms with very small motions. Some would scoop the water like a dog, and those are only the variations I remember right now. And this was with new media students who had instructions on how to use the system. So after discovering that you can't assume people to share your specific idea of an abstract concept such as swimming, especially when brought into a not so natural context, we decided to do something simpler.

We went through a few variations but eventually ended up with a pretty plain pointing metaphor. Pointing forward moves you forward, pointing to the sides turns you, and pointing up and down moves you up and down respectively. The trickiest part was moving down, as resting your hand by your side is also a very natural gesture for just stopping and resting for a while. We ended up requiring that you point forward and down to go downward.

Time will tell if gesture based interaction and interactive exploration are the thing of the future for museum installations or just a curiosity and failed experiment. Whichever it is, however, it has taught me a lot about usability.

By the way, I mentioned below that there are few moments as nice as getting your game to run smoothly on a mobile device. Well, one of them is the moment when your installation is first made available to the public - and they love it.

Kasvi the Game

Summary

One day the Aalto University branch of the Finnish Green Party went to visit Jyrki Kasvi, one of our parlamentarists (at the time) and the most vocal proponent of information technology in the parliament. I had no idea he even knew I was in the game business so it was kind of a shock when he asked, in a sort of an embarrassed way, if I could make a game to promote him in the 2011 election.

It didn't take much to convince me so we went on and made it with a group of people we recruited at the university. We made it in time (barely) and published it, but for some reason Kasvi still dropped out of parliament, a mystery that remains unsolved...

At the time of this posting, the game can still be played here.

The Team

• Jyrki Kasvi acted as the main character of the game.
• Patrick Jensen created most of the graphics.
• Joona Karjalainen composed the music and made the sound effects.
• Olli-Pekka and Tomi-Mikael Kahilakoski tested the game and programmed some features.
• Timo Tuomaala and Antti Mustakangas helped with testing and marketing.
• Maija Koivula helped a bit with concept design.
• I programmed the game and created most of the menu graphics. I was also project manager and coordinated the design process.
• We all participated in designing the game.

Technologies I Used

• Java, Swing

Designing a Political Game

The game wasn't very difficult to code (though optimizing Swing for various platforms turned out to be surprisingly problematic). The inspiring challenge was the design. It had to be fun, easily accessible, and - about politics.

A parody was the obvious choice and we took it. In the game, Kasvi stands on the steps of the parliament building, using a megaphone to shoot down bad laws that fall from the sky, while letting the good ones fall through. The good ones are, of course, the ones supported by Kasvi and the bad ones the ones he opposes.

I believe that especially in a project like this, where you don't have a horde of earlier similar games to base your design on, it's important to do the design as a team. For one, you have to come up with a lot of novel ideas. ("What elements of politics are funny? What should we include?" "I know, how about the 'group whipper', the one who looks after party discipline? She could hit Kasvi with a whip!") On the other hand, you have to weed the bad ones out. ("What do you mean a 'Black Hole of Ignorance' is too obscure?" - I'm glad that early favourite of mine didn't make it into the game.) I think we did a great job and ended up with a good set of game elements that both enhance the basic gameplay and are thematically appropriate.

Even though it didn't save our country from the disaster that was the 2011 election, this was one of the most fun game projects I've been involved in.

Commission Work

Summary

Apart from Unfinished Business, which I still occasionally work on, my first years as freelancer consisted of smaller scale jobs, mainly for the iOS platform. I would get my jobs from the Unity forums, do the job, and get paid either hourly or on a commission basis. Most of these jobs were prototypes and/or are still unfinished due to various reasons, hence the limited number of samples.

Technologies I Used

• Unity Pro, iOS addon
• iPhone, iPad
• various plugins

Mobile Platforms

When I began freelancing, I had never worked on any mobile platform. I didn't even own a device that could run anything more complex than Snake. So I was pretty unprepared for working in such a limited environment. Suddenly, you couldn't just create and trash new objects all the time because that eats up CPU. You had to carefully consider the necessity of every little piece of graphics because of memory limitations. You jump through a number of hoops just to get your draw call count as low as humanly possible. Add to that Apple's convoluted provisioning system, Xcode's mystery errors and seemingly random crashes and performance fluctuations...

So it's a pain, but then again, there are few moments as nice as finally playing your game on your device, and it's running absolutely smoothly. And strangely enough, that usually does happen eventually. And even the optimization itself can be rewarding, as I've always enjoyed a good puzzle - the ruling out of each potential cause until only the answer remains.

It does have its plus sides too. I find the touching / accelerometer interface interesting and inspiring. I'd like to see more games that take full advantage of the accelerometer, where things always fall according to real gravity, for instance. A Lemmings style game where the puzzles are based on turning the device as much as assigning skills. Maybe one day I'll do something like that myself.

Example Games

• Apple iBasket (first version)

Unfinished Business

Summary

On August 2009, I took my first steps towards being a freelance game developer. On the Unity forums, I responded to a request to create a script for a puzzle where the player needs to press a bunch of buttons in the correct sequence. That was only the first of many scripts for Don Gray, and the start of my freelancing career.

Even after working on and off for almost three years, Don has remained secretive as to the exact nature of his game. It's definately an adventure game, played in 3D, and the screenshots suggest a somewhat surreal style. Judged by the puzzles he has ordered, the puzzles rely heavily on pushing buttons, rotating things in correct order, etc, although traditional item based puzzles are also present.

There's a video with Don describing his work with the scripts here.

Technologies I Used

• Unity Pro (PC/Mac)

Freelancing Challenges

I had of course used Unity for a couple of years at that point due to Lies and Seductions. However, there were still many things left to learn about Unity and working on an actual professional basis. Special challenges were caused by the fact that I was providing the scripts but it was Don who applied them. Firstly, I had to make the scripts easy to use and the inputs understandable, but I had already grown decent at clarity. More importantly, even though we often discussed the requests at length, I could never know (and I still don't) how exactly Don was going to use the script. So when testing my script, I couldn't just test them in the context it was used, but also try to think of all the possible contexts in the first place. Plus, Don always seemed to have new ideas on how to use the scripts, different puzzles that needed just one extra feature...

In Don's case, I was lucky to get an agreement of hourly payment, because as I discovered in other projects, my ability to estimate the time required for a task was as lacking as I imagine it's for most programmers. Sometimes I could implement several features in an hour, while at other times I might spend days on a particularly tricky problem. This is amplified by the fact that the clients don't always quite know themselves what they need.