User Involvement, Social Media, and Service Evolution: The Case of Habbo

Page created by Carolyn Reyes
 
CONTINUE READING
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

                 User Involvement, Social Media, and Service Evolution:
                                  The Case of Habbo

                                                   Mikael Johnson
                                         National Consumer Research Centre
                                               mikael.johnson@ncrc.fi

                      Abstract                               the service, and after the research project the author
     Understanding user needs and involving users are        has informally followed the case.
key success factors for service development. However,            In this article, the focus is not on the minute details
it is unclear how to apply old wisdoms from traditional      of a particular user involvement method, but more on a
product development settings in a social media context.      strategic level. During ten years, Sulake developers
This article analyses user involvement practices in a        have explored various user involvement methods
longitudinal case study of the virtual world and social      ranging from informal observation, focus groups,
networking service called Habbo. The focus is on how         market surveys, and new social media possibilities.
particular social media aspects, such as persistent          The aim of the case study was to answer the question
user-created content and developers’ easy access to          of what practices social media developers have
online user action, shaped user involvement and how it       regarding user involvement. In this article, we focus on
evolved over time. In line with previous research, the       two aspects in more detail. First, how the social media
findings indicate that user involvement becomes more         context influences the possibilities for user involve-
formal as the user community grows and the service-          ment, considering especially persistent content created
market combination matures. In addition, user involve-       by users and developers’ easy access to online user
ment regarding social media appears to be an issue of
                                                             action. Second, how user involvement practices change
timing, trans-nationality, and governance. The results
                                                             as the service evolves, with particular interest in the
suggest that the usability practitioner’s ‘toolbox’ needs
tuning regarding social media.                               increased complexity, signified by the increasing
                                                             numbers of both users and developers.

1. Introduction                                              2. User Involvement Approaches

    Involving users in service development is a key               There are many arguments for user involvement:
strategy to build better services and democratize the        reducing risks of failure in technological innovations
innovation process [1-3]. While there are mixed results      [1], democratizing the innovation process [2], building
on the cost-effectiveness of user involvement [4], there     better products, etc. Despite a consensus that under-
is an abundance of methods available to bridge the gap       standing user needs is a key factor in commercial
between developers and users [5-9]. Many user                success [3], this is not easily translated into action in a
involvement methods have been created and applied in         particular innovation context. There is an abundance of
contexts like commercial development for offices,            methods to solve the ‘sticky information’ problem:
government health care, but few consider the emerging        information about users’ needs and developers’ capabi-
social media. Is the standard textbook advice ‘study         lities is highly contextual, tacit and difficult to transfer
first, design second’ applicable in social media             [2]. However, which user involvement approach is
development, or is it the other way around [10]?             applicable in which particular design situation is still
    This article gives an overview of user involvement       an open question.
practices from a longitudinal case study: the virtual             In the same way as using a particular research
world and social networking service called Habbo that        method implies a particular role for a researcher, any
was launched in 2000. The Habbo producer, Sulake             user involvement approach imply a particular user-
Corporation, was involved in a research project in           developer relationship. A researcher can be anything
2003-2006, during which the author followed the user         between a detached and uninvolved observer to a fully
involvement approaches used. In retrospective inter-         engaged participant in a group or organisation under
views developers shed light on the first few years of        investigation. Developers’ information about users

                                       978-0-7695-3869-3/10 $26.00 © 2010 IEEE                                              1
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

                          Table 1: Pros and cons of user involvement methods [12].
 Approach to              Pros                                                Cons
 learning about
 end-users
 Surveys and              Systematic approach to data collection              May not always feed into programme design
 interviews               Surveys provide the possibility to poll             Surveys may be designed to confirm existing
                          representative samples                              preconceptions, may fail to bring up new insights
                                                                              Conducting good research may be expensive and
                                                                              require specialized skills
 Prior research,          Sound theoretical base can guide observations       Strong commitment to prior findings or theories may
 particular theoretical   and help to make sense of energy-related            lead to overlooking contextual particularities
 perspectives             behaviour                                           Overly theoretical background can lead to complex
                                                                              and confusing designs
 Experience from prior    Sound experience creates confidence and             ‘Competence trap’: overconfidence and failure to
 projects and similar     practical skills/solutions that are difficult to    learn new skills in new contexts
 examples                 codify
 User-driven project      End-users know about their needs and                End-users may not be fully aware of their behaviour
 (or pilot project)       circumstances and can contribute to context-        and all the factors underlying it
                          tailored and user-friendly designs                  ‘Upscaling’ from small user-driven pilots to broader
                          End-users are motivated and engaged from the        group of end-users can be difficult
                          start, thus ‘less work’ is left for the programme
                          manager.
 Familiarity and          Informal interactions allow for a rich exchange     It can take a lot of time and commitment to build up
 informal interaction     of information (including non-verbal                the level of familiarity needed to execute a
 with the target group    communication)                                      successful programme
                          Familiarity creates trust and mutual confidence     Contacts may be biased: some end-users are more
                                                                              familiar than others.

originates in similar relations: statistics of user                   standing of the culture and practices of users [12-14].
practices can easily be detached from a particular                    However, while the initiatives recognise the complex-
situation, aggregated and analysed from a distance,                   ity and diversity of user settings, they seem to have had
where as participant observation of situated use implies              modest influence over system design overall [15].
that actual developers and users meet. In the tradition               Stewart & Williams [16] warn against a pitfall they call
of designing for usability, developers refer to these                 the design fallacy: the assumption that from the diver-
situations as usability evaluation with or without users,             sity of user needs follows that extensive knowledge
and methods have emerged that are tuned to diverse                    about user contexts must be built in the product. In
design situations [11,5,6]. Table 1 lists pros and cons               contrast, they report that generic products that can be
of particular approaches to user involvement, taken                   customised and configured seem to have had more
from experiences in energy demand-side programmes.                    impact on system design than user-centred design [15].
     User involvement methods for established markets                     There is no consensus on why potential initiatives
differ from the methods that are developed for new                    such as user-centred design have had little impact.
markets. Leonard-Barton [3] found that traditional                    Recent debates suggest that standard methods such as
market research methods (latent needs analysis, lead                  ethnographic field studies or usability evaluations can
users, surveys, focus groups, mall studies) fit well with             do more harm than good if not applied carefully
an established market and a high maturity of                          [17,18]. This reflective debate can be interpreted as a
technology design. For new markets, more used were                    sign of maturity in the human-computer interaction
methods like market experimentation, future scenarios,                field as it highlights specific problems that must be
and trends exploration. In the grey area between these                addressed when mixing particular research disciplines
typical situations, Leonard-Burton argued for less                    with product development. Another suggestion is that
traditional market research methods under the rubric of               the boundary conditions of product development
empathic design: studying actual observed customer                    projects have not been addressed thoroughly. Svanæs
behaviour, direct interaction between developers and                  and Gulliksen [19] suggest that the design context
users, and considering existing technological capabi-                 needs greater attention, that is, the internal structure of
lities.                                                               the developer and the client organizations, contractual
     Empathic, participatory, and other user-centred                  and tender issues, software engineering tools, and
design approaches emerged because user requirements                   stakeholder agendas and relations.
engineering techniques at the time lacked an under-

                                                                                                                                     2
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

2.1. Social Media                                            the time of writing, there are Habbo hotels in more
                                                             than thirty countries, and 13 million players visit
    New online web services often referred to as digital     Habbo each month [23]. Instead of an entrance or a
media, web 2.0, and social media have changed the            monthly fee, the profit model is based on micro-
possibilities for user involvement. On the one hand,         payments in the hotel. Virtual furniture, mini-games,
people discuss particular products and brands in online      and membership in the Habbo club are bought with
discussion forums and communities. Product deve-             Habbo credits. These credits can be purchased
lopers can more or less openly visit these forums to get     (depending on the country) with pre-paid cards, bank
customer feedback. On the other hand, when people            transactions, credit cards, or special text messages that
use web 2.0 services, they leave an electronic trail.        add a specified amount of money to the customer’s
What people click on, what they search for, what they        mobile phone bill.
select, and how they navigate on a web site can all be           The social interaction in Habbo is truly diverse. In
collected and aggregated. Service developers can use         the design of Habbo, clear winning conditions and
this log statistics to fine-tune the service. Furthermore,   gameplay rules have been avoided, and instead, players
in some services it is customary for users to openly         are encouraged to create their own objectives beyond
share their preferences, bookmarks, opinions, and other      chatting, room decoration, and meeting friends. The
content with other users. This participative web and         provided environment for these activities is a hotel
user-created content [20] sometimes plays a major role       consisting of public and private rooms, where the
in the service and can also be used by service               virtual hotel visitors, called Habbos, chat, buy virtual
developers as inspiration for new features.                  furniture, decorate rooms, and arrange social and game
    Social media is clearly not only win-win situations,     events. Most of the teenage players log on after school,
since many discussions on privacy, advertising,              and according to Sulake, the developer company, on
intellectual property rights, and other conflicts between    average they spend around forty-five minutes per day
commercial actors and consumers have emerged.                in the hotel or on its related discussion forums.
Lehmuskallio et al. [21] observed that these discus-
sions relate to communicational changes in access to,        3.1. Data and Method
scale, distribution, and persistence of computer-
mediated communication. For instance, web 2.0                    The Habbo producers, Sulake, was established in
business models and privacy are often difficult to           2000 and focuses on virtual worlds and social network-
combine, as private information can be used to target        ing. Sulake’s strategy is to be a leader in community
ads, be aggregated and sold to third parties, as well as     based entertainment with a portfolio of properties
get into unexpected hands when big commercial                addressing a wide range of target audiences. During
players buy web 2.0 start-up companies.                      2003-2006, Sulake, together with ten other commercial
    As social media change user-producer relation-           actors of the Finnish gaming industry, participated in a
ships, new research questions related to user involve-       research project coordinated by the Helsinki Institute
ment emerge. Do the short release cycles frequently          of Information Technology [24]. This research context
associated with social media – the infamous ’forever         made different design research studies and collabo-
beta’ approach – give more or less room for user             ration with Sulake’s product development possible.
feedback? How do social media developers apply user              The focus of the Habbo study was user involvement
involvement methods as the service evolves over time?        strategies in the design and management of social
In a discussion of community building, Kim [22]              media. Since its launch, the user communities have
suggests that many online communities start with the         participated actively in the shaping of Habbo [25,26].
developers as central figures in the community, but as       Some as active playmakers [27] and others as equally
the community grows, the developers’ roles and               important participants and audience.
influence decrease. Is such a change in the developer-           The author had the opportunity to conduct both
user relationship reflected in the use of user               quantitative and qualitative research to understand the
involvement methods?                                         Finnish Habbo communities. The project started with
                                                             participant observation in Habbo, pilot interviews, a
3. Case Habbo                                                community manager survey (N=4), as well as an
                                                             explorative survey (June 2004) on the visitor profiles
    Habbo is a virtual environment where children and        (N=10000) as Sulake’s first global youth survey was
teenagers meet, socialize, and play many types of            published two years later. The author participated in
games. It was first launched in August 2000 in Finland       Habbo, explored the features and their affordances,
as Hotelli Kultakala (‘Hotel Goldfish’) and it was           analysed default values and users’ degrees of freedom
based on the developers’ two earlier online services. At     as a routine when new features were launched. The

                                                                                                                         3
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

visits to Habbo occurred on the average about twice                  The data analysis has proceeded in multiple waves
per month throughout the first three years of research,              over the years, including quantitative and qualitative
and a visit every second month from 2007 onwards.                    clustering of Habbo user profiles [29], coding and
    Fansites became an important source of knowledge                 examining interviews with regard to participation and
about the user communities. In 2004, Finnish Habbo                   development histories in Habbo, analysis of fansite
fansites were analyzed in detail [28]. Whereas the                   contents and developer views to them [25,28], and
survey findings provided background statistics, the                  here, constructing the changes in Habbo user-developer
fansites and forum discussions allowed an insight into               relations over time.
active user groups, popular activities, and hotel history.
Since its very beginning Habbo fans have kept several                3.2. Habbo Service Evolution
sites devoted to documenting changes in the hotel, for
instance in kinds and appearance of furniture, charac-                   Like many other digital startups, Habbo is a
ters, mini-games, and bugs.                                          moving target for many reasons. In 2003 there were
    In 2005, the author conducted ten thematic 2-3 hour              four hotels and 1 million users, but at the time of
interviews with ten Habbo developers, or about two                   writing, these numbers are more than tenfold. The
thirds of the game development organization at the                   producer company has grown from a small group to a
time. Six of the interviewed developers (graphical                   300 people corporation, with local offices in many
designers, and client and server developers) had been                hotel countries. To aid the analysis process, a division
in the organization since the beginnings, five years                 of Habbo service evolution into four stages will be
earlier, while four developers had about one year of                 outlined below.
Habbo experience. This was followed by 2-3 hour                          Habbo was based on two previous launches,
individual, pair, and group interviews with 11-16 years              Mobiles Disco (Oct 1999) and a snowball game called
old users (N=6), and 30+ users (N=6) to focus on their               Lumisota (Feb 2000). Mobiles Disco provided the
participation histories, motivations, meanings they give             basic hotel infrastructure in the pixelated retrostyle,
to Habbo, and groups they participate in.                            whereas Lumisota featured payments through mobile
    Habbo has also been a topic of collaboration                     phone messages, which at the time was a practical
between the author and other Habbo researchers. A                    solution for the Finnish market. Starting with the first
Sulake employee’s master’s thesis on communication                   international step to UK, other payment systems were
and action in Habbo provided important secondary                     implemented to suit markets with less wide use of
sources to Habbo data, as did several youth work                     mobile phones among teens. Phone call systems, youth
studies1. Usability research students at Helsinki                    cash cards made of paper, check systems came first,
University of Technology also provided secondary                     but soon followed payment by credit cards, Paypal, and
data, as part of to their efforts on both a usability test           bank transfers.
on new 10-12 year old Habbo users (N=8) in 2004, and                     With the UK hotel also followed the teen invasion
a software engineering project in 2004-2005 to develop               of Habbo [30]. Habbo was not designed for teens at
a fansite starter kit for active Habbo users.                        first and the large number of teen visitors also meant a
    After these activities, the author had the                       large number of concerned parents. To keep all this
opportunity to take part in an intervention study with               together, the community management received a lot of
Sulake. For release 9 of Habbo in 2006, a set of user                attention. A set of guidelines for good behaviour was
feedback methods were explored with different                        established, called Habbo Way, and tools for governing
stakeholders inside Sulake. This included confidential               it were developed.
data sets: database statistics and surveys from two                      During 2002-2003 Sulake developed its business,
countries on the use of a new feature. In addition to                the hotel and the community simultaneously. Strategic
these research activities, Sulake representants                      partnerships were made and brands like Mountain Dew
participated in project partner seminars arranged every              and Britney Spears entered Habbo. Later these brands
6 months, two workshops on virtual economy, and                      have disappeared as Sulake tried another approach with
many project meetings arranged by the research                       Coca-Cola and Warner Cinema. The brands got
project. These meetings made informal discussions and                customised versions of the technical game engine
a continuous dialogue with Sulake possible.                          behind Habbo, which helped fund the development of
    Taken together these bodies of data provide an                   the hotel. The hotel’s technical architecture was
excellent view to the varying forms of dialogue                      stabilized and security improvements were made to
between the users and developers of this virtual world.              package the hotel into a product that could be more
                                                                     easily rolled out in new countries.
1
  These are only available in Finnish, but noted here for further        During 2004-2005 around ten new hotels and local
inquiries: Pietiläinen 2004, Sihvola 2005, Koskinen 2006, Merikivi
2007.
                                                                     offices were established in different parts of the world.

                                                                                                                                 4
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

Habbo actively wanted to become the largest teen           avatars and their descriptions reveal what kind of
brand in the world. The English-speaking UK hotel          Habbo crowd is present. By studying the arrangement
was split into separate hotels for the US, Canada, and     of Habbo furniture and guidelines written on virtual
Australia. Also game development recruited more            post-it notes, the developer learns about how particular
developers in 2004, and new features were packaged         pieces of furniture are used in specific user activities.
into simultaneous releases in all hotel countries.             Developers as users. In the early stages of concept
    Sometimes during 2006 and 2007, the strategies         development and beta testing, the developers were
changed again as Sulake entered the social networking      developing the service for themselves, their friends and
market. Not only did Habbo get new social networking       their new media colleagues. The first prototype was a
features, but Sulake acquired IRC-Galleria, the most       hobby project for a friend’s band. The original group
popular social networking service in Finland. As IRC-      of developers started out as insiders in the user
Galleria users were a little bit older, average age        community, but as the service become more popular,
around 20, the strategy to concentrate on teens was        other insider groups started to form. After a while, the
opened up to a variety of age groups.                      developers became more distanced to the user
    Based on this information, the author has grouped      community as it became more difficult for them to
the service evolution into four stages.                    spend time in Habbo. Each time a Habbo developer’s
                                                           avatar was online, it did not take long before Habbo
   Table 2. Habbo Service Evolution Stages                 newcomers gathered around and wanted to become
                                                           Habbo friends, or even asked for freebies. Habbo
 Concept       Beta         Expansion     Complexity       developers soon invented their own tactics to partici-
 Development   Testing                    Management       pate in cognito: they created new avatars, whose real
 1999-2000     2001-2003    2004-2005     2006-
                                                           identity they only told trusted friends.
                                                               Informal evaluations. Various informal evaluation
    Concept Design refers to the first prototypes in       practices guided the design early on. In interviews, the
1999 and 2000: Mobiles Disco, Lumisota, and Hotelli
                                                           developers talked about slogans such as ”easy access,
Kultakala, while Beta Testing refers to the time period
                                                           easy play” and ”where else”. A developer described the
between 2001-2003, when much of the basic                  latter one as ”We won’t make obvious choices, but
functionality was completed. Expansion refers to 2004-     rather something personal that gives the Habbo world
2005 when the product was packaged so that it made a       something odd and own, which creates an own
roll out possible in more than 10 new countries during
                                                           persona.” (Interview 19 Apr 2005). The “easy access,
one year. Complexity Management refers to 2006 and
                                                           easy play” slogan was not only a slogan, but among the
onwards when the product was extended to a social
                                                           developers it also had a visual form called the Habbo
networking service.                                        Ladder that showed the first steps a new user takes.
                                                               The Habbo Ladder was made to contrast Habbo
4. Habbo User Involvement Methods                          with the comparably higher threshold to start playing
                                                           massively multiplayer online games at the time. The
    The user involvement methods reported here             idea was that during the first few minutes that anyone
attempts to be an overview of the methods used by          is willing to spend on a new web service, one should
Sulake from 1999 to 2009. The criteria for what counts     be able to login and create an avatar easily, learn the
as a method is a more or less organized practice where     basic navigation, and have a chat with someone else.
developers learn about the users, which then shapes the    Each step on the ladder makes more reasons to return
service. The user involvement methods are organized        to the service, as a developer explained:
according to the service evolution stages.                     ”Especially if one gets the first friend on the list of
                                                           friends, then that it is a reason to return, that you have
4.1. Concept Development                                   really got to know someone with whom you might
                                                           have had an interesting discussion or of whom an
    Avatar activities. Avatar activities refer to what     interesting image has been conveyed. Then further on,
users do online in the service. Developers have easy       when you have your own room, well that is of course a
access to users in the sense that the developers can log   real investment, even the notion that you have
on to Habbo and check what is going on. In the same        something own going on there, then that is already a
way as users find out what happens where in the hotel,     good reason to return, especially if you have decorated
developers can also use the list of most popular rooms     the room, really purchased something.” (Interview 19
or tags to find the trends. In the room, a developer can   Apr 2005).
choose to participate in the ongoing activity and take         E-mail feedback. During the first year or so,
note of which users are present. The names of the user     developers received a lot of e-mail feedback by users,

                                                                                                                         5
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

who suggested changes and new features. This e-mail         homepages, and some created totally Habbo-themed
feedback became a handy storage for design inspiration      websites. Being a Habbo journalist was one way of
for the developers, who used to return to it later and      becoming famous, besides participating in various and
browse for good ideas. As the user community grew in        creating own online Habbo activities. Some fansites
size, this direct e-mail feedback became impractical as     featured discussion forums, which also were an
the number of e-mails became too large. Users not only      important source for developer inspiration [25,28].
sent feature suggestions per e-mail, but all kinds of           Official Web Fanzine. For two years, Sulake
support requests, which then as the organisation grew,      published their own official Hotel Magazine in Finland
became the responsibility of a small but growing            (www.kultakalankuvalehti.com), but due to limited
number of volunteer community moderators and                resources they closed it down. It started out with
customer service officers.                                  writings from Habbo developers, and interviews with
    Volunteers. Already before Habbo, in Mobiles            Habbo users, but soon enough some Habbo users
Disco, there was a need to keep the shared meeting          featured as guest authors. In 2004 Sulake launched
places in order, in case a user did not follow the shared   local competitions in each country to give a few
conventions of appropriate online interaction — the         elected large fansites the status of being official Habbo
netiquette. Trusted friends of the developers or other      fansites. While these fansites complemented Sulake’s
insider users received special powers, for instance to      community management, they also helped Sulake in
remove (kick) a disturbing user avatar from a room,         regulating sites that were out to scam the users.
and the responsibility to keep the online place neat.           Summer Meetings. With the responsibility for
These volunteers followed along from the first proto-       teenagers online, Sulake had to develop a policy that
types to Hotelli Kultakala and Habbo. Later it became       discouraged sharing of contact information online and
possible to apply to become a volunteer, so called          meetings in person. To support this policy, Sulake
Hobba. However handy early on, and applicable in            arranged summer meetings where users could meet
many hotel countries, this arrangement involved more        each other in person, as part of an organised event. In
than 1500 volunteers during the expansion phase in          the first meetings in Helsinki active users and
2005. Sulake changed the volunteering practice in           volunteers got to meet the community manager and a
2005, as the special powers were given to paid              few developers. Later on these meetings have been
moderators only, and the notion of volunteer                arranged by active users, without much involvement of
moderators transformed into volunteer guides, who           developers.
could guide online newcomers.                                   Sales Statistics. In contrast to other Habbo stake-
                                                            holders, developers have an additional perspective to
4.2. Beta Testing                                           the online activities in Habbo. They have a back-end
                                                            service that keeps track of what pieces of furniture
    Volunteer forum. To share experiences and mode-         have been sold and how many in which hotel. This
ration policies, the volunteers created an online forum     means that developers can compare Habbo features on
for themselves. Along with the internationalisation and     the basis of their economical behaviour, not only based
more organised volunteer management, Sulake started         on functional or aesthetic properties. Even though the
hosting a local volunteer forum per hotel country. The      use of Habbo is anonymous, the registration process
volunteers soon got an important role as mediators of       asks users to fill in age and gender, which can then be
user opinions: the developers knew that as the              connected to sales statistics on an aggregated level.
volunteers spent the most time in the hotel, they were          Customer service. The need for dedicated custo-
always the first to know about the current user             mer service emerged as the user communities grew and
concerns, wishes and emergent activities.                   the developers could not find time to handle all support
    Weekly newsletter and polls. In an attempt to           requests. Each hotel country got their own country
organise user feedback and create a rhythm of new           office, where a local manager managed the volunteer
content, the hotel community management started             moderators and customer service officers sorted out
creating weekly polls and newsletters. The newsletter       user requests.
contained information about new features and events
organised by the hotel manager (a Sulake employee).         4.3. Expansion
With the newsletter was usually a gallup poll or a
competition that the users could participate in. These         Market survey. With the international expansion
weekly polls were important to gather immediate user        emerged a need to know whether the user communities
feedback regarding new features.                            were similar or different in different hotel countries.
    Fansites. Right from the start, Habbo users started     An outsourced market survey in 2004 generated
writing about and debating Habbo on their own               customer segments and their regional distribution. The

                                                                                                                        6
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

named customer segments and their demographics              pilot country acted as beta-testers and could comment
were important in the communication about Habbo.            on the features in the release before they were
Even though demographics seemed less important than         finalized. The process of selecting the pilot country has
user practices from a game development perspective,         changed several times. In 2005, countries with a
customer demographics played an important role in           relatively small number of users acted as pilot
business negotiations, regarding advertisement for          countries. However, translation and language become
instance.                                                   an issue, which is why piloting returned to Finland
    Focus groups. To prepare for the rapid inter-           sometimes around 2007. However, at the time of
national expansion that happened during 2004-2005,          writing, UK is the pilot country.
focus groups were conducted. The applicability of
Habbo pixel style graphics and use of colours was eva-      4.4. Complexity Management
luated for the Asian market. In addition to traditional
market research focus groups, two variants of the focus         Online user panel. In an effort to gather systematic
group method from different research traditions were        feedback before the implementation of new features,
used, usability evaluation and playability testing.         Sulake recruited 200 volunteers in one country to form
Usability evaluation with users is more like a              an online panel. The online panel was given a weekly
controlled lab experiment in psychology, where users        task consisting of a set of questions regarding design
try out the software with the help of scenarios and         sketches and an opportunity to share opinions
evaluators measure issues like usefulness, ease-of-use,     regarding the sketches in a forum. The online panel
learnability, and memorability [31].                        was popular among developers, who queued to be the
    Usability Evaluation. The first formal usability        one who got to get design-time feedback from the
evaluation in the fall of 2004 was targeted at checking     users.
the usability of service registration and those services        Global youth survey. As both the users and
in Habbo that were subject to a fee, from the viewpoint     competitors change, in 2006 it was again time for a
of 10-14 year olds. The evaluation was outsourced and       new market survey (GHYS’06). This time the
conducted by usability students at the local university     customer segmentation resulted in certain Habbo
of technology (TKK). After the first usability evalua-      lifestyles: achievers, creatives, loners, rebels, and
tion, Sulake established an in-house usability process      traditionals. The report featured extensive country-
through a pilot project. The pilot project was related to   specific information on teenagers’ favourite brands and
a product extension for the mobile phone market, and        media usage patterns.
was influenced by usability standards in the field, such        User, group homepages, tags. In early 2007,
as the J2ME guidelines made by Idean Research [32].         Sulake launched social networking features in Habbo
    Playability testing. While the usability evaluation     as every Habbo avatar got an automatic and
method was appropriate for features such as login, cha-     customisable Habbo homepage. It was also possible to
racter creation, furniture purchases, usability methods     form Habbo groups, which meant a logo, a group
in general are not tuned to assess playability. For these   homepage and a discussion forum. In addition, users
reasons, Sulake’s R&D lab used playability testing          could ”tag” their avatars, which meant that users could
methods developed at Tampere University’s Hyper-            attach a set of clickable one-word descriptors to their
media Lab [33] to assess various playability aspects,       avatar. When a user clicks on a tag, the service
such as gameplay, game mechanics, appearance,               generates a dynamic index of all the users and groups
sound, and social playability.                              with that particular tag.
    CRM system. As some hotel communities grew                  User experience testing. Following current
larger, pressure emerged for customer service to            research terminology, in 2008 Sulake talked about user
automate their responses. For instance, in a country        experience testing in addition to usability or playability
with several hundreds of thousands of users, a new          testing. While the usability testing has evolved from a
feature might spawn several thousands of inquiries per      more stand-alone practice, to a tight integration with
day. In 2005, a new customer relationship management        agile software development, Sulake conducts user
system was introduced. It featured a set of standard        experience evaluations with both new and old users,
questions and responses, which reportedly reduced           internally called ”live tests”, in one country for every
inquiries with 90 percent.                                  major release, which makes about once a year
    Release pilots. As the organic beta testing phase       (Personal communication with User and Market
changed into a more controlled release management           Insight Director, 26 Nov 2008).
process, Sulake started piloting the release for one            Personas. As part of further developing user-
month in one hotel country, before diffusing the release    centred design processes, during Spring of 2009,
to other hotel countries. This means that users in the      Sulake applied the Persona method. Six user

                                                                                                                         7
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

                           Table 3. Overview of Habbo user involvement methods.

  Concept Development     Beta Testing                  Expansion                 Complexity Management
  Avatar activities       Volunteer Forum               Market survey             Online user panel
  Developers as users     Weekly newsletter and polls   Focus groups              Global youth survey
  Informal evaluations    Fansites                      Usability evaluation      User and Group homepages, tags
  E-mail feedback         Official Web Fanzine          Playability testing       User Experience Testing
  Volunteers              Summer meetings               CRM system                Personas
                          Sales statistics              Release pilots
                          Customer service

archetypes have been constructed from data to                 knowledge, which was provided by the use of more
represent the users. The idea is that developers have an      formal methods.
updated reference to the goals and needs of Habbo                 So far the results from this case study is much in
users at hand, which can inform design solutions and          line with previous research. However, the user
evaluations. (Personal communication with User and            involvement practices in this case study stands in
Market Insight Specialist, 2 Jun 2009).                       contrast with the strategy of developing a method and
                                                              then using it in the same repeatable way over and over.
5. Discussion                                                 In this case it has been evident that the history of
                                                              method usage shapes the succeeding use of methods.
                                                              For instance, as the service as a whole had been
5.1. User Involvement and Service Evolution
                                                              usability evaluated, the usability specialists at Sulake
                                                              found no sense in repeating the same evaluation again.
    Based on the Habbo case it seems evident that very
                                                              Usability evaluations turned towards smaller details of
few service characteristics relevant to user involvement
                                                              the service after the overall evaluation. The same
methods remain invariant over time. Some charac-
                                                              finding was found in the market survey practices. After
teristics remained stable: the service concept and the
                                                              the first overall customer segmentation, the following
profit model (after small experiments during the first
                                                              surveys could fine-tune specific issues that remained
few years), the technical access to online user
                                                              open in the previous surveys. This practice can be
activities, as well as the service domain and (life-)
                                                              interpreted as a sensible ongoing tailoring of methods
criticality of the interface. However, a lot has changed
                                                              that becomes necessary when dealing with as complex
over ten years: at least the number of users, the number
                                                              and changing phenomena as in this case.
of developers and resources available, the amount of
money involved, competition on the market, the stage
of design, the skills and experience of the developers.       5.2. User Involvement and Social Media
    As the maturity of the market changed, so did the
relationship between developers and users, much like              It seems that the developers have benefited from
in previous research by Leonard-Barton [3] and Kim            particular user involvement opportunities related to
[22]. In the Habbo case, like with most social media          social media. User created fansites have been an
applications, the developers started developing the           important source for developer inspiration, and the
service for themselves and their friends. While starting      online user panel is an interesting way of making
out as insiders in the user community, with commercial        prolonged contact with a particular user group, which
success, the developers inevitably end up as excluded         complemented both the more informal feedback that
from the user community, if for no other reason, just         developers get in the service itself and the more formal
by the sheer number of users and their practices.             surveys. The social networking features – user and
    The first usability study and succeeding studies          group homepages as well as tagging – provided Habbo
marked two changes in user-developer relations. First,        users with new possibilities to express themselves
as the number of users increased, the margins for errors      within the service and find other interesting users and
in the service were decreased, thus more formal               groups. On the other hand, these social networking
methods were valued. Second, developers’ gut feeling,         features made it easier for Sulake to get an overview of
that is, undocumented previous development expe-              group activities in Habbo as well as search and
rience and participation in the user community, carried       aggregate data on users.
less weight, as not everyone in the growing company               Let us return to one question posed in the begin-
could be involved in the user communities. Business           ning. What happens with user feedback in a “forever
and development decisions needed more documented              beta” approach, is there more or less room for it
                                                              compared to traditional product development?

                                                                                                                         8
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

    In the Habbo case, it seems that user feedback can       6. Conclusions
shape the service most when the design space is open.
How open the design space is, depends on what is                 In addition to the themes discussed in research
going on in development. During concept development          literature on user involvement, user involvement
and early beta testing Habbo was fairly open for fea-        regarding social media appears to be an issue of
ture suggestions, but it did not take many months for        timing, trans-nationality, and governance. It is a timing
the developers to get more ideas than they could handle      issue, because user feedback influences most the
for the foreseeable future. Then the question was no         features under development at a particular point in
longer about getting good ideas, but rather about find-      time. It is a question of trans-nationality, because user
ing projects and funding to implement the ideas and          feedback from different countries is influenced by both
stabilize the service. And, until the service was stabi-     language and market size. It is a question of
lized and packaged enough for manageable duplication         governance, because user feedback might also address
in the expansion phase, the design space was fairly          issues that are not considered open for discussion by
closed. From a Finnish user point of view, the next          the developers.
window of opportunity to shape the service was                   Sulake’s user involvement practices reveal a bias in
probably the implementation of the social networking         common classifications of user involvement methods.
features in early 2007, which again changed the rules        Traditional formal methods and usability evaluations
of the game. Then as the new features were tried out,        are only one small part of the big picture regarding
user feedback again helped fine-tune the service.            learning about social media users. The 23 [!] different
    User feedback varies between Habbo hotel coun-           modes of user involvement reported in this study
tries. Like with any trans-national service, Habbo           suggest that the usability practitioner’s ‘toolbox’ needs
developers experience the dilemmas of language               tuning regarding social media.
regions and differently sized markets. On the one hand,          This case also reveals how the nature of user
one could argue that, because of Habbo’s Finnish             involvement has changed with social media. Before
origins, hotels with close linguistic ties to strong         social media, ‘involving users’ has been a much more
languages in Finland would shape the service most.           explicit effort, as developers and researchers have had
However, there are two factors counteracting this            to go where the users are and ask permission to
simple argument. First, language skills and effective        observe and get feedback. While some ways of
communication among Sulake employees can mediate             involving social media users remain as indirect as
remote user needs. Second, since some user needs are         before, feedback from social media users is at the same
readily communicated through sales statistics, the           time genuinely direct. The very thing people do in and
market size of a particular hotel country might be more      return to the service for, can be interpreted as user
significant than the language in that country.               feedback. The users might not think that they are being
    In a broader perspective, ”what happens with user        ‘involved’ or formally ‘invited’ – they will just not
feedback in social media?” is also a question about          return if the service does not meet their needs.
governance, that is what users get to decide on and the      However, previously it took longer before a decision
form of their influence. Typically users have less           by a user to stop using a product or service reached the
influence on generic products and more influence on          producers.
customisable products. The impact of user feedback is            This ‘direct feedback’ from social media users, in
especially big if the users themselves take the initiative   terms of frequent data points of user actions, has a
to new development. In contrast, social media is not         major consequence to user studies. The techniques
usually developed as a contract between a developer          commonly denoted as evaluation ‘with’ and ‘without’
and a client organisation, but rather on the initiative of   users need to be supplemented with ‘evaluation with
a handful of developers. Habbo users have had a big          database access to users’. As both use and non-use is
influence on early feature development and issues like       logged in the server database, alarms can be set to
community moderation (as volunteers), online discus-         trigger if use practices change. These alarms can then
sion about Habbo (as fansite authors), but have never        be used as starting points for succeeding user studies to
been invited to decide on pricing, advertisements, or        explain the changes in use practices. How strange, or
feature roadmapping. This is not to say that they            perhaps disciplinary inconvenient, it may sound, the
should be, it is just a fact that needs to be stated when    experience from this case study suggests that
outlining the Habbo user-developer relationship. E-          acquisition and analysis of database logs should take
democracy in Habbo has not yet reached the extent of         place before embarking on either quantitative or
the governance experiments in virtual worlds such as         qualitative studies on social media use.
EVE Online or A Tale in the Desert.

                                                                                                                         9
Proceedings of the 43rd Hawaii International Conference on System Sciences - 2010

7. References                                                             Human factors in computing systems, Florence, Italy:
                                                                          ACM, 2008, pp. 111-120.
                                                                 [19]   D. Svanæs and J. Gulliksen, “Understanding the context
[1] A.L. Friedman and D.S. Cornford, Computer Systems
       Development:        History      Organization       and            of design: towards tactical user centered design,”
       Implementation, John Wiley & Sons, 1989.                           Proceedings of the 5th Nordic conference on Human-
                                                                          computer interaction: building bridges, Lund,
[2] E.V. Hippel, Democratizing Innovation, The MIT Press,
                                                                          Sweden: ACM, 2008, pp. 353-362.
       2005.
[3] D. Leonard-Barton, Wellsprings of Knowledge, Harvard         [20]   OECD, Participative Web and User-Created Content:
       Business School Press, 1995.                                       Web 2.0, Wikis and Social Networking, 2007.
                                                                 [21]   A. Lehmuskallio, S. Tamminen, and M. Johnson,
[4] S. Kujala, “User Studies: A Practical Approach to User
                                                                          “Managing Privacy on Social Network Sites,”
       Involvement for Gathering User Needs and
       Requirements,” Doctoral Thesis, Helsinki University                Medienamateure: Wie verändern Laien unsere
       of Technology, 2002.                                               visuelle Kultur?, Siegen, Germany: 2008.
[5] J.T. Hackos and J.C. Redish, User and Task Analysis          [22]   A.J. Kim, Community Building on the Web: Secret
                                                                          Strategies for Successful Online Communities,
       for Interface Design, Wiley, 1998.
[6] M. Kuniavsky, Observing the User Experience: A                        Berkeley, CA, USA: Peachpit Press, 2000.
                                                                 [23]   Sulake,      “Quick    Habbo     facts,”  July    2009,
       Practitioner's Guide to User Research, Morgan
                                                                          http://www.sulake.com/habbo
       Kaufmann, 2003.
                                                                 [24]   M. Turpeinen and K. Kuikkaniemi, Mobile content
[7] J. Hom, “The Usability Methods Toolbox,” 1998,
                                                                          communities final report, Helsinki, Finland: Helsinki
       http://jthom.best.vwh.net/usability/index.htm
                                                                          Institute for Information Technology, 2007.
[8] U.S. Dept. of Health & Human Services,
                                                                 [25]   M. Johnson, “Unscrambling the “Average User” of
       “Usability.gov: Your guide for developing usable &
                                                                          Habbo Hotel,” Human Technology, vol. 3, May 2007,
       useful Web sites,” 2009, http://usability.gov
                                                                          pp. 127-153.
[9] UsabilityNet, “UsabilityNet: usability resources for prac-
                                                                 [26]   M. Johnson and T. Sihvonen, “On the Dark Side:
       titioners and managers,” 2003, http://usabilitynet.org
                                                                          Gothic Play and Performance in a Virtual World,”
[10] D. Norman, “Why doing user observations first is
                                                                          Journal of Virtual Worlds Research, vol. 1, 2009.
       wrong,” Interactions, vol. 13, 2006, pp. 50-ff.
                                                                 [27]   A. Salovaara, M. Johnson, K. Toiskallio, S. Tiitta, and
[11] H. Beyer and K. Holtzblatt, Contextual Design: A
                                                                          M. Turpeinen, “Playmakers in multiplayer game
       Customer-Centered Approach to Systems Designs,
                                                                          communities: their importance and motivations for
       Morgan Kaufmann, 1997.
                                                                          participation,” Proceedings of ACE 2005, Valencia,
[12] P. Ehn, Work-Oriented Design of Computer Artifacts,
                                                                          Spain: ACM, 2005, pp. 334-337.
       Stockholm, Sweden: Arbetslivscentrum, 1988.
                                                                 [28]   M. Johnson and K. Toiskallio, “Fansites as sources for
[13] J. Greenbaum and M. Kyng, Design at Work:
                                                                          user research: Case Habbo Hotel.,” 28th Information
       Cooperative Design of Computer Systems, Hillsdale,
                                                                          Systems Research Seminar in Scandinavia,
       NJ, USA: Lawrence Erlbaum Associates, 1991.
                                                                          Kristiansand, Norge: 2005.
[14] D. Schuler and A. Namioka, Participatory Design:
                                                                 [29]   M. Johnson and K. Toiskallio, “Who are the Users of
       Principles and Practices, Mahwah, NJ, USA:
                                                                          Habbo Hotel?,” Mobile Content Communities, HIIT
       Lawrence Erlbaum Associates, 1993.
                                                                          Publications 2007-1, 2007, pp. 89-107.
[15] R. Williams, J. Stewart, and R. Slack, Social learning in
                                                                 [30]   M. Johnson, “How The Teenage Invasion Changed
       technological innovation: Experimenting with infor-
                                                                          Habbo or Age-Sensitive Design In Inclusive Society,”
       mation and communication technologies, Cheltenham,
                                                                          The Good, The Bad, The Challenging: The user and
       UK: Edward Elgar, 2005.
                                                                          the future of information and communication techno-
[16] J. Stewart and R. Williams, “The wrong trousers?
                                                                          logies: a transdisciplinary conference. Organized by
       Beyond the design fallacy: Social learning and the
                                                                          COST Action 298, Participation in the Broadband
       user,” Handbook of critical information systems
                                                                          Society, B. Sapio, L. Haddon, E. Mante-Meijer, L.
       research: Theory and application, D. Howcroft and E.
                                                                          Fortunati, T. Turk, and E. Loos, eds, Copenhagen,
       Trauth, eds., Cheltenham, UK: Edward Elgar, 2005,
                                                                          Denmark: 2009.
       pp. 195-221.
                                                                 [31]   J. Nielsen, Usability Engineering, Academic Press Inc,
[17] A. Crabtree, T. Rodden, P. Tolmie, and G. Button,
                                                                          1993.
       “Ethnography considered harmful,” Proceedings of
                                                                 [32]   Idean Research, “Series 60 Developer Platform 2.0:
       the 27th international conference on Human factors in
                                                                          Usability Guidelines For Enterprise Applications.”
       computing systems, Boston, MA, USA: ACM, 2009,
                                                                 [33]   A. Järvinen, S. Heliö, and F. Mäyrä, Communication
       pp. 879-888.
                                                                          and Community in Digital Entertainment Services,
[18] S. Greenberg and B. Buxton, “Usability evaluation
                                                                          Prestudy Research Report, Tampere, Finland:
       considered harmful (some of the time),” Proceeding
                                                                          Hypermedia Laboratory, University of Tampere,
       of the twenty-sixth annual SIGCHI conference on
                                                                          2002.

                                                                                                                                  10
You can also read