Design Research for Media - A GUIDE FOR PRACTITIONERS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Design
Research
for Media
Development
A GUIDE FOR PRACTITIONERS
By: In Collaboration with:
Design
Research
for Media
AUTHOR: CREATIVE DIRECTOR:
Panthea Lee Jennifer Thibault
Development
EDITORS: DESIGNER:
Kate Petty, Ethan Wilkes Mollie Ruskin
PRODUCTION MANAGER: ADVISORS:
Patrick Ainslie Zack Brisson, Amanda Noonan
A GUIDE FOR PRACTITIONERS
© 2013 Reboot Design LLC.
The material in this work is subject to copyright. However the authors encourage
the dissemination of knowledge. We invite you to contact us should you seek to
reproduce this work in whole or in part. For all queries on rights and licenses,
we can be reached at hello@theReboot.org.
Reboot // 45 East 20th St, 5th Floor, New York, NY USA 10003
+1 212 388 1010 // www.theReboot.org
ACKNOWLEDGMENTS
With special thanks to Mark Frohardt, Amanda Noonan, and Sam De Silva from
the Internews Center for Innovation and Learning, Oren Murphy from Internews
Asia, and Charmaine Anderson, Arif Khan, Muhammad Ghawas, and the entire
Internews Pakistan Country Office.
The case studies and examples throughout this book are drawn from a design
research investigation commissioned by Internews in Fall 2012. It took place in
the Federally Administered Tribal Areas (FATA) and Khyber Pakhtunkhwa (KP)
regions of Pakistan, with fieldwork executed by a dedicated field team. Reboot
is grateful to our researchers Amjad Ali, Qaiser Ali, Sumaira Asghar, Shakir Ullah
Dawar, Arif Khan, Farooq Ali Khan, Jasim Khan, Kamran Khan, and Rizwanullah
Khan—thank you for your hard work, your perseverance, and your commitment
to finding the real answers.
Finally, to all those in FATA and KP who welcomed our team and shared their
By: In Collaboration with:
lives with us—thank you.
Contents
2 Foreword
4 About This Guide
10 Introducing Design Research
16 PHASE 1: Define
50 PHASE 2: Plan
66 PHASE 3: Collect
88 PHASE 4: Synthesize
110 PHASE 5: Design
122 Looking Ahead
124 About Reboot
125 About ICIL
1
// FOREWORD
Foreword
In 1980, I was working on the Thai- relationships with the communities I was drawn to Internews because We were most delighted to find in
Cambodian border in a refugee camp they are seeking to serve. This is their work has long embodied a Reboot an organization that shares
of over 130,000 displaced individu- a process that goes well beyond similar ethos. Program design is our values. Reboot brings significant
als. This was my first experience in simple question and answer surveys. firmly rooted in the principles of exper tise in formal, as well as
the international humanitarian and This is a process that demands an strong communications and an intuitive, human-centered design.
development space. A number of ethnographer’s ear and a journalist’s intimate relationship with the end This guide was developed to assist
organizations were present providing inquisitiveness to uncover the behav- user. Our evolution into human- those working within Internews to
assistance, and all were effective in ioral insights that drive a community. centered design, therefore, is only challenge our embedded assump-
their own way. But one organization natural. This approach allows us tions and deepen our relationships
impressed me in particular for the I was also fortunate to work with to further refine our engagement with the communities we serve. This
degree of proximity they maintained Fred Cuny of Intertect, a master of with local populations and com- guide is also written for the broader
with those they were working to this approach, during the return of munities to develop solutions fitted community of practice that hopes to
serve. The doctors and nurses of Kurdish refugees to Northern Iraq in to their daily needs and aspirations. understand the continuously chang-
Médecins Sans Frontières would 1991. Although called in to advise This approach also allows us to move ing information ecosystems that
often just sit with their patients and commanders of the multinational beyond a program design process increasingly define and constrain
local staff, beyond the scope of their forces responsible for the repatria- that simply informs our initial development practice.
work and questions—beyond the tion, Fred, as always, first went deep engagement to one that ensures
traditional relationship of aid worker into the camps to develop plans deep and lasting relationships We hope you find the ideas and
and aided. based on an ever-evolving under- throughout the life of a project, methods of this guide valuable in
standing of the needs and capacity keeping objectives based on shift your work and welcome your views
In the over three decades since that of local leaders and individuals. Fred -ing priorities relevant and on target. and suggestions.
first encounter, the organizations was deeply committed to solutions
and individuals I have seen success- that included the “end user” in both Mark Frohardt
fully fulfilling their development mis- conception and implementation. Executive Director
sion are those who have built close Internews Center for
Innovation & Learning
2 Design Research for Media Development 3
// about this guide
About This
Guide
This is a hands-on reference guide This resource is our effort to share Throughout this guide, several special features will
for media development practitioners. our process with our Internews help you better understand and implement design research:
colleagues and among the broader
It is based on principles and practices media development community, in
of design research that have been the hopes that it will inspire practice
long used by the private sector, and and discussion and have an impact
grounded in the experience Reboot far beyond our immediate goals.
has gathered in designing and
implementing international develop- This is not a comprehensive guide
ment projects around the world. to program design. As a field, media
development has a rich history of FATA in Focus use real-life examples Tools signal a resource to help
This guide was born out of a col- innovation, thoughtful leadership, from an Internews project in the FATA practitioners implement the design
laboration between Reboot and and tried-and-true processess— region of Pakistan to illustrate research process. Sample templates
Internews, through its Center for many pioneered by Internews. design research principles in action. and examples from the Internews
They show how the approach both Pakistan Case Study, can be found at
Innovation and Learning, and its Indeed, many ideas in here will
requires adaptability to context and www.design.internews.org/tools.
Pakistan Country Office. Together, we likely be familiar to readers working offers rich contextual insights.
sought to understand the complexity in this systems-oriented field. Thus,
of the information ecosystem in the our hope is that this guide will help
Federally Administered Tribal Areas practitioner s integrate design
(FATA) region of Pakistan. Our ulti- research principles and practices A companion website for this guide can We are eager for feedback and
be found at www.design.internews.org. to hear about your experiences in
mate goal was to design contex- naturally into their existing work,
This online platform will be frequently applying these approaches. Please
tually appropriate programs that stretch their creativity, and find new updated with resources to support send thoughts to hello@theReboot.org
improved access to information solutions. practitioners seeking to integrate and innovation@internews.org.
by communities in this region. design research into their work.
4 Design Research for Media Development 5
// about this guide // Principles in Practice
Harnessing Design Research in FATA
This guide features a running case studie to illustrate
how design research can apply to media development.
All examples are drawn from an investigation conducted
in the Federally Administered Tribal Areas (FATA)
region of Pakistan in September 2012.
About FATA
Socially, economically, and politically, Politically, FATA has traditionally Yet opportunities for change are But the region remains media dark,
FATA is the least developed region been ruled, not governed, by on the horizon. The upcoming 2013 with variable access to mobile
in Pakistan. Sixty-six percent of the unelected tribal leaders. The region general election will be the first time networks and Internet and tight
population lives below the poverty is strongly conservative and cultur- that political parties are able to form restrictions on journalists. Although
line; 1 its key industries are farm- ally isolated, constraints that are and operate in FATA, meaning the there has been an increase in media
ing, smuggling, and illegal drugs; 2 heightened by geographic and first chance for FATA communities access and channels in recent years,
unemployment is estimated at 60 security factors. United States mili- to have their interests represented little is known about the informa-
to 80 percent;3 and the literacy rate tary operations using unmanned on the national stage. To leverage tion and media behaviors of FATA
is 22 percent (and 1.5 percent for aerial vehicles (drones) have tar- this moment, independent and communities.
females).4 geted the region since 2004, in an good quality information about the
attempt to defeat Taliban and Al- political process will be vital to its
Qaeda militants. Drone strikes have residents.
killed up to an estimated 3,400
people.5
6 Design Research for Media Development 7
// about this guide // Principles in Practice
HARNESSING DESIGN
RESEARCH IN FATA
Project Background Project Approach
Given the FATA’s communications the elections? Furthermore, what did It was determined that design Due to the security restrictions,
landscape and its population’s low they think about politics in Pakistan? research—a research and analytical Reboot and Internews staff did
literacy rates, Internews was inter- And finally, how should informa- process that combines ethnograph- not travel in FATA. Reboot and
ested in the potential of interactive tion be presented? What sources of ic, journalistic, and systems think- Internews managed field research
voice response (IVR) technology to information were perceived as trust- ing approaches—could help answer from Islamabad, and conducted
get information about political pro- worthy, and how did people evaluate these complex questions, and help training and synthesis workshops
cesses to FATA communities, particu- the credibility of information? Internews identify opportunities to in Islamabad and Peshawar.
larly around the upcoming elections. improve the information ecosystem
But as teams began planning such Internews realized a more nuanced for FATA communities.
an initiative, they realized they need- understanding of the information The research findings will be
ed answers to a few key questions to and communications landscape in A design research investigation released in early 2013.
enable strong program design. the region was necessary to ensure was conducted over four weeks
programs would be appropriate and in September 2012. A team of 10
To start, it was critical to understand impactful on the national stage. To researchers from FATA conducted
what were the technology behaviors leverage this moment, independent the fieldwork, with training and man-
of FATA communities, and whether and good quality information about agement from Reboot. The study
IVR was even a realistic option. And the political process will be vital to covered 7 agencies in FATA and
beyond the choice of communica- its residents. And, in recent years, Khyber Pakhtunkhwa (KP) and 125
tions channel, what kind of informa- there has been an increase in media respondents were interviewed.
tion did people want? What did FATA access and channels within the FATA.
communities already know about
1 Shuja Nawaz and Arnaud De Borchegrave, FATA - A Most Dangerous Place,
(Washington, D.C.: Center for Strategic and International Studies, 2009).
2 Shahid J. Burki, “Economics and Extremism,” DAWN.com, n.d.
3 “FATA: Inside Pakistan’s Tribal Regions,” New America Foundation, n.d.
4 Government of Pakistan, United Nations Children’s Fund, and the World Food Programme, Federally
Administered Tribal Area (FATA) Multiple Indicator Cluster Survey (Peshawar: Government of Pakistan, 2009)
5 “The Year of the Drone,” Counterterrorism Strategy Initiative, New America Foundation, 2012.
8 Design Research for Media Development 9
// INTRODUCING DESIGN RESEARCH
Introducing Key Principles
Design
of Design Research
1. Understand humans and institutions.
The best solutions meet the needs of users, while respecting
their capacities and constraints, and they do so at the
Research
community and institutional levels. Understanding both,
therefore, is key.
2. Discard assumptions.
“Design by ToR (Terms of Reference)” is tough. The best
solutions come from the real world, so learn to recognize
personal biases and assumptions, as well as those of others,
to see the way forward.
3. Remain flexible.
It is impossible to predict the solution (or even the best
path for finding the solution) at the start of a project.
Good program design requires That is why design research is a Change is the norm, so be open-minded and adaptive.
two things: empathy for people valuable and important starting point
and understanding of place. for media development. Design
research is more than an exercise 4. Consider diverse stakeholders.
Design research is a valuable A program is only as good as the sum of its parts. Examine
in knowledge gathering; it is a pro- the needs and capacities of stakeholders along your program
tool to achieve both.
active, rigorous, and intellectual delivery chain, such as regulators, media outlets, journalists,
process tailored to create concrete and audiences, to ensure the design accommodates each
Design research is useful whether
programmatic interventions. of them.
you are designing an iPhone in Palo
Alto or a media development program
Blending practices from ethnography, 5. Be responsible.
in Pakistan. But for projects like
journalism, and systems thinking, Anticipate and mitigate the unintended negative impacts
the latter—whose designers often
design research helps practitioners a project may have on communities. From selecting
come from vastly different back-
understand the causes, relationships, respondents to introducing the research, consider how
grounds than their users, and where
and human dimensions of complex actions and words may impact local dynamics.
the context is extremely complex—
contexts—and then provides tools
developing empathy and under-
to incorporating this knowledge into
standing requires greater effort and
the design of innovative and realistic
unique skillsets.
interventions.
10 Design Research for Media Development 11
// INTRODUCING DESIGN RESEARCH
Design research is a process of The process of design is fluid and
discovery, not of verification. It is
creative, it is constantly refined
particularly useful in the early stages
and adjusted for different contexts—
of a project, where opportunities are
design research is similarly versatile.
vast and the paths to seize them To that end, while design research Design Research is Not....
are unclear. It is a process to discard
is inherently valuable for program
assumptions, and a means to design, it can be integrated into Design research is a powerful tool for understanding complex
design interventions that are truly fit
other approaches already used by human systems and surfacing key factors that are critical for
for context. media development professionals. program design.
Just as no two contexts are alike,
Because, ultimately, while success- there is no one formula for good It is not:
ful programs require funding, tech- research and design.
nology, and management, under- A substitute for quantitative research.
standing of context is paramount. Beyond the field of media develop- Design research is not intended to yield statistically significant
findings. Yet it complements quantitative research by surfacing
ment, design research has been
key insights that are invisible to less nuanced methods. As nec-
widely utilized to answer a range of essary, qualitative insights can be probed further for represen-
questions relating to governance tativeness through survey work.
and international development pro-
Integrate
gramming. These include: A replacement for stakeholder engagement.
design
Design research emphasizes deep engagement with communi-
research • How can we design a social ties and stakeholders with the aim of eliciting data to inform
into M&E. accountability system to program design. It is not intended to replace a program’s
maximize citizen engagement stakeholder consultation and sensitization activities.
This guide focuses on the in the design and delivery of basic
applications of design research services, and compel government A solution for every program.
in program design and develop- response based on citizen input?
Design research is most useful when the challenge or context
ment, but the process is equally is new, complex, or involves diverse stakeholders. Certain
valuable for M&E efforts. • How can we help diverse and often
inaccessible human trafficking types of programs, such as infrastructure development or
victims find the means to escape policy advocacy, may demand other approaches, with design
The ability to assess and explain research playing a secondary, or even non-existent, role.
project outcomes in terms of their situations?
context is essential to learning—
• How can the international community
particularly when projects fall short.
leverage new technologies to
support good governance,
social cohesion, and economic
development in a country
that has undergone a popular
revolution?
12 Design Research for Media Development 13
// INTRODUCING DESIGN RESEARCH
The Design Research Process
There are five key phases in the design research process.
The following chapters will explain each of these in detail,
along with practical advice and case studies.
PHASE 1 PHASE 2 PHASE 3 PHASE 4 PHASE 5
Define Plan Collect Synthesize Design
Articulate a program’s Develop an intellectual Work to understand Make sense of the Distill findings into program
ultimate objective, and framework for informa- the people the research, and draw design recommendations
define the questions that tion that is needed to program seeks to patterns and insights and guidelines for use by
must be answered to help meet the design serve, and the actors from the data. Ensure the implementing team.
achieve it. Set clear goals challenge, and an that will be serving that findings are It is also helpful to present
to establish boundaries operational plan for them. Dive into the useful and actionable findings in a format
for what falls outside how to get that context. for program design. accessible to wider
immediate relevance or information. audiences to benefit the
current capacity. larger community
of practice.
Design research can and should has its limitations when examining inform programs spread across
be used in conjunction with other complex systems. To design a good areas with geographic variance, or
approaches, such as participatory survey, you need to know what you programs that will be expanding to a
rural appraisal (PRA), social network are looking for and how that information significant scale.
analysis, the Delphi Method, or may manifest in the particular context.
audience research surveys. In fact, Qualitative research, on the other Resources, human and financial,
practitioners will find similarities, or hand, excels at building rich portraits as well as time will also be factors
even shared tools, between some of of a people or place, and at mapping in selecting research approaches.
these approaches. the relationships between people, Thankfully, design research scales
places, cultures, and institutions. well. It can be effective as a small-
In combining research approaches, it But insights surfaced by design scale process as well as a much
is a matter of sequencing and scale. research may merit additional deeper expansive investigation.
Quantitative research, for example, surveying, especially if they will
14 Design Research for Media Development 15Phase 1 // DEFINE
Define the
Challenge
Phase 1:
Define All development projects
start with a challenge. But too
often, a project will define that
challenge by presupposing a
These kinds of assumption-based
projects risk failure, no matter how
many times or where else they
have worked before, because they
do not consider how people in a
solution. Design research helps
specific context actually behave or
us start with the right questions. the unique characteristics of the
environment. Even if a proposed
If, for example, a challenge people solution, such as the SMS information
face is poor information or media service, may suit a particular context,
access, practitioners might ask: it is impossible to know how to design
“How can we support independent a service that is broadly accessible,
community radio?” or “How might has locally relevant content, and
we develop an SMS-based informa- which can be cost-effectively and
tion delivery service?” sustainably maintained.
16 17Phase 1 // DEFINE // Research Framework
Research
Framework
Framing and defining a design chal- Once a precise and concise design The research framework is a living In practice, priorities tend to shift
lenge, therefore, is key for program challenge has been determined, the document that defines your research from what was originally put on
design, and for the design research team should be able to determine themes and questions, but its focus paper. All practitioners know about
that will inform it. Start from the the information it will need to be will evolve and narrow over time, first the importance of flexibility and
wrong perspective, or with a set of able to tackle the challenge. This is as feedback is gathered from stake- striking a balance between planning
assumptions, and you can spend where design research comes in. A holders, and then even more so over and operations. The research frame-
your whole project working on the strong solution for any context starts the course of field research. work creates a way to adjust to shift-
wrong thing. with a strong understanding of that ing donor and stakeholder priorities,
context. Inspired by the design chal- While not a record of insights or find- while also keeping the on-the-ground
Thus, a good design challenge lenge, we are now ready to build our ings, it is reflective of an increas- realities in focus.
should: Research Framework. ingly sophisticated understanding of
the context, honing in closer to the
Speak to a user need.
actual truth as the research unfolds.
Over time, more targeted and more
Do not speak to an organizational
need, a technological opportunity, or detailed questions arise—this is a
a donor directive. good sign.
Be open-ended in terms of how
that need might be addressed.
It should not predefine a solution.
Include the constraints that the
design needs to satisfy.
These may include meeting certain
timelines, collaborating with certain Use the
partners, or building upon existing framework
programming. Research
to align
Framework
stakeholders.
It can help stakeholders The complete
develop shared understanding Research Framework
of the project goals, processes, from the Internews
and expected outcomes. During Pakistan Case Study
check-ins, use it to reflect the can be found at:
evolving understanding of the www.design.internews.
operational context, and thus org/tools.
program priorities.
18 Design Research for Media Development 19Phase
Phase 1 //
1 // DEFINE////Research
DEFINE ResearchFramework
Framework
Principles in Practice
Key Components
of a Research Framework:
Project Objectives.
Specific goals, both concerning the questions to explore
and the projected deliverables based on research findings.
Defining the Challenge: A Collaborative Process
Key Opportunities and Challenges.
From institutional priorities, desk research, and expert
consultations.
Guiding Themes and Questions. Coming up with a robust and for low-literacy users, like many
Also grounded in desk research and expert consultations, realistic design challenge was FATA populations, and the upcoming
these drive the field research. no easy task. Our project had elections were certainly a factor in
planning timelines.
Target Respondents and Sampling Plan. eight people representing four
Outline of different user and stakeholder groups to consult, teams from two organizations
Ultimately, however, the team
and how they will be recruited. spread across four cities (and agreed to keep the prompt as broad
that is not including travel). as possible: “Identify opportunities
Methodology.
This is what we did: to improve information access
Summary of the research methods to be used,
along with the rationale, role, and specific focus of each. and relevancy for communities in
Based on Internews’ existing work the FATA.” While technology and
Operational Plan. and team members’ experience new media were of interest, they
Field logistics, including locations, timelines, research working in the region, everyone were excluded from the challenge
schedule, team composition, and roles. agreed that FATA communities needed definition to not prematurely bias
better information channels to get findings. And while Internews had
Question Guides and Scripts. information about the world, and to timelines and programming that
Concrete plan for the research team to explore key share information about themselves.
themes in respondent interviews.
could be strategically integrated with
But there were differing opinions the project, it wanted to untether
as to how this should be done, and the research of institutional priorities
what timelines it should be accom- to truly benefit from the generative
plished within. nature of design research. The challenge,
therefore, was intentionally free of
Interactive voice response (IVR) constraints to encourage research
technology showed promise as that may contribute to the broader
a news and communications tool development community.
20 Design Research for Media Development 21Phase 1 // DEFINE // Research Framework
Principles in Practice
STEP 1:
Map the Issues
To understand a challenge, examine it, systematically, from all angles.
Write down your design challenge, then write down all the questions you
might have to answer to be able to tackle the challenge.
Then, group the questions to determine key themes that will drive the
research.
For information- and media-related research, practitioners
Determining Key Themes
may want to consider:
Trends over time. Contours of trust and influence.
In addition to current attitudes and To the extent that trust and The work in Pakistan aimed to We began by defining ‘information
behaviors, examine longitudinal influence are relative, dynamic understand the behaviors usage’ and realized we were interested
patterns—how have cultural, political, concepts, explore the drivers and in the following question:
boundaries of trust. This should
of FATA communities relating
and technological changes impacted
user behavior over time? Such a be done within communities, as to information usage—a tall
How are FATA populations access-
focus can help anticipate future well as between communities and order, to be sure.
ing, sharing, evaluating, and creating
shifts, and allow program designs a) institutions,
Through a mapping exercise, information, and how might their
that accommodate them. b) diverse information sources,
we determined the key themes habits change over the next two
and
Given the pace of technological that the study would focus on. years?
c) the media.
change, practitioners must under-
stand the forces driving change, not To the extent that effective Beyond current habits, we wanted
only how that change is manifest interventions always, at some to understand the needs, capacities,
today. level, rely on trust, understanding constraints, and sensitivities that
the nature of trust in a particular
Multiplicity of media. influence the information-seeking
context is paramount.
People’s usage of and relationship and media consumption behaviors
to media is complex and diverse, of FATA communities.
and research should examine
diverse information sources Through a mapping exercise, we
(formal and informal), as well determined three key themes that
as the complex, ever-changing would drive the research, presented
relationships people have to them. here as an excerpt from the Research
Framework of the Internews Paki-
stan Case Study.
22 Design Research for Media Development 23Phase 1 // DEFINE // Research Framework
DEFINING KEY Principles in Practice
THEMES
Theme 1: Theme 2:
Understanding Trust & Influence Impacts of Displacement
How do FATA communities evaluate not specifically oriented towards What has been the impact of con- how FATA populations access, evalu-
the credibility and trustworthiness disseminating information but that flict and internal displacement on ate, and trust different information
of different information sources may do so as a secondary activity the information-seeking and media sources and media channels. To the
and media channels, and how do or naturally as a result their role in behaviors of FATA communities? extent possible, the research will
these factors impact the influence communities (e.g. service providers map how these attitudes and behav-
and reach of different source and such as schools or clinics where Since 2007, FATA has experienced iors change over people’s timeline of
channels? people gather, or entrepreneurs that significant population migration as displacement.
wield economic influence). There a result of government operations
Both formal (e.g. newspapers) and will be a specific focus on emerging against non-state armed actors and
informal (e.g. community leaders) sources of information. sectarian violence. The research
sources of information will be seeks to understand how migration
considered, as well as sources and displacement has impacted
Key questions include: Key questions include:
• What are the social norms around gathering, sharing, • How do displaced FATA populations access information?
and assessing different types of information? How does this compare to communities that still reside within the FATA?
• What are the metrics FATA communities use to determine • How does the experience of migration and displacement,
whether a particular piece of news or information—or a source— both conflict-related and seasonal or work-related, impact how FATA
is credible, trustworthy, or worth passing on to their networks? populations evaluate the credibility and importance of different
media sources?
• Specifically, what are the uses and perceptions of secular versus
religious information sources? • How does information pass to and from FATA residents and
connections outside?
• Based on the above metrics, along with reported and observed user
experiences, can we map the relative influence and reach of different
types of information sources among FATA communities, how information flows
between them, and how these patterns have changed over time?
24 Design Research for Media Development 25Phase 1 // DEFINE // Research Framework
DEFINING KEY
THEMES
STEP 2:
Review Existing Research
Theme 3: Desktop research is a critical early As part of the literature review,
Usage of New Media & Technology input to the design research process. include work in the specific geo-
Literature reviews, expert consulta- graphic context as well as the sector
As technology is vastly changing Beyond analyzing indicators such tions, and discussions with institutional (and review both case studies and
the communications landscape in as technology penetration (e.g. partners ground the potential future evaluations).
Pakistan, and will continue to do so, radio, television, or mobile), size of in the wisdom of what has been
the research will have a particular audiences for particular programs, done before. In consultations, seek out three
focus on perceptions of and behaviors dollars invested in specific channels, different types of experts:
around new media and digital com- or other market metrics, the study Desk research sets the stage for 1) Operational experts,
munications tools. will examine the human factors that design. With such an expansive
impact technology behaviors and body of innovation and achievement 2) subject matter experts, and
Given the potential of information choices. in the field of development, any work 3) institutional experts.
and communications technologies should build on what has been done.
(ICTs) to enable easier and broader
access to information, the study
will seek to understand specific
technologies that have high potential
to reach FATA communities, as well
as challenges around and limitations Operational experts have worked extensively in a
of ICTs. geographic context. Their insight will be especially key
in planning the logistics of a research trip and developing
a risk management plan.
Key questions include:
Subject Matter experts have implemented or evaluated
• What are current behaviors around the use of technology to access, share, similar or complementary programs, and can share their
create, and evaluate information? Can we extrapolate larger trends around opinions and understanding about how and why their
technology usage beyond specific channels and media sources? research and program approaches succeeded (or fell short).
• How does literacy, in all its forms — textual, technological, and media —
Institutional experts from organizations, donors, or other
impact usage of technology among FATA populations? What are ways
stakeholders can help focus goals and deliverables. They can
in which FATA communities currently circumvent their own capacity
help understand what outputs are most useful and feasible,
constraints around technology to get and share the information they need?
and how best to present them.
• What are the threats to the use of technology, and how do FATA
populations respond? Does the cultural appropriateness of technology
vary with its intended purpose (e.g. mobile phones for communications
versus mobile phones for listening to music or the news)?
26 Design Research for Media Development 27Phase 1 // DEFINE // Research Framework
Principles in Practice
Consulting Experts
Before fieldwork, the team consulted with experts on key project
themes and to gain better understanding of the operational Category Role Affiliation Profile
context. After fieldwork, experts helped interpret, analyze, and validate
Subject Academic University DC-based Pakistan expert
research findings. This is redacted sample of those consulted. Matter Researcher with on-the-ground perspective,
with focus on political stability
and Islamic politics. Advise on
research framing questions,
Category Role Affiliation Profile and institutional interests.
Operational M&E Donor Well versed in operating M&E Subject Co-Founders Local ICT4D Islamabad-based technology
Officer Organization programs in FATA, familiar Matter Firm and civic participation firm that
with Internews. Advise on has deployed mobile-based
operational and risk planning information services across
and management. Pakistan. Advise on technology
behaviors and service design.
Operational Chief of International Extensive on-the-ground experi-
Party Development ence in FATA, and with media- Subject M&E International Worked with Internews on research
Contractor related programming. Advise on Matter Program Development and M&E in several countries.
operations, logistics, and security. Advisor Contractor Advised on complementary and
adjacent research initiatives to
Operational/ CEO Local Works on audience research with draw from and feed into.
Subject Research Internews in Pakistan. Extensive
Matter Organization experience media behavior in the Institutional Senior Donor Deep understanding of media
region. Advise on gaps in current Leadership Organization landscape in FATA, has managed
research. Internews programming. Advised
on donor priorities and interests
Operational/ Various Staff Local Experts in freedom of information in the region and for the research.
Subject Civil Society and expression issues in Pakistan,
Matter Organization and the cultural and political fac- Institutional Various Across Advised on Internews’ needs and
tors that influence the conversation. Internews Operational goals around research, perceived
Advise on research questions, and Staff and Country opportunities and gaps in current
the role of technology in shaping Teams research, program development,
the information landscape. and M&E processes to help refine
outputs.
Subject Journalist Various news Pakistani journalist with extensive
Matter outlets experience covering FATA and
Given the aims of the research, local or locally based experts
international news. Advised on
research framing, questions, were prioritized over international ones. Institutional and international
and managing sensitivities. perspectives still very much informed the research to ensure it would
be useful to a range of stakeholders.
28 Design Research for Media Development 29Phase 1 // DEFINE // Research Framework
STEP 3: Their values and needs will impact This list becomes your respondent
program outcomes, thus their per- group, and each group’s needs and
Identify the Stakeholders spectives must be understood and challenges will need to be addressed.
accommodated, as appropriate, in While the map to it is not yet clear,
Media development seeks to improve human livelihoods through information.
the program design. the places where these many view-
So, it is logical that humans are at the heart of good program design. They
points and needs intersect is where
both inform and inspire program designs, and their needs are the metrics by
List all the stakeholders who the the most sustainable interventions
which programs should be evaluated.
program must consider. Starting will be found. And be realistic: There
Yet there are many actors that impact the design, delivery, with the end-users, list all those who will be actors your program will not
and ultimate effectiveness of a program. influence their behavior, as relevant have access to or influence over.
to the program: What people and Still include them as variables to be
These may include: institutions do they interact with? aware of, and to recognize the gaps
Which other stakeholders have the in your own data.
End-Users Community Leaders greatest impact on their actions? Do
Target individuals and communities Individuals with both formal and
this for all the stakeholders identified. Do not get intimidated. This process
(often called “beneficiaries”) the informal distinctions as leaders
activities and outputs of the within their community and that
can be overwhelming. Any map of
program are intended to benefit. have particular influence. Think through the perspective of the conflicting interests of dozens
each listed stakeholder: How do of people and institutions will look
Service Providers Contextual Influencers they impact the experience of the unsystematic and unsustainable.
Individuals or organizations that
Individuals and organizations that end-users? What are their priori- Getting through the mess is what
have impact or control over the
serve as functional touchpoints for ties and goals? How do they view the research is for.
environment or the individuals
reaching end-users—not all may be the other stakeholders on the list?
in which a program is targeting
formally recognized as information How can they contribute to program
sources or service providers. These Policymakers objectives?
may include radio stations, primary Creators of the environments in
schools, or agricultural extension which a program exists through
workers. legislation, rhetoric, or other
Administrators forms of power.
Functional managers that oversee Institutional Partners Predict users’
the actions of service providers. Institutional donors, private sector needs, not Know who to
partners, or other organizations solutions. include when.
that define, resource, and influence
program design.
Think broadly and fundamentally Naturally, stakeholders include
about users’ needs. For example, funders, partners, colleagues,
users may not need “a radio and others who are contributing
station” or “better information”; to or otherwise directing the
rather, they may need security, work. Include their perspectives
stability, or connection with when planning the research,
loved ones, all of which can be but once fieldwork begins,
addressed in multiple ways. your allegiance is to end-users.
30 Design Research for Media Development 31Phase 1 // DEFINE // Research Framework
Principles in Practice
A Note on Sample Size
In the age of “Big Data”, it is hard to resist the idea
that survey data, and lots of it, is better. It is true that
quantitative data is a valuable tool for many aspects
of program design: Large datasets can help identify
sweeping trends across a community, or determine how
human well-being has changed in a region over time.
Defining Target Respondents
The strength of such analysis comes in part, however, by
reducing nuance.
To complement data analysis, design research is
intrinsically about understanding complexity through
human nuance. This means aiming for a focused and Pakistan has a vigorous and diverse media ecosystem, and
manageable sample size. The goal is not statistical below are the stakeholder groups that our research targeted,
representation; there is no need to touch every potential as well as the target percentage of the overall respondent
type of stakeholder relevant to the program. Rather, the population for each group, based on our mapping exercise.
goal is enough representativeness to draw meaningful
insights on patterns of behavior and context.
Early on, we agreed that the focus Further, researching information
The most valuable insights often come from making would, first and foremost, be on di s tri b utor s ( 5 % ) a n d reg ul a -
surprising thematic connections between seemingly information consumers (60%): FATA tors (5%) would help Internews
disparate behaviors and cultural norms. These can only communities who seek information identify the oppor tunities and
be generated by human evaluation of collected data. through diverse outlets. Under- challenges for information dis-
Further, as the best analysts and program designers are standing their behaviors and atti- semination. Finally, media-related
usually those that conducted the field research, their tudes—and their underlying ratio- organizations (5%) including NGOs,
facility with the entire data set is important. As powerful nale—and capacities would allow watchdog groups, and others will
as large data sets can be when run through NVivo or Atlas Internews to develop content and help guide the research process,
TI, human creativity is also a powerful tool—and for it to programs that will reach and be both in data collection as well as
work well, the capacities of human memory should be understood by these populations. analysis. On the following pages is
considered when determining sample size. Understanding a broad range a condensed version of the study’s
of information producers (25%), stakeholder analyses:
both formal and informal, and the
motivations, capacities, and self-
conceptions of each actor would be
critical to mapping information flows
and networks of influence and trust
among FATA communities.
32 Design Research for Media Development 33Phase 1 // DEFINE // Research Framework
DEFINING TARGET Principles in Practice
RESPONDENTS
60% 25%
Information Consumers Information Producers
Examples of Entities Key Questions Examples of Entities Key Questions
• FATA populations in and outside of • How do they access, share, and • Journalists: local and foreign • What are their goals in participating
FATA, a diverse sample in terms of assign value to different types in the information landscape, and
• Bloggers
gender,* age, economic background, of information? in producing information?
education, location (both rural and • Traditional leadership
• What is the role of information/news • How does each actor see its role,
urban) • Informal community leaders
in the context of life in FATA? (Does and how does it compare to other
• Particular attention will be paid to it play role in creating change? How • Civil society and/or special interest actors in the ecosystem?
a) influencers, as their habits and FATA populations perceive informa- groups (e.g. youth organizations, • What sort of training have they had
choices have outsize impact on tion/media? What types of informa- NGOs, fundamentalist groups) to produce information, and what are
their larger communities tion do they want to access?)
• Religious media outlets (e.g. mullah the standards they hold themselves
b) youth, as their behaviors can • What is their access to and usage of radio) and leaders (e.g. imams)* to?
suggest trends in information technology?
• Government officials • What is their sense of the security
consumption and technology • How do they evaluate the credibility (e.g. Political Agents)* issues involved in their work?
habits and trustworthiness of information How do they mitigate security risks?
sources? • Government and state-controlled
c) low-literacy populations, as ICTs media outlets (Pakistani and foreign)* • What other information sources
have the potential to uniquely • For influencers: What is the do they draw upon and use?
serve them relationship between these individu- • University journalism departments
• What do they see as the biggest
als and those they influence? How • Press clubs constraints, for themselves and
do they gain and use their influence?
* Note: It may be difficult to access • Political parties and activists their listeners/viewers?
How does information and com-
female populations. Little is known munication impact or facilitate their • Businesses and/or entrepreneurs • How do they evaluate their success
about this demographic in the influence? as information producers?
region, much less specific media
• For youth: What are the tools they • For businesses: How do they
and technology behaviors. * = May not be possible
use to access and share information? approach marketing in FATA, and
is it any different from how they
• For low-literacy populations:
advertise in the rest of the country?
What are the ways in which they are
accessing information, despite their
lack of literacy?
34 Design Research for Media Development 35Phase 1 // DEFINE // Research Framework
DEFINING TARGET Principles in Practice
RESPONDENTS
5% 5% 5%
Information Information Media-Related
Distributors* Regulators/Controllers* Organizations*
Examples of Entities Examples of Entities Examples of Entities
• Media groups • Government agencies • NGOs and civil society groups
(e.g. Jang, Dawn, Lakson) (e.g. FATA Secretariat, Pakistan (e.g. Rural Media Network Pakistan,
Broadcasting Corporation, Pakistan Pakistan Centre for Development
• Local media outlets
Electronic Media Regulatory Authority) Communication)
• International media outlets
• Traditional leadership and tribal • Journalist unions
• Television broadcasters elders
• Crowdsourced content sites
(e.g. Hosh Media)
• Aggregated content sites
(e.g. Teabreak)
Key Questions Key Questions Key Questions
• How do they determine the • What are their key goals and • What do they think are the key * For this study, it was determined
content that they will include in concerns regarding to the production opportunities and challenges in that perspectives of information
their outlet? How do they prioritize and dissemination of information? securing greater information distributors, media-related
different pieces of content, and • How do they determine what types
access and better quality of organizations, and information
choose what to feature? of information can or cannot be
information for FATA communities? regulators/controllers would
• Do they track responses to the shared, and how they will be shared? • What are their thoughts on our largely be drawn from existing
content they put out? If so, what research framing and plan, as well research and expert consultations.
tends to be more/less popular? as our findings/analyses? Field research would largely target
Do they have ideas as to why? information consumers
and producers.
36 Design Research for Media Development 37Phase 1 // DEFINE // Research Methods
Research
Methods
Many instruments are available to Naturally, practitioners must choose
help researchers for getting a holistic instruments based on what is fea-
view of a target community. sible in the environment. Consider The Value of Variety
culture as well as security, timelines,
Group interviews provide a broad and other challenges. Any research program worth its salt will use several
but basic understanding of key different instruments. Multiple methods can help
research themes, and sur face Over time, the methods used may researchers triangulate data surfaced by different tools
threads that require further probing. change, either to focus on an and address the inherent deficiencies within each tool,
emerging theme or to overcome helping overcome research bias.
Individual, in-context depth challenges (of bias, logistics, or
interviews allow a deeper under- other) that crop up in the field. Often, Everyone carries biases, and different instruments surface
standing of people’s frustrations, a discovery in the course of research different biases in both researchers and respondents. For
vhopes, and day-to-day lives, especial- makes one or more instruments example, due to natural social dynamics, respondents
ly when used over time. obsolete; for example, a service in a group interview may be eager to appear more pious,
trial of a government information more hard-working, or otherwise impress each other with
During a home stay, as the guest line is impossible if the phone line their answers; individual follow-up interviews may elicit
of a respondent, researchers gain has been shut down for the past six more self-reflective answers. Similarly, during an in-depth
greater empathy and insights into months. Governments and devel- interview, a respondent may be embarrassed to admit
how people think and operate, as opment organizations do not always challenges they have experienced in using a service, for
social barriers are broken down have updated information on the fear of appearing incompetent, but a service trial or an
through sustained engagement. service availability, especially for observation of that same person using the service can
rural areas, so plan accordingly and raise these frustrations loudly and clearly.
Service trials yield first-hand expe- remain flexible.
rience of how programs and services Using several instruments helps correlate findings, check
work—and thus insights about the biases, and better understand the subtext of people’s
relationship between institutions answers.
and people—which is critical for
effective program design. Keep it
interesting.
The common thread to all of these
instruments is immersion. Design
Always question whether a
research tools open structured space research tool is the right tool for
where researchers can discuss, the situation, and whether the
observe, test, experience, and mix keeps things engaging for
understand the mechanics and the team. Research should be
emotions of daily life in the way that stimulating, and using a range
respondents do. of tools help unlock creativity,
which is critical for program design.
38 Design Research for Media Development 39Phase 1 // DEFINE // Research Methods
Research Tools
GROUP INDIVIDUAL
INTERVIEWS DEPTH INTERVIEWS
Facilitated discussions with a Semi-structured conversations that
group of respondents. Respondents probe into an individual’s attitudes
may be grouped for any number of and thought processes. Best when
reasons: Similar demographics or conducted in private and in a setting
profiles, or diverse viewpoints. that respondent is comfortable with,
such as their home or workplace.
Used To: Challenges: Used To: Challenges:
• Gain a broad understanding of • Group environments will impact • Gain a deep and nuanced under- • Getting rich data is time consuming.
a new issue area and uncover respondents’ responses. The desire standing of how a person thinks and The team may have to visit a respon-
key themes to probe further. to appear in a certain light amongst behaves. dent several times to establish the
ones peers can influence answers. trust required for them to speak
• Understand the dynamics within • Learn about the environments in
openly and honestly.
a community, quickly profile • Certain individuals may dominate, which respondents live, work, and play.
several individuals, and identify and if they have some kind of • People may not want to reveal their
• Develop relationships with potential
respondents to follow up with at formal or informal status within true feelings or may simply misreport
target users, with whom the team can
greater depth. the community, their presence may details for any number of reasons.
return to help design, prototype, and
prevent others from speaking their Thus, it is important to validate
• Establish shared research test solutions.
minds. findings across several respondents.
experiences and common
reference points among the team.
Requires:
• A private setting to minimize the
Requires: influence of other people.
• A facilitator who is socially Be • A significant amount of time, Keep it
perceptive, firm, able to synthesize inclusive. at least one hour. small.
quickly, and has the ability to spark
and sustain discussion. • A skilled interviewer that has deep
The value of group interviews di- familiarity with the research For individual interviews,
minishes if there are more than 10 topics, and ideally an accompanying there should be ideally one or
respondents. But when entering a note-taker/photographer. two, and certainly no more than
new context, group interviews that three researchers, present.
are open to all signals interest in
all perspectives, and helps diffuse Any larger a group and it can
tension when specific individuals become intimidating for the
are chosen for interviews. respondent.
40 Design Research for Media Development 41Phase 1 // DEFINE // Research Methods
OBSERVATION HOME STAYS
Observing the activities in a Extended (and usually overnight)
particular context, or the actions stays in the homes of your target
of an individual or group of users, where interview and
individuals, to gain insight into observation methods integrate
the subtle forces that shape a with natural social interaction.
place or a people.
Used To: Challenges: Used To: Challenges:
• Gain personal, experiential under- • Time constraints often do not allow for • Deepen understanding of a target • Homestays can raise expectations
standing of a context, community, long-term observation—only when the user group by observing the finer among respondents—after living for
or individual helps build an intuition researcher starts getting bored do they details and nuances of their lives. several days (or even a single night)
about that context; this then enables start noticing the minute contextual People tend to discard with pretenses with someone, the line between
you to make better design decisions insights that are most revealing. and act more naturally the longer a researcher and respondent can blur.
for that context. Yet project timelines rarely afford the guest spends with them. It is critical to address expectations
space for a team member to ‘grow upfront, before the interaction begins.
• Identify otherwise unarticulated needs • Achieve a more holistic understanding
bored’. Thus, researchers must be
and actions that may be subconscious of community dynamics. By living in a • There may be personal risks to the
constant, vigilant observers that are
to respondents themselves; what community for several days, 24 hours researcher: Depending on the con-
attuned to, and documenting, the ebbs
people think they do is often different a day, greater exposure to diverse texts, there may be health concerns
and flows of the environment.
from what they actually do. rituals and opportunities for engage- (e.g. hygiene, food preparation) and
ment will present themselves. security challenges, so heed the
advice of the local team.
Requires:
Requires:
• Patience. Observation takes a long
time. It is impossible to visit a hospital • A high degree of sensitivity to how
for an hour and understand all that Try your presence may impact your hosts.
happens in that environment, and why. shadowing. The presence of a guest can be a
financial and even emotional burden,
• Confidence. Observation requires
Shadowing is much like Observa- in low-income communities. Thus,
researchers to trust their own instincts,
tion—it requires immersion and the be sensitive to your hosts’ cues, and
and to believe their own insights have
focused study of people. Unlike compensate for any inconvenience
value. Such confidence grows with
Observation, Shadowing asks users you pose (e.g. bringing food as a gift)
experience.
to be active participants in the or understanding when to abandon
research process by “thinking out (e.g. if it seems that it is straining the
loud”; that is, by articulating their family’s resources and energy).
thoughts as they conduct actions.
While an imperfect window into the
human mind, it yields oft hard-to-
get insights into factors that direct
or influence behavior.
42 Design Research for Media Development 43You can also read
