From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab

 
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
From Design to Impact Assessment
Prof. Rafael A. Calvo

           @Rafael_A_Calvo
                                   1
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
Go to http://menti.com
                use the code 11 35 66

What is your background?
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
Agenda

• HISTORY – From unbridled progress to human impact
• APPROACH – How do we assess human impact?
• PROCESS – A process for responsible technology
• METHODS – Values sensitive design and other methods for understanding impact
• CASE STUDIES – Examples in practice

                                                                                 3
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
HUMAN IMPACT ASSESSMENT

     HISTORY
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
7
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
9
From Design to Impact Assessment - Prof. Rafael A. Calvo @Rafael_A_Calvo - Frankfurt Big Data Lab
Industrial Revolution

                                INPUTS (depletion)

   ENVIRONMENT                                                          ECONOMY

NATURAL RESOURCES                                                    TECHNOLOGY
   land, water, air

                              OUTPUTS (impacts)

      Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
Circular Economy

                                INPUTS (depletion)

   ENVIRONMENT                                                           ECONOMY

NATURAL RESOURCES
                                                                     TECHNOLOGY
   land, water, air

                               OUTPUTS (impacts)

      Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
OUTPUTS + WASTE (impacts)

Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
Infographic: Wikimedia Commons, image credit: RawPixel   13
"Data is the new oil"
         - Clive Humby

                         14
15
INPUTS (depletion)

          ENVIRONMENT                                                          ECONOMY

NATURAL RESOURCES          HUMAN                                            TECHNOLOGY
   land, water, air       attention,
                        behavior, data

                                     OUTPUTS (impacts)

             Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
HUMAN IMPACT ASSESSMENT

   APPROACH
Human Impact Assessment for Technology - HIAT

         Calvo, R.A., Peters, D. & Cave, S. Advancing impact assessment for intelligent systems. Nat Mach Intell, Vol 2, 89-91 (2020).
Differences between traditional EIA and HIAT

                               EIA    HIAT
    • Slow or static once installed   • Dynamic, Self-learns
       • has boundaries, regional     • Unbounded
            • Largely anticipatory    • Continuous & Iterative
            • Nature as resource      • Humans as resource
                     • Long history   • New to technology
Go to http://menti.com
                        use the code 11 35 66

       What values do you think a
Human Impact Assessment should protect?
Go to http://menti.com
                                  use the code 11 35 66

What would you measure to judge if the values are being
                    satisfied?
Technologists & Users              Doctors and Patients

                     •   Do no harm
                     •   Wellbeing
                     •   Human Autonomy
                     •   Fairness

Engineering Ethics                        Biomedical Ethics

                                                         22
“All Technology
should be designed to support                                             POSITIVE
                                                                            COMPUTING
psychological wellbeing”                                                  Technology for Wellbeing
                                                                          and Human Potential

                                                                                                     Rafael A. Calvo
                                                                                                     Dorian Peters

           R.A. Calvo and D. Peters. (2014) “Positive Computing: Technology for wellbeing and human potential” MIT Press.
Speed-dating with key psychological theories

    MODELS OF PSYCHOLOGICAL WELLBEING
    1.   Clinical DSM Global Assessment of Functioning (e.g. CES-D < 16)
    2.   Positive Psychology (e.g. Fredrickson, Seligman, Lyubimorski,
         Huppert, etc. )
    3.   Socio-emotional intelligence (e.g. Salovey, Mayer & Caruso)
    4.   Self-determination Theory (Ryan & Deci)
    5.   Subjective well-being (e.g. Diener, Kahneman)
    6.   Neuroscience and Physiology (e.g., Panksepp, Davidson)
Basic Psychological Needs & SDT

     1.   Autonomy - feeling willingness and volition in action

     2.   Competence - feeling able and effective

     3.   Relatedness - feeling connected to others
The Spheres
of Technology
Experience

                                                                                         Credit: Dorian Peters, Rafael A. Calvo, Richard, M. Ryan, 2019
           Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience.
           Frontiers in Psychology, 9(MAY).                                                                                       @Rafael_A_Calvo
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
Adapted from: Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital
experience. Frontiers in Psychology, 9(MAY).
METUX
                SPHERE                                PSYCHOLOGICAL NEEDS IN CONTEXT                                        DESIRABLE OUTCOMES                              MEASURES

   Adoption                                • To what extent is adoption of the technology autonomously              • Adoption of the technology (purchase,       • ACTA
   The decision-making experience            motivated?                                                               download, use)
   between becoming aware of a new         • To what extent does a potential user anticipate need satisfaction
   technology and acquiring it.              in relation to technology

   Interface                               • To what extent does direct interaction with the technology (i.e.,      • Sustained engagement with technology,       • TENS-Interface
   The experience of interacting with a      via the user interface) support psychological need satisfaction?       • High usability scores,
   technology via its interface during                                                                              • User satisfaction
   use.

   Tasks                                   • To what extent does engagement in technology-specific tasks            • Sustained engagement with tasks and         • TENS-Task
   The experience of engaging in a           support psychological need satisfaction? (e.g., tracking, text           technology,
   technology-specific task.                 chatting, etc.)                                                        • User satisfaction

   Behaviour                               • To what extent does the technology improve psychological need          • Engagement                                  • Assessments of psychological
   The experience of engaging in a           satisfaction with respect to the behavior that the technology is         (with behavior)                               need satisfactions in relation
   behaviour (that a technology is           intended to support? (e.g., exercise, managing a chronic disease,      • Satisfaction                                  to behavior (e.g., PNSESi)
   intended to support).                     communicating with friends, speaking a second language.)                 (with the behavior)                         • Assessments of behaviour-
                                                                                                                    • Behavior-specific outcomes (e.g., weight-     specific outcomes (e.g.,
                                                                                                                      loss, symptom control, learning               weight measure)
                                                                                                                      outcomes)
                                                                                                                    • Greater wellbeing during behavior.

   Life                                    • To what extent does the technology influence the user’s                • Greater life satisfaction,                  • BPNSii,
   An individual’s overall experience of     experience of psychological need satisfaction in their life overall?   • Increased wellbeing,                        • Other validated measures of
   life including all that is outside or                                                                            • Thriving/flourishing                          flourishing
   beyond the technology.

   Society                                 • To what extent does the introduction of the technology impact          • Increased measures of societal wellbeing,   Population measures such as the
   The experiences of all members of a       on societal wellbeing?                                                 • Societal flourishing                          FSiIi or SLWS iv
   society beyond the users of a
   technology.

               Peters, D., Calvo, R. A., & Ryan, R. M. (2018). Designing for motivation, engagement and wellbeing in digital experience.
               Frontiers in Psychology, 9(MAY).
METUX

Providing evidence for the utility of the model in practice is ongoing.
HUMAN IMPACT ASSESSMENT

    PROCESS
Go to http://menti.com
                        use the code 11 35 66

What development process do you follow?
A design process to support wellbeing, and ethics

                                                    38
A design process to support wellbeing, and ethics
             Research: investigate the needs, preferences, contexts,
             and lives of the people who will be served or, otherwise,
             impacted by a technology.

             Direct and Indirect Stakeholders

                                                                     39
A design process to support wellbeing, and ethics
                     Insights: analysis of the data from the
                     research phase, and synthesis into specific
                     insights for design.

                     • Wellbeing theory & opp for design
                     • Ethics: identifying potential biases,
                       ethical risks, and tensions.

                                                                   40
A design process to support wellbeing, and ethics
Ideation: divergent generation of
ideas for design solutions.

Ethical reflection can be integrated into
the ideation phase through framing.

                                                    41
A design process to support wellbeing, and ethics
Prototypes: Converge and build various design
solutions.

Responsible impact analysis involves collaborative
speculation on the wellbeing and ethical impacts
(good and bad) to which a particular design concept
may lead.
This will ideally involve a wide range of stakeholders
including end users.

                                                         42
A design process to support wellbeing, and ethics
Evaluation: during actual use

Wellbeing impact evaluation involves evaluating the impact of
technology use on a user’s psychological experience during and
after use.
Ethical impact evaluation involves evaluating the ethical impacts of
a technology’s use, not just on its users, but often, also on those
indirectly affected, such as their friends and families, communities,
society as a whole, and the planet.

                                                                        43
Environmental Impact Statements
Think before you do it.
EIA Process – a comparisson

1.   Screening. Does the proposal require an EIA?
2.   Scoping. What environmental impacts need to be examined?
3.   Prediction. What is the size & magnitude of the likely impact?
4.   Assessment. Is the impact significant?
5.   Mitigation. What can be done to address the adverse impact?
6.   Review. Is the assessment adequate?
7.   Approval decision. Should it be approved?
8.   Follow up. What occurred since the implementation?
Software ‘implementations’ Which on we do we
                           regulate?
                      Adaptation
                                        Enterprise
                       to vertical
                                           user
                         market
                                        e.G Gov A
                      e.g security
      AI algorithms
     and framework
        e.g. facial   Adaptation
                      to vertical      Enterprise
       recognition                        user
                        market
                      e.g health     e.g Hospital X

                      Adaptation
                      to vertical       Enterprise
                        market             User
                          e.g             Corp Z
                      marketing
S: software project
Innovation costs                                 I: Infrastructure project
         Core Tech           Market                  Client
                          Adaptation
Cost €                                                        Enterprise
                           to vertical
                                                                 user
                             market
                                                              e.G Gov A
              S           e.g security
          AI algorithms
         and framework
                                                        I
                          Adaptation
            e.g. facial
           recognition        S
                          to vertical                 Which
                                                          Enterpriseon we do we
                                                             user
                            market
                              I
                          e.g health                  regulate?
                                                        e.g Hospital X

                          Adaptation
                          to vertical
              I                                               Enterprise
                            market                      S        User
                              e.g                               Corp Z
                          marketing
                                  Time / phase
Algorithmic Impact Assessment (AINow)
Public agencies urgently need a practical framework to assess automated
decision systems and to ensure public accountability

Its focus is on procurement, and the process:
• Pre-acquisition review: allows the agency and the public to identify.
• Initial agency disclosure requirements. evaluate potential issues of
  inaccuracy, bias, and harms to affected communities, and address these
  potential impacts
• Comment period. Opportunity for the public to engage
• Due process challenge period. path for the public to challenge an agency
• Renewing AIAs. renew AIAs on a regular schedule
                                              https://ainowinstitute.org/aiareport2018.pdf
Digital Catapult - Machine Intelligence Ethics

1.   Be clear about the benefits of your product or service
2.   Know and manage your risks
3.   Use data responsibly
4.   Be worthy of trust
5.   Promote diversity, equality and inclusion
6.   Be open and understandable in communications
7.   Consider your business model
Screening and Inclusion Lists

• Technology that promises to support health and wellbeing.
• Technology for which there is academic evidence that it may cause
  harm to humans or the environment
• Technology targeted to vulnerable populations

Positive: easy to apply
Negative:
• New tech might not be included
• Perceived as adding bureaucracy
• Salami slicing – “Make a rule and people will find ways to break it”
HUMAN IMPACT ASSESSMENT

    METHODS
Learning from Environmental Protection

Lessons from EPA:

     The process matters: it elicits behavioural change by
             proponents and other stakeholders

    See Jones, M., & Morrison-Saunders, A. (2017). Understanding the long-term influence of EIA on
  organisational learning and transformation. Environmental Impact Assessment Review, 64, 131-138.
Impact on direct stakeholders

Lessons from health:
          Measuring impact is difficult and expensive,
                       but necessary.
Impact Measures and stakeholders

Value sensitive Design provides useful methods to identify and work with
stakeholders

Direct. Those who will interact with the technology

Indirect. Others (including organisations, society)
HUMAN IMPACT ASSESSMENT

 CASE STUDIES
Case Study: Ada Lovelace Institute & Covid19 report
Ada Lovelace Institute (20 April 2020) Exit through the App Store? A rapid evidence review of the
technical considerations and societal implications of using technology to transition from the COVID-
19 crisis.

Examines the potential development and implementation of technical solutions to support
• symptom tracking,
• contact tracing and
• immunity certification

Evidence from experts for:
• Government
• Parliament
• Technology Provider and Developers

                                                                                                   56
Ada Lovelace Institute (20 April 2020) Exit through the App Store…
Contact Tracing
“The technical limitations, barriers to effective deployment and social impacts demand more
consideration before digital contact tracing is deployed.” ~Screening
Recommendation (Mitigation):
•         Government must establish an independent Group of Advisors on Technology in
    Emergencies to oversee the development and testing of any prospective digital contact
    tracing app.

“The effectiveness of a digital contact tracing app will be contingent on widespread public trust
and confidence, which must translate into broad adoption of the app. “
• In order to increase public trust and confidence, and guard against abuse and scope creep,
  Government should advance primary legislation ….

                                                                                                    58
Case Study: Headgear
Mental fitness in male-dominated workplaces

          USER RESEARCH   WIREFRAMES            MOCKUPS             PROTOTYPE      LAUNCH, ITERATE
          Participatory   Interaction design.   Look & feel, User   User testing    Pilot, clinical trial
          workshops       Wellbeing design      feedback
Participatory process
 •   Co-design Workshops with 62 Firefighters & Transport
     workers in urban, suburban and rural locations.
 •   106 additional workers filled out a questionnaire
 •   Focus groups held with postal workers
 •   Ongoing feedback from organization representatives.

   Peters D, Deady M, Glozier N, Harvey S, Calvo RA “Worker Preferences for a Mental Health App Within Male-Dominated Industries: Participatory Study” JMIR Ment
   Health 2018;5(2):e30
  M. Deady, D. Peters, H. Lang, R. Calvo, N. Glozier, H. Christensen, S. B. Harvey; Designing smartphone mental health applications for emergency service workers. Occup
  Med (Lond) Volume 67, Issue 6, 1 August 2017, Pages 425–428
                                                                                                              @Rafael_A_Calvo
Impact: RCT results
         2,271 participants – Largest RCT of a workplace mental health app

                   Cases of Depression (PHQ)
Study 2: Youth Mental Health
NLP, Social Media Classification & Conversational systems
Project: Moderator Assistant (版主助手)
Helping those who help others

                     Liu, Calvo, Davenport, Hickie “Moderator Assistant: helping those who help via online mental health support groups”.
                     Social Technologies for Health and Wellbeing’ workshop at OzChi 2013.
Some results
• The algorithm achieved 84% f-measure in identifying content that required a
  moderator response.
• Between prelaunch and post-training periods, response ratios increased by 0.9,
  4.4, and 10.5 percentage points for messages labelled as crisis, red, and green,
  respectively, but decreased by 5.0 percentage points for amber messages.
• Response latency was significantly reduced (P
Global Initiatives
An enormous list
• AI Now Institute
• IEEE Ethically Aligned Design (IEEE Standards P7000, P7010, P7005 etc)
• Z-inspection: Towards a process to assess Ethical AI (Frankfurt Big Data Lab)

                                                                                  67
Failures in responsibility
CannyAI VDR; Face Replacement as a Service or “Fake News as a service”

https://www.fxguide.com/quicktakes/cannyai-vdr-face-replacement-as-a-service/
https://www.youtube.com/watch?v=y_hthFKMxgw

                                                                                68
How can we prevent irresponsible design
and engineering from happening?

         • Regulation,
         • Technology Push &
         • Market Pull

                                          69
The importance of organizational capabilities

 In addition:

 Firms need the organisational, technological and marketing capabilities to
 respond to regulatory pressures and customer demand related to environment
 with innovation whilst

 Others fail in the face of external pressures (regulation and customer demand).

Demirel, P., & Kesidou, E. (2019). Sustainability-oriented capabilities for eco-innovation: Meeting the
regulatory, technology, and market demands. Business Strategy and the Environment, 28(5), 847-857.
You can also read