The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
The Art of Data Portraiture: Enabling a Public Debate on
Self-surveillance
Dr. Ralph Kenke
Lecturer, University of Newcastle
ralphkenke@gmail.com
Dr. Elmar Trefz Mark Roxburgh
University of Newcastle Associate Professor, University of Newcastle
elmar@semnon.net mark.roxburgh@newcastle.edu.au
Mario Minichiello
Professor, University of Newcastle
mario.minichiello@newcastle.edu.au
Abstract The name implies changes in artistic practice,
This research is a practice-based speculative influenced by industrialisation, and the resulting
design inquiry into the emerging field of data mass production of consumer goods, which has
portraiture. Humans’ use of the networked digital influenced Western societies’ understanding of
environments that are now so much part of life art and creativity in a broader sense. Warhol’s
leaves a massive data trail stemming from portraits explored different mediums and often
individuals’ everyday interactions with these showcased celebrities and other artists. They
environments. An increasing amount of this data challenged the conventional understanding of
trail remains invisible. Although we spend a the portrait as a single, still image.
significant amount of time participating in digital He understood that the portrait at its time of
network activities, we have just begun to discover creation is fundamentally connected to the
the potential of visualizing personal data as a medium that carries the image, which influences
graphical representation. This research into the the interpretation of the portrait. Warhol
emerging field of data portraiture seeks to envisioned that the medium and the lifecycle of
understand the role of the “artist” as a creative a portrait as an image would take on new forms,
practitioner in interpreting qualitative data into when he said his famous words that in the future,
image experiences, and to offer insights into the everybody would have 15 minutes of fame. It
behaviour and interests of individuals engaging seems as though he almost predicted a future in
with such work. Through a number design which everybody has access to sophisticated
iterations, this research investigates and reveals technology, such as computers and
the importance of participant contribution to the smartphones. Through contemporary network
“datafication” of social life and the emergence of technology and smartphones, today’s social
“self-surveillance” that can shape data portraits. media platforms enable individuals to broadcast
their opinions, photographs, and videos
The transformation of photographs into immediately to the world. The internet provides
Big Data access to a large volume of image data, such as
This practice-based project was named Selfie selfies, from smartphone users. The Instagram
Factory to reflect the phenomenon of the application programming interface (API)
“selfie” photograph as an act of self- enabled us to access publicly available selfies,
surveillance, as well as Andy Warhol’s art which added a reader-driven element to our
studio, known as “the Factory.” Many of prototype. The phenomenon of selfies has
Warhol’s artworks were created in the Factory. occurred in part because of the Instagram online
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 63
Part I. Full Papers
platform. Today #selfie is the most popular preserved; they are a temporary statement in a
hashtag used to describe a self-portrait taken telematic society. However, selfies endure
with a smartphone and posted on Instagram. because their data are stored online and cannot
Network technology offers access to be erased. Sharing and at the same time
sophisticated infrastructure that enables recording social activities online—by posting
individuals to take self-portraits and immediately selfies, for example, is a contribution to the
share them with millions of users on their datafication of social life. The datafication
smartphone screen. This phenomenon enables surrounding social activities in the online
users of these devices to briefly experience environment is perceived critically: even “for
mediated fame, similar to what Warhol media scholars . . . the shift to a data-rich
envisioned—except perhaps that users gain only environment poses challenges for a robust
15 seconds rather than 15 minutes of fame, understanding of how agency and expression
considering the volume of selfies and the might still work within that environment”
frequency at which they are posted every few (Couldry and Powell 2014). In particular, one
seconds. Selfie Factory does not reveal many of may ask what agency individuals have over their
its computational features as an installation. personal data use, captured online; and whether
Rather, it hides the intelligent parts that operate its there is a way to express one’s concern about the
mechanical and technological features. The use or abuse of personal data other than within
intention behind the installation is that the the temporary cycle of sharing personal data on
observer can explore the images and their the platform from which it may be gathered.
automated process, and experience a telematic Some critics have questioned whether selfies
performance with as little distraction as possible. are more than simple fun and a method of
To achieve this, we provided a “reader-driven” expression. From a psychological perspective,
element to it, enabling exploration (Segal and Sunitha (2016, 151) considers that “the present
Heer 2010), which according to Cairo (2016, 31), generation is technology-addicted and selfie-
allows us to “conceive a data-driven obsessed. Efforts need to be made to educate
representation as a tool that lets people extract everyone about how social networking is
their own conclusions from the data.” To facilitate leading to a culture of ‘popularity’ based on
such an approach and at the same time prevent materialism and giving way to unreal standards
distraction, we decided to hide the Raspberry of physical appearance.” Although we agree
computers and any tactile features that indicated with the concerns raised by Sunitha regarding
technological equipment and to enable the use of selfies and the addictive nature of
engagement only via a smartphone. Some technology, we are fascinated by the immediate
individuals post on Instagram to increase their experience and the need to share one’s data,
popularity, while others use Instagram to stay in given that millions of people take self-portraits
contact with friends or follow their idols. Selfies with their smartphones. Our research is less
are often an expression of the moment—a way concerned with the selfie as a visual artefact;
of sharing that the individual is experiencing rather it views the selfie as a reference frame of
something worth photographing: datafication resulting from “the ubiquitous
quantification of social life” (Mayer-
unlike traditional portraiture, selfies don’t Schönberger and Cukier 2013, 78). The selfie is
make pretentious claims. They go in the a vehicle that carries personal data freely shared
other direction—or no direction at all. online through platforms such as Instagram,
Although theorists like Susan Sontag and with little resistance. We question what might
Roland Barthes saw melancholy and signs happen to image data once an image’s
of death in every photograph, selfies temporary message of personal expression
aren’t for the ages (Saltz 2014). expires. The selfie is a great example and an
appropriate reference for the increase in the
As author Jerry Saltz (2014) points out, selfies volume of personal data shared online. However,
are important right now; they are not made to be this increasing trend of sharing personal data on
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
64 School of Creative Media, City University of Hong Kong 10-14 June 2021
The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
free platforms such as Instagram is a the individual’s willingness to freely share
transactional exchange between individuals personal data online.
taking advantage of a convenient way to upload The massive increase in the volume of data
selfies and a company obtaining valuable data available for our prototype seemed to present
on users. This disconnect in the transactional both a challenge and an opportunity to explore
relationship taking place, which operates within the dynamic dimensions of Big Data. It allowed
a sophisticated system, can be difficult for users us to explore a new medium and investigate the
to comprehend. Moreover, the issue is not indexical properties of portraiture in an
unique to image data from selfies: “Big Data automated context using selfies. According to
technologies and the growing relevance of Rubinstein (2015a), “the self is not permanent
algorithms may disconnect system and and solid; rather it is dependent on continuous
experience . . . because the traces of data people reinvention and adaption.” Considering this
leave behind are often unconscious and not argument, it occurred to us that selfie data on
meaningful to them, and the insights generated Instagram were appropriate for the prototype.
by companies or governments are not, or only They would enable us to utilise the
partially” (Couldry and Powell 2014, 5), “folded undistinguishable qualities of selfie data and the
back into the experience of everyday life” adaptive nature of the self in a data portrait
(Baack 2015, 2). capable of demonstrating our divided self; that
Regardless of the factors that motivate is, a self that is a split representation involving
individuals to post and share images, an the online self and our actual self, emerging as a
individual with a large follower group (known result of today’s telematic world in which we
as an “influencer”) is more likely to be displayed live. As selfies are shared constantly, we were
on Instagram’s ‘most recent’ page, than are confident they would provide a flow of data as
others. Rubinstein (2015a, 165) states that part of the automated process. As Rubinstein
“classifications of ‘good’ and ‘bad’ or ‘original’ (2015a, 175) indicates, “The selfie opens up a
and ‘copy’ are less important online than such possibility of a discourse about self and about
questions as how many ‘likes’ does a selfie get.” photography that is not bound to indexicality,
It is interesting that Instagram enables its users representation or memory but instead suggests a
to view each other’s images, which I believe meditation on the forces of network expressed
changes the digital Instagram platform into a through the plurality of fragments.” While
digital curated public space, where people Donath’s (2014) work reminded us that portraits
observe themselves and each other through are situated within a community, or even emerge
photographs. from a community, Rubinstein’s (2015a) view
We are intrigued by the fact that millions of reassured us about our decision to use selfies and
selfies are shared online everyday (Glum 2015) their dialectic meaning as a vehicle to iterate and
and people are willing to provide their data— learn through prototyping. We decided to
even via a digital portrait to correlate with their display the data because they were creating the
personal identity—with little concern. Thus, we image and displaying the time and date of selfies
began to conceptualise the direction of the Selfie as shared on Instagram. The literature identifies
Factory prototype. However, it is not just visual two forces at play in a selfie: it points to
data that are contained in a selfie; the literature someone in space and time, and yet it is instantly
suggests that a selfie carries a rich set of data, distributed among different spaces, appearing
both qualitative and quantitative. Rubinstein on the screens of multiple Instagram users.
(2015a, 173) suggests that “the defining quality Our intention was to highlight the high
of the selfie is its instant shareability: Its logic frequency of selfies taken and shared online, as
does not distinguish between the act of ‘taking,’ well as how accessible the data is to the public.
‘making’ or ‘snapping’ and the act of uploading By printing a selfie, observers can experience
and sharing.” The simple act of creating a selfie the data in the physical form of paper. However,
relies on sophisticated network technology and a printed selfie is only a by-product of the data
that we aimed to highlight; that is, the time and
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 65
Part I. Full Papers
date of the shared selfie entering the Instagram designed software began to connect to Instagram
database and its relationship with the qualitative via WiFi, the five units began sequentially
data. As Rubinstein (2015a, 171) states, “the printing in a 10-minute cycle (to prevent
selfie . . . encapsulates the present moment as overheating), which continued for about 24
‘ecstatic’ temporality.” The printed image to hours. The thermal printer began to print the
which we applied “direct visualisation” to rasterized greyscale selfies as they were shared
maintain context was like a signature confirming publicly online, and the thermal paper slowly
that the event of the selfie had occurred. In made a long trail of selfies flowing from the
Rubinstein’s (2015a, 175) words, “what the units, elevated two metres above the floor. As
selfie shares is the possibility of detaching the the five paper trails dangled down, people were
image from its foundation in platonic able to explore the selfies until the paper trails
metaphysical unity and the chance of eventually curled up in a pile on the floor.
overcoming the representational force of People could also use their smartphones to share
photography.” We partly agree with a selfie via Instagram and experience the
Rubinstein’s view and suggest that the selfie’s installation from a participant’s point of view.
photographic indexicality is tied to a The installation was exhibited in four galleries:
representation of a social phenomenon, rather Watt Space Gallery, Testing Grounds,
than abolishing its indexicality completely. Campbelltown Arts Centre, and the National
While Rubinstein describes the loss of the Portrait Gallery.
indexicality of the photograph through the
mediation of network technology and the
plurality of fragments of the selfie, we argue that
it acts as a facilitator pointing to a social
phenomenon, rather than functioning as causal
indexicality of a single individual. With this in
mind, we began to explore the scale and data
traffic of selfies shared on Instagram, which
eventually became the source of personal data
for a third (and final) prototype. Selfie Factory
underwent three major iterations, described in
the following section. Each iteration reached a
point of exhaustion in terms of ideas, designs,
software, or hardware capabilities until the final
stage was reached as a functioning media
installation.
Designing a data portraiture machine
Fig. 1. Selfie Factory, Ralph Kenke and Elmar Trefz, 2016,
Before we explain in further detail the Sydney (Campbelltown Arts Centre), Australia. Mixed media.
conceptual approach linked to our theory and the Copyright Ralph Kenke, 2016, all rights reserved.
installation’s iterative process, we provide a
brief overview of the installation’s function that The first attempt to design Selfie Factory
determined its final physical format. The final soon revealed (fig. 2) that the format or medium
installation consisted of five small to represent the dynamic nature of the
computational units, each of which contained a accumulation process would require a different
Raspberry Pi computer connected to the internet approach than simply displaying images on a
via WiFi, and a 30 cm diameter roll of thermal screen. In a way, Selfie Factory acts as an
paper, which fed into a thermal printer (Figure artefact—an image experience, pointing to a
1). The five units were mounted parallel to each possible future speculative design. It presents an
other traversing along the wall with a 20 cm alternative present and suggests replacing the act
space between them. As the custom-made and of taking and looking at photographs made with
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
66 School of Creative Media, City University of Hong Kong 10-14 June 2021
The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
smartphones with the act of recording and that facilitated debate about the possible future
sharing data. It aims to shift our perspective of data portraits, rather than proposing a ready-
away from the behavior we associate with made solution. We wanted to build a prototype
photography and perceive it as self-surveillance. that demonstrated in real time the scale of data
It presents a speculative future in which we all being shared, which we believed was essential
participate in self-surveillance through data. in the debate about future data portraits
Rubinstein (2015b, 17) expressed a similar and self-surveillance.
viewpoint:
An iteration of an interactive art installation
As photography becomes encoded in a to free the artist from reliance on vision
networked object, the emphasis shifts from The first iteration involved a prototype that used
considering it in visual terms . . . this in turn an automated accumulated procedure for data,
establishes photography as a kind of scrapping the Instagram API. This was
unstable surface that produces meanings important to gain a sense of the data volume
not through indexicality or representation, available online. With each iteration, the
but through the aggregation and the prototype and its conceptual manifestation
embodiments of data. refined the installation’s execution. Influenced
by decision making along the way, the final
result was a mixed media installation that
operated at intervals. People using their
smartphones to capture selfies often refer to the
act of taking a photograph when they point the
device at themselves. However, we learnt from
Rubinstein (2015a, 173) that the selfie “does not
distinguish ‘taking,’ ‘making’ and the act of
‘uploading and sharing;’ rather it encapsulates
the present moment as ‘ecstatic’ temporality,
while it is in fact the aggregation of data and the
embodiment of data shared online” (Rubinstein
2015a). Since the selfie’s characteristics are
distinct from the idea of taking a photograph,
our intention was to provide a form of tangible
evidence of personal data being shared on
Instagram, thereby demonstrating that the
aggregated data are automatically accessible to
the public, and providing a form of receipt that
the data have been noticed by the public after
Fig. 2. Selfie Factory, Ralph Kenke and Elmar Trefz performing a selfie.
(Prototype) 2016, Sydney, Australia. Mixed media. Copyright In some instances, challenges arising at the
Ralph Kenke, 2016, all rights reserved.
time influenced the prototype, and later
provided the effect we were seeking. For
Selfie Factory suggests a shift away from the example, the nature of automation and printing
tradition of photography, indicating that data in a real-time process (which was vital for us to
portraits facilitate the sharing of personal data emphasise the telematic experience in the
on a more comprehensive level than gallery space) forced us to use the cost- and
photographs can through social network time-efficient medium of thermal paper. The
platforms. The successes and failures of our first nature of the high volume of data shared meant
attempt made us realise that we had emphasised that we had to experiment with the paper supply
problem solving during the process and and elevate the thermal printer (fig. 3) to enable
forgotten our initial intention to create a project the printed trails to display enough images for
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 67
Part I. Full Papers
observers to notice the printer’s real-time with images on display eventually create a pile
response as the paper slowly made its way to the of paper that is messy and chaotic. This short
floor, while also allowing enough room for an lifecycle of the data in the form of an image is
audience in a gallery to explore the installation manifest in the kinetic nature of the
and its printed images. installation’s chosen medium. Second, thermal
The transition of the telematic experience paper is a non-permanent medium, meaning that
from the mobile screen (which is commonly printed images fade over time (within three to
used to interact on Instagram) to the temporary seven years, depending on the temperature at
medium of thermal paper (fig. 2) was a which the paper is kept). This non-permanence
statement about the images and their use; they is a statement on the uncertainty around what
fade from sight shortly after being posted on happens to the data we leave online. We know
social media. Although they are stored on Cloud we can view the data online, yet so can others,
servers, images such as the selfie disappear from and wondering what others do with these data
most smartphone screens shortly after they are can cause discomfort. We have only a vague
posted, and become just another image in a understanding of the degree to which Cloud
massive stream of data. storage keeps track of our data and how the
accumulated data may be used in the future.
When viewed from a distance (fog. 3), the
media installation appeared the same at each
venue, yet its displayed content was never the
same when viewed in detail. The selfies hanging
from the wall at the Campbelltown Arts Centre
were not the same as the selfies shown at the
National Portrait Gallery, which means that an
observer would never see the same image in
both galleries, although the media installation
remained the same. The artist was not in control
of the continuously changing content appearing
Fig. 3. Selfie Factory, Ralph Kenke and Elmar Trefz
(exhibition at Testing Ground) 2017, Melbourne. Mixed
in the gallery because the content was entirely
media. Copyright Ralph Kenke, 2017, all rights reserved. the responsibility of the individual subjects
exercising self-surveillance by taking and
The installation created awareness of this sharing selfies online. The subjects viewed their
topic in two ways. First, it constantly printed selfies on a smartphone and approved their
images posted on Instagram from all around the visual appearance without me—the artist—
world, moving slowly down a thermal paper trail. being present.
The installation displayed the images in an I argue that in this instance the artist is, in a
orderly manner on five paper trails next to each sense, free from a core reliance on vision. We
other, thereby enabling easy comparison of the essentially provided a framework for an existing
images. The Selfie Factory revealed little about cultural activity—the selfie—to be documented
the mechanics of the installation and focused and displayed in an altered format through a
simply on the ever-changing content on display. different medium than anticipated by the
In a debate at the Campbelltown Arts Centre, participant and observer. This transformation in
participant Majidah told her mother Malika, “I medium and content situates the selfie as a
love that it uploads itself as things get posted. portrait in a new context. In Donath’s (2014)
Because that’s what it is; everything is so words, “the photographer’s eye and intention
instantaneous in our culture.” Malika agreed, remained actively involved, but creating the
stating, “After a minute is gone, it is interesting image itself become the job of the machine. The
and then it ends up in a pile; it’s from fifteen installation invited individuals to post their own
minutes of fame to one minute of fame” (quoted data by sharing their images on Instagram (fig.
in Ayres 2018). However, the fragile paper trails 4) using the hashtag #selfiefactory while in the
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
68 School of Creative Media, City University of Hong Kong 10-14 June 2021
The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
gallery to experience their own image being viewing the images of others. This playful
printed by Selfie Factory. exploration of their own embodiment within a
group of other bodies or portraits through
interaction with an artwork is what media artist
Lonzan-Hemmer described as (quoted in Stern
2011) “tele-absence” in his participatory media
installation Body Movies (Stern 2008). The
experience with Selfie Factory amplified the
often-isolated experience of sharing and
viewing a selfie on the small screen of a
smartphone.
Fig. 4. Selfie Factory, Ralph Kenke and Elmar Trefz
(interacting participant) 2016, Watt Space Gallery. Mixed
media. Copyright Ralph Kenke, 2016, all rights reserved.
The images appeared on Instagram’s online
platform, while simultaneously being
documented on thermal paper dangling on the
gallery wall. As intended at the beginning of my
research, Selfie Factory functioned as a
perceptual bridge (Auger 2013) across the gap
between the online and physical image
experience. According to Nathaniel Stern’s
reading of Brian Massumi’s writing, Parables
for the Virtual, on the subject of the body: “It
moves as it feels, and it feels itself moving”
(2008, 22). In this way, Selfie Factory provided
observers with a different perspective of a
familiar activity on a larger scale and displayed Fig. 5. Selfie Factory, Ralph Kenke and Elmar Trefz
in an altered medium. (participant posting his selfie on Instagram) 2017, Watt Space
More importantly than documenting and Gallery. Mixed media. Copyright Ralph Kenke, 2017, all
rights reserved.
transforming data in the form of an image (fig.
5) via a screen-based setting into a tactile media
installation, Selfie Factory places the experience
in a new context by making it available online
and changing it into a tangible group portrait of
Instagram users performing selfies in a gallery
setting (fig, 4). It amplifies the usual palm-sized
image of a selfie (fig. 6) into a sculpture-like
media installation.
The transformation of data from an
intimately sized experience on a smartphone
screen held in an individual’s hand suddenly
enabled groups of people to view the posted
images in a new context—a representation
where everything was seen. Participants
interacting with Selfie Factory online via
Instagram while at Watt Space Gallery
considered the two different layers of Fig. 6. Selfie Factory, Ralph Kenke and Elmar Trefz
interactivity that invited visitors to participate by (participant re-posting his selfie on Instagram after the print
either posting their own image or simply
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 69
Part I. Full Papers
appeared) 2017, Watt Space Gallery. Mixed media. Copyright speculates on an alternative future (Dunne and
Ralph Kenke, 2017, all rights reserved.
Raby 2013) with the potential to comment on
The difference between the framework of self-observation through data. More importantly,
Selfie Factory (which made the observer a it explains what Flusser referred to as “the
participant) and the framework of Instagram on magical nature of images,” which is that they
which the work was built, is that Selfie Factory replace events by states of things and translate
automatically employed the motion of scrolling. them into “scenes.” The scenes not only emerge,
On a smartphone, an Instagram user participates but can also be re-enacted by the participant,
by taking and observing their selfies by using which in Selfie Factory was demonstrated by
their finger to scroll through them on the screen. people pointing their smartphones towards
In the case of Selfie Factory, this act of scrolling themselves to take and share a selfie (fig. 8).
was automated through the kinetic experience of This combination caused friction between the
images slowly moving down a paper trail. embodied experience and the temporary image,
The combination of selfies being printed thereby creating what Deleuze referred to as a
instantly as people posted them on Instagram “time-image” (1988).
and the movement of the printed selfie amplified
the automation (fig. 7) to the point where it
removed the scrolling gesture from this cultural
activity. With its simulation of smartphone
participants scrolling on Instagram, the work
commented on a gesture in the telematic culture
that has emerged in our society.
Fig. 8. Selfie Factory, Ralph Kenke and Elmar Trefz
(participants: mother and daughter performing a selfie at the
exhibition, Campbelltown Arts Centre, Sydney). Mixed
Media, Copyright Ralph Kenke, 2017, all rights reserved.
When participation in self-surveillance also
contributes to an artwork
The installation captured the evolutionary
development of social engagement and
communication in the increasingly digitally
dominant environment (fig. 9) that shapes our
current state of online behaviour. It is no longer
a matter of identifying the act of self-
surveillance, but more a question of negotiating
between the act of photography
and self-surveillance. The level of connectivity
created by digital network technology has forced
individuals to operate and perform in everyday
Fig. 7. Selfie Factory, Ralph Kenke and Elmar Trefz, National
Portrait Gallery, Canberra. Mixed media. Copyright Ralph
life, and today has reached an irreversible height:
Kenke, 2017, all rights reserved. “Statistics confirm the assertion that the
datafication of almost everything is growing
This aligned with our research method of relentlessly” (Kennedy 2018).
speculative design, where the creative work is in
fact speculative design—a future artefact that
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
70 School of Creative Media, City University of Hong Kong 10-14 June 2021
The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
“More than 200 million people obtained their and Selfie Factory aimed to highlight this state
first mobile device in 2017, and two-thirds of the to participants through the re-enactment of
world’s 7.6 billion inhabitants now have a taking and sharing a selfie. Therefore, the
mobile phone” (Kemp 2018).Well over half of installation Selfie Factory demonstrated our
the world’s population is now connected online, collective acceptance of self-surveillance, while
with the latest data showing that almost a quarter questioning the rules we, as a society, establish
of a billion new users came online for the first around network technology.
time in 2017. Baudrillard (1994, 12) continued:
You are information, you are the social,
you are the event, you are involved, you
have the word . . . no more violence or
surveillance: only “information,” secret
virulence, chain reaction, slow implosion,
and simulacra of spaces in which the effect
of the real gain comes into play.
In the instance of Selfie Factory, observers who
volunteer as participants view their own
simulacrum—a simulation of existence in a
telematic society that is fluidly crossing from the
physical experience to the online environment,
back and forth—without any great effort,
providing context to an otherwise abstract
experience.
Fig. 9. Selfie Factory, Ralph Kenke and Elmar Trefz (collage
of 15 participants posting their selfies to engage with Selfie
Factory in 2016–18; screenshots from Instagram searching Artwork as a facilitator of public debate on
the hashtag #selfiefactory). Mixed media. Copyright Ralph self-surveillance and data portraiture
Kenke, 2017, all rights reserved.
Selfie Factory is aligned with the idea of “the
In addition, the web is expanding: “the practices and imaginaries of open data activists
Internet of Things, connected ‘smart’ devices which are centred around the distribution and use
that interact with each other and us while of data and thus linked to datafication, the
collecting all kinds of data, is exploding from 2 ubiquitous quantification of social life” (Mayer-
billion devices in 2006 to a projected 200 billion Schönberger and Cukier 2013). The aim of the
by 2020 and is one of the primary drivers for our installation was to expose the transformation of
data vaults exploding” (Marr 2018). As a the value of the individual —from consumption
consequence, the world and its people who to commodification—and the realization that
operate within this connected network have our digital identity is reduced to a product.
already abolished the critique that our world has By highlighting the tactile features of the
become a large panopticon. Baudrillard pointed medium, the thermal paper’s temporary nature,
out that the observer in front of a television (an fading over time, carried a message that is
earlier visual format of a telematic experience) relevant for the age of information in which we
has become a participator without noticing it: “a live today. Receipt paper is designed to carry
switch from the panoptic mechanism of information about sales transactions and is
surveillance (Discipline and Punish [Surveiller offered as proof to tax institutions of the
et punir]) to a system of deterrence, in which the exchange of goods and services for money. It is
distinction between the passive and the active is an item in a consumer system that has been
abolished” (1994, 11). Instagram abolishes this fundamentally important to the growth of
“distinction between the passive and the active,” capitalism in the Western world. In a sense, it is
a representation of capitalism: receipts are not
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 71
Part I. Full Papers
required for barter—where one person swaps an While industrialisation had to rely on the
egg for an apple, for example. Receipts are synchronisation of working hours, in the age of
crucial for tracing transactions and monitoring information, this centralised approach is no
exchanges. Capital systems rely on these little longer needed. The physical dimension in a
pieces of paper as much as they rely on the dollar telematic society connected through computers
bills that are used to communicate the exchange leaves behind the old structure of punch clocks.
of money for goods and services. Unlike ink-
based print techniques, thermal paper does not
absorb ink, but instead turns darker on the
surface when exposed to high temperature. As a
consequence, the printed information on
thermal-paper receipts fades after some years,
and the information is eventually lost. The
receipt is a mechanical product, which in the
context of Selfie Factory, references the passing
of the industrialisation that dominated social and
cultural behaviour during the twentieth century.
Selfie Factory (figs. 10, 11 and 12) is a
Fig. 10. Selfie Factory, Ralph Kenke and Elmar Trefz
manifestation of the transitional stage that (Campbelltown Art Centre exhibition, shortlisted for the
society is currently experiencing while Fisher’s Ghost Award) 2017. Mixed media. Copyright Ralph
romanticising industrialisation and struggling to Kenke, 2017, all rights reserved.
reach a digitalised age of information. During
industrialisation, machines were praised for
freeing individuals from exhausting labour and
elevating them to engage in more enlightened
work, thereby reducing physical stress on the
human body and enabling companies to
manufacture cost-effective products in larger
quantities. Applying this thinking to the age of
information, questions arise in relation to
personal data—for example, are our selfies
somehow enslaved to the gig economy of Big
Data and its markets, thereby causing us
existential stress? Lehtiniemi points out that
individuals may be unaware that they are
targeted by companies that capitalise on their
selfie data: “Surveillance capitalism monetises
data acquired through surveillance. It operates
on data extracted from users, turns extracted
data into behavioural predictions, and often
monetises them through markets that users
cannot participate in” (2017, 2). In hindsight,
industrialization had both benefits and Fig. 11. Selfie Factory, Ralph Kenke and Elmar Trefz
drawbacks; however, it was not the intention of (participant exploring Selfie Factory at the exhibition at Watt
Space Gallery) 2017. Mixed media, Copyright Ralph Kenke,
Selfie Factory to comment on this argument, as 2017, all rights reserved.
such an undertaking would be too ambitious.
The installation was a “perceptual bridge”
towards the potential of data portraits and their
future meaning and appearance, rather than a
statement on our history.
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
72 School of Creative Media, City University of Hong Kong 10-14 June 2021The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
Fig. 13. Selfie Factory, Ralph Kenke and Elmar Trefz (with a
gallery visitor at the National Portrait Gallery, Canberra)
Fig. 12. Selfie Factory, Ralph Kenke and Elmar Trefz 2017. Mixed media. Copyright Ralph Kenke, 2017, all rights
(participant capturing a re-posted selfie in Selfie Factory) reserved.
2017, Watt Space Gallery. Mixed media. Copyright Ralph
Kenke, 2017, all rights reserved.
However, this transactional relationship
In the online space, we experience the between the installation and the participant
expansion of the public space; hence, Selfie raises an important question: who or what was
Factory brought the online space into the public programmed—the media installation or the
space to facilitate interaction within the public participants directing their smartphone cameras
space of the gallery (fig. 13). Kennedy discussed towards themselves? One can think of this
the relevance of engaging the public on a scenario in a different way—it was not only
personal level in her 2018 publication Living Selfie Factory that was programmed, but also
with Data, in which she explained: the participants. Flusser (1983, 64) argues that
“we are manipulated by photographs and
We need to listen to the voices of ordinary programmed to act in a ritual fashion in the
people speaking about the conditions that service of a feedback mechanism for the benefit
they say would enable them to live better of the camera.”If this is the case for the camera,
with data, and in so doing, arm ourselves it may also be the case for computational power
with knowledge which advances data and its network technology, in which case we
studies and serves the interests of data are engaging in a ritual of data contribution on a
activism. global scale without much discussion of the
possible outcomes or disadvantages that such a
The installation Selfie Factory aimed to development may have for our society. Our
facilitate public debate using art (fig. 13) as a creative practice resonates with contemporary
manifestation of the future of our online philosophy regarding automation and
identities and the commodification of our data. datafication through computation. It implements
Posting and sharing selfies online can be an elements of data activism as a driver to inform
innocent act with no intention other than to the public on complex cultural matters such as
display a self-portrait online. As we saw with surveillance capitalism, datafication, and self-
Selfie Factory, only a smartphone and a few surveillance.
button clicks are required to enable the entire As Baack (2015) suggests, “we can turn to
world to view a photograph. The simplicity and research on its broader cultural significance and
popularity of posting a selfie online renders this influence beyond software development.” It was
act of self-surveillance with a selfie a telematic the intention of my research to use publicly
gesture that can be the start of a broader debate available data and open-source tools to develop
via Selfie Factory. an artistic project to include citizens in a
discourse on the subject of data. Selfie Factory
was an instrument of “data activism”—an
artefact that functioned as a mediator (fig. 14) of
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 73Part I. Full Papers
quantified social life (open data) to capture an technology today. More importantly, the process
image experience while facilitating self- and the engagement with Selfie Factory in
surveillance through a small-scale simulation of public showcased the complex nature of shaping
the way Big Data in action may appear. The data portraits and the nuances of enacting self-
artistic installation invited participation in the surveillance with technological means.
experience, as well as in a public debate. This Selfie Factory has influenced and provided a
was achieved partly by including a level of new perspective on our increasingly important
reader-driven elements, which is a common dual identity in two ways. First, the project
feature in the field of data-journalism and is used gained success and recognition by engaging
to demonstrate how the participation of an with public galleries and festivals, and earned
audience can result in different interpretation in several awards, which indicates its relevance in
a design. The artist’s or author’s role is to create the field. Second, the project attracted media
such a reader-driven framework so that coverage shortly before evidence emerged of
participants become their own protagonist, mass data surveillance conducted by the
steering away from a visualisation of company Cambridge Analytica via some of the
“prioritized expert knowledge or institutional most popular social media platforms, such as
knowledge over what Jeremijenko called Facebook. The company successfully recorded
‘layerperson knowledge’” (quoted in Abrams approximately 50 million user accounts and
and Hall 2003, 225). Because of its observer- their personal data, which are used to deliver
driven design, Selfie Factory is capable of acting targeted messaging (also known as
as a “perceptual bridge,” enabling participants to “psychographic targeting”) to influence political
experience the transitional nature of selfie data elections and referenda through social media. In
from the perspective of protagonists who can this final section, I provide a brief overview of
extract facts, connect events, and shape their these two levels of influence and how my
own conclusions. The installation thereby relied research may offer knowledge to the community
on “layperson knowledge” gained from their and the broader public.
prior experience with selfies and their ability to Selfie Factory was shortlisted for a Fisher’s
act within a technology network environment, Ghost Award at the Campbelltown Arts Centre
which is an essential part of our lives in a before receiving its most prestigious
telematic society. recognition—the Digital Portraiture Award at
the National Portrait Gallery in Canberra. This
award culminated in Selfie Factory being
exhibited at the National Portrait Gallery,
alongside the works of other Australian artists.
Selfie Factory’s impact reached an
unexpected magnitude, spreading the idea and
message to the public for further debate. This
Fig. 14. Selfie Factory, Ralph Kenke and Elmar Trefz, (at the message is relevant at a time when personal data
exhibition opening on the left, and approximately 24 hours
later with a pile of paper on the floor on the right) 2017, Watt are accessible to everyone—including
Space Gallery. Mixed media. Copyright Ralph Kenke, 2017, companies such as Cambridge Analytica—and
all rights reserved. can be used to influence our decisions and future
identities. Considering the facts mentioned
Conclusion above, Selfie Factory influenced media art
because of its reader-driven and participatory
In conclusion, this paper describes the iterative elements concerning the genre of portraiture,
design process for the interactive installation encouraging gallery audiences to engage with
Selfie Factory, and a shift in artistic practice art in the context of data portraits. Therefore, the
away from the reliance on vision and towards participants and observers could experience the
the use of automation, which can facilitate or digital divide our telematic society currently
trigger a debate on our relationship with inhabits through a “perceptual bridge” (Auger
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
74 School of Creative Media, City University of Hong Kong 10-14 June 2021The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
2013), titled Selfie Factory. As a result of its Baudrillard, Jean. 1994. Simulacra and
timely relevance, the installation enabled Simulation. Ann Arbor: University of
members of the public to draw their own Michigan Press.
conclusions around the notion of self- Cairo, Alberto. 2016. The Truthful Art: Data,
surveillance and data portraiture, and actively Charts, and Maps for Communication.
shape their own future, potentially with more United States: New Riders Publishing.
consideration of personal data accessibility. Couldry, Nick and Alison Powell. 2014. “Big
In light of Flusser’s (1983) critique of Data from the Bottom up.” Big Data and
photography as part of a feedback mechanism Society, 1–5. doi:10.1177/20539517145 392
that programs humans to perform rituals, we can 77.
see how in some cases, we are not aware when Deleuze, Gilles. 1988. Bergonism. Tanslated by
we perform self-surveillance or when we H. Tomlinson and B. Habberjam. New York:
capture a photograph. It becomes clear with this Zone Books.
research that distinguishing self-surveillance is Donath, Judith. 2014. The Social Machine. MIT
not an easy task in a world intertwined with Press, Cambridge.
network technology. By agreeing with Flusser Dunne, Anthony and Fiona Raby. 2013.
(1983) that photographs are, in essence, the Speculative Everything: Design, Fiction, and
transformation of scenes, we can confidently Social Dreaming. MIT Press, Cambridge.
propose that Selfie Factory is an installation Flusser, Vilém. 1983. Towards a Philosophy of
where photographs turned into scenes, and Photography. London: Reaktion Books.
scenes were part of a re-enactment that revealed Glum, Julia. 2015. “Millennials Selfies: Young
the presence of self-surveillance both inside and Adults Will Take More Than 25,000 Pictures
outside the art gallery. of Themselves During Their Lifetimes:
As demonstrated here, the negotiation Report.” International Business Times. 22
between photography and self-surveillance is in September 2015. http://www.ibtimes.com/
its infancy because the technology of the millennials-selfies-young-adults-will-take-
medium continues to shift and evolve. Further more-25000-pictures-themselves-during-
investigation into the practice of data portraiture 2108417
and the design process of art installations Kemp, Simon. 2018. “We Are Social, Digital in
utilising automation will provide more rigorous 2018: Worlds Internet User Pass the 4 Billion
findings on the subject and help us understand Mark” (blog). 30 January 2018.
the use of technology in an artistic context. https://wearesocial.com/blog/2018/01/global
-digital-report-2018
References Kennedy, Helen. 2018. “Living with Data:
Abrams, Janet and Peter Hall, eds. 2003. Aligning Data Studies and Data Activism
Else/where Mapping: New Cartographies of through a Focus on Everyday Experiences of
Networks and Territories. Minneapolis: Datafication.” Krisis: Journal for
University of Minnesota Press. Contemporary Philosophy, 1, no. 1: 18–30.
Auger, James. 2013. “Speculative Design: Lehtiniemi, Tuukka. 2017. “An Intervention in
Crafting the Speculation.” Digital Creativity Surveillance Capitalism?.” Personal Data
24, no 1: 11–35. Spaces, 627–39.
Ayres, Tony (writer). 2018. Everyone’s a Critic. Marr, Bernard. 2018. ‘How Much Data do we
ABC Television, 16 August, Television Create Every Day? The Mind-Blowing Stats
broadcast. Everyone Should Read’. Forbes, 21 May,
Baack, Stefan. 2018. ‘Civic Tech at mySociety: https://www.forbes.com/sites/bernardmarr
How the Imagined Affordances of Data /2018/05/21/how-much-data-do-we-create-
Shape Data Activism’. Krisis: Journal for every-day-the-mind-blowing-stats-
Contemporary Philosophy 1: 44–56. everyone-should-read/#95cdb0760ba9
Mayer-Schönberger, Viktor and Kenneth Cukier.
2013. Big Data: A Revolution That Will
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 75Part I. Full Papers
Transform how we Live, Work, and Think. exhibited at Galleries such as the Campbelltown
London: John Murray. Art Centre, The Copper Union and the National
Rubinstein, Daniel. 2015a. “Gift of the Selfie.” Portrait Gallery(Australia). Ralph’s work was
In Ego Update, edited by Alain Bieber, recognised with awards such as the Digital
Düsseldorf: NRW-Forum,. Portraiture Award (Australia), Museums
Rubinstein, Daniel. 2015b. “What is 21st Australian Multimedia & Publication Award
Century Photography?” (blog). 3 July 2015. (MAPDA), several Australian Graphic Design
https://thephotographersgallery.org.uk/conte Association Awards and distinctions from the
nt/what-21st-century-photography#:~: New York Type Directors Club (USA).
text=21st%20Century%20photography%20i
s%20this,frames%20%2D%20these%20coff An expert on the interactions between humans,
ins%20of%20photography.&text=As%20su computers, urbanism and art, Elmar Trefz
ch%2C%20photography%20is%20the,art% began his multimedia fascination by
20in%20the%20current%20time. programming Commodors in the ’80s. In 2016,
Saltz, Jerry. 2014. “Art at Arm’s Length: A he was awarded a PhD for his research into
History of the Selfie.” Vulture. 26 January urban user experiences in the digital age.
2014 https://www.vulture.com/2014/01 Along the way, he has blended data and
/history-of-the-selfie.html. media art for projects with Futurefarmers in San
Segel, Edward and Jeffrey Heer. 2010. Francisco, Spinifex in LA and Sydney, Art-for-
“Narrative Visualization: Telling Stories Innovation initiative Disonancias in San
with Data.” IEEE Transactions on Sebastian, Spain, and Germany's ZKM in
Visualization & Computer Graphics 16, no. Karlsruhe. He taught Big Data Visualisation at
6: 1139–48. the University of Sydney and directed the
Stern, Nathaniel. 2011. “The Implicit Body as Electrofringe festival in Newcastle, Australia.
Performance: Analyzing Interactive Art.” Having co-created AMP’s 360Goals project,
Leonardo 44, no. 3: 233–8. doi:10.1162/ he is currently exploring how decentralised
leon_a_00168. finance is reshaping our interactions with money
Sunitha Lobo, Sandra and Yamini Gowda. 2016. and financial products. When not solving thorny
“The Selfie Phenomenon: Self-presentation problems, Elmar is up a mountain, on a
and its Implications.” International Journal skateboard, or enjoying the ocean with his
of Computational Research & Development friends and family.
1: 147–53.
Mark Roxburgh, Ph.D., is an associate
professor of design, a management stooge, and
Biographies a fading would be indie rock star (google Joeys
Ralph Kenke is a visual communication Coop) in the School of Creative Industries at the
designer and media artist. He studied Visual University of Newcastle, (Newcastle, New
Communication as an undergraduate at South Wales) Australia. He received his
Swinburne University and later graduated with a undergraduate degree in Visual Arts from the
Masters in Design at the University of New College of Fine Arts, Australia; his Masters in
South Wales. He received his PhD at the Communication and Cultural Studies from the
University of Newcastle on his study of University of Western Sydney, Australia; and
prototyping data portraits, where he is teaching his PhD in Design from the University of
as a Design Lecturer and researcher at FASTlab. Canberra, Australia. He has been a design
Ralph has held multiple positions as a Design educator and researcher for over thirty years. His
Consultant and Senior Designer working scholarship in those fields has been focused on
alongside companies such as Deloitte, M&C design research methods; design theory; photo-
Saatchi and Imagination, delivering conceptual observation in design research; research-based
design applications that bridge art technology inquiry and learning; work integrated learning;
and design. His installations and designs were the relationship between tacit and explicit
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
76 School of Creative Media, City University of Hong Kong 10-14 June 2021The Art of Data Portraiture: Enabling a Public Debate on Self-surveillance. Ralph Kenke, Elmar Trefz, Mark
Roxburgh, Mario Minichiello
knowledge and the role of visual communication
in mediating those domains.
Mario Minichiello, Ph.D. studied at the
University of Leicester, Saint Martins, and
Loughborough in England. He has over thirty
years of experience in industry and academia
including leading roles in the academy at
Leicester DMU University, Birmingham City
University, Loughborough University in the
UK, and the University of Newcastle in NSW.
As a researcher, he has investigated and
prototyped new ideas, experiences and products
using tacit and explicit knowledge with creative
practices. He is Professor of Design and Director
of the Future Arts Science and Technologies
Laboratory (FASTlab) at the University of
Newcastle. Minichiello has the rare distinction
of working closely with Birmingham Children’s
Hospital (UK), a world-leading teaching
hospital, and with the Hunter Medical Research
Institute, one of the foremost medical research
centres in the world. He is the author and co-
author of a number of books, journal articles as
well as developing creative outputs and
intellectual property.
Proceedings of Art Machines 2: International Symposium on Machine Learning and Art 2021
School of Creative Media, City University of Hong Kong 10-14 June 2021 77You can also read
