FREEDOM OF THOUGHT: LEGAL PROTECTION FROM MANIPULATION - Index ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
DOI: 10.24234/wisdom.v14i1.310
Davit HARUTYUNYAN,
Lilit YEREMYAN
FREEDOM OF THOUGHT:
LEGAL PROTECTION FROM MANIPULATION
Abstract
The freedom of thought is stipulated as a fundamental human right in main international human
rights instruments at universal and regional levels. Freedom of thought is also guaranteed at the national
level in constitutions of many states. It might seem that the legal regulation of freedom of thought is more
declarative by its very nature: prima facie, it cannot be limited or violated in practice. Thus, one might as-
sume that it does not need any legal protection. In this paper, we argue that the rapid scientific and techno-
logical evolution urge the necessity of rethinking the legal content of the freedom of thought and elaborat-
ing mechanisms at national and international levels for its effective protection. In particular, we discuss the
lawfulness of manipulation as means of influencing the freedom of thought in the age of high technologies
and argue that the large-scale intensive manipulation by using special big data processing tools (including
artificial intelligence) with the aim to shape the information receivers‘ decision-making process in order to
reach a certain outcome motivated by self-interest should be viewed as unlawful interference into the free-
dom of thought under International Human Rights Law, consequently creating positive obligations for
states.
Keywords: freedom of thought, manipulation, ethics, human rights, national and international regula-
tions.
Introduction anything more important for personal autonomy
and individual self-determination than the free-
―Cogito, ergo sum‖ (I think, therefore I am): dom of thought? Thus, freedom of thought can
these words were the centre of Rene Descartes‘s also be regarded as a form of expression of hu-
philosophy and the cornerstone element of Wes- man dignity. This leads to the question: what is
tern rationalism. In this context, thinking should freedom of thought? Is it merely a metaphysical
be understood not just as the ability of a human category, moral, philosophical term, a supreme
being to think, but actually also as the readiness value or can it be classified as a legal concept? Is
to exercise this ability via questioning, doubting, freedom of thought axiomatic, and is its applica-
reasoning, choosing, via free will, as well as the tion absolute in practice? Does freedom of
ecosystem, which allows a person to freely exer- thought require legal protection? What does the
cise this ability. Indeed, is there anything more freedom of thought in legal instruments, such as
vital to Homo sapiens than thinking, and is there universal and regional international human rights
131 WISDOM 1(14), 2020
131D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
treaties, or constitutions of different states, stricting access to information and means of ex-
mean? Is it envisaged in legal documents as a pressing information against the will of the po-
proclamation that should never be forgotten, or tential ―thinker‖. Thoughts concerning the 1984
does it have a practical meaning? Is it possible to raise the question of whether hard power, i.e.
limit the freedom of thought and how? oppression, is the only feasible tool for restricting
One might recall the Orwellian 1984 and the freedom of thought? A mechanism that Or-
the oppression of freedom of thought through the well did not discuss in the 1984 is soft power in
so-called Thought Police that was created to pun- the form of targeted information control policy
ish thought-crime. The 1984, often referred as and large-scale manipulations using new tech-
science-fictional drama, in our opinion, is in fact nologies and artificial intelligence (AI) under the
exaggerated description of real repressive socie- aegis of freedom of information and freedom of
ties as was, for instance, Stalin‘s totalitarian re- expression in the atmosphere of love and solidar-
gime, with only elements of science fiction. But ity.
is totalitarianism capable of restricting the free- In the first section of this paper, we analyze
dom of thought per se or just it‘s manifestation the regulation of the freedom of thought as en-
by restricting the freedom of expression? ―Don‘t visaged in universal and regional human rights
you see that the whole aim of Newspeak is to instruments, inter alia, by drawing a comparison
narrow the range of thought? In the end, we between the wordings and revealing the legal
shall make thought-crime literally impossible, content of the right to freedom of thought and its
because there will be no words in which to ex- protection under International Law. The second
press it. …Every year fewer and fewer words, section is devoted to the analysis of the right to
and the range of consciousness always a little the freedom of thought at the national level as
smaller. Even now, of course, there‘s no reason stipulated in state constitutions of randomly se-
or excuse for committing thought-crime. It‘s lected countries from the European, American
merely a question of self-discipline, reality- and African human rights systems. Here we
control. But in the end, there won‘t be any need show that at the national level there is no uni-
even for that…‖ (Orwell, 1949, p. 67). Thus, in form approach towards regulation of the freedom
the ―Orwellian‖ scenario freedom of thought of thought and that belonging of the given state
could be limited either by self-discipline or by to a certain system of human rights protection
―rubbing‖ the means of expression of thought. does not significantly influence the content of the
Both these mechanisms described in 1984 are in constitutional level regulation of the freedom of
essence examples of hard power. The first one thought. In section three of the current paper, we
requires the will of the ―thinker‖ for self-discip- focus specifically on large-scale manipulation as
line assumedly under the fear to be punished by a means of influencing the freedom of thought in
the Thought Police. But this mechanism hasn‘t the age of rapid scientific and technological evo-
proven to be successful. The collective character lution. This thesis calls for the necessity to re-
of the 1984 - Winston - is the best proof of its think and revise the legal mechanisms of protec-
inefficiency, which called the necessity for a new tion of the right to freedom of thought at interna-
oppressive mechanism aimed at limiting the pos- tional and national levels in the light of risks
sibility to exercise freedom of thought by re- posed by widely used new technologies and al-
WISDOM 1(14), 2020 132 132Freedom of Thought: Legal Protection from Manipulation
gorithmic tools in the data processing. Some of rights and freedoms of others or for the pur-
such legal mechanisms and relevant recommen- pose of just requirements of morality, public
dations are discussed in section four of the paper. order and the general welfare in a democrat-
ic society. This is the most reasonable inter-
Freedom of Thought in International pretation of the applicability of the UDHR
Human Rights Treaties article 29 deriving from common sense.
UDHR is not binding for states in and of it-
The Universal Declaration of Human self, but it has been largely argued that it is
Rights (UDHR)1in its article 18 states: ―Every- legally enforceable as a reflection of cus-
one has the right to freedom of thought, con- tomary international law or general princi-
science and religion; this right includes freedom ples of law (Shaw M.N., 2014, p.204). This
to change his religion or belief, and freedom, argument, however, seems as applicable to
either alone or in community with others and in the rights and freedoms reflected in UDHR
public or private, to manifest his religion or be- as the minimum that states should be
lief in teaching, practice, worship and ob- obliged to ensure, rather than being applica-
servance‖. While article 29 envisages grounds ble to the limitations clause.
for possible limitation of rights as ―determined The International Covenant on Civil and
by law solely for the purpose of securing due Political Rights (ICCPR)2 generally follows the
recognition and respect for the rights and free- UDHR approach when envisaging the right to
doms of others and of meeting the just require- freedom of thought in its article 18, regulating
ments of morality, public order and the general this right together with freedom of conscience
welfare in a democratic society‖. and religion with further detalization only of the
As we can see from the UDHR provisions: right to freedom of conscience and religion. This
a) Freedom of thought is referred to as a right, raises the question of whether the mentioned
thus, implying state obligation to protect it. provision puts an equation between thought and
b) Freedom of thought is regulated in the same conscience or whether it views thought as a wid-
clause with conscience and religion. The er concept. According to General Comment N
clause further opens the legal content of 22 the ―right to freedom of thought, conscience
freedom of religion, leaving the freedom of and religion… is far-reaching and profound; it
thought without interpretation. encompasses freedom of thought on all matters,
c) The wording of article 29 represents a ge- personal conviction and the commitment to reli-
neric regulation referring to all rights stipu- gion or belief, whether manifested individually
lated in the UDHR. However, its analysis or in community with others‖.3 From the word-
leads to the conclusion that it cannot be ap-
plicable to the right to freedom of thought. 2
International Covenant on Civil and Political Rights
Adopted and opened for signature, ratification and
Logically by no means can limitation of the accession by General Assembly resolution 2200A
freedom of thought per se be necessary to (XXI) of 16 December 1966 entry into force 23
March 1976.
secure due recognition and respect for the 3
UN Human Rights Committee (HRC), CCPR General
Comment No. 22: Article 18 (Freedom of Thought,
Conscience or Religion), 30 July 1993, CCPR/C/21/-
1
UN General Assembly,Universal Declaration of Rev.1/Add.4, at: https://www.refworld.org/docid/-
Human Rights, 10 December 1948, 217 A (III). 453883fb22.html.
133 WISDOM 1(14), 2020
133D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
ing of the General Comment N 22, it could be the exigencies of the situation, provided that such
concluded that the freedom of thought as envis- measures are not inconsistent with [states‘] other
aged in article 18 should be interpreted as includ- obligations under international law‖. This ap-
ing but not limited to personal beliefs or religious proach, in our opinion, might as well be condi-
thought. At the same time, ICCPR establishes tioned by the narrower interpretation of the free-
the permissible grounds for limitations in article doms enshrined in ECHR Article 9. In any case,
18 itself with a very precise wording applicable the absence of case-law interpretations on free-
only to the limitation of the freedom to manifest dom of thought, in our opinion, should not be
religion or beliefs. While ICCPR article 4, clause considered as leaving room for its limitation by
2 declares inter alia article 18 as non-derogable. states but rather as a sign that the natural freedom
At the regional level, the European Con- is taken as granted and not requiring any specific
vention of Human Rights (ECHR)4 regulates regulation or interpretation.
freedom of thought in article 9 akin to analogical The African Charter on Human and Peo-
provision of ICCPR. However, the analysis of ples Rights (ACHPR)7 guarantees the freedom
the relevant literature, as well as the case-law of of conscience, the profession and free practice of
the European Court of Human Rights (ECtHR) religion, freedom of information and freedom of
leads us to the conclusion that concept of free- speech, freedoms of association, freedom of as-
dom of thought in the meaning of ECHR is dif- sembly (articles 8-11) and other freedoms, re-
ferent from that of article 18 of ICCPR. While maining silent on the freedom of thought. Does
treating freedom of thought, conscience and reli- this mean that the guarantee of the freedom of
gion as representing one of the foundations of a thought is perceived as meaningless? In the dis-
―democratic society‖ in the meaning of the cussion of the legal content of the right to free-
ECHR,5 it implies from the relevant analysis that dom of thought, opinions have been expressed
the freedom of thought is viewed as a narrow that freedom of thought should be understood as
concept embracing only the religious aspects of a declarative norm rather than having a practical
thought and moral convictions (White, Ovey, & meaning. Under such interpretation, the freedom
Jacobs, 2010, pp. 402-424)6. As to limitations, of thought at best would be constituting freedom
unlike the ICCPR article 4 on derogations, article implying no obligation of the state rather than a
15 clause 1 of ECHR does not include the provi- right with corresponding obligations. The
sion on freedom of thought, conscience and reli- ACHPR, however, regulates the thought neither
gion in the list of non-derogable rights, at the as a right nor even as freedom.
same time envisaging that derogations should be Unlike the ACHPR the American Conven-
allowed only ―to the extent strictly required by tion on Human Rights (ACHR)8 stipulates the
protection of the right to the freedom of thought
4
Convention for the Protection of Human Rights and and does so with an ―Orwellian approach‖ in its
Fundamental Freedoms (ECHR), Rome, 4.XI.1950,
CoE.
5 7
Kokkinakis v Greece, 25 May 1993, App. 14307/88. Organization of African Unity (OAU), African Char-
6
Also see in general: Guide on Article 9 of the Europe- ter on Human and Peoples‘ Rights (―Banjul Charter‖),
an Convention on Human Rights: Freedom of 27 June 1981, CAB/LEG/67/3 rev. 5, 21 I.L.M. 58
thought, conscience and religion Council of Europe/ (1982).
8
European Court of Human Rights (2019) at: https:- Organization of American States (OAS), American
//www.echr.coe.int/Documents/Guide_Art_9_ENG.p Convention on Human Rights, ―Pact of San Jose‖,
df. Costa Rica, 22 November 1969.
WISDOM 1(14), 2020 134 134Freedom of Thought: Legal Protection from Manipulation
article 13 titled ―Freedom of thought and expres- gency that threatens the independence or security
sion‖, thus, directly linking the protection of the of a State Party [… and only…], provided that
freedom of thought to the freedom of expression. such measures are not inconsistent with its other
Article 13 clauses 1 and 2 state: obligations under international law and do not
―1. Everyone has the right to freedom of involve discrimination on the ground of race,
thought and expression. This right includes free- colour, sex, language, religion, or social origin‖
dom to seek, receive, and impart information and as envisaged in ACHR article 27.
ideas of all kinds, regardless of frontiers […]
2. The exercise of the right provided for in Freedom of Thought in
the foregoing paragraph shall not be subject to Constitutions
prior censorship but shall be subject to subse-
quent imposition of liability, which shall be ex- The analysis of randomly selected constitu-
pressly established by law to the extent necessary tions of different states shows that there is no
to ensure: a) respect for the rights or reputations uniform approach towards regulation of the free-
of others; or b) the protection of national securi- dom of thought at the national level. It also
ty, public order, or public health or morals‖. seems that belonging to the given state to a cer-
As we can see, the first sentence of article tain system of human rights protection does not
13, clause 1 guarantees the right to the freedom significantly influence the constitutional level
of thought, implying state obligation for its pro- regulation of the freedom of thought. Thus, for
tection. Then the second sentence may seem to instance, in the constitutions of Bulgaria (article
narrow down the protection to the freedoms of 37), Estonia (article 40), Latvia (article 99), Ma-
expression and information. In our opinion, lawi (article 33), Namibia (article 21), Rwanda
however, this would be just the prima facie de- (article 37), Slovakia (article 24) the freedom of
termination of the scope of the freedom: because thought is regulated in the same provision with
it does not limit the freedom of thought and ex- the freedoms of conscience, religion, faith or be-
pression only to freedom to seek, receive, and lief. In article 54 of the Cuban Constitution, the
impart information and ideas but rather includes freedom of thought is regulated in line with free-
the latter freedoms into the legal content of the dom of conscience and freedom of expression.
freedom of thought and expression. The ACHR The Constitution of Morocco regulates the free-
article 13 clause 4 does not allow any prior limi- dom of thought in the same clause with the free-
tations to the freedom, except public entertain- dom of opinion and expression (article 25). In
ments by law for the sole purpose of regulating article 30 of the Constitution of Niger the free-
access to them for the moral protection of child- dom of thought, opinion, expression, conscience,
hood and adolescence, ―any propaganda for war religion and worship are regulated together. The
and any advocacy of national, racial, or religious Constitution of Chile guarantees the freedom of
hatred that constitute incitements to lawless vio- conscience, expression of belief and the manifes-
lence or to any other similar action against any tation of religion (article 19), and there is no
person or group of persons on any grounds…‖ as mentioning of the freedom of thought. The Ger-
stipulated in article 13 clause 5 of ACHR, and man Constitution envisages inter alia the free-
―in time of war, public danger, or another emer- dom of faith and conscience in its article 4, free-
135 WISDOM 1(14), 2020
135D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
dom of expression, arts and sciences in article 5 rights standards. A similar approach is undertak-
with no mentioning of the freedom of thought. In en in the Constitution of the Russian Federation,
the constitutions of Greece, Peru, Poland, Portu- also providing the possibility of restricting the
gal, Belarus, Cameroon, Tanzania, a similar ap- relevant freedoms in emergency situations (arti-
proach is undertaken, and the freedom of thought cle 55, clause 3 and article 56). The Constitution
is absent from regulations. Freedom of thought is of Cuba envisages in article 54 as a limitation
not specifically regulated also in the Constitution clause that ―conscientious objection may not be
of the USA, however, in several instances, the invoked with the intention of evading compli-
freedom was acknowledged to get absolute pro- ance with the law or impeding another from the
tection by virtue of the First Amendment to the exercise of their rights‖.
US Constitution.9 The wording of the relevant In the given context, it is interesting to ana-
provisions of the constitutions of Latvia (article lyze also the evolution of the legal regulation of
99), Malawi (article 33), Niger (article 30), South the right to freedom of thought in the Constitu-
Africa (article 15) explicitly envisage the free- tion of the Republic of Armenia (RoA). Freedom
dom of thought as a right, in constitutions of of thought has been envisaged by all three edi-
some other states, it is stipulated as freedom tions of the RoA Constitution: namely article 23
however clearly implying the corresponding ob- in the 1995 edition, article 26 in the 2005 edition,
ligation of the state to guarantee it.10 Different and article 41 in the 2015 edition. The RoA Con-
approaches are undertaken by states also with stitution of the 1995 edition stipulated: ―every-
regard to restriction and derogation clauses. one is entitled to freedom of thought, conscience,
Thus, for instance, in Bulgaria (articles 37 and and religion.‖ The freedom of thought was stated
57) and Estonia (articles 40 and 130), the consti- as non-derogable and not subject to any re-
tutions provide no possibility for derogation or strictions, while the freedom to exercise religion
restriction of the freedom of thought. The consti- and beliefs could be restricted by law on the
tutions of Malawi (article 44) and Namibia (arti- grounds prescribed in the Constitution. This pro-
cle 21) envisage general grounds for reasonable vision was amended in 2005, and then new
restrictions of rights and freedoms by law, which wording envisaged: ―Everyone shall have the
are in consistence with international human right to freedom of thought, conscience and reli-
gion. This right shall include freedom to change
9
For example, US Supreme Court Justice Black in his religion or belief, and freedom — either individ-
concurring opinion in Wieman v. Updegraff decision
of 15 December 1952 stated: ―Framers rested our First ually or in community with others — to manifest
Amendment on the premise that the slightest suppres- them in preaching, church ceremonies and other
sion of thought, speech, press, or public assembly is
still more dangerous‖ (p.194), and in dicta of the deci- rites of worship. The manifestation of this right
sion Lawrence vs Texas of 26 June 2003 the US Su-
preme Court mentioned that ―liberty presumes an au-
may be limited only by law, where it is necessary
tonomy of self that includes freedom of thought, be- to protect the public safety, health, morals or the
lief, expression, and certain intimate conduct‖ (pa-
ra.1). rights and freedoms of others.‖ Prima facie it
10
This is the case, for instance, in article 54 of the Con- might seem unclear from the wording of the
stitution of Cuba, article 25 of the Constitution of Mo-
rocco, article 37 of the Constitution of Rwanda, article RoA Constitution of 2005 edition whether the
24 of the Constitution of Slovakia, article 29 of the provided grounds for limitation are applicable
Constitution of the Russian Federation, article 33 of
the Constitution of the Republic of Belarus, articles 14 only to the conscience and religious freedom, or
and 40 of the Constitution of Estonia.
WISDOM 1(14), 2020 136 136Freedom of Thought: Legal Protection from Manipulation
to the freedom of thought as well. However, the however, seems to be an unreasonable interpreta-
keyword of the restriction clause, in our opinion, tion first and foremost because such interpreta-
is the word ―manifestation‖. In other words, the tion would be viewed as conflicting with the
limitation clause provides grounds only for limit- RoA international legal obligations in the
ing the manifestation of the right rather than the framework of article 5 clause 3 and article 81 of
right itself. According to the authoritative doctri- the RoA Constitution. Thus, despite the vague-
nal interpretation of the given constitutional pro- ness of the wording we are inclined to apply to
vision thought is described as a natural character- article 41 the same interpretation as in case of the
istic of human beings, the basis for their spiritual wording of article 26 in the 2005 edition of the
life and spiritual freedom, which cannot be sub- Constitution.
ject to legal regulation as such simply because
the thought per se cannot be limited (Harutyun- Restriction of the Freedom
yan & Vagharshyan, 2010, pp. 297-298). Thus, of Thought
these commentaries also lead us to the conclu-
sion that the mentioned limitations clause cannot Hobbes (as cited in Lemetti, 2012, p. 174)
be interpreted as applicable to the freedom of describing the natural freedom of thought in his
thought. The latest amendments to the RoA Con- Leviathan wrote: ―internal faith is in its own na-
stitution were adopted in 2015, also altering the ture invisible and, consequently, exempted from
wording of the provision on the right to freedom all human jurisdiction.
of thought, conscience and religion. Article 41 of Through centuries the thought has been
the RoA Constitution stipulates: perceived as an element of the inner world of the
―1. Everyone shall have the right to freedom ―thinker‖, the intangible product of the mental
of thought, conscience and religion. This right process in the brain of a person known only to
shall include the freedom to change religion or that person until the moment of its expression in
belief and, either alone or in community with one way or another, and, thus, reasonably not
others and in public or in private, the freedom to subject to any regulation or limitation by state
manifest them in preaching, church ceremonies, (Bublitz, 2014, pp.1-3). Such perception of
other rites of worship or in other forms. 2. The thought served basis for leaving the freedom
expression of freedom of thought, conscience without regulation in the basic laws of some
and religion may be restricted only by law for the states. Such perception might also be the reason
purpose of state security, protecting public order, for limited doctrinal analysis on the right to free-
health and morals or the basic rights and free- dom of thought. The rapid evolution of science
doms of others‖. and technology, which is already merging with
It is difficult to judge what the legal content once science fiction, make us rethink the use-
of article 41 clause 2 is: i.e. does the term ―ex- lessness of the right to the freedom of thought
pression of‖ apply to the freedom of thought and and the necessity to open a discussion on its legal
conscience or is it linked only to the freedom of content, hypothetic grounds for and lawfulness
religion? It could be assumed that the freedom of of its limitation, as well as the evolution of the
thought is envisaged as potentially subject to re- law on the protection of the freedom of thought.
striction by law for certain listed purposes. This, There might be different hypothetic scenar-
137 WISDOM 1(14), 2020
137D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
ios of intervention into the freedom of thought – We may witness or experience manipula-
all interesting subject for discussion, however, in tion in our everyday life in the workplace, when
the current paper, we would like to focus on ma- doing shopping, making (facilitating) choices
nipulation as means of influencing the freedom about the government structure or participating
of thought in the age of high technologies. in elections. Manipulations are always intention-
Manipulation in and of itself is not a new al and manipulators also often acknowledge the
phenomenon: it has always been used in order to unethical nature of their attempt to interfere with
influence the decision-making process at all lev- the freedom of thought of a third person, and yet
els. It is said that our reality is subjective, our it has always been and remains present in our
expectations often shape the reality: we see what lives. So it would be naïve to assume that one
we expect to see. Manipulation in our under- day, because of its unethical nature, manipula-
standing is the process of misleading the infor- tions would stop taking place. But what has
mation receiver by information control or selec- made current-day manipulation significantly dif-
tive processing, and by this also determining the ferent from that of Wilde‘s age is the information
behaviour of the information receiver, in order to and communication technologies and utilization
make the latter form a certain opinion or idea, of AI for collecting and processing data, tracking
make a certain decision or act in a certain way, and predicting individual and collective behav-
and constituting direct intervention into the area iour, which makes psychological manipulation
of freedom of thought. Manipulation becomes easier and less obvious (hidden), the expected
easier if manipulator possesses comprehensive outcome more precisely predictable, the outcome
information about the preferences, personal more targeted and large-scale, capable even of
character, expectations, his/her belonging to a affecting public relations and government struc-
certain group based on certain parameters, and tures. So if in case of single instances of manipu-
about the environment of the information receiv- lation, i.e. intervention into the freedom of
er. Regardless of whether the aim or cause of the thought domain of one person by another person,
manipulator is good or bad manipulation is al- the state should not reasonably be expected to
ways unethical. One of the most illustrative fic- have an obligation to protect the freedom of
tional examples of manipulation could be con- thought of the person being influenced, in our
sidered that of Dorian Grey by Lord Henry in opinion, the opposite assumption should be true
Oscar Wilde‘s The Picture of Dorian Grey. when dealing with certain cases of large-scale
―There is no such thing as a good influence. All manipulation conducted by using new technolo-
influence is immoral - immoral from the scien- gies. In casual interpersonal manipulation both:
tific point of view. Because to influence a person the information receiver and the manipulator are
is to give him one‘s own soul. He does not think in horizontal and symmetric relations. Both have
his natural thoughts or burn with his natural pas- equal opportunities in seeking and receiving in-
sions. His virtues are not real to him. His sins, if formation, choosing what information to impart,
there are such things as sins, are borrowed. He what to believe in, and what to ignore, with
becomes an echo of someone else‘s music, an whom to interact, etc. New information technol-
actor of a part that has not been written for him‖ ogies and almost limitless possibilities of AI put
(Wilde, 2011, p. 28). manipulator into a better position, transform the
WISDOM 1(14), 2020 138 138Freedom of Thought: Legal Protection from Manipulation
nature of manipulator-information receiver rela- pp. 217-225).11 One of the approaches to address
tions into vertical and asymmetric. This, in our the issue of privacy is accepting the consequenc-
opinion, calls the necessity for the state interven- es of the rapidly developing world and adapting
tion to protect the ―weak‖ side from possible in- to the new situation. As the co-founder of Sun
tervention into the freedom of thought. While Microsystems Scott McNealy famously said yet
any manipulation in and of itself is immoral, we in 1999 to a group of reporters: ―You have zero
argue that large-scale intensive manipulation privacy anyway… Get over it‖ (Sprenger, 1999).
(interference into the freedom of thought do- Later argued that these words were taken out of
main) by using special big data processing the context, however, in essence, this approach
tools (including AI) with the aim to shape the not just has fairgrounds, but it has become even
information receivers’ decision-making pro- more topical with the development of the new
cess in order to reach a certain (approximate- information technologies. The opposite ap-
ly pre-calculated) outcome motivated by self- proach, i.e. attempt to regulate the use of new
interest is not just immoral but also unlawful. technologies and balance it with the right to pri-
Big data is a term used to describe large and vacy, has also been considered. One suggested
inter-connected data with high volume and a solution has been the ―privacy by design‖ con-
wide variety of information, as well as high cept aimed at ensuring privacy protection ex-ante
speed of collecting and processing (McGregor, and included in the design of the new technolo-
Calderón, & Tonelli, 2013, p. 1). Data mining gy. This concept got pioneered by Ann Cavouki-
and pooling tools allow to aggregate and com- an, formerly Ontario‘s Information and Privacy
bine the large quantity of information from many Commissioner, who also elaborated seven prin-
different sources and collected by different, in ciples of privacy by design: proactive and pre-
one way or another, interconnected agencies, the ventative, privacy as the default setting, privacy
so-called data warehouses, to then integrate and embedded into the design, full functionality –
analyse it, categorize the useful information and positive-sum, end-to-end security – full lifecycle
identify individual and collective characteristics protection, visibility and transparency, user-
and features, as well as trace behavioural trends, centric approach (Birnhack, Toch, & Hadar,
while these patters, in their turn can be either fo- 2014, pp. 55-114). Another very striking exam-
cused or not, either applied on a wide-scale pop- ple of the balanced protection approach has been
ulation or targeted at a certain group depending the adoption of General Data Protection Regula-
on the aim of data analysis (Kulhari, 2018, pp. tion (GDPR) implemented in May 2018 and ap-
26-27; McGregor, Calderón, & Tonelli, 2013, plicable to the territory of all European Union
p.1). It has been widely discussed that data min- (EU) member states.12 After the GDPR adoption,
ing, pooling and its subsequent (automated or
human) processing often fairly raise questions in 11
See also Data mining, dog sniffs and the fourth
amendment, Harvard Law Review Association, HLR
the context of posing risks to person‘s privacy Vol. 128, No.2 (December 2014), pp. 697-698.
12
(Fienberg, 2006, pp. 143-154; Pavolotsky, 2013, European Union (2018). General Data Protection
Regulation, 2016/679 of the European Parliament and
of the Council of 27 April 2016 on the protection of
natural persons with regard to the processing of per-
sonal data and on the free movement of such data, and
repealing Directive 95/46/EC, at: https://eurlex.euro-
pa.eu/eli/reg/2016/679/oj.
139 WISDOM 1(14), 2020
139D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
many states also outside the EU have taken steps the main tool for generating government struc-
to improve respective legislation on data protec- ture changes, like Arab Spring dubbed as ―Revo-
tion. This movement is reaching even the USA lution 2.0‖ (Hochwald, 2013, pp. 21-27).
(Serrato, Cwalina, Rudawski, Coughlin, & Far- These trends reasonably give rise to the le-
delmann, 2018). gal discussions concerning the protection of the
With massive discussions still ongoing freedom of thought. Interestingly, unlike the Or-
states can choose between the ―zero privacy‖ wellian approach to possible limitation of the
approach and regulation of the field to protect freedom of thought through restricting the free-
privacy, inter alia, in accordance with interna- dom of expression, nowadays a shift took place
tional human rights standards. The situation is often arguing the lawfulness of factual limitation
different with balancing the application of new of the freedom of thought by the necessity to en-
information technologies and mental autonomy. sure the freedom of expression. This argument,
Discussions on the necessity to view the applica- however, does not seem to be well-grounded. In
tion of automated data processing techniques our opinion, the protection of the freedom of
inter alia in the light of freedom of thought, and thought does not need to be realised by imposing
the ―challenges to cognitive sovereignty‖ have unbalanced or unlawful restrictions to the free-
started to take place recently with a focus on dom of expression, but rather imposing only rea-
such aspects as possible future AI application, sonable, necessary and proportional limitations.
for instance, equipment capable of reading the A number of organizational and legislative
mind of a person through the interface (McCar- measures at national and international levels
thy-Jones, 2019; Gilbert, 2019) or manipulation could be undertaken by the state and businesses
strategies used by social media (Băncău-Burcea, in order to ensure full and effective protection of
2017). The legal regulation of the first scenario the right to freedom of thought.
would be necessary but more pre-emptive and
proactive in nature, and the second scenario, in What could and should be done to Protect
our opinion, remains in the domain of social me- Freedom of Thought: Summing up
dia ethics. In the meantime, ―Big Techs‖ like
Facebook and Google already today have very The absolute nature of the right to freedom
significant power exerted from the bottom up, of thought in International Human Rights Law
via the advanced technologies, the so-called sur- (IHRL) and the states‘ obligation to ensure and
veillance capitalism, which ―allows them to di- protect it against violations call for the necessity
vide and conquer us in ways that the oligarchs of of specific measures at international and national
the past could only dream about‖ (Foroohar, levels to address the risks in the context of wide-
2019). ―Domination!‖ was the proclamation with spread utilization of new technologies and algo-
which, as reported, Mark Zuckerberg used to end rithmic processes.
meetings in the early years of Facebook At the universal level, the ICCPR regula-
(Kuchler, 2019). Indeed, recently we have often tions ensure full protection of the freedom of
been hearing that social media is influencing the thought, while the regional level protection
results of elections, such as Cambridge Analytica seems to have room for improvement and updat-
scandal (Confessore, 2018), or that it served as ed interpretation.
WISDOM 1(14), 2020 140 140Freedom of Thought: Legal Protection from Manipulation
The necessity of enhancing the freedom of cial emphasis on political and electoral process-
thought protection has already been emphasized es, research and raising awareness campaign, etc.
by the Council of Europe (CoE) member states. The Draft Recommendations, in their turn, stipu-
Thus, acknowledging that machine learning late details of the suggested cope of the obliga-
technologies and tools can be used to predict tion of the states and responsibilities of the busi-
choices, influence thoughts and subject people to nesses with respect to rights and freedoms in the
manipulation, CoE has called on member states context of advanced algorithmic systems.
to address the risk in February 2019. As a result, Additional measures at European level, in
a Declaration on manipulative capabilities of al- our opinion, should include: broader interpreta-
gorithmic processes (Declaration)13 was adopted tion of ECHR article 9 in the framework of EC-
along with further draft recommendations of the tHR case-law to define ―thought‖ as including all
Committee of Ministers (Draft Recommenda- aspects of thought not limited to a religious con-
tions) to member-states addressing the impacts text. As it was first acknowledged in Tyrer case,
of algorithms on human rights14. The Declara- ECHR is a ―living instrument‖, and its interpreta-
tion, as reflected in its paragraph 9, encourages tion should be adopted to the current-day condi-
member states to address the risks to human tions.15 Such approach also referred as the prin-
rights by, inter alia, ―initiating, within appropri- ciple of effectiveness, is dictated by the necessity
ate institutional frameworks, open-ended, in- to give the rights and freedoms of the ECHR
formed and inclusive public debates with a view ―fullest weight and effect consistent with the
to providing guidance on where to draw the line language used and with the rest of the text and in
between forms of permissible persuasion and such a way that every part of it can be given
unacceptable manipulation. The latter may take meaning‖ (Merrills, J. as cited in White, Ovey, &
the form of influence that is subliminal, exploits Jacobs, 2010, p. 73). It seems to be the right time
existing vulnerabilities or cognitive biases, and/ to give meaning to the word ―thought‖ in article
or encroaches on the independence and authen- 9 via case-law. Such developments could, cer-
ticity of individual decision-making; [as well as] tainly, be facilitated by the evolution of CoE soft
taking appropriate and proportionate measures to law.
ensure that effective legal guarantees are in place As the ACHPR does not specifically regu-
against such forms of illegitimate interference‖. late the freedom of thought, regional initiatives
The Declaration also stresses the need to put in could be launched aimed at analysing and as-
place a stronger regulatory body or to take other sessing the data on the use of specific technolo-
measures for oversight and monitoring with spe- gies or algorithmic processes with capacities to
influence decision-making. The results of such
13
Declaration by the Committee of Ministers on the
analysis could then serve as the basis for drafting
manipulative capabilities of algorithmic processes legal regulations ensuring full and effective pro-
(Adopted by the on 13 February 2019) Decl.
(13/02/2019)1. tection of the freedom of thought.
14
Committee of experts on human rights dimensions As to the ACHR, it might be interpreted as
of automated data processing and different forms of
artificial intelligence (MSI-AUT), Addressing the having relevant guarantees for ensuring full pro-
impacts of Algorithms on Human Rights Draft
Recommendation of the Committee of Ministers to
15
member States on the human rights impacts of al- Tyrer vs United Kingdom, ECtHR (App.5856/72) 25
gorithmic systems (status – draft, 2018). April 1978, para. 31.
141 WISDOM 1(14), 2020
141D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
tection of the freedom of thought embedded in dignity. States, where freedom of thought is
article 13. Thus, the wording of the third para- guaranteed by the constitution should carefully
graph of the ACHR article 13 could be interpret- choose the wording making sure that it leaves no
ed broadly as a relevant tool for protecting the room for misinterpretation of the scope of pro-
freedom of thought. According to the mentioned tection of the right to the freedom of thought in
clause ―indirect methods or means of restriction, consistence with their international legal obliga-
including abuse of government or private con- tions.
trols over newsprint, radio broadcasting frequen- States should, in our opinion, also take addi-
cies, or equipment used in the dissemination of tional efforts to enhance their national legislation
information, or by any other means tending to with mechanisms aimed at full and effective pro-
impede the communication and circulation of tection of the freedom of thought. One such
ideas and opinions‖. This regulation is, prima measure could be envisaging legal responsibility
facie, aimed at ensuring full exercise of the right (liability) for large-scale intensive manipulation
to information rather than the freedom of thought (interference into the freedom of thought do-
as such: dealing with the freedom of thought main) by using special big data processing tools
from the Orwellian perspective. However, taking (including AI) aimed at shaping the information
into account that the development of ACHR receivers‘ decision-making process and by that
takes place through its interpretation, which in its resulting in a certain (approximately pre-calcu-
turn shall keep pace with the current develop- lated) outcome and motivated by any form of
ments, it could as well be argued that the prohibi- self-interest. The described situation imposes
tion of ―equipment used in the dissemination of positive obligations on the state to protect people
information or any other means tending to im- from possible violations against freedom of
pede the communication and circulation of ideas thought. This approach seems to be necessary,
and opinions‖ should be broadly interpreted as reasonable, proportionate and consistent to the
applicable to and relevant for imposing re- international human rights standards. And im-
strictions, inter alia, on targeting technologies posing liability for the already realized conduct
and software which are technically capable of would ensure no unnecessary interference with
selectively suiting people on the basis of data- the freedom of expression.
driven profiling and by this indirectly impeding Other preventive legislative measures at the
the circulation of ideas and opinions. state level could include: envisaging legislative
At the national level, states could also un- requirement for the technology companies to
dertake certain measures aimed at protection of embed the freedom of thought guarantees into
the right to the freedom of thought. Thus, state the design of a technology or an algorithmic sys-
constitutions represent the highest legal act by tem similar to ―privacy by design‖ concept. This
which the freedom of thought could be guaran- could be done by default disclosure of all third
teed. If the constitution of a given state is silent parties with whom the information provided by a
on the right to freedom of thought, it should be person may be shared, all purposes of processing
interpreted as acknowledging that right by the the data and receiving the periodic informed con-
force of IHRL as a customary norm or a general sent of the platform user, or by means of inform-
principle of law constituting the core of human ing him/her of the origins of the advertisement,
WISDOM 1(14), 2020 142 142Freedom of Thought: Legal Protection from Manipulation
video or any similar materials and indicating the nowledged that the freedom of expression is not
parameters that served as a basis (reasons) for an absolute right and states have a certain margin
showing the given content to the given platform of appreciation in lawfully limiting it, including
user, the patterns identified, including the latter‘s via legislation (this approach is reflected, inter
belonging to a certain category of users based on alia, in the ICCPR article 19 clause 2, ECHR
the relevant parameters – labelled as a targeted article 10 clause 2, ACHR article 13 clause 2).
advertisement in every such case. By contrast, as analysed supra, the freedom of
In our opinion states should also proceed thought, encompassing all matters, is an absolute
with stipulating restrictions or prohibition on the right to which no limitations and from which no
dissemination of targeting advertisement (con- derogations should be allowed. States have posi-
tent) concerning political and electoral processes, tive obligations to secure this right and protect it
as well as the adoption of do-not-track laws, fake from violations not only by state agents but also
news laws, and establishing independent over- by private persons or entities (White, Ovey, &
sight and monitoring mechanisms as comple- Jacobs, 2010, pp. 99-102). In other words, the
mentary measures. necessity to ensure the right to freedom of
Acknowledging the significant role of fake thought, in our opinion, could serve a legitimate
news in facilitating large-scale manipulation (in- ground for imposing restrictions on the right to
ter alia with the use of automated data pro- freedom of expression given certain conditions
cessing tools) a number of states have already are fulfilled.
taken action against online manipulation via fake The practice of do-not-track bills has al-
news. Such actions include elaborating state ready been adopted in the USA. Thus, as a
strategies and action plans, launching media lit- mechanism to ensure privacy and data protection
eracy campaigns, setting up government task Assembly of California has passed a bill, the so-
forces, signing public-private agreements, setting called Do Not Track Bill (AB 370), on amending
online reporting portals and fact-checking sites, California Online Privacy Protection Act.16 The
adopting special legislation (Funke & Flamini, main purpose of the Act is to provide the Internet
2019). One of the toughest regulations on online users with the opportunity to opt out of online
manipulation is the ―Protection from Online tracking schemes (McGregor, Calderón, & To-
Falsehoods and Manipulation Act‖ (POFMA), nelli, 2013, pp. 3-4). In May 2019 a similar piece
passed in May 2019 by the Parliament of Singa- of legislation to enact a national do-not-track sys-
pore (Wong, 2019). Singapore has invoked tem and to limit the targeted advertisements was
POFMA for the first time against Facebook introduced to the Senate (currently pending) by
(Palma, 2019). POFMA and similar laws are privacy and freedom of choice pioneer Senator
treated with caution for the risks to the freedom Josh Hawley.17 These kinds of initiatives not on-
of expression. Indeed special care shall be taken
by states to ensure that any action aimed against 16
State of California Assembly (2013). An act to amend
spreading online misinformation does not consti- Section 22575 of the Business and Professions Code,
relating to consumers, available at: https://leginfo-
tute a violation of the freedom of expression and .legislature.ca.gov/faces/billNavClient.xhtml?bill_id=
is in consistence with international human rights 201320140AB370.
17
Hawley, J. (2019). A Bill to protect the privacy of in-
standards. At the same time, it should be ack- ternet users through the establishment of a national
Do Not Track system, and for other purposes (S.
143 WISDOM 1(14), 2020
143D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
ly enhance privacy protection but also serve as tection against violations by the state or private
supplementary tools for ensuring the right to companies.
freedom of thought. Moreover, in the protection It should be acknowledged and ensured that
of freedom of thought Senator Hawley has also the freedom of thought remains the right of every
suggested passing a bill ―to prohibit social media person. ―Think for yourself and let others enjoy
companies from using practices that exploit hu- the privilege of doing so too.‖ Voltaire (Essay on
man psychology or brain psychology to substan- Tolerance).
tially impede freedom of choice, to require social
media companies to take measures to mitigate Conclusion
the risks of the Internet addiction and psycholog-
ical exploitation, and for other purposes‖.18 De- Freedom of thought is recognized as a fun-
spite the almost no chances for the draft to be damental human right in a number of interna-
approved, Josh Hawley‘s action is one more sig- tional and regional human rights instruments. At
nal drawing attention to the necessity of thinking the same time at the national level, there is no
of special mechanisms or regulations aimed at uniform approach towards regulation of the right
enhancing the protection of the freedom of to freedom of thought. Thus, in constitutions of
thought. some of the randomly selected states, the regula-
In addition to the above-mentioned mea- tion of the freedom of thought is directly linked
sures, in our opinion, action should also be to the freedoms of conscience, religion, faith or
launched to make businesses and state agencies belief, or to the freedom of expression, in other
responsible for data processing via specific algo- states the freedom of thought is at all absent from
rithmic systems to elaborate and adopt business constitutional regulations. Similarly in some
ethics rules specifically addressing the mecha- states, where the freedom of thought is guaran-
nisms that are used to ensure the freedom of teed under the basic law, the relevant constitu-
thought. An example of such mechanism could tional provision explicitly envisages the freedom
be, for instance, providing every user with a right of thought as a right, in other states the freedom
to access an interactive online chart with the en- of thought is stipulated as freedom however
tire massive of depersonalized, categorized data- clearly implying the corresponding state obliga-
bases and datasets with identification only of that tion to effectively ensure it. Some of the ana-
given user. And last but not least, states should lyzed state constitutions provide no possibility
ensure that the civil society is well aware of the for derogations or restriction of the freedom of
legal content of the right to the freedom of thought, and others envisage generic grounds for
thought in the context of new technologies, as reasonable restriction of all rights and freedoms
well as familiarized with the means for its pro- in general (including the right to the freedom of
thought) by law provided that such restrictions
are in consistence with international human
1578), available at: https://www.congress.gov/bill/11- rights standards.
6th-congress/senatebill/1578/text.
18
Hawley, J. (2019). Draft Bill ―Social Media Addiction While the right to freedom of thought is un-
Reduction Technology Act‖ (SMART), LYN19429, doubtedly perceived as an absolute right to
(para.1), available at: https://www.hawley.senate.-
gov/sites/default/files/2019-07/Social-Media-Addic- which no restrictions are possible in practice, the
tionReduction-Technology-Act.pdf/.
WISDOM 1(14), 2020 144 144Freedom of Thought: Legal Protection from Manipulation
rapid technological and scientific development, Birnhack, M., Toch, E., & Hadar, I. (2014). Pri-
in our opinion, calls for the necessity of evolu- vacy Mindset, Technological Mindset.
tion of the law on the protection of freedom of Jurimetrics, 1(55), 55-114.
thought, inter alia by revealing the legal content Bublitz, J. C. (2014). Freedom of Thought in the
of the right to the freedom of thought in the pre- Age of Neuroscience: a Plea and a Pro-
sent-day realities, as well as determining the posal for the Renaissance of a Forgot-
scope of positive obligations of the state to en- ten Fundamental Right. Archives for
sure its full and effective protection. This argu- Philosophy of Law and Social Philoso-
ment becomes specifically topical in the context phy, 1(100), 1-3.
of using machine learning technologies and tools Confessore, N. (2018, April 4). Cambridge Ana-
for large-scale data processing aimed at reaching lytica and Facebook: The Scandal and
a certain outcome motivated by self-interest, i.e. the Fallout So Far. The New York Ti-
large-scale manipulation facilitated by the use of mes. Retrieved from: https://www.ny-
new technologies (including AI). We conclude times.com/2018/04/04/us/politics/cam-
that while any manipulation in and of itself is bridge-analytica-scandal-fallout.html.
immoral, the large-scale intensive manipulation Fienberg, S. E. (2006). Privacy and Confidential-
(interference into the freedom of thought do- ity in an E-Commerce World: Data
main) by using special big data processing tools Mining Warehousing, Matching and
with the aim to shape the information receivers‘ Disclosure Limitation. Statistical Sci-
decision-making process in order to reach a pre- ence, 2(21). A Special Issue on Statisti-
calculated outcome and motivated by self- cal Challenges and Opportunities in
interest should also be interpreted as unlawful Electronic Commerce Research, 143-
under IHRL and, consequently, reflected in na- 154.
tional legislation. Such approach should be ad- Foroohar, R. (2019, December 8). How to Take
vanced by developing the interpretation of the Back Control from the Big Tech Bar-
right to freedom of thought regulations of rele- ons. Financial Times. Retrieved from:
vant international treaties addressing the risks https://www.ft.com/content/1274db2e-
posed by technological development, as well as 1276-11ea-a7e6-62bf4f9e548a.
by ensuring states undertake appropriate Funke, D., & Flamini, D. (2019). A Guide to
measures to fulfil their positive obligation in pro- Anti-Misinformation Actions around
tecting the absolute right to freedom of thought. the World. Retrieved January 2020
from: https://www.poynter.org/ifcn/an-
REFERENCES ti-misinformation-actions/.
Gilbert, B. (2019). Facebook Reportedly Spent
Băncău-Burcea, A. (2017). Social Media and More than $500 Million to Buy a
Freedom of Thought. Proceedings of Mind-Reading Technology Startup -
the RAIS Conference: The Future of and People are Calling the Move
Ethics, Education and Research. Re- ―Gross‖ and ―Scary‖. Business Insider.
trieved from: https://ssrn.com/abstract- Retrieved from: https://www.business-
=3085972. insider.com/facebook-mind-reading-
145 WISDOM 1(14), 2020
145D a v i t H A R U T Y U N YA N , L i l i t Y E R E M YA N
tech-ctrl-labs-acquisition-backlash- law-todayi.
2019-9. Orwell, G. (1949). 1984. Retrieved from:
Harutyunyan, G. G., & Vagharshyan, A. G. https://www.planetebook.com/free-
(Eds.). (2010). Hayastani Hanrapetut‘- ebooks/1984.pdf.
yan Sahmanadrut‘yan meknabanut‘- Palma, S. (2019, November 29). Singapore Gov-
yunner (Commentaries to the Constitu- ernment Orders Facebook to Correct
tion of the Republic of Armenia, in Social Media Post. Financial Times.
Armenian). Yerevan: ―Law‖. Retrieved from: https://www.ft.com-
Hochwald, Th. (2013). How Do Social Media /content/1274db2e-1276-11ea-a7e6-
Affect Intra-State Conflicts other than 62bf4f9e548a.
War? Journal Connections, 12(3), 9- Pavolotsky, J. (2013). Privacy in the Age of Big
38. Data. The Business Lawyer, 69(1), 217-
Kuchler, H. (2019, March 28). How Facebook 225.
Grew Too Big to Handle. Financial Serrato, K. J., Cwalina, Ch., Rudawski, A.,
Times. Retrieved from: https://www.ft.- Coughlin, T., & Fardelmann, K.
com/content/be723754-501c-11e9-9c- (2018). US States Pass Data Protection
76-bf4a0ce37d49. Laws on the Heels of the GDPR. Com-
Kulhari, S. (2018). Building-Blocks of a Data pliance and Risk Management. Re-
Protection Revolution: The Uneasy trieved from: https://www.dataprotec-
Case for Blockchain Technology to Se- tionreport.com/2018/07/u-s-states-pass-
cure Privacy and Identity. Nomos Ver- data-protection-laws-on-the-heels-of-
lagsgesellschaft mbH. the-gdpr/.
Lemetti, J. (2012). Historical Dictionary of Shaw, M.N. (2014). International Law (7th ed.).
Hobbes‘s Philosophy. The Scarecrow Cambridge University Press.
Press Inc. Sprenger, P. (1999). Sun on Privacy: ‗Get Over
McCarthy-Jones, S. (2019, October 21). Free- It‘. Wired News. Retrieved from: https:-
dom of Thought is Under Attack – //www.wired.com/1999/01/sun-on-pri-
Here‘s How to Save Your Mind. The vacy-get-over-it/.
Conversation. Retrieved from: https://- White, R. C. A., Ovey C., & Jacobs, F. G.
theconversation.com/freedom-of-tho- (2010). Jacobs, White and Ovey: The
ught-is-under-attack-heres-how-to-sa- European Convention on Human
ve-your-mind-124379. Rights (5th ed.). Oxford University
McGregor, V. K., Calderón S. H., & Tonelli R. Press.
D. (2013, November). Big Data and Wilde, O. (2011). The Picture of Dorian Gray.
Consumer Financial Information. Busi- Retrieved from: http://www.planetpub-
ness Law Today (pp. 1-9). Retrieved lish.com/wp-content/uploads/2011/11-
from: https://www.jonesday.com/en/- /ThePictureofDorianGrayNT.pdf.
insights/2013/11/big-data-and-consu- Wong T. (2019, May 9). Singapore Fake News
mer-financial-information-ibusiness- Law Polices Chats and Online Plat-
WISDOM 1(14), 2020 146 146You can also read
