A Literature Review of Music in Computer Science - UOB ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
International Journal of Computing and Digital Systems
ISSN (2210-142X)
Int. J. Com. Dig. Sys. #, No.# (Mon-20..)
A Literature Review of Music in Computer Science
Harco Leslie Hendric Spits Warnars1 and Winston Rusli2
2
Computer Science Department, BINUS Graduate Program- Doctor of Computer Science, Bina Nusantara University, Jakarta,
Indonesia 11480
1
Computer Science Department Global Class, School of Computer Science, Bina Nusantara University, Tangerang, Indonesia 15143
E-mail address: spits.hendric@binus.ac.id, Winston.rusli@binus.ac.id
Received ## Mon. 20##, Revised ## Mon. 20##, Accepted ## Mon. 20##, Published ## Mon. 20##
Abstract: Music has been a big part of human life since ancient civilizations and listening to music is fun and besides for the benefit
of humans themselves. Over time, computer science continues to develop and music begins to be recorded digitally and belongs to the
group of unstructured data whose data needs to be managed so that it becomes structured data. Much academic work has analyzed how
music is digitized and there is also mixed analysis of the negative effects of music, but most research agrees that digitization has many
positive effects in terms of the computer science-backed world of music. A large number of papers analyze what people think about
the use of technology in music education and several have proposed several different ideas that can be applied. This paper analyzes
and reviews papers that discuss the application of computer science technology that plays a role in managing digital music data, which
is better known in the form of sound or sound. The literature review in this paper is carried out in 4 steps, namely paper collection,
selective screening, classification, and summary analysis. The literature review discussed in this paper is divided into three parts,
namely the rise of digital music, how computer science can be integrated into music education, and music creation through computer
science.
Keywords: Music in Computer Science, Computer Music, Music Pattern Recognition, Computational Music Thinking
blowing or produced with one or a combination of several
1. INTRODUCTION musical instruments designed to produce a musical sound.
Computer science has developed rapidly in the last few In addition, music can also be accompanied by a human
centuries and has influenced people's lives so much that choir consisting of six types of which there are 3 types of
most of our daily activities are confirmed to be using female voices, namely alto, Mezzo-Soprano, and Soprano,
computer science. Through computer science, many as well as 3 types of male voices, namely bass, baritone,
technical marvels have been created and enabled the and tenor.
emergence of large corporate management, social We conduct a literature review as a starting point for
networking sites, and much more. Nowadays the researching the knowledge of computer science which
smartphones that we use are some of the gadgets that use limits the use of unstructured data, namely data in the form
this computer science technology and it is hard to imagine of sound. What this paper does is that first, we did a Google
how to live without using computer science technology. Scholar search for papers of all time with the phrase
The development of research in the field of computer "computer science music" and got 2,900,000 search results.
science is still being developed and explored and in its However, when each search result link is opened and read,
development, it is combined with many things according to it turns out that not all search results represent the searched
its uses and applications, including music. Music is an art phrase. Then the same search for papers all the time with
form that has been a big part of human cultural activity the same phrase but limited to the search only on article
since time immemorial and over time, it is only natural that titles and returns only 44 papers containing that search
we should explore the possible applications of computer phrase. This shows that research on music data processing
science to music. with computer science or the use of computer science for
Meanwhile, the term music is a sound that is arranged music is still interesting, few have done it and interesting
in such a way that it can be heard by humans which to observe.
contains rhythms, songs, tones, and sound harmonies that
can be produced naturally, such as the sound of the wind
E-mail: spits.hendric@binus.ac.id, Winston.rusli@binus.ac.id
http://journals.uob.edu.bh
2 Winston Rusli, Harco Leslie Hendric Spits Warnars: A Literature Review of Music in Computer Science
In addition, the reason this literature review was created papers with titles that seem related to our main topic and
is that very few literature reviews have been conducted on some of which have special requirements if we want to
this particular topic. We believe that it is important that a access them. In this case, we find and limit the total to 120
literature review on this topic is conducted to gather and papers.
condense the knowledge and information we have on this
topic. Furthermore, through this paper, we want to convey B. Selective filtering
this to people in an easy-to-understand format where we try After collecting papers, in this second stage, we do not
to understand the problem based on understanding previous use all of them and limit them. We only want to use papers
research. Hopefully, we can develop interpretations of past that are relevant to the research we are doing, which is
papers and use this literature review as fuel for future related to the application of computer science technology
research related to the use of computer technology for voice to unstructured data in the form of sound, so we do
data. selective filtering. This stage is done by skimming all
academic papers obtained from the next stage and if the
In order to complete this literature review, it was done topic is not related to music in computer science, then we
in four steps as seen in Figure 1 where the activity was discard it. So that after selective selection, there were 62
carried on started from the paper collection, selective papers left.
filtering, classification, and analysis summarization. The
next section will explain in more detail the steps. C. Classifications
In this third stage, the classification stage is carried out
Paper Collection in 2 sub-stages starting with subtopic classification where
the classification will be divided into three sub-topics,
namely the rise of digital music, how computer science can
Selective Filtering be integrated into music education, and the creation of
music through the use of computer science. The next stage
is paper quality sub-classification which is divided into 2
Classification sub-stages, namely subtopic classification and paper
quality
1) Subtopic classification
Analyze and Summarize Although all the remaining papers have topics related
to music in computer science, we feel that each paper
Figure 1. Literature Review Creation Steps
individually focuses on a different subject matter, and if we
were to make a literature review of it, it would not be
2. LITERATURE REVIEW IMPLEMENTATION systematic and structured. Therefore, This is why we need
a way to divide the filtered papers. In order to review the
This section will be explained the four steps of literature academic paper most effectively and cohesively, we have
review activities carried in this paper as seen in figure 1 divided it into three parts, namely:
where there are four steps such as paper collection,
a. The rise of digital music
selective filtering, classification, and analysis and
summary. The newest form of music media is digital music and
in this section, we will explore and limit it to only
A. Paper collection papers that discuss how digital music emerged and
The first step in this paper is to collect a large number what impact it has on the music industry. In addition,
of academic papers using Google Scholar, which is a free we also explore some of the interesting uses of
web search engine that can show us scientific literature of computer science in music.
various formats and disciplines. To keep work current and
relevant, we use a filter that will only show academic b. How computer science can be integrated into musical
papers from 2015 to the present year 2021. To keep it as a education
generic search, we use the search term "music in computer Music education has become an integral part of
science" but return very limited results, with only 20 papers keeping our musical culture alive. This section will be
for publication published at any time. limited to papers relating to how computer science
To address this, we use more specific search terms with technology can be used to assist the music education
different levels of focus. Next, the search terms used are process.
digital music, music technology, music education c. Music creation through the use of computer science
technology, computer music, music generation, music
composition technology. From the results of filtering and In this section, we limit ourselves to papers that show
searching for papers using Google Scholar, we collect how computer science technology can help people
http://journals.uob.edu.bhInt. J. Com. Dig. Sys. #, No.#, ..-.. (Mon-20..) 3
with or without musical composition experience literature review. This step was explained in the third
become more creative in creating music. section as the next section where the explanation will refer
to the three sub-topic classification as mentioned before
2) Paper quality classification such as the rise of digital music, how computer science can
The second form of sub-classification carried out is to be integrated into music education, and music creation
separate papers based on the type of paper quality as shown through the use of computer science.
in table 1 below which specifically for journals will refer to
the scimago rating journal at https://scimagojr.com. Figure 3. LITERATURE REVIEW
1 is a graph of paper qualification classification referring to As mentioned before where this section will explain in
the data in table 1, where there is 7 paper type such as B.S detail the three sub-topic classifications mentioned before
Thesis, Ph.D. dissertation, Journal Q1, Journal Q2, Journal such as the rise of digital music, how computer science can
Q3, Journal no Q and conference proceeding where each be integrated into music education, and music creation
paper type has a total number of papers and refer to the through the use of computer science.
number of paper in the references in this paper. Paper type
Conference proceeding as the most references with 32 1. The Rise of Digital Music
papers where journal no Q as the least reference with only Music has become a big part of human life and has
1 paper. played a large role in human life for a long time and Tracing
its origins means understanding many things that can be
TABLE I. PAPER QUALITY CLASSIFICATION extracted from the music itself. In line with the sound, the
Paper type References Total
almighty certainly created the ear as a tool that can be used
by humans to listen to this sound or music. The formation
B. S. Thesis [1], [14], [28], [29], [34] 5 of the human ear when referring to the theory of evolution
Ph. D dissertation [2], [43] 2 is When mammals evolved, where they formed rib bones
from the jaw spine, where this would be used to receive
Journal Q1 [7], [12], [22], [25], [32], [51], [60] 7
sound waves in the air and of course these ribs would
Journal Q2 [3], [4], [5], [6], [8], [9], [11], [18], 11 become part of the ear. With a better level of intelligence
[19], [35], [52] and sense of hearing, humans learn to do vocalizations to
Journal Q3 [10], [13], [20], [24] 4 increase the beauty of the sound produced both with sound
and its combination with musical instruments which can
Journal no Q [40] 1
change the mood or please the mood when hearing certain
Conference [15], [16], [17], [21], [23], [26], [27], 32 types of music.
proceedings [30], [31], [33], [37], [38], [39], [41],
[42], [43], [44], [45], [46], [47], [48], Over time, humans began to experiment in creating a
[49], [50], [53], [54], [55], [56], [57], harmonious sound. Traces of this can be found in our
[58], [59], [61], [62]
ancestors, where early hominids would form groups to sing
and bond together. Finally, they began to make instruments
that can produce sounds and so that these sounds which are
called musical instruments are combined with the human
35 voice to produce sound sounds which are called music and
30 in essence can bring human emotions and feelings when
25
20 listening to music. certain. In his paper, M. Williams
15 reviews research conducted by other researchers and
10 concludes that after humans evolved to utilize language,
5 music was created to take advantage of the greater
0
efficiency of language as a bonding mechanism [1].
According to H. Sun, the most popular way of
recording music before the digital era was to use a
technique called analog recording. This allows music to be
played back at a later time. The shift from analog to digital
is believed to have started with the creating of the compact
disc, or CDs for short, in 1982. And when Peer-To-Peer,
P2P for short, a file-sharing service was introduced, the
Figure 2. Paper Quality Classification Graphic music industry began to thrive. forever changing.
Responses to these changes are varied and complex. Some
D. Analyze and summarize major labels are trying to defend themselves by reaffirming
In this last step, each of the papers was analyzed stronger copyright enforcement while independent labels
thoroughly and summarized and key points from each are embracing new opportunities. As the transition to
academic paper were taken and used as material for this digital took place, the first important advances were made
http://journals.uob.edu.bh4 Winston Rusli, Harco Leslie Hendric Spits Warnars: A Literature Review of Music in Computer Science
by Apple, a technology company. They developed the The encounter
iPod, a media player, along with iTunes, a media
management application. It created a platform for people to The first proposed idea is the ‘encounter’. It is defined
enjoy digital music and soon after, several other companies as the process that people use to gain non-conformist
created similar platforms, however, the nature of their knowledge on the Internet. To test this, Popham and
system left some to be desired [2]. Volpe use an adapted version of the Computer Self
Efficacy Scale.
Aguiar and J. Waldfogel describe their findings on how
unpredictable music quality uses digital sales in the United Shielding
States, Canada, and 15 other European countries by time The Internet acts as a shield for the perpetrators.
and vintage. From there, they concluded that the Meaning that they hide the negative behaviors of the
digitization of music has brought many positive side people. The scale of the measure was derived from a
effects. They point out that the overall quality of music has questionnaire by Lwin and Williams.
improved worldwide after the change. their research note
notes that products with modest release prospects will Imitation
occupy the top positions despite the concentration of The Internet enables the formation of custom networks
growth in sales and product numbers[3]. for digital gatherings focused on counter-hegemonic
Aguiar and Martens found that there are major changes exchange.
in the way people consume music based on data obtained Affect
from the IFPI, known as the International Federation of the
Phonographic Industry. From 1999 to 2014, sales of ‘effect’ concerns the person’s assessment of their
recorded music declined while sales of digital music behavior. A singular measure of effect was used, based
increased exponentially. With the advent and development on the procedures used by Panas & Ninni.
of digital music, there is a fear of music piracy and it is
feared that most people will choose to pirate music instead Moral disengagement
of buying licensed music. Therefore, it reduces music The people feel as though what they are doing is not
revenue and, in turn, eliminates the incentive to create too morally wrong A cumulative and singular measure
innovative music. An analysis of 16,500 clickstream was used based on the methods from Bandura et al.
activities of internet users in 2011 showed no evidence of
unauthorized music downloads on digital music purchases. They concluded that the discontinuous nature of the
On the contrary, it was found that there is a positive Internet may cause people to do digital music piracy [6].
relationship between them. From their statistics, it can be In their paper, J. Waldfogel proposes the golden age of
understood that digital music purchases would drop by 2% music as an elaboration how digitalization where most
if pirated music websites did not exist. It also shows that people assume that digitization to the industry would cause
different countries are affected by pirated music a great reduction in their revenue, which is especially true
differently. In addition, they need to take into account that for the music industry. However, it was observed that the
the music industry continues and will continue to grow [4]. main effect of digitization was reducing the creation of new
works to market. But it would also seem that digitization
Several people assume that the digitization of music has made music more widely accessible to people.
would cause the music industry to lose intermediation
between music producers and consumers. However, P. Digitization has indeed led to piracy, which in turn
Galuszka argues against this, as the music industry is causes decreased revenue. Threats to reduced revenue
exceedingly complex and intermediation provides several result in reduced cost. It is unclear whether copyright is
benefits to both producers and consumers. Music doing its function and needs further assessment. It’s a good
aggregators such as iTunes or Spotify allow major record thing to enforce rules against piracy, but it should not
company, independent labels, and artists to put their hinder creative output [7].
products on digital music stores for consumption [5]. R. Fleischer and P. Snickars took software studies and
The interaction order by Brewer and Goldsmith is digital humanities approach towards streaming media,
tested by several hypotheses by J.F. Popham and C. Volpe, where in 2017 Spotify has been using almost 100 million
where correlated with action in terms of piracy in terms of users. In their time spent on the research project, they
digital music. This provides an argument that the discovered that it was difficult for them to grasp the
hypothesis obtained in the study is that the internet is used definition and the nature of Spotify. This is why they made
to commit cybercrimes, especially in terms of piracy in an effort to localize and historicizing Spotify as a company.
terms of digital music without the permission of the owner, They attempted to localize Spotify as a company and
either on YouTube in the form of music videos. For this found it to be difficult as there are a lot of grey areas. In the
paper, five theories and five forms of measurements were year 2012, Snickars conducted a discussion together with
used: Sophia Bendz, who was Spotify’s Head of Marketing at the
http://journals.uob.edu.bhInt. J. Com. Dig. Sys. #, No.#, ..-.. (Mon-20..) 5
time. She was asked what type of company is Spotify Crooke expressed his research opinion in the hope of
classified as and she replied that it was a tech company. changing the way people view music technology in music
However, after some analysis, Fleischer and Snickars therapy. Where on the one hand music therapy can affect
stated that Spotify has become more of a media company. the development of the human psyche and besides that
there is a negative side where certain music can also
Aside from localizing Spotify, they also took a crack at damage the human psyche and morals with descriptions of
historicizing Spotify. They felt that this was important invitations to violate the law and crime. What's more, Hip
because Spotify is constantly changing and so their
Hop music is also strongly associated with social justice
research and findings should be based on historical acts which are because they are a product of minority
information. Due to its immense number of users and culture [11].
growing valuation, Spotify is often considered a prosperous
company. However, the researchers explored multiple Throughout the continuous development of the
other perspectives on how to view Spotify’s history and digitization of music, one of the most notable applications
noticed that it was filled with unachieved visions and regarding music is Shazam. A. L. Wang attempts to break
predictions [8]. down how Shazam functions. Shazam is an application that
can help people identify music that’s currently being
Contestability in digital music players was analyzed by played. Using the device’s microphone, Shazam takes in a
Dai and Yu and they hypothesize that due to threats from query containing the small and short ambient audio sample.
new entrants, the power of the firm’s market is held at bay. The smallest being as short as two seconds. Then, it
Others argue against this point as they think that a searches through its large database containing around 40
contestable market is a rare occurrence. At the beginning million songs. There may be distortions in the query audio
of the digitization of music, the market for portable music
and they may be filtered out. The algorithm Shazam utilizes
players was dominated by Apple. Apple developed the is comparatively better than previously available
iPod, a portable personal media player released in 2001.
algorithms. Firstly, it computes many times faster, with
Since 2004, the iPod was the dominant digital music player processing time as fast as a millisecond. It is also more
with its sales being 90% of the market in the US. quality-
tolerant towards distortion to the query audio by about 20
adjusted price index estimates as part of a study conducted to 30 db SNR. When comparing the efficiency. It was
by Dai and Yu, who used data between 2002 and 2010 in
found that Shazam is approximately 1e + 9 times more
which they found that there was a decline of about 20% efficient [12].
annually for the market price index. They consider this as
a supporting idea that proves their hypothesis to be correct Recording companies and music promoters make use
[9]. of data from sales reports to make strategic decisions
makings. In their paper, E. Kristiani et al. discuss data
Consumers of digital music usually keep their library of warehouse design techniques. To simulate real data, they
songs on their hard disks. One issue they may encounter is use chinook, which is a data sample available to the masses.
the duplication of music files. This is not optimal as they During the design process, they use Object Relational
take unnecessary space in the hard disk. H. I. Hamzah, T.
Mapping. The creation of the data warehouse design is to
J. Low, and A. A. Mutalib decide to tackle this problem. help relevant stakeholders in making important decisions
There have been existing applications that detect duplicate
[13].
music files but they still leave something to be desired.
They propose the Cognitive and Constructive Learning M. I. Fadhil, R. Magdalena, and I. N. Ramatryana
(CCL) approach. They came up with the Digital Music wanted to make a database containing the reff and verses
Archiving Model (DMAM). To test the effectiveness of of songs. However, doing separating these reff and verses
their model, they compare it to existing applications that manually would be very work-intensive and time-
detect duplicate music files. The analysis shows that the consuming. To help overcome this, the research on ways to
DMAM is better than the other applications [10]. automatically do these separations. They achieved a way to
do this by using correlation calculations between frames.
The article written by A. H. D. Crooke contextualize
music technology in Hip Hop. They start their paper by The system receives an mp3 song as an input and
explaining the historic significance that music technology converts it into small frames in the framing process. Each
has had to the Hip Hop tradition of beat making. As time of the frames is transformed using the Harmonic Fast
went on, the beat-making culture starts to expand and Fourier Transform method. The result determines the
diversify. This diversity extends to the equipment used. correlation pattern of the frames. Then, it can determine the
The article gives a typology of these beat-making location of the reff and verse. Lastly, the reff and verse are
equipment. It shows the varying music technology and taken from the song depending on the previously
states how they should be considered as instruments on determined location. The researchers conducted this
their own. It also explains how these technologies are process on 25 songs and analyzed the result The study
integrated into music therapy. produces an accuracy of more than 50% in determining the
location of the reff and verse. The best computation time
http://journals.uob.edu.bh6 Winston Rusli, Harco Leslie Hendric Spits Warnars: A Literature Review of Music in Computer Science
produced in this final project is 86 seconds with a 1000ms that this time, most of the participants are those with
frame to cut a single song mp3 file [14]. musical backgrounds, allowing them to more easily
understand HexaChord. The results showed that most were
Ubimus introduced in 1999 by Weiser is a research
impressed and astounded by the integration of computers
focus that combines Ubiquitous (Ubicomp) computing and on music learning and how mathematics can be used as a
music, which literally means Ubicomp calculates learning tool as well [16].
anywhere, anytime, anything, and every day, including the
definition of Invisible computing. , Silent technology, or In the process of learning a musical instrument, it is
Pervasive. computing. This Ubicomp examines the use of usually of great significance that the external instrument
software and hardware in implementing its benefits in the becomes one with the musician’s consciousness. M.
music field, which uses special software to produce digital Myllykoski et al. think that obtaining such skill requires
music following the selection of special hardware that can patience and perseverance. As an alternative, they
produce variations of the musical art itself [36]. developed SmartHand, which is a wearable device for
musical education. The idea was to reduce the need for
2. How Computer Science Can be Integrated to Musical external instruments and use something we are already
Education familiar with in terms of our sensory-motor skills. The
ARPiano, an abbreviation of Augmented Reality Piano, developed prototype was a wearable glove with touch
is an augmented reality keyboard designed for fun and sensors. One hand wears the glove and acts as an
efficient music learning. F. Trujano et al. discussed that instrument while the other hand plays it. The glove’s note
music sheets may be difficult and off-putting for beginners mapping uses the western tonal system but it may be
because they need to understand the musical theory before switched with other systems.
being able to read them. Using augmented reality, a
They used several different approaches to studying the
different method could be used by using a system where parameters needed for their project. They use cooperative
the user plays the note to correspond with the bars that
design to conceptualize and design the project. As for
appear. ARPiano can locate a physical keyboard and sketching the interaction, bodystorming was used. Use
overlay digital objects on individual keys and around the
scenarios were utilized to further explain the particulars of
keyboards. Furthermore, it has other features such as being the experiment. Interviews were also conducted to gain the
able to provide analysis on mistakes made during the
requirements of the project and user experience [17].
performance or play the song at different speeds [15].
P. R. Webster suggested that the use of computing
In their paper, S. Cannas took an interesting approach technology in music should be more than just using
as they analyzed how pedagogical methods can be used on hardware but also includes software that is combined to
learning both music and geometry. There exist several
create and arrange music so that humans are familiar with
musical graphs such as the Tonnetz, a note-based graph, it and assisted by computing technology in designing and
and the chicken-wire torus, a chord-based graph. There are making music. They outline that there is a newfound
also circular geometric models such as the circle of fifths respect for computer-assisted teaching. Interest in
and the tone-clock. These graphs are sometimes used in
constructionism as an idea underlying learning created
musical theory to help visualize and convey the relations more contemporary research on music technology. Such an
between musical objects.
approach focuses on creativity and motivation to learn
Festival of Science (2017) was held at the Vaisseau of using activities [18].
Strasbourg. There, they set up an IT-mathemusical G. Waddel and A. Williamon conducted research that
laboratory specially designed for children. Cannas used is focused on the usage of digital technologies by a student
two main methods for musical education: a software named in individual settings. An exploratory survey was made to
HexaChord and a wooden model of the Tonnetz and provide insight and knowledge on this topic. It had a total
chicken-wire torus. HexaChord was utilized due to its of 338 respondents, all of whom were musicians. The
ability to visualize the progression of a song in a geometric survey was shared online and contained several sections.
model. It was also possible to connect the software to a The first one is regarding the characteristics of the
piano keyboard to record a musical arrangement. The respondents. The second section is about their use of
children were interested in the wooden model more than technology in their daily life. The third section asks about
the software. One reason is that the children recognize the
technology in their learning process. The fourth and final
model to be entertaining. In comparison, the piano section asks them how they view future technologies.
keyboard and HexaChord were viewed as foreign items
they don’t understand. The results also showed that the From the result of the survey it was found that while
mathematical skills required in making chords were less musicians have several issues with technology, they
difficult compared to the musical skills required. generally have a positive attitude towards technology. They
actively use technology such as smartphones, laptops, and
A second activity was conducted in another Festival of desktops for video and audio recording and playback. The
Science (2017) held in Cagliari, Italy. What’s different is majority also prefer to use applications for music tools such
http://journals.uob.edu.bhInt. J. Com. Dig. Sys. #, No.#, ..-.. (Mon-20..) 7
as tuners, metronomes, and video or audio recording level of comfort with using technology, the average rating
instead of the actual equipment. It also shows that was 4.85 out of 7. Metronomes and audio recorders were
generally, musicians are optimistic towards future among the most commonly used tool. Overall, the teachers
hypothetical technologies [19]. were divided in their opinion if the use of technology helps
their students or not. It is believed that a large number of
A study conducted by S. Nart aims to explore the teachers refuse to use technology in their teaching because
software that may be used by music teachers and how they don’t need it to successfully teach their students
effective they are in educating the students. By the time of
music. The level of comfort with technology didn’t seem to
writing, there is much software that fits inside the category matter as they don’t see it as a requirement in their teaching
of musical education. There are many possible ways on process [21].
how the teacher may use this software as a tool in spreading
musical knowledge, composing, and performing music. W. I. Bauer and R. J. Dammers observe how collegiate
However, it is of utmost importance for the teacher to have music teacher education programs help preservice teachers
a good understanding of the software’s features. use technology in teaching the K-12 music curriculum.
From 250 NASM schools, half of them were randomly
In their study, Nart highlights five types of musical selected and from each of them, the head of music
software. Tutorial software, which is a less interactive education was asked to complete an online survey. A total
medium that displays detailed explanations on music. Drill of 89 respondents participated. After reviewing the result,
and practice software, which allows the students to practice approximately 80% of the respondents said that there is a
at whatever level they want. Game software educates music technology class for the student. 77% of them said
students through interactive computer games that may have that those classes included integrated information about
an evaluating system. Notation software, which lets the
pedagogical uses of technology. Bauer and Dammers feel
students arrange or edit music. Sequencing and recording that a single class is not enough. Technology, musical
software, which records the music and allows editing via
content, and pedagogy should be integrated into music
sequencer section. learning [22].
The main goal of software implementation in musical
Studio One is software used as a digital audio
education is to positively impact both the educator and workstation. In a paper by R. Putra, he proposed Studio
students. The expanded teaching methods support group
One as an alternative to traditional music education for
activities and allow an increase in scientific thinking skills. students of Sendratari PGRI Palembang. Initially, he
It also provides encouragement and motivation towards explains the importance of musical education, especially
those learning music [20]. regarding traditional music. The reasoning for this is that it
Similarly, a study was done by R. Uptis, P. C. Abrami is important we do not lose our tradition. He emphasizes
and K. Bose seek to understand how music teachers that the medium for education plays an important role.
consider the significance of self-regulation and how do Innovations in technology have allowed students to learn
they implement digital tools in their teaching process. music through a digital medium. This brings us to the main
Evidence from other researches shows that digital tools can point of the paper. Using software such as Studio One as an
help students learn music, especially correlating to alternative way of teaching music. Studio One is a product
supporting their self-regulation, given that they are guided created by the company Presonus, that connects MIDI
along the way by a teacher or mentor. The writers of the controller and sequencer audio digital for Windows. It has
paper wish to know if this is the case with independent comprehensive features that allow for productive audio and
music teachers. music creation. An experiment was conducted where it
uses the following equipment as Laptop/CPU with Studio
To answer the question, they created a survey to gather One installed, mouse, MIDI-Controller, and speaker.
information. In short, the survey consisted of questions Participants were then taught to use the Studio One
regarding the demographic of the participants, which were software to create musical melodies. From the gathered
all teachers. Their opinions and views on digital tools, results, it can be concluded that Studio One can be an
studio practices, digital tools usage, and development effective tool in aiding students in learning traditional
activities. The survey was conducted in Canada and music [23].
distributed across the dataset of The Royal Conservatory
along with music schools and conservatories. They also Another paper by F. D. Pramanta, A. Rohman, and M.
invited the teachers to reply to a prompt. In total, they R. Kurniawan tries to promote traditional music education
amassed 1,468 submissions and 443 responses for the with the use of modern technology. They explained the
prompt. difficulties in traditional music education, including getting
access to the instruments, which may cost large amounts of
The results show that the teachers think that it is very money. Their proposed solution was to use Realsense
important to have the existence of self-regulating activities based technology to simulate real instruments. It can be
as they have numerous amount of benefits both for the used to visualize traditional instruments such as the
teachers and the students. When asked to describe their Javanese gamelan. The output of the application will be
http://journals.uob.edu.bh8 Winston Rusli, Harco Leslie Hendric Spits Warnars: A Literature Review of Music in Computer Science
displayed on a mobile device for the students to see. They Nuterpea is a programming language used for teaching
hope that using this method, students at primary school and the art of music (www.nuterpea.org) so that students focus
middle school levels can gain a better understanding of on the music rather than programming languages
traditional instruments [24]. developed at the state university of the western state of
Colorado USA, where students who do not have a
Hhuang et al. explored how elementary school students, computer or music background are invited. to study
who are not familiar with music theory, experience algorithm coding with a programming language as a new
automated music composition tools. This research creates
model for music education. The curriculum is designed to
software that is used by students to compose and arrange include computer science in learning music theory around
their music which is an automatic music composition the music genre that is easily programmed with the
software in terms of music education. The melodies and
programming language and follows topics that contain
rhythm will be up to the students, but more abstract stuff music theory and structure and besides that programming
such as chord configuration and arrangement are generated language syntax is given in this case [41].
automatically by the software. In order to test the software,
students from the non-experimental school in their sixth EarSketch is free web-based learning accessible at
year are gathered to use the software. They were observed http://earsketch.gatech.edu developed at the music school
in the level of reaction, learning effectiveness, and the at the Georgia Institute of Technology, the USA which
correlation between them. The conclusion is that students aims to provide an understanding of computer science
using the ACME software gained improvements in terms through art learning. music for students in the university
of reaction level and learning level [25]. without any previous knowledge in music or computer
science[42]. The application of Earsketch is carried out
Whilst most propose technology to be incorporated in
using the Python or JavaScript programming language
the teaching process, T. Siburian, R. Dewi and Widodo which focuses on types of hip hop and dubstep music to
propose to use technology to help recommend course
learn the basics of programming languages such as
places. First, they explained that music has been a popular variables, loops, options, functions, strings, recursion, and
interest for many people and how many varying types it
others [43][44].
has. Due to this, people who are interested in learning
music may struggle on deciding where to start. A good Another study developed at the Lawrence University of
place to study music is in a course place but even then there Technology in the USA demonstrates an idea of teaching
are many types of course place for different kinds of music. mathematics and computer science to elementary and
middle school students by integrating programming
Their proposed method is to use a recommendation languages, dance, music, and the arts known as the
system based on the Multi-Factor Evaluation Process, or
MathDance & Music project. Teaching was piloted to
MFEP for short. MFEP takes in a large number of criteria study it using the scratch app version 2.0 which showed that
that will be used for decision-making. It then uses a students could easily learn math and computer science
weighting system calculation to give a score to each while simultaneously learning music and art. The
criterion. The scores allow for qualitative comparison of
MathDance & Music application is the same as any other
the different factors [26]. programming language that has variables, repetition,
Computer technology with special software and selection, functions, and others, where students learn to
hardware used to teach music to students with visual code with this application in composing music. [45].
impairment implemented in a rehabilitation center in A study conducted by the University of Applied
Leningrad, Russia. In its application, braille letters are used Sciences and Arts Northwestern Switzerland linked
to make it easier for students who have vision problems, teaching computer science and music science using five
and besides that, they are equipped with a 19-inch touch Computational Musical Thinking (CMT) patterns such as
screen MIDI keyboard, 2 active speakers, and a interpreters, interactions, opportunities, hierarchies, and
microphone are used to make it easier to see and rewriting rules in game design and simulation. The
communicate with the system[37][38]. In addition, the application of CMT greatly helps students in understanding
system is equipped with speech applications such as computer science and music simultaneously and it is
JAWS, NVDA for voice access to the system, including the proven in its application that students become more
use of games to increase student interest in learning [39].
enthusiastic in studying computer science and music at the
A project called L2Ork was developed at Virginia Tech same time even though they do not have the basics of either
University, the USA in 2009 with the aim of integrating computer science or music [46].
arts and science into primary and secondary school A study looked at the correlation between female
education in the USA where there are ten female and male gender in the relationship between music and computer
students aged 11 to 13 years. This project aims to make science education carried out in Brazil, in which this study
students proficient in creating and designing complex focused on female senior high school students who were
digital music arts while learning computer interested in continuing their education to university. These
programming[40]. female students were taught the basics of programming and
http://journals.uob.edu.bhInt. J. Com. Dig. Sys. #, No.#, ..-.. (Mon-20..) 9
the art of music including playing musical instruments This methodology combines the lego brick game as a
simultaneously and in addition, this project was carried out means for elementary school students to learn basic music
using games with the aim of attracting female students to notation and in practice, these students are required to play
be more interested in studying simultaneously, namely the lego brick which is equipped with basic music notation
music science and computer science. This research shows on each brick and played physically as well as on the
that female students are successful in learning the arts of second block. Step. students work in teams to create music
music and computer science simultaneously and the coding software by laying virtual LEGO bricks [54].
application of games makes it more interesting and easier
to follow the subject matter presented[47]. Google Blockly is a visual programming language that
has been implemented in many other fields such as its
A study of multiculturalism in the US was undertaken application to create applications so that girls like to learn
to identify the teaching of computer science and its effects programming languages and the application was
on black music and its application for new computer successfully implemented for research on 24 young girls in
science students and non-computer science sophomore Germany [55], besides that, it was applied to children
students. This study aims to increase diversity in the field primary school in Bucharest, Romania in understanding
of computer science where the 2015 data shows only 8.6% computer hardware theory studied the Internet of Things
black / African American, 19.6% Asian, and 6.8 Hispanic / (IoT) and its application is named Wyliodrin [56]. In
Latino have jobs in their respective fields in computer addition, research in Japan was carried out by making
science. Therefore teaching the basics of computer science prototypes for the SPIN process to increase software
combined with the application of black music such as Hip- reliability in reducing bugs and errors [57]. In addition,
hop, R&B to increase interest in learning, especially for there are many more applications, but not all of them
black or African American students in the field of computer intersect with applications in the world of music learning.
science[48].
3. Music Creation Through the Use of Computer
A study conducted at the College of Charleston, USA Science
where data used between 2010 and 2014 showed brilliant
As computer science is continuously being
results when course delivery in programming basics was
linked to music theory and practice for first-year students implemented in music, the next logical step is on the
automated generation of music. However, achieving a
from multiple disciplines. The subjects provided include
computers, the art of music, music appreciation, and the realistic generation is not an easy challenge. Prior works on
basics of music as well as the following subject matter such music generation focused mostly on symbolic
as introduction to computer music, music programming, representations such as scores, MIDI1 sequence, etc. Due
to this, several aspects of music are removed, resulting in
computer music performances, and others[49].
an impact on the music quality. One example of such an
A study developed an open-source tool called aspect is the imprecise timing of the notes played by a
BlockyTalky that can run on low-cost processors such as musician.
the raspberry pi interactive computer music device for
A proposed solution for generating music is to model
teens between the ages of 11 and 13 to program music
behavior and user interaction and for that matter. case, the music in the raw audio domain. Models that are capable
of doing this are the autoregressive models. To be exact,
proving that computer music supports computer science
autoregressive discrete autoencoders are used to capture
learning as part of integrating computer music into the
computer science curriculum[50]. local signal structure in waveforms which are then used to
generate piano music. They experimented on using
BlockTalky is available in a web-based form with real- autoregressive models to create the long-range structure for
time sensors, message reception, and sending, which in its the audio signals. They modeled this structure on
development using the Google Blockly toolkit together approximately 400,000 timesteps, which allows them to
with Javascript and the programming languages Elixir and create stylistically consistent music. The result shows
BlockyTalky have been successfully applied to music successful generation with stylistic bits [27].
theory learning and practice in playing musical instruments
and have proven very easy. implemented even though they In their paper, R. Manzelli et al discuss the approach to
do not have knowledge of music or computer combining symbolic and raw audio models. They explain
that symbolic models can be used on note generation and
science[51][52]
raw audio models can be used to generate the
A study in Italy was proposed using the LEGO brick expressiveness and nuance of performed music.
game approach in the context of learning algorithmic
programming concepts and the application of Google They used two different types of deep learning models
Blockly visual programming in the context of learning to generate music. The first of which utilizes raw audio
music through programming concepts [53]. models. Initially, a raw audio generation used LSTMs.
Then WaveNet was developed by DeepMind. It was
capable of interacting with raw audio. The other deep
http://journals.uob.edu.bh10 Winston Rusli, Harco Leslie Hendric Spits Warnars: A Literature Review of Music in Computer Science
learning models used are the symbolic models. A L. Yang et al found the results of the experiment to be
commonly used one is MIDI however many other exists: quite satisfying. They feel that their model can be a great
Chord2Vec, piano roll representation, Blues Melody alternative to RNNs. They have plans on expanding
Generation and many more. MidiNet to generate music with multiple tracks as well as
velocity and pauses. Moreover, they also hope that the
A work-in-progress model was developed to achieve model would be able to implement music theory and
this by working with WaveNet architecture and the LTSM receive input from music information retrieval models [30].
network. In the future, they plan on combining them into a
coupled architecture, omitting the requirements to J. Granger et al. presents Lumanote as an interactive
synthesizing the files before usage [28]. music composition tool intended for beginner or
intermediate composers. After analyzing existing tools for
J. Cruz experimented to compare deep learning
musical composition, they have decided on four main
algorithms and Markov models on their effectiveness in requirements:
music generation. Early on in the deep learning model
development, a decision was made to use a Recurrent Build an efficient composition software
Neural Network (RNN) model as the basis. The final
architecture used is a Biaxial Recurrent Neural Network The software can be used with or without a musical
with several modifications. The model was separated into instrument
four and trained on music collections by four piano Show appropriate chords and notes to users with little
composers. Each of the models was instructed to generate music composition experience
100 songs. The resulting songs were very distinct and
pleasant to listen to but they do show apparent evidence of Guide uses to create songs without being forceful
being computer-generated as there are repetitive patterns
The resulting product can be accessed on the website
and abrupt silence. As for the Markov model, a Hidden
www.lumanote.net. It scans for any MIDI-type devices
Markov Model (HMM) was used. It undergoes the same
connected to the hardware. Then the user will be introduced
procedures as the deep learning model on generating
to the main interface and may begin composing. To see the
music. To compare the results between them, the models
effectiveness of Lumanote an evaluation was conducted on
were passed to a classifier that attempts to classify the
20 people. They were asked to create a simple piece of
music based on the composer they were trained on. Results music with and without Lumanote. Results showed that all
show that the songs made by the deep learning model were participants improve their composition’s quality and
more accurately classified compared to the Markov model
musicality after using Lumanote, regardless of their
[29].
musical experiences [31].
L. Yang et al have created a model called MidiNet, A new way of conducting computational calculation is
which is capable of music generation using a Convolutional through the use of biocomputers. Bio computers are
Neural Network (CNN) model. The model uses random
computers that use electronic components from biological
noises as input and generates the music in musical bars.
materials. With the development of biocomputers also
Distribution of melodies was possible due to the model also comes the development of bio-memristors, jointing passive
being a Generative Adversarial Network (GAN). Previous
components that serves as a resistor with memory. E. R.
knowledge can be utilized to achieve music that follows a
Miranda, E. Braund, and S. Ventakesh are interested in
chord sequence. A total of 1,022 MIDI tabs of pop music using the bio memristor to build a music generation system.
was acquired from TheoryTab and used as the training set. The particular bio-computer they use is the PhyBox. For
To reduce the complexity of the data, they filter out some their music generation system, they use a voltage that goes
of the chords and group the remaining tabs. Their model through the bio-memristor to represent music input. The
uses the TensorFlow library to help complete its training current obtained from the system’s memristive behavior is
and tasks.
the music output [32].
MidiNet was compared with three MelodtRNN models
Choi, Fazekas, and Sandler used the Repetitive Neural
by eight bars of melody generated by each model. Network model to introduce the use of automatic jazz
Analyzing them were 21 participants, 10 of whom have chord progressions in their use and implementation of
musical backgrounds of varying degrees. The
character and word-based rock drum tracks using LSTM
questionnaire was conducted online. They were asked to be units. Two LSTM layers were used, they both had 512
in a quiet room and were given random melodies generated
hidden units in them. The Keras framework was also used.
by the models. Even though all of the melodies were For the training data, musical pieces were taken from The
generated by a machine, the participants were told that Realbooks and The Fakebooks. The character-based RNN
some of the music may be or may not be real. Several
failed to create structure when composing drum tracks so
different questions were asked and by the end of the they proceeded with word-based RNN, which was able to
experiment it was found that MidiNet was capable of
perform well on both tasks. This proves that LSTM may be
generating interesting melodies in addition to being on par used as a tool in assisting music composition [33].
with MelodyRNN on being realistic and pleasant to hear to
http://journals.uob.edu.bhInt. J. Com. Dig. Sys. #, No.#, ..-.. (Mon-20..) 11
In their paper, R. B. Dannenberg explores the idea of MIDI files. Moreover, they would like to improve the
Languages for Computer Music as a language distinct from user’s interaction by incorporating other devices like the
ordinary programming languages. This software helps mouse or joystick [35].
composers to use computers effectively even if they have
no prior knowledge as software engineers. They point out In Brazil, early year students for computer science and
some of the differences between regular programming engineering study programs are directed to make music as
languages and computer music languages which include: a musical instrument for students who are interested in
researching the field of games which for them is part of a
• Programming languages focus on getting quick final project and where theory and practice about music are
answers while computer music generates 'answers' also delivered to the student[58]. In practice, they use the
within a set time. Java programming language as part of the object-oriented
programming concept and use the Greenfoot Introductory
• It is important To enable the processing of large Learning Environment concept [59].
capacity audio files, where the audio is counted
incrementally and constantly to pass the recorded A study was conducted at The Australian National
audio samples through the use of generators and University (ANU) reports on the activity of LENS: ANU
operators. Laptop ENSemble which combines musicians and brings
musicians and computer scientists into an ensemble laptop
• The application of musical arrangements requires where a musician or a computer scientist can hold a concert
accurate and dynamic calculations in which the with his laptop ensemble as a musical instrument[60].
preparation requires the use of human language which
is more expressive in conveying the meaning of the Another study was conducted at the College of
music than the human voice produced. Charleston, USA to develop a free and open-source
software called JythonMusic (http://jythonmusic.org)
They also show examples of existing computer music which was developed using the Python programming
languages: Music N, Max/MSP and Pd, SuperCollider, and
language with the Jython library which can be run with the
many more Even after many years of development, Java Virtual Machine (JVM) to create interactive music
computer music language is still constantly being
that helps composers with manipulation. music, pictures,
developed because of the large range of users with a large graphical interface, and includes MIDI support and OSC
range of needs [34].
library[53]. There have been several projects developed
In their paper, M. N. Caceres et al proposed a different starting in 2010 called Laptop Orchestra, in 2011 a project
idea for automated music generation. A system would called Monterey Mirror, in 2014 a project called Time
generate musical melodies while being supervised and Jitters, in 2015 a project called Diving into infinity and the
guided by a user using a mechanical device. Essentially, it Migrant Project, and lastly, in 2016 a project called
works by capturing the user’s movements with the SoundMorpheus. [54].
mechanical device and translating them into melodies. The Research conducted in the department of computer
machine can improve itself by learning from previous science at the College of Charleston, USA developed
compositions because it uses the Case-Base Reasoning
Specter which performs music information retrieval (MIR)
architecture. Furthermore, the machine can change the by combining sound spatialization for musical
user’s movement to create greater musical melodies.
arrangements and live music orchestras using Kinect
A mechanical device was incorporated because it is one sensors and interfaces such as hand tracking, smartphone-
of their goals to encourage human-machine collaboration. based OSC library controllers, and voice. in three
When using it, users are asked to create a profile and select dimensions. This system was developed with the open-
several parameters. Based on these parameters, the system source JythonMusic (http://jythonmusic.org) as software
may search from their internal memory or the Internet for for music processing, graphics, animation, and audio or
previous solutions. These solutions along with the user’s sound manipulation [61][62].
movement will be used as training sets for note probability
calculation. Based on how good is the result, it may be
stored in the memory for later melody generations. 4. AUTHORS’ OPINION
There were three experiments conducted during this Conducting this literature review has been an
research. The first one is to check the learning architecture. enlightening experience for us as it has taught us about
The second checks the features of the system. The last one how music and computer science can be integrated. It has
conducts a usability and listening test. Feedback from users given us a new appreciation for how far the music industry
indicated that the melodies generated by the system are has evolved as well as for how influential technology is to
good and follow a single style. In the future, they would our life. We agree with the notion that digitization has
like for the system to take in emotions as another input for brought a positive impact on the music industry and how
the music generation. They would also like to introduce a we consume it. Regarding using musical education
way to include musical styles not depending on tagged
http://journals.uob.edu.bhYou can also read
