Abstract
In this paper, a system to build music in an intuitive and accessible way, with Lego bricks, is presented. The system makes use of the new powerful and cheap possibilities that technology offers for making old things in a new way. The Raspberry Pi is used to control the system and run the necessary algorithms, customized Lego bricks are used for building melodies, custom electronic designs, software pieces and 3D printed parts complete the items employed. The system designed is modular, it allows creating melodies with chords and percussion or just melodies or perform as a beatbox or a melody box. The main interaction with the system is made using Lego-type building blocks. Tests have demonstrated its versatility and ease of use, as well as its usefulness in music learning for both children and adults.
Similar content being viewed by others
Explore related subjects
Find the latest articles, discoveries, and news in related topics.Avoid common mistakes on your manuscript.
1 Introduction
Music and technology are part of the everyday life for many people in the whole world. There is a growing number of original ways of creating, interacting, and playing with music using different possibilities technology can offer. For several years now, new systems appear constantly in which the musical experience is more and more immersive and does not rely on physical instruments, but generic commercial or specific systems for motion detection [15, 16, 24, 34, 40] or the interaction with elements of a physical desk (pencil, pen, table, etc.) [12] or other interaction means. In addition, more and more studies are emerging to understand music in relation to the way people interact with it [11, 13]. Another good example of the work carried out within this framework is the H2020 project #MusicBricks [37] that presents many and diverse interfaces of interaction and music creation oriented to both entertainment and music professionals.
Music composition is commonly seen as a complex task that can only be accomplished by experienced, well-trained individuals or by specialized, automatic systems [14, 20, 25, 32, 33, 41]. This fact makes this creative activity to be beyond the reach of most people.
However, music composition systems for children are beginning to emerge as a creative element in their education [1], starting in kindergarten [18]. These systems use Lego pieces as a manipulative element, but the generation of sounds and the actual composition must be done on a computer [2]. There are other Lego-based music learning systems that use piano roll-like notation [21, 26, 28].
In this context, we present a new way of music creation using modified actual Lego-Duplo building blocks to enable most people to create music melodies with a rhythmic base and even chord accompaniment by building with bricks. The target audience is especially children, but it can also be used with elder people to train in therapeutic psychomotricity. The system consists of three main parts: one to synchronize (Synchronism Box), another one to set the rhythm (beatbox), and a third one to create the melody (melody box). Each part of the system is controlled using a Raspberry Pi (RPI) [31]. Note that our approach does not aim to give aids for the composition of musical melodies or automatically create music [14, 20, 25, 32, 33, 41] but to provide a different playful way to approach basic concepts of music and music creation.
The paper outline is as follows: in Sect. 2, the block diagram of the whole system and the description of the three main subsystems and the communication between subsystems are presented. Specifically: in subsection 2.1 the data communication protocol between the three subsystems is presented; subsection 2.2 describes the synchronism subsystem used to synchronize signals and processes for the whole system; next, in subsection 2.3 the beatbox subsystems that create the rhythmic base of percussion to accompany the melody is presented; the description of the subsystems ends with subsection 2.4, in which details of the melody subsystem are exposed, this part is responsible for creating the melody and accompanying chords. Section 3 displays several usage examples, results, and a discussion on the evaluations carried out by external users. Finally, the last section presents the conclusions drawn from this work.
2 System design
Figure 1 shows a block diagram of the whole system. This is an interactive music demonstrator that allows creating a melody with chords and percussion easily and intuitively using Lego-type building blocks. The system can work as just a beatbox or just a melody box, by simply removing the melody box or the beatbox, respectively, as the diagram in Fig. 1 illustrates.
The complete block diagram including the systems, data and control lines and external devices: sound card, computer and speaker of the devised system is presented in Fig. 2. In this figure, it can be observed that three main subsystems are designed: synchronism box, beatbox and melody box. These parts are interconnected by MIDI cables using standard MIDI protocol. Three separate Raspberry Pi’s are used to control and process all the signals and ensure modularity. Finally, in order to convert the standard MIDI signal to sound, an external sound card connected to a computer running a VST plug-in is needed. Raspberry Pi’s are used to control all the parts of the system separately. Its versatility, processing capacity, and the different available physical ports included make it possible. In Fig. 3, the input/output ports of the Raspberry Pi are shown; these include 26 GPIO (General Purpose Input Output) pins. These pins can be used as input, output, or both and are controllable by the user at runtime. Among the GPIO pins, \(5\) of them allow communication through the SPI (Serial Peripheral Interface) protocol [7] (note that SPI needs to be enabled), 2 pins that allow communication through the I2C (Inter-Integrated Circuit) protocol [27] and 2 others intended for UART (Universal Asynchronous Receiver-Transmitter) connection [30] for serial port communication. Note that, it must be taken into account that the GPIO pins offer a voltage of 3.3 V and are not tolerant to voltages of 5 V; this fact is relevant for the system design.
2.1 Data communication
Data communication between the three subsystems shown in Fig. 1 and Fig. 2 is carried out by using the standard MIDI protocol. This choice has inherent hardware implications, specifically: the use of 5-pin DIN connectors and the voltage levels employed by MIDI signals, which require voltages between 3.5 V and 5 V to represent the high logic level. Recall that Raspberry Pi works at 3.3 V maximum. In order to solve this issue, additional hardware was designed for the MIDI out and MIDI in connections. The schematics for both the MIDI in and MIDI out connections are shown in Fig. 4 and Fig. 5, respectively [23].
The hardware created for MIDI out is shown in Fig. 4. This design allows to convert the 3.3 V output of pin UART-Tx of Raspberry Pi (RPI) to a 5 V output that can be driven to the 5-pin DIN MIDI connector.
The hardware for MIDI in is shown in Fig. 5. Observe that this interface needs a more elaborated design, and additional hardware since it is necessary to isolate the input signal, which ranges between 0 and 5 V, from the UART-Rx input of RPI, which handles up to 3.3 V. This is achieved by the utilization of an opto-isolator, as shown in Fig. 5 [23].
2.2 Synchronism subsystem
The first subsystem shown in Fig. 2 is devoted to provide with the synchronization signal for the whole system. This subsystem creates a MIDI clock signal that will be sent through the whole system to allow for synchronized tempo for music composition.
In this subsystem, beats per minute parameter (BPM) is set, together with the time signature, and the maximum number of quarter notes the melody and the rhythm signal will have before the end of a cycle.
In Fig. 6, the MIDI clock signal is shown conceptually. In this case, the maximum amount of quarter notes considered is \(16\).
For each quarter-note, a Continue signal is sent. During each quarter-note period, \(24\) Timing Clock signals are sent. For every \(16\) quarter-notes, a Start signal is sent. The time between each quarter note is set by means of the selected BPM, so that the duration of a quarter note is \(=60/BPM\) \(sec\). The use of the Continue MIDI message is standard in sequencers and it is needed in order to decide with quarter note, among the \(16\) in the board, must be read. The maximum \(BPM\) value is limited to 208; this limitation is due to the processing time of the RPI.
The clock signal is sent through GPIO14 (UART0_TXD) UART (Universal Asynchronous Receiver/Transmitter) of the RPI (see Fig. 3). This is an asynchronous serial communication protocol.
2.3 Beatbox subsystem
After the Synchronism subsystem, the beatbox subsystem is described. The aim of this subsystem is to create the rhythmic base of percussion to accompany the melody. Note that this subsystem can be removed from the chain to let the melody subsystem alone with the synchronization one, or it can be used without the melody subsystem as well (Fig. 1). In order to achieve this behaviour, the output of this box, MIDI Rhythm, must be added to the MIDI clock.
It must be taken into account that, in General MIDI standard streams, channel 10 is reserved for percussion instruments [9]: each MIDI note in this channel corresponds to a single note of a percussion instrument. In the system presented, four different percussion instruments, with total duration corresponding to 16 quarter-notes, are considered.
As it is shown in Fig. 2, the beatbox is divided into tree blocks:
-
Beatbox composer: this is the hardware structure for building the rhythmic pattern in a visual and intuitive way.
-
Beatbox controller: this part includes a MIDI-out port, and a MIDI-in port. These pieces, described in Sect. 2.1, and Figs. 4 and 5, are designed to allow the connection of the Lego beatbox hardware to the RPI.
-
Beatbox RPI. The RPI is connected to the beatbox controller by one 40-flat cable grid.
These three blocks are described in deeper detail next.
2.3.1 Beatbox composer
The structure for building the rhythmic pattern is shown in Fig. 7. It is a printed circuit board of dimensions \(16\times 23.5cm\), on which the components are soldered. The logical mapping of the hardware shown in Fig. 7 into its musical counterpart is exposed in Fig. 8.
As shown in Fig. 7, the composer includes a push-button with light for each of the 16 quarter-note periods of the 4 percussion instruments selected. This leads to 16 × 4 push-buttons. So, the output of the beatbox composer is formed of 16 × 4 lines. When a button is pressed the line connected to ground and the output is \(0\) V, otherwise, it is \(3.3\) V. This output is connected to a controller to carry all data to the RPI.
2.3.2 Beatbox controller
Each instrument in the beatbox composer allows for 16 quarter-notes. Since the RPI does not have 64 entries, several multiplexers are needed to adapt the input to the controller. Specifically SN74LS151N [38] multiplexers are selected for the 8 to 1 multiplexing and SN74LS157 [39] multiplexer for the four 2 to 1 multiplexing. Figure 9 shows the circuit diagram for each percussion instrument. The control signals A, B and C are common to the two multiplexers 8 to 1. The control signal D that reaches the multiplexer from 2 to 1 determines if the definitive output of the channel belongs to the first 8 quarter-notes or the last 8 ones. The scheme in Fig. 9 is repeated for each of the 4 percussion instruments.
2.3.3 Beatbox RPI
The Beatbox RPI receives the MIDI clock through GPIO15 (UART0_RXD) (see the physical ports of the RPI drawn in Fig. 3).
The function of the beatbox RPI is to turn the rhythmic base of the percussion into a MIDI stream, and combine it with the MIDI clock signal.
The combined structure of the MIDI sequence of the clock signal and rhythm signal is represented in Fig. 10. The rhythmic base is included after the Start signal and the first 24 Timing Clock events. The MIDI sequence for the rhythmic pattern for each quarter-note period includes Mode change to channel 10 (percussion instruments) [23], All Notes Off (to turn off the previous percussion instruments), and, finally, the active notes. Figure 10 presents an illustration in which in a certain quarter-note period all the four instruments (bass drum, box, charlie 1 and charlie 2) are active.
2.4 Melody subsystem
This subsystem is aimed to create the melody and accompanying chords. So, the output of this subsystem will be a MIDI stream composed of the melody, accompanying chords, rhythm, and clock. The MIDI channel 1, configured as polyphonic, is used to send the melody, and the accompanying chords to the VST plug-in.
The Melody subsystem is composed of tree blocks (see Fig. 2):
-
Lego melody composer: this is the structure created for building the melody in a visual and simple way.
-
Melody controller: this block includes a MIDI-out port, and a MIDI-in port with the adaptation hardware presented in Figs. 4 and 5 and the necessary hardware that allows to connecting the Lego melody composer to the RPI.
-
Melody RPI. The RPI runs the software to create the music from the constructions built by using the Lego melody composer scheme. It is connected to the melody controller by a 40-line flat cable.
2.4.1 Lego melody composer
Figure 11 shows and example of construction of music, including melody and accompanying chords.
Figure 12 shows the transcription into a music score of the melody and accompaniment built in Fig. 11.
The base for the construction of melodies, Fig. 11, consists of a Lego Duplo board with dimensions \(9.5\times 51cm\), containing \(4\times 16\) pairs of studs, although just \(3\times 16\) pairs of studs are employed in our design. The three construction rows of pairs of studs used in our design are clearly shown in this figure; each of them has a different function:
-
The first row is used to build the melody. Figure 13 shows how the pitch of each quarter-note is encoded by building pitch-towers with Lego-type pieces. It can be observed that the note range considered is \(11\): from \(C4\) to \(B4\). The lowest note (\(C4\)) is encoded using one Lego piece. Then, the pitch is raised half a tone for each new piece stacked. If no piece is placed, a quarter-note rest is interpreted.
Lego pieces need to be modified to build a melody so that the RPI can identify the pitch desired at each quarter-note period. The pieces are modified as shown in Fig. 14; each piece contains a \(50K\Omega\) resistor connected in such a way that stacking pieces means to connect resistors in parallel. Therefore, the equivalent resistance of certain number \(n\) of stacked pieces will be determined by equation (1):
Then, the stacked pieces are connected to a voltage divisor structure as shown in Fig. 15, where \({Z}_{1}=10K\Omega\). Thus, the voltage, \({V}_{out}\), for each quarter-note period depends on the equivalent resistance of the stacked pieces according to Eq. (2):
-
The second and third rows are used to determine if the melody note has a minor or major chord as accompaniment, respectively. If both pieces are connected, the system always decides major chord. Lego pieces have been customized for chord selection as shown in Fig. 16. These pieces are different to the ones used to build the melody since their only function is to determine whether a chord is enabled or not; so they are built to simply create a short circuit when placed on the board.
The Lego melody composer is connected to the melody controller with four \(16\)-line flat cables; the additional row with \(16\times 1\) pairs of Lego studs provide capacity for further extensions.
2.4.2 Melody controller
The Raspberry Pi must convert the sequence of stacked Lego pieces in the first row of the Lego melody composer into the corresponding musical sounds to be inserted into the MIDI stream. However, recall that the Raspberry Pi only has digital inputs so, in order to interpret the inputs from the composer, an analogue to digital converter is used. Specifically, the MCP3008 has been chosen [22]. The MCP3008 is a low-cost 8-channel 10-bit analogue to digital converter that performs the conversion presented in Eq. (3):
where
-
DOC stands for the Digital Output Code of the converter.
-
\({V}_{out}\) is the voltage that depends on the equivalent resistance of the stacked pieces, defined in Eq. (2).
Note that, due to the tolerance of the resistor (5% for Lego pieces and 10% for reference resistors), the Digital Output Code from Eq. (3) must be considered a value in the interval corresponding to a certain note \(i\), within the range of \(DOC\) values for each note in the system. Due to the fact that the notes range from C4 to B4, the note identifier, \(i\), is in the interval \([\mathrm{1,11}]\). Thus, note \(i\) will be decided if \(DO{C}_{i}\) is in the interval expressed by equation (4):
where \(DO{C}_{i}\) is the centre of each decision interval obtained assuming ideal resistor values.
Since the maximum length of the melody is \(16\) quarter-note periods, two MCP3008 are employed, and their outputs are driven to a multiplexer. The Raspberry Pi will control which signal goes to the line MELODY and convert the \(DOC\) into the corresponding MIDI channel. Figure 17 shows the overall block diagram: the control of the two MCP3008 chips is done by using one set of GPIO outputs, and only one pin is used to control of the SN74LS151N multiplexer [38].
The second and third rows of the Lego melody composer have simpler operation since the connection of each Lego piece simply changes the voltage level form \(3.3V\) to \(0V\); consequently, only two levels of the AD converter are needed.
2.4.3 Melody RPI
The melody RPI uses the information received from the composer through the controller to create and insert the corresponding MIDI events into the MIDI stream. The structure of the MIDI sequence including the clock, rhythm, melody, and chord is represented in Fig. 18.
The melody and accompanying chords are inserted after the rhythm signal. The MIDI sequence for the melody and accompaniment for each quarter-note includes Mode change to channel 1 piano, All Notes Off, to turn off the previous notes, and, finally, the active notes. Figure 18 shows the case in which a certain quarter-note period contains both melody and accompanying chord (\(4\) notes). Note that, thanks to the use of MIDI protocol, the sound of different instruments [23] can be simulated by simply changing the channel number.
The system as it is built does not allow for immediate expansion. In order to extend the system to create longer melodies, it would be necessary to modify the RPI programs accordingly, as well as to extend the hardware of the beatbox controller and the melody controller, and to build a beatbox composer, and a Lego melody composer with more connectors, and an adequate multiplexing scheme.
3 Results and discussion
This interactive music demonstrator allows creating melodies with accompanying chords and percussion in an easy and intuitive way. Figure 19 and Fig. 20 show examples of use of the Lego-based music composition system developed. As the examples show, it is an easy-to-use system that allows the intuitive generation of melodies and rhythms in real time. Moreover, it should be noted that a simple snapshot of the Lego construction with the system designed represents the score of the melody and accompanying chords, and constitutes a simple way to save the creations; Figs. 19 and 20 illustrate this fact and the relation between the constructions built and the corresponding scores.
This system was presented at Transfiere 2020, obtaining great public success [8]. Transfiere 2020 is the main R + D + I meeting in Southern Europe to share scientific and technological knowledge, promote innovation and connect science and business. This edition was attended by about \(4600\) professionals from more than \(1800\) different entities, representing \(30\) different countries.
The comments received from the subjects playing with the system were all very positive. The system was tested by many of the attendees at Transfiere 2020, since it attracted a lot of attention as it was generating music continuously and its main means of interaction was the Lego pieces. All the participants testing this system found it appealing and easy to use; we think this is due the fact that the system designed and built is based on a very common and well-established construction system, and to the immediacy playing the music being created. People with musical knowledge required neither help nor previous explanations to use this music construction system, whereas non-musicians were able to construct their own melodies and music after a brief explanation.
The general feeling of all participants was that the system was useful for learning music in a very entertaining way. It is noteworthy that those who asked about the system spent some time constructing melodies, listening to them, making variations, etc. Also, it became clear that some training is required to achieve satisfactory results in terms of melody and rhythm working together, using the full system: the rhythmic pattern builder and the Lego melody and accompanying chords composer; however, the learning curve was not observed to be steep, on the contrary, it seemed really accessible for subjects with certain musical knowledge. On the other hand, the utilization of the composition subsystems separately was considered very simple, and lead to reasonably satisfactory results almost immediately.
In general, the attendants that played with this music composition system recognized and emphasized its usefulness for entertaining young children while creating music. Moreover, its convenience for occupational therapy was also highlighted, specifically for elder people in need of both motor and cognitive rehabilitation or training and in therapies with autistic children.
There are some other advanced systems that use Lego pieces and sensors that got some social interest and relevance; those schemes are aimed at creating musical instruments in a way that encourages creativity and learning engineering concepts [10, 17]; even a micro-tonal guitar with Lego pieces has been designed [29]. However, a different perspective is considered in the design and construction of the system presented in this manuscript; in this case, our system, for the creation/construction of music, doesn’t rely on the design of new instruments but on building music in a manipulative and simple way, linked to the natural interpretation of rhythm and height of pitch.
Music and building with Lego pieces are two well-differentiated tasks employed to stimulate different motor and intellectual functions of children [3, 4]. Considering this context, the system presented in this manuscript allows working both skills: to learn music and to improve fine motor skills simultaneously, and with high degree of interactivity and immediacy.
Music and Lego, separately, are also used as therapy to reduce post-operative pain in children [35]. With the system presented in this paper, music and Lego can be used together, and this kind of therapies could be improved by their simultaneous interrelation.
Also, for several years now, work has been done on new therapies for improving inclusion and social skills among children and youth with autism. Two commonly used elements to attain these objectives are Lego therapy [19] and music [36]. A system based on Lego, which allows creating music in an interactive and collaborative way, as the one presented in this paper, can be very useful for the treatment of children and youngsters with autism.
Regarding occupational therapy for the elderly, new apps and games are appearing [5] lately, as well as many works related to cognitive stimulation with music [6]. Precisely, this is one of the application environments highlighted by some users testing the system at Transfiere 2020 regarding the benefits that could be attained by using this system; the benefits specifically considered were related to psychomotricity and cognitive skills (Table 1 Appendix).
4 Conclusions
A system for building music melodies with a rhythmic base and chord accompaniment with Lego bricks using Raspberry Pi has been designed. The target audience is formed mainly of children, but the system developed is considered to be entertaining and beneficial for people of all ages.
The system consists of three main parts: one to synchronize the whole scheme (synchronism box), another one to set the rhythm (beatbox) and one to create the melody and accompanying chords (melody box). Each part of the system is controlled by a separate Raspberry Pi (RPI).
The modular structure designed allows the use of the complete system (rhythm and melody) or only a part of it (only rhythm or only melody). This gives the system a wide range of utilization possibilities.
The design of the system presented is radically different from other approaches found in the literature, which use Lego and music from the perspective of the construction of new instruments; our novel approach is linked to music representation in terms of rhythm and score, and the natural interpretation of the height of pitch. This approach makes the system designed easy to use, and together with is immediacy creating music, suitable for the rehabilitation of elderly people or for the cognitive stimulation of young children.
The use tests that have been carried out with the system in a real-world scenario have unveiled great interest, and discovered the usefulness of this system for learning and entertainment for both young and elder people. Some direct applications identified are: joint stimulation of children’s motor and intellectual functions, therapy to reduce pain in the postoperative period of children, improvement of inclusion and social skills among children and youth with autism, and occupational therapy for the elderly.
With all this, it has been found that the design and implementation done are really satisfactory, although improvement and extensions could be considered, like the implementation of a simple hardware interface to set the BPM parameter, the possibility of cascading several subsystems like the one shown in this manuscript (however, note that this would require major redesign of some schemes, and further limitation of the BPM parameter), or the capacity to build with other musical figures.
Code availability
Code can be made available upon reasonable request.
References
Baek Y, Taylor K (2020) Not just composing, but programming music in group robotics. Music Educ Res 22(3):315–330. https://doi.org/10.1080/14613808.2020.1767558
Baratè A, Ludovico LA, Malchiodi D (2017) Fostering computational thinking in primary school through a LEGO®-based music notation. Procedia Computer Science. Knowledge-Based and Intelligent Information and Engineering Systems: Proceedings of the 21st International Conference, KES-20176–8 September 2017, Marseille, France, 112:1334–1344. https://doi.org/10.1016/j.procs.2017.08.018
Bugos JA, DeMarie D (2017) The effects of a short-term music program on preschool children’s executive functions. Psychol Music 45:855–867
Bugos JA, DeMarie D, Torres MR, Lamrani D, Gbadamosi AA (2023) The effects of a multimodal music program on young children’s facial expressions during controlled singing tasks. Music Sci 27(1):54–69. https://doi.org/10.1177/10298649211021463
Cabrera-Tigre P, Villa-Matute V, Lema-Condo E, Parra-Luzuriaga K, Robles-Bykbaev Y, Robles-Bykbaev V, León-Gómez P, Tapia-Rivera C. An interactive system based on personal area networks, serious games and data mining to provide rehabilitation activities for older adults with cognitive decline. In 2020 IEEE World Conference on Engineering Education (EDUNINE), pp 1–5. https://doi.org/10.1109/EDUNINE48860.2020.9149507
LDA Ferreira. Musiquence-Design, Implementation and Validation of a Customizable Music and Reminiscence Cognitive Stimulation Platform for People with Dementia. PhD thesis, Facultade de Ciências e Tecnologia. Universidade Nove de Lisboa. http://hdl.handle.net/10362/116417. Accessed 2023-06-19
Freescale Semiconductor: AN991D. Using the Serial Peripheral Interface to Communicate between multiple microcomputers, 2002. Freescale Semiconductor
FYCMA: Transfiere 2020. https://transfiere.fycma.com/transfiere-edicion-2020, 2020. Accessed: 2021–09–13 (2020)
General MIDI Standard. Prince George’s Community College. http://academic.pgcc.edu/njudy/mt/MIDI/gm.html. Accessed 2021–09–13
Gold NE, Purves R, Himonides E (2022) Playing, constructionism, and music in early-stage software engineering education. Mult J Educ Soc Technol Sci 9(1):14–38. https://doi.org/10.4995/muse.2022.16453
Greenberg DM, Decety J, Gordon I (2021) The social neuroscience of music: Understanding the social brain through human song. Am Psychol. Advance online publication. https://doi.org/10.1037/amp0000819
Herrero G, Barbancho I, Tardon J, Rosa-Pujazon LA, Barbancho AM (2015) Drumkit simulator from everyday desktop objects. Multimedia Tools Appl 74:10195–10213. https://doi.org/10.1007/s11042-014-2159-z8
Holland S, Wilkie K, Mulholland P, Seago A (2013) Music interaction: Understanding music and human-computer interaction. In Music and Human-Computer Interaction. Springer, London, pp. 1–28. https://doi.org/10.1007/978-1-4471-2990-5
Huang C-F, Huang C-Y (2020) Emotion-based AI music generation system with CVAE-GAN. In 2020 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE), pp 220–222. https://doi.org/10.1109/ECICE50847.2020.9301934
A Ilsar, G Kenning. Inclusive improvisation through sound and movement mapping: From DMI to ADMI. In The 22nd International ACM SIGACCESS Conference on Computers and Accessibility, Association for Computing Machinery.pages 1–8, New York, NY, USA, 2020. https://doi.org/10.1145/3373625.3416988
Jaime J, Barbancho I, Urdiales C, Tardon LJ, Barbancho AM (2016) A new multiformat rhythm game for music tutoring. Multimedia Tools Appl 75(8):4349–4362. https://doi.org/10.1007/s11042-015-2478-8
Jakobsen KB, Stougaard J, Petersen MG, Winge J, Grønbæk JE, Rasmussen MK (2016) Expressivity in open-ended constructive play: Building and playing musical LEGO® instruments. In Proceedings of the The 15th International Conference on Interaction Design and Children, IDC ’16, Association for Computing Machinery. pp 46–57, New York, NY, USA, 2016. https://doi.org/10.1145/2930674.2930683
Johnson-Green E (2018) Musical architects: Immersive learning through design thinking in a kindergarten music composition curriculum. In Visions of Research in Music Education. Vol. 31, Article 2. https://opencommons.uconn.edu/vrme/vol31/iss1/2
Lindsay S, Hounsell KG, Cassiani C (2017) A scoping review of the role of LEGO® therapy for improving inclusion and social skills among children and youth with autism. Disabil Health 10(2):173–182
Li H (2020) Piano automatic computer composition by deep learning and blockchain technology. IEEE Access 8:188951–188958. https://doi.org/10.1109/ACCESS.2020.3031155
Ludovico LA, Malchiodi D, Zecca L (2017) A multimodal LEGO®-based learning activity mixing musical notation and computer programming. In Proceedings of the 1st ACM SIGCHI International Workshop on Multimodal Interaction for Education, MIE 2017, Association for Computing Machinery. pp 44–48, New York, NY, USA
Microchip Technology Inc.: MCP3008 datasheet. https://www.microchip.com/en-us/product/MCP3008. Accessed 2021–09–13
MIDI Manufacturers Association. The Complete MIDI 1.0. Detailed Specification, 1996. Los Angeles, CA, MIDI Manufacturers Association
Mitchell TJ (2011) Soundgrasp: A gestural interface for the performance of live music. In International Conference on New Interfaces for Musical Expression. Oslo, Norway, pp. 465–468
Muñoz E, Cadenas JM, Ong YS, Acampora G (2016) Memetic music composition. IEEE Trans Evol Comput 20(1):1–15. https://doi.org/10.1109/TEVC.2014.2366871
Muller J, Oestermeier U, Gerjets P (2017) Multimodal interaction in classrooms: Implementation of tangibles in integrated music and math lessons. In Proceedings of the 19th ACM International Conference on Multimodal Interaction, Association for Computing Machinery. pp 487–488, New York, NY, USA
NXP Semiconductors. UM10204. I2C-bus specification and user manual. Rev. 7.0, 2021. NXP Semiconductors. https://www.nxp.com/docs/en/user-guide/UM10204.pdf
Oestermeier U, Mock P, Edelmann J, Gerjets P (2015) Lego music: Learning composition with bricks. In: Proceedings of the 14th International Conference on Interaction Design and Children, pp 283–286. https://doi.org/10.1145/2771839.2771897
Perks R (2023) Transforming the microtonal fingerboard: ‘Small’ frets, Lego and robots-interview with Tolgahan Çogulu. In Perks R and McGrath J, editors, 21st Century Guitar. Bloomsbury Publishing, London, UK
Philips Semiconductors. SCC2691. Universal asynchronous receiver/transmitter (UART), 2006. Koninklijke Philips Electronics N.V. https://www.nxp.com/docs/en/data-sheet/SCC2691.pdf
Raspberry Pi Foundation. Raspberrypi. https://www.raspberrypi.org/. Accessed: 2021–07–30
Roig C, Tardón LJ, Barbancho I, Barbancho AM (2014) Automatic melody composition based on a probabilistic model of music style and harmonic rules. Knowl-Based Syst 71:419–434. https://doi.org/10.1016/j.knosys.2014.08.018
Roig C, Tardón LJ, Barbancho I, Barbancho AM (2017) A non-homogeneous beat-based harmony Markov model. Knowl-Based Syst, 142. https://doi.org/10.1016/j.knosys.2017.11.027
Rosa-Pujazon A, Barbancho I, Tardon LJ, Barbancho AM (2016) Fast-gesture recognition and classification usign Kinect; an applications for a virtual reality drumkit. Multimedia Tools Appl 75:8137–8164. https://doi.org/10.1007/s11042-015-2729-8
Shahrbabaki RM, Nourian M, Farahani AS, Nasiri M, Heidari A (2022) Effectiveness of listening to music and playing with Lego on children’s postoperative pain. J Pediatr Nurs. https://doi.org/10.1016/j.pedn.2022.11.023
Simpson K, Keen D (2011) Music interventions for children with autism: narrative review of the literature. J Autism Dev Disord 41:1507–1514
Stromatolite, Sigma-Orionis, Ircam-Centre Pompidou, Music Technology Group, Vienna University of Technology, and Fraunhofer IDMT. Musicbricks. https://mtflabs.net/MusicBricks/. Accessed: 2023–03–16
Texas Instruments: SN74LS151N datasheet. https://www.ti.com/lit/ds/symlink/sn74ls151.pdf. Accessed: 2021–09–13
Texas Instruments. SN74LS157 datasheet. https://www.ti.com/lit/ds/symlink/sn74ls157.pdf?ts=1643012533007. Accessed: 2021–09–13
Torre G, Andersen K, Baldé F (2016) The hands: The making of a digital musical instrument. Comput Music J 40(2):22–34
Wiriyachaiporn P, Chanasit K, Suchato A, Punyabukkana P, Chuangsuwanich E (2018) Algorithmic music composition comparison. In: 2018 15th International Joint Conference on Computer Science and Software Engineering (JCSSE), pages 1–6. https://doi.org/10.1109/JCSSE.2018.8457397
Acknowledgements
Alejandro Villena worked on several stages of the prototype for melody construction and synchronization. Joaquin Cáceres worked on the beatbox and synchronization.
Funding
Funding for open access publishing: Universidad Málaga/CBUA This publication is part of the project PDC2021-120997-C33 funded by MCIN/AEI/10.13039/501100011033, and European Union “NextGenerationEU/PRTR”. This publication is part of the project PID2021-123207NB-I00 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE. This work was done at Universidad de Málaga, Campus de Excelencia Internacional Andalucia Tech. Funding for open access charge: Universidad de Málaga/CBUA.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest/Competing interests
The authors have no financial or non-financial interests that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Barbancho, A.M., Tardón, L.J. & Barbancho, I. Building music with Lego bricks and Raspberry Pi. Multimed Tools Appl 83, 10503–10523 (2024). https://doi.org/10.1007/s11042-023-15902-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-023-15902-z