Keywords

1 Introduction

This paper presents a computer system and its interface for musicians and composers to analyze musical pieces described as a sequence of decision making process during the composition of musical pieces. This is a novel system, and no conventional computer system is able to process and visualize musical pieces from the viewpoint of creation process. Such information is valuable for both composers and musicians. For composers, it is valuable to verify her own composition techniques and creative process. And for musicians, it offers different viewpoints to understand the musical piece that results in better execution of the musical piece.

With the wider use of computer scoring system, the so called digital audio workstations, to compose music, which allows annotations of musical pieces in digital format, searching similar parts or by keywords and comparison of partial pieces are becoming easier and faster on daily basis, due to the advance of computer processing speed. The main factor that influences the usefulness of the software to study musical pieces is the type and quantity of available data in the system. Many music production software exist, such as LogicFootnote 1, CubaseFootnote 2 and PerformerFootnote 3. Although these software are created to compose music, these are also useful to study musical pieces. However, data available by these software are limited to data directly represented in the musical sheet, such as pitch, duration, chord type, among others. Surely these information are essential to study, we recognize these are insufficient.

Our system displays the music sheet together with the decisions involved in composition of phrases and passages that constitute the musical piece. Users are able to change the visualization detail of the decisions and to compare selected decisions, and edit the descriptions of decisions.

2 System

The hypernetwork model is extended from the bipartite representation of the hypergraph [5]. The hypernetwork model has more representation power than conventional knowledge representation models that are based on graph [4]. The main difference is the capability to represent N-ary relationships, the property of duality, and creation of relationships among relationships. The hypernetwork model follows basic definitions of semantic networks, where a node is connected to other nodes (1) to specify the nodes or (2) when nodes are related by some relationship.

A uniqueness of the hypernetwork model is the existence of three types of description elements, equivalent to the types of nodes. Graph and hypergraph models consist of nodes and links connecting the nodes. In the basic representation of a decision, a node represents any fact or concept, and a link connects two or more concepts based on causal relationship. In a visualized diagram, a link connects two or more nodes. The generated representation is then converted automatically to bipartite representation, where links that represents relationships also become nodes, and links of a new type are inserted to connect the nodes and nodes converted from links. The bipartite representation consists of two types of nodes: (1) the vertex node that represents nodes that originally are nodes in basic representation, and (2) relation node that is converted from a link in basic representation. The vertex node serves to represent substances or phenomena or concepts, and the relation node to describe relationships among them.

Details or properties of a concept represented by a vertex node can be specified in two ways: by attachment of attribute nodes, or by relating to other vertex nodes through relation nodes. The attribute node exists to specify any of three node types. Two connections are prohibited: between vertex node and vertex node, and between relation node and relation node, constraint imposed from their role in hypergraph.

In order to represent decisions in music composition, we use the text description of decisions involved during the composition process. The text is written after each work stage defined by the composer himself, written by the composer himself. The number of stages depends on the composer’s work style and musical piece being composed, as some pieces take years to be accomplished. Therefore, a stage is anything with varied work amount, number of created and edited notes, and working time durations. In other words, a stage corresponds to the amount of composition work between subsequent intermissions defined by the composer.

In each intermission and after the completion of composition, the composer reviews the modifications since the previous version of the music piece, enumerating every single alterations. Then the composer writes the Decision List Report, a text explaining each modification points, describing the decision type and the details. The decision type should be chosen from (a) Theoretical, (b) Selective, and (c) Intuitive. Theoretical decisions denote decisions based on Music theory. Empirical (heuristic) foundations are excluded because they are empirical and lack theoretical bases. The second type and the third type are used when multiple options exist. It is possible that a decision is theoretical and simultaneously either selective or intuitive, when multiple options exist. The selection of a theory is chosen from multiple possibilities or intuitively. Only one type is associated with decisions, however.

In order to homogenize the granularity of decision sizes, each decision description is analyzed to subdivide into smaller decisions or to join with other decisions depending on the explanation text. Two types of decisions exist, (1) Framework decisions and (2) Component decisions, differing on the extent affected by decisions. Framework decisions are global decisions, and affect the entire musical piece, such as tempo and instruments used. Component decisions are local decisions, modifying passages or a part of musical piece. Basically a component decision consists of a single modification on a single region of an instrument part. A region may contain any number of notes, between a single note, a single chord, or dozen of notes encompassing multiple measures. It may not involve any notes.

The next step is the generation of hypernetwork representation of extracted decisions. The hypernetwork model is explained in next section. The sizes of hypernetwork representation of all decisions are uniform, because the granularity of size of decisions are standardized in the previous step.

Then decisions are interconnected based on: Type-I: decision sequence, subdivided into Type-IA: Global order and Type-IB: Order within overlapped target region; Type-II: Overlap on target region (notes, measures, phrases, among others); Type-III: Identical element node (decision component); and Type-IV: Semantic relationship among element nodes (decision components). This connection process is semiautomatic using computer program. The Type-I connections generate sequence relationships among decisions. The second type of relationships, Type-II, connects decisions affecting at least one identical musical element. It connects multiple decisions that generate N-ary relationships, which are impossible to be generated using conventional representation models. The overlapped element is described in relationship entity, which also functions as a “concept” entity when a person reads the music score. Connections based on same musical element are used to connect multiple decisions if they contain identical elements. The hierarchical level of elements may differ in each decision. For instance, the composer refers to musical elements in other region to employ a variation of these elements. In this case, the hierarchical level of referred elements in decision structure will be different. In other cases, a same thematic element may be used multiple times, and the element description appearing in relevant decisions are linked. The semantic connections, Type-IV, are based on semantic relationship among decision elements. The semantic relationship types used in our representation are: hierarchy of concept, hierarchy of target region, antonym (opposite concept), and synonym concept.

3 Discussions

The disclosure of description of intermediate composition process is useful for both composers and musicians. For composers, it is valuable to overview and clarify his own composition process to improve the composed opus, besides the benefit to reorganize his ideas. For musicians, the acquisition of background and underlying phylosophy is invaluable, because deeper understanding of musical piece is fundamental and crucial for good execution. Before the execution, every musician analyzes the musical piece he/she will perform. During the analysis, musicians investigate every note and their context, their raison d’être, and instructions on execution indicated by the composer. Our method differs from conventional works because the musical piece is represented by a temporal sequence of decisions. Such a creation history is more valuable than static structures generated by conventional methods, such as Generative Theory of Tonal Music (GTTM), due to reasons discussed before.

Musical score is the de facto representation of musical pieces. Musical score encompasses every aspect of the musical piece, and it describes what to be performed, how to be performed, and composer’s intentions. Everything is in the score, as some say. John Cage once said that by looking at the music sheet, one can judge the composer’s talent, but not by listening to the performance of a musical piece. Music composition process involves a wide range of fields, and the list of fields depends on the music style. Even limiting to fields directly related to music, a composer should be familiar with many disciplines of musical theory inclluding Harmony, acoustics of musical instruments, and genre-dependent articulations of each musical instrument.

Our system treats the creation process, or composition process, from a blank music sheet to the final work. This is a “creation history” of musical piece. Obviously if the data on intermediate process is absent, the representation will only be about the final status of the music. Composers input and annotate each decision making during the music composition. Each decision making is represented as causes, details of the decision, and results. For each musical phrase or passage, decision making structure that originated the relevant musical passage is visualized in our computer system.

Figures 1–3 are examples of representations of the same passage shown in the upper part of the figures. Other representations are also possible. The representation in Fig. 1 focuses in the relationships among musical elements, highly similar to conventional representations that treat directly the musical elements. On the other hand, representations in Figs. 2 and 3 are unique to the proposed system, where the descriptions are based on the composer’s decisions during composition, unavailable in conventional systems.

Fig. 1.
figure 1figure 1

Representation-1 of a passage (measures 19–20) of music \(\beta \)

Full size image
Fig. 2.
figure 2figure 2

Representation-2 of a passage (measures 19–20) of music \(\beta \)

Full size image
Fig. 3.
figure 3figure 3

Representation-3 of a passage (measures 19–20) of music \(\beta \)

Full size image

Obviously such information is unavailable in most musical pieces. Howerver, preliminary uses by professional composers and musical instrument players indicate that information related to composers’ decisions are useful and important for the understanding of the musical piece. Once the usefulness of such information is established, composers might describe such information during their composition.

The visualization of decision making is useful for musicians and also for composers. The proposed system detects dissimilar passages but are similar in composer’s decision. Consequently, musicians have better understanding of these two passages, which results in better execution and performance. Detecting similar decisions is useful to analyze musical pieces and results in valuable information for the musical instrument players. Furthermore, similarity among decision sequences is more important than comparison of single decisions.

The proposed system also offers mechanisms to facilitate the input operation of decisions involved during composition, to be activated in parallel to DAW software for composers using DAW, and solely activated for those using pencil and music sheets.