In the following, the construction of digital twin behavior models by means of subject-oriented modeling embedding socio-emotional aspects, and in this way generating digital selves, is considered from a development perspective. We start with providing the fundamental concept before showing the bottom-up and top-down approaches to creating subject-oriented digital selves’ representations. Thereby, we structure the adjustment of functional and social modeling for development.

Digital selves form a networked universe of discourse consisting of behavior entities. Hence, their digital twin representations are based on behavior abstractions, also termed subjects. They represent the behavior of any active entity in a socio-technical setting. A specification of a subject does not imply any human, system, or technology that could be used to execute the described behavior. It is a description of setting actions, either in terms of functional operation or communication with other subjects.

Typical examples of subjects are functional roles in organizations and community settings related to some task, such as facilitator, learner, facility manager, back office, and customer service. Less typical behavior abstractions are generic sets of activities, as they refer to cross-organizational or cross-sectoral roles of a domain, however may be critical for operation. Samples of that kind are information collector, privacy management, emergency case handler, or security.

As subjects represent abstractions, any instantiation of a twin model of a digital self needs to be linked to implementation in a socio-technical setting. Sample implementations are robots taking care of facilities in case of emergency, information services provided by concierges, security staff, customer service collaborative robots on how to install IoT sensor systems in a smart home setting, or Enterprise Resource Planning module handling orders.

Digital selves in terms of subjects represent nodes of a network scenario. They communicate with each other by exchanging messages. Typical exchange processes are role-specific subjects, as in a classroom setting a subject “Facilitator” sending the subject “Learner” a learning assignment or in business operation for a subject “Billing” sending subject “Customer” the billing message. Each message has an identifier in terms of a name and a payload. The name should express the meaning of a message informally. The payloads are the data (business) objects transported along the message exchange.

Digital twin representations as (part of) subject networks encapsulate behavior of entire systems, with each network node or subject characterized by naming internal behavior representations. Internally, each subject either executes local activities (“doing”), or sends messages to other subjects, or expect messages from other subjects (“communicating”). Hence, each subject encapsulates behavior composed of three types of fundamental activities. When recognizing emotional aspects, each activity can be linked to a subject influencing the behavior in subsequent steps. Such “influencer” subjects can be provided with parameters referring to previous activities and deliver decision support for selecting further activities of the calling subject (see also above).

Subjects performing internal activities can be combined in a system behavior specification with separated subject behaviors. Functional samples comprise work tasks, such as handling an order, moderating a discourse between different stakeholders, preparing a decision, or communicating to others. Examples referring to the emotional state of a subject are:

  • A subject “Learner” gets uncomfortable with classmates (i.e., other subject carrier of type “Learner”) and activates a corresponding pattern of activities, such as questioning the contribution of other subjects “Learner.”

  • The subject “Facilitator” feels the need to settle an upcoming conflict among subjects of type “Learner” who in turn expect a message “Clarification” from subject “Facilitator.”

Communication when based on sending and receiving messages (including data) can lead to patterns of information exchange, including socio-emotional states, which might even lead to overrun institutional regulations requiring formal approval, such as the following example reveals:

  • A subject “Facility Manager” feels the need to accelerate evacuation in case of an emergency case and directly requests security guards for help, sending the subject “Safety Unit” directly a message “Clear Building.”

Since digital selves and their digital twin representation are embedded in some context, we also need to define that concept for their model representations. The context of each subject is defined by the operating organization and environment it is part of, which in turn is a set of (connected) subjects. Context is also provided by the technological infrastructure by which a subject-oriented behavior specification can be executed.

The System-of-Systems perspective should be implemented, whenever particular concerns, such as handling emergency case, need to be addressed (cf. Heininger & Stary, 2021). It allows to bundle requirements for that concern, e.g., recognizing regulations affecting all actors of the network, and to connect to each actor implementing these requirements. Finally, monitoring and dynamic adaptation of requirements can be performed, and resulting behavior changes can be put to operation by System-of-Systems subjects.

1 Representing Case-Sensitive Semantics

As the construction of a digital twin and digital selves by subject-oriented modeling is based on connecting behavioral entities or abstract components involved in a dynamic setting, subjects and their exchanges of messages (interactions) need to be specified in a specific sequence. A complete subject-oriented model requires the specification of the following:

  • Concerned universe of discourse, such as a business operation case

  • Active components or subjects involved in a specific case or process

  • Interactions the active components or subjects are part of

  • Messages the active components or subjects send or receive through each interaction

  • Behavior of each subject encapsulating functions and interaction activities in terms of sending and receiving messages

Figure 13.1 shows the generic steps how subject-oriented concepts are applied when embedding socio-emotional aspects into functional/cognitive digital representations of digital selves.

Fig. 13.1
figure 1

Generating socio-cognitive digital twin representations (digital selves)

Full size image

Assume the setting is a classroom and the case to be handled is Emergency Management. The subjects that are involved are at least for functional operation Teacher, Student, Facility Management, and Safety Unit. Sample interactions in case of emergency are Teacher-Facility Management and Facility Management-Safety Unit. Thereby, the exchanged messages are Emergency notification, Emergency confirmation, Evacuation request, and Evacuation plan delivery. The flow of control and information included is as follows: A Teacher sends an Emergency notification to the Facility Management of the building provider. He is going to receive an Emergency confirmation and the Evacuation plan by the Safety Unit.

The corresponding communication structure of that process is represented in a Subject Interaction Diagram. It is the most abstract diagrammatic level of describing behavior encapsulations. Hence, for each active entity, a subject as part of a Subject Interaction Diagram has to be specified. The set of subjects and their message exchanges establish a network as considered universe of discourse. Typically, all relevant actors and components involved in a scenario or operational case are captured by a Subject Interaction Diagram (SID).

At that stage of development, it is not specified whether a subject or role behavior model is implemented by a digital system, a human, or a Cyber-Physical System. For instance, a teacher could either be a human facilitator, a piece of software, or a cyber-physical artifact, including actuators in the classroom. In either case, algorithms that support social behavior decision can be used for decision-making for the next action, in the case of both routine behavior, e.g., supporting learners when accomplishing their assignments, and non-routine behavior, e.g., acting in case of emergency in the classroom. For each activity of a subject detailed in a Subject Behavior Diagram, an algorithm can be activated. This capability enables evaluating socio-emotional behavior patterns, e.g., proposing action in case of an emergency, and thus taking the lead how to proceed in the classroom, when recognizing student herding in a certain direction.

Each behavioral entity (subject) of the SID is refined by its corresponding Subject Behavior Diagram (SBD). A subject’s behavior is described by three states (send, receive, internal function) and transitions between these states. Hence, when specifying the behavior of each subject, a sequence of sending and receiving messages and activities to be set for the goal to be achieved or the task that needs to be accomplished need to be represented. The states of a SBD represent operations. They are active elements of the subject description and are typically implemented by services. They enable exchanging and manipulating data or business objects between subjects, including information sharing as part of socio-emotional behavior.

Taking the example of an emergency case, the behavior of each involved subject, such as Facility Management and Safety Unit, needs to be addressed. In the first state of its behavior, the recognizing subject, e.g., Teacher in the classroom, executes the internal function “Prepare emergency notification.” When this function is finished, the transition “Notification call” follows. In the succeeding state “send call,” the message “emergency call” is sent to the subject “Facility Management.” After this message is sent, the subject “Teacher” goes into the state “wait for guidance.” If this message is not available, the subject stops its execution until the corresponding message arrives or sets an action independent of the incoming conformation depending on the socio-emotional behavior of the students. Upon receipt of the emergency guidance, the subject follows the transition into state “Wait for safety unit support.” In case of changing environmental conditions, e.g., herding of students toward exits instead of waiting for Safety Unit support, the Teacher could take the lead and guide the people to the outside, as captured in the SBD model.

Operations can be of the type “send,” “receive,” or “internal function”: internal functions deal with specific objects, as required in terms of reporting when an emergency case occurs. As a consequence, at least one operation needs to be assigned to each state. Detailing the operations is not necessary at the modeling stage. It is a matter of an abstract object specification or of the integration of an existing component or application. Typical examples of operations are transactions of an Enterprise Resource Planning system processing data, e.g., stemming from updated sensor components when monitoring a classroom situation. Objects are data and/or applications affected by internal functions or operations of a subject and processed through services. For instance, an internal operation “prepare notification” uses internal data to prepare the data for the notification message. This notification data is sent as payload of the message “emergency notification” to its recipient.

2 Refining General Behavior Designs

Besides the stepwise refinement of subjects arranged in a Subject Interaction Diagram, an alternative approach to modeling has been developed (see also Fleischmann & Stary, 2012). It starts with an overall generic network model. This generic model represents some kind of chaotic setting: every network element communicates with every other element whenever it wants.

The initial modeling task is therefore to identify the number of elements required for a digital selves’ model. This means modelers have to decide upfront how many actors (subjects) are involved in the application case to be described. Starting with a generic network template that is defined by the number of involved actors (subjects), the behavior of a digital selves’ ecosystem can become more concrete step by step. The procedure requires several restriction steps:

  1. 1.

    Specify a generic template according to the number of subjects involved in handling a certain operation or business case.

  2. 2.

    Name each subject of the remaining generic template for handling that operation or certain business case, e.g., by identifying its relevant actors.

  3. 3.

    Stepwise reduce the interactions for each subject until the operation or business case can be handled from both a technical (functional) and a socio-emotional perspective.

  4. 4.

    Name each remaining interaction (i.e., message connection) between subjects which are required for handling the business case.

  5. 5.

    Introduce message types according to the functional and socio-emotional modeling needs.

  6. 6.

    Adapt the specification of subject behavior according to the case at hand.

  7. 7.

    Refine the structure of the objects of operation transmitted by the various messages.

For the generic model, the question “Who Needs to be Involved?” needs to be answered. For Emergency Management, a generic subject-oriented model with three involved actors (Teacher, Facility Management, Safety Unit) fits to the number of subjects a modeler has to expect for the respective scenarios and related processed (Fig. 13.2). Each actor exchanges messages with another one.

Fig. 13.2
figure 2

Subject-oriented representation scheme for a case involving three actors

Full size image

Each subject can send messages with the name “message” to any other subject any time. Figure 13.3 shows the behavior of the subject with the name “subject1.” In the select state, a subject decides whether it wants to send or to receive a message. To initiate a system’s operation, a subject needs to start sending messages. Choosing the send transition, the subject goes into the state “prepare notification” and provides object data that is transmitted by the message “notification message.” After that, the subject decides to which other subject the message with the data as content will be sent.

Fig. 13.3
figure 3

Generic behavior of the subject “subject1”

Full size image

In the select state, a subject can also decide whether it wants to receive a message1. If there is a message for the subject available, it can be accepted, and a follow-up action can be executed. It is not specified what the follow-up action is. This is like receiving an e-mail. The receiver can interpret the content of an e-mail and knows what the corresponding follow-up action is. The abort transitions back to the select state enable to step back in case a subject has made the improper choice.

Utilizing the message “notification message,” a data or business object is sent. The structure of this object corresponds to the structure of a traditional e-mail with extensions like the topic, keywords, and signature.

With each restriction step, the behavior specification is becoming more stringent for the subject holders to their actual situation and way of accomplishing tasks. However, it needs to be noted that bottom-up and top-down modeling not necessarily result in identical models. Nevertheless, both approaches to modeling need to deliver the results from a system behavior perspective.

Addressing the “clock is falling off the wall” case, a facilitator or teacher could be modeled as a digital self by this model. In this case, the behavior is not implemented by a human role taker but some cyber-physical artifact with socio-emotional intelligence. The start state of behavior specification is for both sending and receiving messages “The clock has fallen off the wall.” The subsequent figures contain the interaction from both perspectives, sending and receiving information from the facilitator’s or teacher’s point of view.

The receiver side, as shown in Fig. 13.4, contains receiving hints from students what to do next in the situation that just occurred to the class. In case the facilitator anticipates hints from the student crowd (Kahneman’s System 1) (Kahneman, 2011), no further sensing of inputs is performed by the facilitator. The digital self does not set further reactions to received inputs. In case the facilitator does not know the crowd and cannot anticipate the inputs to be received from the students (Kahneman’s System 2), a novel type of reaction (solution) needs to be identified, and further analysis of each hint is performed. The development of an action includes simulation of actions and reaction patterns through evaluation before adaptation, e.g., to test opportunistic behavior in terms of common sense reactions to an emergency, such as leaving the classroom to be on the safe side (from various perspectives including safety and role responsibility).

Fig. 13.4
figure 4

Creating a digital self by restricting behavior of Teacher interacting with Student from a receiver perspective

Full size image

The sender side, as shown in Fig. 13.5, refers to Kahneman’s System 2 looking for a socially compatible solution and has its focus on preparing for an acknowledge action by “testing” social appropriateness of possible actions. The “testing” process could be a complete simulation as described in Part II of this book. Depending on the proposed feedback, the Teacher sets the next action.

Fig. 13.5
figure 5

Creating a digital self by restricting behavior of Teacher interacting with Students from the sender perspective

Full size image

3 The Trans-humanist Development Framework

As demonstrated, digital selves can be modeled by construction or restriction. Socio-emotional consciousness can emerge due to cognitive computing models embodying social behavior models. They lead to digital substrates that either can be grasped in the physical world like Cyber-Physical Systems or remain digital artifacts after being represented as digital twin.

Digital artifacts will increase in everyday life. Hence, we adopted the human-technology-organization framework to show how the individual cognitive perspective on trans-human developments can propagate to digitally enhanced communities.

Figure 13.6 shows the concept we propose for socio-technical development of trans-human settings. The modeling represents the abstraction from biological and technological substrates. The Complex Adaptive Systems perspective recognizes the particular dynamic nature, and the System-of-Systems perspective allows for reducing complexity for the sake of operating and acting in trans-human settings.

Fig. 13.6
figure 6

Digital self-development as socio-technical endeavor in the context of behavior representation, Complex Adaptive Systems, and System-of-Systems

Full size image

For further developments, it seems to be important to exploit the systems engineering skills so that multiple CAS can be modeled in SoS environments properly. The ultimate goal is the incarnation of human-like behaviors as digital selves.