“To raise new questions, new possibilities, to regard old problems from a new angle requires a creative imagination and marks the real advance in science”

Albert Einstein

In pursuing the debate on ethics in the emerging convergence of the cyber-physical-biological worlds, it is time to move forward from academic arguments on regulatory models to the creation of human-centred ethical frameworks that find a coherence between technological innovations, professional practices and societal aspirations. Recent advances in biomedical technologies may be considered as an exemplar of this convergence, posing a major challenge to societies and raising fundamental philosophical, ethical, legal and political questions. Some of the questions relating to this challenge raised at a recent Philomathia symposium (2016) on Body Politics were: Should the law allow patenting of DNA or not? What are the implications of intervention to rectify ‘errors’ in DNA? What are the implications of a shift to individualised treatment based on analysis of the patient’s DNA? These questions intersect with intervention in reproduction, both to assist infertile couples to have children, and to deal with genetic disease. What are the limits of intervention, and who should make the decisions? And how should the shortage of organs for transplant be overcome? Should there be a presumption that organs are donated unless an individual opts out? How should surgeons balance the risk of using sub-optimal organs against the risk of death?

Commenting upon the regulation of organ transplant, Saeb-Parsy et al. (2016, Symposium ibid.) argued that medical ethics is much more than legal regulations. What is important is to create a regulatory framework of professional ethics that finds a coherence between different perspective of ethics, for example how lawyers see ethics, how the public sees ethics and how medical professionals see ethics. On the regulation of sub-optimal organ transplantation, he noted that organ transplantation is not an exact science; here sub-optimality is a norm rather than the exception, and in this scenario, the knowns and unknowns cannot be easily quantified. There may be thousands of permutations before the transplant is accepted, declined or rejected. Further issues arise such as those of how to deal with bias in both the organ data and in the algorithmic process of transplantation; how to prove that something has gone wrong; what price are we prepared to pay for trust in the machine learning process; and how to regulate the biases in information? For example, a clinical consultant takes account of the well-being of the patients, both the donor and recipient, but lawyers may know very little of the work and practice of how clinicians work. Also one clinical centre may reject the organ and the other may accept it, depending upon the patient–clinician consultation, as well as on the data on the organ. What also matters in an organ transplant situation is the availability of the organ and speed of its transfer to the clinic. In the world of practice, what matters is the clinician–patient trust, the well-being value, and the dialogue and engagement of patients, rather than mechanistic evidence and justification of the clinician’s work. Moreover, an organ transplant is not about abstract ideas or scientific judgements, it is about making a contextual human judgement that takes account of multiple factors (clinical, social, cultural, policy), in addition to factors impacting the individual patients, their family and friends and the community in which they live. Organ transplantation is thus a process, a performance of data, information, knowledge, skill and wisdom of clinical practice, where judgement is about the performance of clinical practice, and ethics is about the performance of human judgement that we may call performing ethics. Those who seek regulatory solutions to ethical problems of medical practice need to be very mindful of the limit of regulation and the danger of over regulation, generating further fear of litigation, thereby undermining the ethical tradition of clinician–patient trust. The ethical challenge here is how to find accommodation between public science, professional practice and the regulatory regime.

On the role of public science in society, Jasanoff (2015) argues that science and technology, especially public science, is itself socially and politically constructed through rules of delegation, deference and accountability that are seldom openly acknowledged or systematically analysed in legal or political theory. For example, it is crucial to understand the impact and implication of the governance of environmental risk and biosciences and biotechnologies. She says that it is important to explore the uses and accommodations of science in public reason. Institutionally mediated relations between science and politics affect what democratic publics accept as evidence, how they perceive the common good, and how they contemplate their futures. Science in this way participates not only in making societies as they are but in shaping imaginaries of societies as they ought to be.

Broadening the ethical debate on the role of science in society, Jasanoff (Symposium ibid., 2016) argues that ‘science’s role in public life is not simply to provide facts and truths but to help create meaning. The facts of science have to make sense in lived contexts in order for them to carry moral weight as truths to live by. Reason, not science, is the vehicle through which communities around the world seek to integrate knowledge and values, and societies differ in the ways they judge what counts as good reason. Shifting our attention from science to reason in public debates might allow us to break through recent stalemates, enabling productive cosmopolitanism in place of stale debates about facts and counter-facts’. We are further alerted to the danger of the suggestion prevalent in certain circles that it is the not fault of science or scientific knowledge but that ‘fault lies in our own imperfect brains, and that human minds are incapable of grasping knowledge that seems new, unfamiliar, or fearful’. She notes that ‘The Enlightenment ideal of humanity united in a common vision of the good, based on growing scientific knowledge and understanding, lies in tatters’, and argues that we are due for a second Enlightenment, one that respects ethical differences whilst embracing knowledge and truth.

However, this aspiration of a second Enlightenment needs to be envisioned in the era of emerging convergence of cyber-physical-biological worlds that brings a more troubling dimension of technological complexity. We observe that scientific knowledge is increasingly being transformed into algorithms, and public science and professional practice into an algorithmic process, adding a further dimension of opaqueness to this complexity. Gowing and Langdon (2015) note that the growing dependency on complex technological systems generates a source of vulnerability. For example, if any major component within the systems that support modern life ceases to function for a significant period of time, our prosperity, freedom and comfortable lives are threatened. This vulnerability tends to generate a culture of risk aversion and conformity, and consequently to ‘deresponsibilisation’ of both the self and the social due to the fear that the personal cost of standing one’s ground on facts and arguments is real, and therefore limiting first the imagining then tabling ‘unthinkables’. They further argue that in many public organisations and institutions, we see ‘how what was once inspirational leadership acting on insight and vision has been replaced by a new bureaucratic imperative for “managerialism”’. In turn, this has created an institutional basis of world politics that is increasingly dysfunctional. The danger is that the price of this growing culture of reluctance complacency and resistance to imagination is likely to be high, and destructive of both the self and society. Or as the mindfulness people would say, this culture of spun imagination is becoming so mindless that what could and should be thinkable turns into the unthinkable. In this mindless presence, we first fail to see and even hear the coming of the unexpected, and are unable to predict the consequences and thereby act on them. Perhaps it is this state of mindlessness that gives rise to a type of myopia, where over time we either fail to remember or ever learn to know that things could unexpectedly turn to disaster. It was thus ‘no accident that the crisis of 2008 came at a time when there wasn’t a single risk manager on the planet who could remember the last really big crisis, the really big one’ (ibid.).

This brings us back to the debate on a new ethical framework for professional practice and envisioning a new Enlightenment of shared differences. We note that saturation of public policy and professional practice with algorithms and algorithmic processes could have profound implications for ethics. The ethical debate needs to go beyond the public policy and academic preoccupation with legal regulations, data protection law, and the science of ethical frameworks, and bring into focus the human condition and human values as the locus of the ethical debate. During the panel session of the symposium (op.cit.), it was pointed out that although there has been a great surge in science and in the professions, there has been little change in human nature. The question then is how can these two live together in harmony? Since science cannot be unlearned, we should be wary of accepting the received wisdom of science without judgement. It was also pointed out that science as we know it may not be fit for purpose in the era of big data, neuroscience and biological science, as we encounter ethical issues of the primacy of consent and collective responsibility. It was suggested that we need to think coherently about ethical policy making by engaging the public in conversation, and recognising that neither science nor society are homogenous as both are living entities exhibiting ongoing and multiple relationships. In the living world, ethical policy making is to recognise that ethics in the day-to-day practice is to do with making moral judgements, and what we need is to scaffold a coherent policy around these judgements. Such a scaffolding requires the creation of an ethical framework for ethical policy making that is situated in the human-centred vision of what is possible, what can be done, and what should be done.

This human-centred vision (Gill 1996) has been a core guiding principle of many of AI&Society debates on socially responsible technologies from its very inception in the 1980s. The focus of these early debates has been on seeking a symbiotic relation between man and machine, between objective and tacit knowledge. This notion of symbiosis is rooted in the belief that human decision making is an ongoing and evolving process, and this process involves the transformation of data into information, information into knowledge, knowledge into wisdom and wisdom into action. The notion of performing ethics here resides in making judgements of the progression of this data-action cycle. For example, in the organs transplant process described above, judgements may be made about the accuracy, relevance and acceptability of data that is coming from many sources. Once information from the data is aggregated, this information itself may be scrutinised by the medical and health professionals based on their experiential knowledge and medical practice. Once this information is turned into knowledge building upon the clinical knowledge and contextual knowledge of transplantation, judgement may be made as to the wisdom of either abandoning or proceeding with the transplantation process. This scenario of organ transplantation illustrates that there is always a scope for human intervention at various levels of the data to action cycle and that an intervention reflecting the many overlapping contexts would bear witness to situated judgements, in contrast to an intervention based upon deterministic calculations and measurements. The performance of ethics here is visualised in terms of an evolving judgement-making process culminating in action. This transformational process from data to action, encompassing feedback loops and human intervention, provides a human-centred model of judgement that is contrary to the computational model of ‘calculation to judgement’, in which data are used to compute judgement. What has now changed is the radical departure from this ‘calculation to judgement’ model to that of turning judgement into calculation, the machine learning algorithm. It may be tempting to argue that nothing has fundamentally changed in the data-action cycle except for the availability of an abundance of data (big data) and the exponential processing speed of computers. The argument then revolves around the idea that only if we have an abundance of data and exponential processing speed of the computer, we can construct machine learning algorithms that can outstrip human cognition to the extent that machines can become far better than humans in processing a wide variety and large number of data sets and working in different ways to those of humans in reaching analytical judgements. However, this calculation-centred view of judgement fails to recognise that human judgement is about the process of finding a coherence among often conflicting and yet creative possibilities that cannot be reduced to calculation. Moreover, human judgement resides in and reflects the dynamic and evolving nature of professional and social practices, enriching human experience, knowledge, skill and cognition. From this human-centred perspective, performance of ethics lies in the performance of practice of the ‘data-action cycle’, in other words the performance of inter-relations between data, information, knowledge, wisdom and action. This view of performing ethics perceives and experiences the world around us, and seeks to understand the nature of the interface between the physical, cultural and our experiential worlds. The nature and practice of the interface here is fundamentally relational between, in-between, and across knowledges, experiences and practices of contextual domains, and not transactional in the sense of ‘cause and effect’ calculation. This view shifts our attention from a purely technological fascination of machine learning to the evolving interaction of biological systems and technology, thereby providing a symbiotic horizon of an ethical platform.

This perspective of performing ethics, ethics in action and practice, is situated in the European tradition of human-centred systems, which cultivates the idea of the humanistic tradition of ethics that transcends the legalistic and the analytical model of ethics. Ethics in this tradition is about the performance of knowledge and skill in designing and producing socially useful technology and systems that enhance living and working conditions of people rather than using science and technology to control people and exclude them from engagement in shaping their own human conditions. The ethos of this performing ethics has its recent roots in the socially responsible technology movement of the Lucas Plan of the 1970s (Cooley 1987). This movement, building on the experiential knowledge and skills of artisans, engineers and crafts people together with the scientific knowledge of socially conscious academics and researchers, set up an ethical framework for socially useful production. The essence of this movement was to proactively provide an alternative work life model to that of structured unemployment. The purpose was to make a creative and purposeful use of the experiential knowledge and skill of engineers, artisans and craft people for producing socially useful products, for example for enhancing the lives of disadvantaged and disabled people, and providing a sustainable living for people in the developing world, as well as for improving the environment. This social production model later laid the foundation for Technology Networks, a collaboration between communities in London, trade unions, universities and hospitals, to launch socially useful technology initiatives ranging from health care, community cooperatives and the environment (ibid.). It is only recently with the emergence of artificial super intelligence, big data and the Internet of the things that human-centred ideas are being taken on board by the mainstream scientific community, for example by those who are concerned with wider issues ranging from existential risk, surveillance, and social robotics to professional ethics. However, it remains to be seen whether the scientific community would go on keeping adherence to their faith in the ‘calculation to judgement’ model, in building predictive models and seeking machine learning solutions to complex ethical challenges, or would they move beyond the limit of ‘calculation’ towards shaping symbiotic and coherent ethical frameworks. The inherent risk in the scientific model of ethics is that it abstracts the very human factors away from concrete realities of the human condition (where ethics is a living and engagement process), thereby making it a matter of mere ‘calculation’, an algorithm. This abstraction may not cause concern in domains of science ‘where concepts, ideas and principles are all inter-related in a carefully structured matrix of mutually supporting definitions and interpretation of experimental observations’. But the problem arises when this scientific model is applied to societal domains such as health care and medical practice, where the diversity, interrelatedness and complexity of factors is so great that the abstraction of these factors may lead to severe damage and unacceptable results. It is welcoming, though, to note that performing ethics rooted in the human-centred movement is being creatively cultivated by distributed networks of socially active scientists, community engagement groups, tech for good entrepreneurs and communities for public good, especially those concerned with social engagement, equity and fairness, health care, well-being, social inequality and governance frameworks.

Whilst recognising the re-emergence of a human-centred ethos of ethics, we should be mindful of the technological lore of machine learning, logics of analytics, deep learning, probabilistic reasoning, and models of quantification and measurement, which may seduce us towards accepting the technological inevitability of ‘The human becomes the machine, the machine becomes the human’. van Est (2015) in his article on Intimate Technology gives us an insight into why we should be concerned with the implication of two technology trends: ‘biology is becoming technology’ and ‘technology is becoming biology’. The first of these is reflected not only in biological interventions such as those of ‘cloned sheep, cultured heart valves and artificially reconstructed bacteria’, but also in technology interventions such as the ‘deep brain stimulation to influence brain processes’. The second trend of ‘technology becoming biology’ is reflected in artefacts such as social robots that ‘increasingly appear more lifelike or seem imbued with human behaviours’. van Est further reflects on these technological trends that may lead to a transformation in which ‘human beings are more and more seen as machines’, and ‘machines become more and more humanoid’. Further more, as machines increasingly penetrate ‘our privacy and social life’, the danger is our interaction with machines may become so interchangeable and indistinguishable that we begin to visualise ‘man as machine’. These technological and biological trends raise the question of what is it to be human in the era of ‘sliding boundaries’ between these concepts? This question should make us reflect and rethink the impact of the convergence of these trends on performing ethics.

AI&Society warmly welcomes reflective contributions to the debate on performing ethics in the era of the cyber-biology convergence. In pursuing this debate on the impact of technological inventions in complex societal situations, we should take note of Cooley’s reminder that ‘the more we give to the machine, the more we diminish ourselves’.