Earlier this year, Sydney hosted the World Conference of Computational Neurosurgery. The inaugural event was testament to the increasing interest in the way neurosurgery employs computational methods such as AI, surgical robotics and neurotechnologies, including brain-computer interfaces (BCIs), and the way all of that may enhance the care and future prospects of patients who require neurosurgery. It culminated in the Declaration of Sydney, a set of principles relating to the ethical use of a variety of forms of AI in neurosurgery, and a related white paper.
I was part of the Task Force which drafted the Declaration as well as a co-author of the white paper, and since I have written extensively about ethical, social and legal issues relating to neurotechnology, of particular interest to me is the inclusion of an article relating to the governance of brain-computer interfaces and other forms of neurotechnology which reads as follows:
The development, validation, and application of brain-computer interfaces (BCIs) and other forms of neurotechnology to advance patient care require informed consent for surgical intervention, rigorous regulatory oversight, and strong safeguards to prevent non-consensual data extraction, processing, and misuse, including manipulative, coercive, malevolent, or criminal interference (art 14).
In what follows, I outline my perspective on some of these issues. I speak for myself here and should not be taken to be speaking for other members of the Task Force.
Benefits and risks
As well as producing benefits, the abovementioned technological advancements come with risks and generate some issues that may not traditionally have been at the forefront of the minds of those engaged in surgery, or of those who produce or regulate the computational biomedical devices that are used. This short commentary will focus on some human rights risks and relatedly informed consent, in the context of one kind of computational device that is now an option for some neurosurgeons –- implanted neural devices. It will first consider the patient perspective and then speculate about a possible future with human rights risk to neurosurgeons themselves.
It is important to draw attention to the remarkable and very significant upsides to an increased adoption of these and other computational devices such as surgical robots, and technology that can make AI-powered inferences from data revealed by brain scans – they seem very likely to alleviate much suffering and we should think carefully about whether any responses to concerns outlined here may thwart the upside.
To take a striking neurotechnological example we might think of a company that has its origins in Australia -– Synchron. This BCI company is a rival to Elon Musk’s brain-implant company Neuralink and has implanted devices in a number of participants in clinical trials. Spawned by research from the University of Melbourne, the device enables people who, as a result of forms of paralysis, cannot effectively use their musculature to control technical devices such as cursors in order to interact with the web and smart homes. Synchron’s technology requires a minimally invasive endovascular procedure rather than open brain surgery: a stent-like device, the ‘Stentrode’, is delivered through the jugular vein and into the superior sagittal sinus, a large vein running alongside the motor cortex, from where it records signals without being implanted in brain tissue. The procedure is performed by neurointerventionists, who may include neurosurgeons.
Computational methods instantiated in sophisticated forms of device-engineering as implanted by or with the involvement of neurosurgeons make this possible, and the participants in Synchron’s study have joined a small number of participants in other clinical trials around the world who, as a result of the implant, require less support from carers through the increased autonomy that comes from a new way of interacting with the world – the ability to control devices by mental acts as interpreted from associated neural activity using computational methods.
At this point it is worth reiterating that these and other computational advances in the options available for neurosurgeons have a very significant upside but what are the ethical and legal issues relating to the computational shift in neurosurgery, and what are some downsides that go beyond more standard risks of medical procedures?
Ethical risks
Different computational methods and devices bring different issues, but I will focus solely on implanted neural devices here. Let us assume a person is to be implanted with an AI-powered neural device that monitors and stimulates the brain surgically to address a neurological condition. Rather than use the BCI examples above that are still in clinical trial stage, I will consider devices such as those addressed at drug-resistant epilepsy that have already been approved by medical regulators in some parts of the world and are in clinical use. An example would be the surgically implanted NeuroPace RNS System, which operates by continually monitoring the brain of a person with epilepsy and automatically stimulating it to avert an impending seizure when it ‘notices’ the neural precursors to it. Its detection is closed-loop and automated, but it is worth being precise about what is doing the work: the implant applies threshold-based detectors whose parameters are programmed by the treating clinician, rather than machine learning running on the device. Machine learning is applied to the recordings the device generates, away from the patient, to refine those settings and individualise care. The system is therefore computational and automated, and only loosely AI-powered.
Epilepsy has joined other conditions such as Parkinson’s disease, OCD and depression where, at least in some parts of the world, therapeutic intervention by way of neural device is possible. The abovementioned BCI devices that address issues relating to forms of paralysis form part of a wider neurotechnological program to treat a variety of neurological and psychiatric issues by way of neural device that is fuelled by very considerable economic excitement.
However, if we think about what is happening with, say, a smart epilepsy device of today we can identify a number of features:
- It monitors the brain
- It stimulates the brain
- A black box AI makes the ‘decision’ about when to stimulate
- The device at some stage provides information and data to the therapeutic advisors and thus at least at times connects to other networks rendering it possibly susceptible to hacking
The ability to monitor the brain gives rise to concerns about privacy. So, of a neural device we might ask where is the data stored? On the device or in the cloud? Who might potentially get access to it? The patient? The therapeutic advisors? The company who makes the device? Business associates of the manufacturer? Or, as noted at point 4 above, hackers?
Further, what kind of inferences are permissibly made about the data? Is it only health-related? Or might the data one day be re-mined or perhaps even combined with other repositories of data such as those from social media, search engines, or chatbots to make inferences about likely consumer or even political behaviour?
This of course brings concerns about manipulation. Knowledge is power and one can imagine a dystopian world where someone has a neural implant with somewhat divided loyalties, and it surreptitiously transmits real-time information about its host’s mental states to a remote recommender algorithm as that person is engaged in online shopping so that the seller has an unfair advantage over the consumer in the transaction.
Neural devices that treat depression stimulate the brain, and of course if depression is treated then behaviour is altered (hopefully in a positive way). But what about a device that might surreptitiously stimulate to make a person a little more impulsive in a consumer context to someone else’s benefit? Or in a political context?
Neural devices have even been seen as a risk factor in the context of cognitive warfare and some have envisaged a military adversary hacking the neural devices of military and civilian populations during a conflict as part of their cognitive warfare strategy.
Human rights and beyond
Worries including those related to privacy and manipulation have given rise to a vibrant scholarly debate about whether novel ‘neurorights’ are needed. It is clearly the case that when the international human rights framework was created in the post-war period, the drafters were not thinking about brain-implants that extract data and manipulate people – they were understandably more focused on the possibility of the use of more rudimentary technologies such as gas chambers recurring.
In light of this and for other reasons some have proposed novel ‘neurorights’ are needed, such as the rights to cognitive liberty or even free will. Others think that the existing human rights framework, which includes rights such as those in the International Covenant on Civil and Political Rights, such as privacy (art 17) and freedom of thought (art 18), together with mental integrity as found in the Convention on the Rights of Persons with Disabilities (art 17), is sufficient even if those rights require some reinterpretation for the contemporary technological context.
The discussion about the adequacy of the human rights framework has now spread into policy discussion with contributions from bodies such as the UN and Australian Human Rights Commission and concerns about neurotechnology have even had a role in leading to constitutional change in Chile.
This brief overview is not the place for a discussion of the adequacy or otherwise of the existing framework but it is worth noting that as well as the right to mental integrity1 (which relates to actions that detrimentally affect the mind, including non-consensual therapy, and is particularly relevant to those with disabilities) and the right to privacy, there is another right which is now starting to attract more attention.
The erstwhile neglected right to freedom of thought has been singled out as particularly important in the context of neurotechnology, and it is included in the Universal Declaration of Human Rights, the International Covenant on Civil and Political Rights, and a variety of other national and international human rights instruments. But for most of its life it attracted little attention, and it has not been clear what its contours are.
As a result of developments in AI and neurotechnology in the context of democratic backsliding, this has changed and people are now considering how it might be elaborated and one contribution to this discussion has come from part of the UN. Amongst other things, the former UN Special Rapporteur on Freedom of Religion or Belief, Ahmed Shaheed, in his report on the right to freedom of thought, was of the view that it included the right not to reveal one’s thoughts or to have them impermissibly modified.
It may be that further development of the right to freedom of thought and other rights could go some way to addressing the concerns about privacy and manipulation without the need for novel neurorights (which may be politically infeasible in any case). Whilst the best route to tackling worries about the adequacy of the human rights framework given advances in neurotechnology is more a concern for lawyers than for computational neurosurgeons, it is useful for the latter group to know about the human rights threats. As demonstrated, some lawyers have been thinking about how they are going to respond to these risks, and those involved in computational neurosurgery need to think about them too, hopefully with input from groups representing patients.
One initial thing to consider is how devices (such as forms of neurotechnology) that make use of computational methods are designed. What happens to data? How safe is it? Is it kept on the device? What happens to it after it has served its therapeutic goal? If the device stimulates the brain how well-protected is it from external influence?
Devices might be designed in ways which make them more or less risky from a human rights perspective, and that is an issue for those biomedical engineers who are involved in producing the computational devices that are used by neurosurgeons. It will be important to consider the perspective of end users of the devices or their organisations such as the BCI Pioneers Coalition when deciding how to design future devices.
Something for neurosurgeons and their professional bodies to consider is how a person can give informed consent in respect of a neural device. Outside the context of neural implants and setting down broad principles in relation to informed consent, the leading Australian authority remains the 1992 High Court decision in Rogers v Whitaker. According to that case, a medical practitioner has a duty to warn a patient of a material risk inherent in a proposed treatment. A risk is material if, in the circumstances of the particular case, a reasonable person in the patient’s position, if warned of the risk, would be likely to attach significance to it, or if the medical practitioner is or should reasonably be aware that the particular patient, if warned of the risk, would be likely to attach significance to it. That test now operates alongside the civil liability legislation enacted in each Australian jurisdiction in the early 2000s, which preserves the Rogers materiality test for the duty to warn of risk while modifying the standard of care in other respects. The test places patient autonomy at the centre and means that there needs to be an inquiry into the particular patient’s situation and concerns.
But what should be said about risks such as hacking, manipulation (by manufacturers or others) and loss of privacy? It is not a standard part of medical consent practices to discuss the risk of a patient being manipulated to the benefit of another person (natural or corporate), or to note that the risks of the device, even for civilians, might change during possible future military conflict or as a result of AI-assisted advances in hacking capabilities. However, such issues may be thought of as significant for reasonable people and perhaps of particular concern to some patients and thus may need to be raised. These risks may be hard to quantify.
Another thing to consider is the role of AI in this. The epilepsy device mentioned earlier doesn’t check with a neurologist when it is time to deliver brain stimulation; this is automated. Its settings are revised by clinicians rather than learned by the implant, so its autonomy is presently very limited. One can readily imagine successors that are updated in their software from time to time, and that become increasingly autonomous as a result of more sophisticated AI. The upshot of all of this is the device changes in its manner of operation over time and a patient would need to give informed consent for a system that is continually in flux and is somewhat unpredictable – it is not clear precisely how this will be achieved but given that consent must be informed, discussion would need to be had about this to ascertain the patient’s perspective on accepting a system that is likely to change in its operation over time.
Given that the operation of devices is likely to be somewhat of a black box, questions such as why the device ‘chose’ a particular moment to stimulate will be hard to answer. However, people may be prepared to accept some algorithmic mystery for health benefits. It seems that for consent to be fully informed a patient should know that questions relating to the timing of stimulation may be unanswerable even if the device goes wrong at some stage and it seems that it may be reasonable to want to know about this if deliberating about whether to have surgery.
Human rights are important and one needs to think of a variety of rights including those already mentioned and of course the right to health itself, but the list of ethical and legal challenges goes well beyond this. Beneficial devices might be somewhat unfairly distributed in society raising issues of equality. Companies producing devices might go bust, and neurosurgeons and their patients should consider what happens in that eventuality. Clinical trials of neurotech come to an end and those involved in these trials need to consider post-trial support where a device remains implanted in a person’s brain or peripheral nervous system. Developers need to consider whether there is a bias in the sample groups for clinical trials and whether the operation of their algorithms is in some way biased.
Device approval by regulators is not normally a central concern in human rights discourse but these regulators need to think about the human rights risks, as well as what they will do should a device have the potential to go beyond a therapeutic/restorative orientation towards enhancing people beyond the normal ranges of human capacity (for example in relation to memory). It is also worth noting that if medical device approval practices are attuned to rights such as the right to mental integrity, privacy and the right to freedom of thought when they evaluate safety and risk, some of the abovementioned issues will be mitigated.
Neurorights for computational neurosurgeons
Beyond debates concerning new interpretations of existing rights or the creation of novel neurorights, neurosurgeons might one day need to think about their own rights in the face of advances in computational neurosurgery.
In the context of operating potentially unsafe mining equipment it has been reported that some employers require their employees to make use of wearable forms of brain-monitoring neurotechnology in an effort to ensure safety. Whilst neurotechnologies might alert a worker if they become drowsy or inattentive, their use has led to some concerns about the mental privacy of workers. Perhaps if neural devices become more pervasive in society, then an expectation might emerge that neurosurgeons be neurally monitored in order to avoid slips of attention during surgery. This would bring the privacy issues home to them.
The abovementioned reference to the possibility of memory enhancement raises the more general question of the ethics of cognitive enhancement. There is military interest in the prospect of neurotechnology to cognitively enhance military personnel and it seems possible that this one day may become feasible. Elsewhere it has been suggested that lawyers might at some stage face a pressure to cognitively enhance by way of forms of neurotechnology. If concentration can be enhanced by a neural device it might at some stage be thought of as negligent for a person engaged in important legal work (such as arguing a case in court) not to cognitively enhance thereby creating a pressure to do just that. Another pressure might come from competing with peers. If one’s peers are using neural devices in ways that enhance their workplace performance, then it might be hard to resist ‘voluntarily’ engaging in enhancement. Finally, as AI systems become more capable of engaging in legal work, lawyers might decide to use cognitive enhancement by way of neurotech to keep up with the parts of the workplace that are wholly mechanised by becoming partly mechanised themselves.
If one thinks about the neurosurgical context, neurosurgeons might too face a pressure to cognitively enhance themselves — to beat their peers by keeping up their good surgical record of success, to avoid litigation, or to stay relevant in the context of other parts of the computational neurosurgical program that might start to compete with humans in the workplace, such as perhaps more autonomous surgical robots (which bring a whole other set of ethical issues for computational neurosurgeons to consider!).
In this way the surgeons might need to worry about their own brains and their rights to privacy, mental integrity and freedom of thought, as well as those of their patients.
But perhaps all this anxious thinking just comes from getting gloomy lawyers involved in the project of computational neurosurgery, and we must not forget the very considerable upside of neurotechnologies, and other computational methods and devices involved in medicine. Sometimes sensible courses of action can be hijacked by downside thinking and that would be disastrous if suffering that could have been alleviated is prolonged or left unaddressed as a result of overblown anxieties about risk.
Nonetheless neurotech-related risks do need to be acknowledged, communicated and mitigated.
Acknowledgements
Thank you to Christopher Rudge and Hugo Layard Horsfall for very useful comments. Any remaining errors are the responsibility of the author.
- For a discussion of this right, see Christoph Bublitz, ‘Neurotechnologies and Human Rights: Restating and Reaffirming the Multi-Layered Protection of the Person’ (2024) 28(5) International Journal of Human Rights 782, 786–788 https://doi.org/10.1080/13642987.2024.2310830. ↩︎
Leave a Reply