An uncomfortable question the health tech will have to face in the future is the question of accountability. If AI fails to perform a surgery, who will be held responsible or who will be blamed for it. We do not know.

Recently during research, an AI system successfully performed a gall bladder surgery from start to finish autonomously. It was like a glimpse into the future of medicine but in the middle of the celebration of progress comes an uncomfortable question that what happens when AI fails and who will bear the responsibility- the supervising doctor, the hospital or the AI developer? This is the ethical storm health tech is about to face. This is not just a medical question, it is an ethical one too, the kind of question that will define the future of AI in healthcare.

And in a country like India, where innovation often sprints ahead while regulation tries to catch up with it, that ethical storm I told you about may arrive sooner than we think. When a surgeon decides on a treatment plan, those actions are backed by human judgment, professional license and moral responsibility. But when an autonomous system makes a surgical decision, who do we hold accountable when things go wrong?

Who gets sued is not the real question

Whenever new technology disrupts a field, the first instinct for us is to ask: “If it fails, who gets sued?” But in health tech, that question is too narrow as we will see accountability as not just about legal blame but also about failed trust, transparency and clarity. To build more safer systems that every single person can trust, we need more than liability frameworks, we need moral and operational ones.

AI is not a human doctor and it cannot feel remorse or make ethical choices. It only optimizes for objectives it has been trained on by us. So, the real priority is not figuring out who to punish after failure. It should be about designing systems that anticipate failure responsibly.

Shared accountability framework

You walk into a hospital knowing that your surgeon is AI, the first question will not be how advanced it is, but how safe it will be? That is why trust should become the primary goal in health tech care. Patients should have the right to know whether an AI was involved for the treatment and get consent on it before starting the procedure.

We cannot and should not cling to the illusion that one person alone can bear responsibility. The solution lies in shared accountability. Every model used in healthcare should be auditable and its decision pathways should be transparent enough for medical boards to review. And the developers must also be accountable for all the regular updates, checking on performance and also retraining the developed models.

Even if an AI performs surgery autonomously, a licensed human practitioner must oversee the operation which is ready to intervene as there is ethical, or patient related specific judgment needed.

Hospitals and clinics must ensure bias checks, informed consent from patients and independent trial certifications so that the same shared accountability approach can make sure that no one escapes from responsibility but no one shoulders it alone either.

Trust is The Real Operating System of Healthcare

We know that no technology can transform healthcare sustainably without building trust. Patients trust doctors not because they are perfect, but because they are accountable. That moral agreement is sacred in medicine. When AI enters this space, it inherits that responsibility even though it cannot fully express it. That means we, the people building AI and developers carry a moral weight. We must design for trust and not just for efficiency. And trust does not come from saying that AI is safe. It comes from showing how safety is maintained throughout the process.

A future where AI operates independently will only succeed if people believe that the system is governed with full fairness, responsibility and clarity.

Towards a responsible future

The potential of AI in healthcare is huge like it provides shorter surgeries, low cost and faster recovery. But to unlock it, we need safer and clear policies. We do not need to bring an abrupt stop or fear AI in healthcare or surgery but we need to build moral and legal ways for a safe and trustworthy innovation.

Until we build systems that give us a clear answer to the question of accountability, every success story of AI in healthcare will carry a quiet warning, that is, progress without accountability is just like taking risk wearing a lab coat.

Without clinical validation protocols, transparency mandates and ethical boards that are overseeing AI deployment in critical healthcare, we are innovating into a vacuum of responsibility and India cannot afford to lag behind on this.

If such a system fails, who do you think should be responsible?