Constitutional Dimensions of AI-Driven Judicial Decision-Making

1 Introduction

AI is defined as follows: “[Artificial intelligence] (AI) refers to systems designed by humans that, given a complex goal, act in the physical or digital world by perceiving their environment, interpreting the collected structured or unstructured data, reasoning on the knowledge derived from this data and deciding the best action(s) to take (according to pre-defined parameters) to achieve the given goal; AI systems can also be designed to learn to adapt their behavior by analysing how the environment is affected by their previous actions”.¹

The struggle for justice has been a foundational element of human civilization. The legal system has been evolved over the centuries. The development of justice finds its base in the idea of equity, fairness, natural justice and the preservation of public order. The advent of Artificial Intelligence (AI) has affected various aspects of day-to-day life of human civilization including its applicability over judicial decision making. The judiciary has been long regarded as an institution of human discretion but is now facing technological transformation. Artificial intelligence, once used only in science fiction, is being actively used in courtrooms around the world nowadays. AI is now mediating decisions on automated mode that is remarkably affecting individuals’ lives.

Machines are taking over the work of people everywhere. In the not too distant past, spell-checking and search engines were regarded by many as ‘intelligent’ information technology. Today, facial recognition routinely checks travellers at our airports. Google Maps gives unsolicited advice about destination: “the restaurant may be closed”. Tablet and mobile phone answer spoken questions with friendly spoken replies. The newspapers talk about ‘robot’ justice. There are claims that algorithms can accurately predict court decisions, and that we won’t need human judges anymore. We enjoy talking about things that don’t exist yet, and fantasize about how they will make our lives easier.

²AI can be described as “allowing a machine to behave in such a way that it would be called intelligent if a human being behaved in such a way”.³ This is the definition that John McCarthy, considered to have invented the term “Artificial Intelligence”, gave to AI in 1956. This is important to establish, defining human intelligence as the measure of what AI does. Intelligence is the ability to reason abstractly, logically and consistently, discover, lay and see through correlations, solve problems, discover rules in seemingly disordered material with existing knowledge, solve new tasks, adapt flexibly to new situations, and learn independently, without the need for direct and complete instruction.⁴

This use of AI is expected to reduce human error and enhance access to justice. AI is capable of analysing vast datasets, identifying patterns and making predictions and is hence used in the judicial process now. To recommend a sentence it can analyse evidence and conduct legal research regarding the suitability of the sentence for a particular category of offence and an offender.

Artificial Intelligence (AI), powered by advanced algorithms and machine learning, has become increasingly vital in modern society. It has been integrated into criminal justice systems worldwide, significantly influencing the administration of justice. The adoption of AI is expected to grow substantially in the future, reshaping how justice is delivered. While AI is valued for its perceived reliability and objectivity, it also presents notable ethical and legal challenges. Despite its benefits, concerns have emerged that machine learning may adversely affect the fairness of judicial processes. The incorporation of cutting-edge technology and robotics in AI holds potential to enhance economic and social aspects of human life.⁵

However, its deployment raises certain questions regarding the foundation of constitutional law, like: is it possible for a machine to follow the constitutionally guaranteed justice? Can it reassure the principle of equal protection under the law? Can it be a threat to impartial judiciary? In aggregate it can be said that excessive reliance on AI raises complex constitutional questions that can be a threat to the democratic governance and the rule of law.

In March 2025, Justice B.R. Gavai, who later served as Chief Justice of India from May to November 2025, expressed serious reservations about usage of AI in legal organizations. At a Nairobi seminar on “Leveraging Technology within the Judiciary,” Justice Gavai said that although technology has many advantages, its use needs to be carefully considered in light of its drawbacks.⁶

While AI promises enhanced proficiency, stability and access to justice, it also constitutes significant constitutional obstacles. China has been a pioneer in incorporating AI into its court system, hence the constitutional framework of China with special reference to the role of AI-driven judicial decision-making needs to be analysed to frame an outline of Indian law. Apart from it comparative insights from global jurisdictions shall also be considered. AI is the system that produces intelligent results. Nowadays, using AI in the administration of justice has become common. AI has a significant role in the routine activities of tribunals and courts.⁷ While using AI it must be ensured that the pursuit of technological advancements does not come at price of cherished legal principles and individual rights.

2 AI in the Judiciary: Application and Capabilities

AI can be useful in many different ways to meet different requirements. Sales talk on AI for courts is abundant. It has been argued that ‘it would make it fairer, and moreover, unlike human judges, AI does not get tired and does not depend on its glucose levels to function’.⁸ The working speed of the judiciary is enhanced due to use of AI and expedites dispensation of justice. AI assists judges by predicting vital information regarding the case. AI is not only used in the judiciary, but police also use it for investigation purposes. The police can acquire the requisite information by use of AI without any difficulty. The police acquired vital information using AI, i.e., information about the accused, charge sheet filing date, number of witnesses examined during evidence, the reason for adjournment, the quantum of punishment, and compensation awarded.⁹ Therefore, analyzing all information through AI helps judges make better decisions and strategies. Across the world, different countries used AI in their judicial proceedings, especially the UK, Austria, France, Finland, and the Netherlands. AI is used in the administration of justice.¹⁰ AI use in the legal domain can give extraordinary assistance to the judges. However, it cannot replace a judge’s expertise. Algorithms help to analyze a large set of data to find solutions and make predictions as the AI has predictive nature. This has led to an increase in its use in criminology, forensics, and legal areas. AI technology helps in the translation of documents and legal research. AI assistance supports judges’ decision-making and helps in judicial decisions. It has been suggested that robot judges could replace human judges and that hearings could be conducted automatically.¹¹ AI tools are used in the criminal justice system. AI introduced algorithms informing the decisions about parole, probation, sentencing (acquittal or conviction), and bail.¹²

AI technologies in judicial systems include processing language along with predictive analysis of cases to enhance decision-making and efficiency. These tools access vast datasets to identify patterns, but their reliance on set historical data can lead to bias, often called a “black box”. Although AI can simplify tasks and minimize human mistakes, its complex nature requires careful examination to ensure compliance with law and justice. Estonia is the country that has made an experiment with AI-based judges to handle small claim cases. Common applications of AI include:

2.1 Case Management and Court Administration

As per the information provided by the Supreme Court of India, Artificial Intelligence (AI), and Machine Learning (ML) based tools are being deployed in case management. They are being used in transcribing of oral arguments in Constitution Bench matters. The AI assisted transcribed arguments can be accessed from the website of the Supreme Court. The competent authority has directed to consider the transcribing of oral arguments on regular hearing days i.e. Thursdays.¹³ It helps to reduce the burden of administration and improves the efficiency of judiciary.

2.2 Organising Information

Recognising patterns in text documents and files can be useful, for example when sorting large amounts of cases, or in complex cases that contain a lot of information. An example from the United States of America is ‘eDiscovery’, an automated investigation of electronic information for discovery, before the start of a court procedure. eDiscovery uses machine learning AI, which learns through training what the best algorithm is that is capable of extracting the relevant parts from a large amount of information. Parties agree which search terms and coding they use. The judge assesses and confirms the agreement. This is a method of document investigation recognised by the courts in the United States and the United Kingdom. The method is faster and more accurate than manual file research.¹⁴

2.3 Legal Research as well as Document Analysis

AI can be used for legal research when we have to go through thousands of documents, which can be very time-consuming. AI can do the same task within a few minutes and obtain accurate and relevant information.¹⁵ AI can process enormous amounts of judgments and statutes more efficiently and accurately. It can understand and interpret legal language, allowing lawyers and judges to perform more detailed research in less time.

2.4 Predictive Analytics and Sentencing

Predictive analytics has become a cornerstone of data-driven decision-making, enabling businesses and organizations to forecast future trends, behaviours, and outcomes. With the integration of Artificial Intelligence (AI), predictive analytics has reached new heights of accuracy and efficiency. AI-powered predictive models can analyse vast amounts of data, identify patterns, and generate actionable insights, transforming industries like finance, healthcare, and retail.¹⁶ Predictive analytics uses historical data to derive outcomes of proceedings of the court. These tools are based on probabilities in a case or estimate the time required to resolve a dispute.

2.5 Virtual Hearings and Online Dispute Resolution

Online dispute resolution (ODR) consists of online alternative dispute resolution (OADR) and online courts. OADR is dispute resolution outside the courts, which originally emerged in the mid-1990s as an adjunct to various forms of alternative dispute resolution (ADR) and as a response to disputes arising from the expansion of ecommerce. As a result, it focussed on using technology to resolve customer complaints and sought to support negotiation, mediation and arbitration. Today it may go further and give rise to innovative ways to resolve disputes beyond the traditional categories of ADR. OADR may be privately run or state-sponsored, such as when it forms part of a consumer redress scheme. It may be synchronous (the participants are all present at the same time) or asynchronous (the participants engage with the process at different times), or a combination at different steps in the process. In contrast, online courts form part of the justice system and are therefore subject to institutional norms and legal requirements derived from the nature of the judicial function.¹⁷

2.6 Translation and Accessibility

AI-powered translation tools can work efficiently to break down language barriers and work accurately in multilingual legal systems. AI can provide real-time translation of court proceedings, documents or testimonies. This feature helps to ensure that all parties understand the proceedings regardless of the language understood by them. In addition, AI tools can generate audio or visual content for individuals with disabilities, making the judicial system more inclusive and equitable.

3,89,41,148 cases are still ongoing at the Taluka or District levels, besides 58,43,113 cases are still waiting at higher courts, according to the most recent NJDG (National Judicial Data Grid). The effectiveness of the judiciary suffers as a result of such pendency, which eventually limits people’s availability of justice.¹⁸

2.7 AI Related Start-Ups¹⁹

• •
  Spot Draft: This is an AI based start-up by Shashank Bijapur, who is a Harvard Law School alumnus and Madhav Bhagat an ex-Google employee. This AI driven business has the ability to scrutinise legal documents and it reduces the paperwork by giving the customers an option to create business contracts. This revolutionary platform offers its clients an option to draft and sign contracts; it also has features of automatic reminders and payments.
• •
  CaseMine: This is a legal research platform. This start-up aims at using AI technology in order to draw links between different case laws and thus makes it simpler for the legal researcher to have an in-depth and comprehensive research.
• •
  CaseIQ: This machine learning software acts like a legal assistant in researching case laws; it also analyses the legal language and works like an assistant by pointing out any potential points of law which might be missing, suggests alternative arguments, highlights relevant judgements and case laws for a comprehensive legal research.
• •
  NearLaw: This Mumbai based start-up offers AI based solutions to legal practitioners and law firms. NearLaw is said to be using NLP technologies to assist in legal matters by understanding case rankings.
• •
  Practice League: This is a Pune based legal tech law firm which has used AI capabilities to transform the working model for more than 8,000 lawyers. Reports highlight that this firm is working with tech giants like Google and Amazon in order to weave AI abilities into its working solutions.

2.8 Benefits of AI in the Judiciary

Nowadays, use of data in an electronic form (E-Form) is gaining importance in the community. The electronic form of data is easily and quickly searchable and processable. Across the world, the development of technology is rapidly increasing. Today, AI is used in many fields, including the judicial system. AI is a system designed by humans; this system achieves complex goals. AI is used in the judicial system in different ways; firstly, AI as a tool in the hands of the judge; secondly, AI as an ADR method; and thirdly, AI as the autonomous decision maker.²⁰

AI is used in the justice system for “transparency, justice, fairness, non-discrimination, safety and security, privacy and data protection, and human responsibility for the decision”. These all are AI’s core ethical principles and are universally accepted. When AI is used in judicial procedures, it redefines work and improves work conditions for judicial officers. AI is applicable in numerous areas with positive results. On the other side, AI also carries some risks. In AI technology, the self-generated algorithms contain more complex risks.²¹

The use of AI tools in Criminal Justice Proceedings provides several opportunities to ensure the legal system’s predictability, standardization, and transparency. Law enforcement officials and legal bodies involved in law and criminal proceedings must be aware of opportunities AI might provide for effectiveness and development in real-time. The use of AI in judicial field has positive and negative aspects. When the AI algorithms are used in judicial proceedings, they also take some risk factors, e.g., as a judge has to decide upon release of a pregnant woman offender who is at risk of reoffending. In this case, the AI algorithm should rationally provide the decision against such a woman about her release. Still, on the other side, the human judge has the opportunity to make the decision of this case based on new hierarchy of the victims that would be created once offender becomes a mother; hence, this example shows that the AI cannot resolve all issues which are raised in the criminal proceedings. The principles of AI used in criminal judicial proceedings should be ensured or comply with rule of law, the presumption of innocence, and other general principles established in Article 6 of ECHR. Thus, in criminal proceedings, the interested and aggrieved party has the right to claim or challenge the scientific validity of the use of AI Algorithms.²²

AI can easily access and analyse information and enables courts to resolve cases more quickly and efficiently. AI can reduce inconsistencies in judicial decisions by relying on data and precedent and can improve access to justice for individuals. AI supports judges with comprehensive data, legal research and precedents to enable courts to make reasoned decisions. AI offers innovative solutions to enhance efficiency, transparency and accessibility.

On the other side, when AI is used in the judicial system, it also has some inherent limits. When algorithmic decision-making is used, it is not easy to achieve precision and objectivity. When algorithmic decision-making is used in difficult nature and complex cases, then it is difficult for AI to achieve substantive justice. AI may be able to promote only formal justice.²³

In the Court proceedings, the AI can advise on the matter. The advisory AI could be useful for the people, the general public, the parties of the case, and legal professionals. AI technology in the Courts not only looks for relevant and legal information. But, this technology also provides answers to questions. Later on, it is the choice of the user whether the user will act on the advice provided by the AI tools or technology. Hence, this advisory function of the AI tools helps people to resolve their disputes by themselves (outside of the court). For example, the Civil Resolution Tribunal (CRT) was established in British Columbia, Canada. This CRT Tribunal deals with disputes related to subsidized housing and strata. This tribunal was provided free public legal information, available 24/7, calculation aids, and solutions explorer. In 2019, it gradually extended its jurisdiction after it proved successful. The CRT builds its expert system after every three months. Thus, updating this system (CRT) is still done by human experts.²⁴ Another significant benefit of AI is that it can fix the language barrier in between diversities of 22 official languages of India by translating the text in desired language.

2.9 Constitutional Principles and AI

Principle of Fairness, Impartiality and Transparency is the fourth fundamental principle of the European Ethical Charter on using AI in the judicial system. This principle of the Charter is broader as compared with other ones. This principle deals with use of AI in judicial system with transparency, fairness, and impartiality. There is still a need to pay further consideration and attention to the explainable AI and take initiatives regarding fairness, accountability, and transparency in judicial AI. For example, COMPAS Software (algorithm) was also criticized for its bias. According to the Charter, fairness and transparency are the broader concepts, and when AI is used in the judicial system, many rights of people are concerned in the case. Thus, the algorithmic software and another tool of AI used in the judicial system must act fairly, transparently and impartially.²⁵ There must be some check and balance or any form of accountability on the tools of AI used in the judicial system.²⁶ Principle under user control is the 5th fundamental principle of the Charter on using AI in judicial proceedings. This principle supports a lawyer’s work in applying AI in the judicial system. According to this principle, the algorithms and software may not be used as prescriptions.²⁷

Use of AI in courts sparks constitutional debates about fairness, privacy and justice. Risk of bias in AI might weaken independence of judiciary and hinder handling of private information of individuals. Equality is at risk if AI reflects bias in making its decisions. To uphold constitutional values AI must support and not replace judges. The constitutional dimensions of AI in the judiciary are as follows:

2.9.1 Due Process and the Right to Fair Trial

The right of availing fair trial is enshrined within the provisions of Indian constitution. Fundamental theories of natural justice include entitlement to receive a reasoned decision and right to be heard i.e. audi alteram partem. When AI systems participate in judicial decisions there is a risk that these rights may be compromised. If the reasoning behind an AI’s recommendation or prediction is inaccessible, litigants may be unable to understand or contest the basis of the judicial outcome. Article 21 of the Constitution of India guarantees both entitlement to fair trial and due process. AI techniques challenge the due process when their decision-making processes are opaque. There is a probability of denying the ability to understand or challenge the basis of decisions affecting the liberty. Trust may be damaged by this lack of transparency in the legal system and violate the fundamental principle that justice must not only be done but must also be seen to be done. Furthermore, the reliance on statistical correlations identified by AI might not adequately suit individual circumstances and may lead to decisions that are procedurally unfair.

2.9.2 Equality Before the Law

The principle of identical protection mandates that all individuals must be treated equally under the law. One of the core promises of AI is the elimination of human bias. However, if trained on historic information that reflects societal bias and prejudices related to race, gender, socio-economic status, etc., AI systems may establish rather than eliminate discrimination. This poses a direct threat to the constitutional guarantee of identical protection under the law. Articles 14–18 of our constitution deal with equality before law. AI used in sentencing might disproportionately recommend harsher penalties for individuals. Reducing algorithmic bias is essential to prevent AI from worsening discrimination in our judicial system. Our constitution disallows difference on basis of place of birth, race, religion, sex, caste, etc. An investigation conducted in 2016 found that COMPAS was biased against African American defendants, flagging them as higher-risk at committal of crime nearly twice the rate of white defendants.²⁸

Judicial values such as impartiality and equality intersect with AI tools. AI systems have been used in biased ways in relation to marginalized people historically. AI tools’ decision-making led to bias and discrimination. Bias in digital systems may enhance in multiple ways. For instance, systems may overfit training data that is not representative of the broader population. Technologies such as intelligent speech processing could be used to replace court reporters and keep a live transcript of the court proceedings. The lack of interpretation creates a risk that financially capable parties would better understand technological systems. Principles of equality and impartiality require not only that like cases are treated alike but also that different cases are treated differently. Digital decision-making through machine mechanisms clusters the cases together that need to be treated differently.²⁹

2.9.3 Separation of Powers

AI in the judiciary raises questions about the balance of power among all other branches of the government. It has been embodied in Article 50 of the constitution to prevent the concentration of power and safeguard individual liberties. The increasing dependence on AI in jurisdictive decision-taking could potentially blur the lines between the judicial, legislative and executive branches, thereby violating the principle of separation of powers. The separation of powers doctrine ensures that each branch of government operates within its constitutional limits. If algorithmic tools are developed or controlled by the executive agencies or private corporations, there is a high risk that judicial discretion may be curtailed, undermining the independence of the courts.

2.9.4 Judicial Independence or Impartiality

The Indian Constitution establishes an independent and impartial judiciary as a cornerstone of fair adjudication.³⁰ AI systems, if overly relied upon, may erode the judicial discretion and produce biased decisions. If judges rely upon AI-generated recommendations without critical review they risk becoming mere spectators rather than justice deliverers.

2.9.5 The Right to Privacy and Data Security

Use of AI in judicial processes involves gathering and analysing large amount of sensitive personal information. It raises concerns regarding right to privacy and data security. Strict legal frameworks and technical safeguards are compulsory to prohibit unconstitutional access of and misuse of sensitive data used and processed by AI.

2.10 Comparative Approaches to AI and Judicial Oversight Worldwide

Across the world, different law enforcement agencies used the “crime forecasting software”. This software includes the algorithms, i.e., Precobs, HunchLab, PredPol, MapRevelation, etc. These algorithms of AI are also used in the work of criminal courts. Several algorithms of AI are also used to prevent or stop the commission of criminal acts. These algorithms have been identifying where the criminal act can happen or people who can commit a criminal act.³¹

Different countries are using AI in their legal systems in different ways. Some countries are using AI to make judgments more effective. Other countries are experimenting with using AI in more complex ways. Some other countries are using AI to help judges in making decisions in certain types of petty cases. Few countries have created specific laws and guidelines about how AI can be used in their legal system. These laws might set standards for things like fairness, transparency, and accountability. In some other countries judges themselves are taking help of AI. There are some common principles which are important in all cases; they are human control, due process, explainability and ongoing evaluation. These approaches illustrate the need for specific and principled regulatory frameworks that uphold constitutional norms. Different nations balance innovation concerns and safety through distinct legal frameworks.

Article 10 of UN UDHR, Article 14 of the UN ICCPR, and Article 47 of Charter of Fundamental Rights of EU provide the right to access justice and guarantee fundamental right of “fair trial within a reasonable time and establishing independent and impartial tribunal.”³² Article 14 of ECHR and Article 21 of Charter of Fundamental Rights of EU stated that any discrimination based on religion, language, belief, genetic feature, political opinion, property, age, sexual orientation, colour, race, ethnic, social origin, sex, birth and membership of a national minority should be prohibited. Thus, AI tools in the judicial system are bound to not discriminate against anyone.³³ Article 6 of ECHR also provides guidelines regarding ethics. This Article of ECHR also provides the rules and standards regarding the proper procedure. The main requirement is that AI is useful in the courts and provides a transparent procedure, equality rights provided to parties during proceedings, and well-founded judgment. Applying AI in the courts could reduce the complexity of the judicial system.³⁴

In the European Union (EU) a risk-based regulatory framework governs AI. EU’s “Artificial Intelligence Act 2024” is the first ever legal AI framework worldwide. It categorises AI systems in 4 levels as per risk, i.e. minimal, limited, high and unacceptable. High-risk system faces dire need of alterations in AI for transparency and accountability.

The United States adopts a decentralized approach and no comprehensive AI law exists there. U.S. courts play a significant role in tackling issues of facial recognition and misidentification. States like California and New York do not have AI in judicial system but introduced AI-specific laws, on privacy and transparency.

China’s approach is state-driven and it prioritises national security and social stability. The Cyberspace Administration of China (CAC) enforces strict AI regulations. It requires approval of the government for data affecting public opinion. China’s AI governance emphasizes real-time monitoring of AI systems which reflects a collective approach. In India, AI regulation is still somewhere evolving. The government’s National Strategy for AI focuses on its adoption in healthcare and agriculture fields but judicial oversight remains fragmented. The Punjab & Haryana High Court referenced ChatGPT in the Jaswinder Singh v. State of Punjab³⁵ case, for general view on bail jurisprudence.

In 2019, the state of Kazakhstan also implemented “AI to predict the court decisions”. This system of AI includes the 1.2 million judicial acts and 120 thousand statements of claims. This system of AI helps the judges regarding decision-making. When the judges feel difficulty in decision-making, they will turn to this system for help. The judges put the request in the system. After that, the system will provide 10 top court cases related to the request (facts, application of law, and related circumstances). Then, the judges consider this system’s recommendations and apply them to the case’s decision-making. In 2016, the Ministry of Justice of the UK introduced the bill known as “Transforming our Justice System”. The primary aim and objective of this bill are to digitize the Court proceedings, and it will “play a significant role in ensuring that legal systems in England and Wales provide a rapid and certain Judgments; this saves the time and cost of people and also reduces the impact of legal proceedings”.³⁶

Courts around the world face a tough task in making sense of complex technology while keeping laws fair. Different countries handle this differently. The European Union plays it safe, focusing on preventing problems. The United States leans toward letting the market decide, with less strict rules. China uses strict government control, while India is still building its system. Bringing these approaches together is hard because each country has its own priorities. Still global talks are working to find common and responsible AI.

3 Case Studies and Examples

While deciding a case, the Manipur High Court noted that it relied on Google and ChatGPT 3.5 for further research, indicating a growing trend of AI use in Indian courts, albeit with a general hesitancy similar to worldwide judicial sentiments over AI integration.³⁷ AI tools cannot replace human judgment and must be used subject to judicial review; this was held in State v. Loomis.³⁸ The European Court of Human Rights has addressed AI in cases involving automated surveillance and data processing. While fully autonomous AI judges are not yet a reality, various countries are experimenting with AI tools that assist in different stages of judicial processes. The following case studies highlight both the potential benefits and the constitutional challenges:

Germany’s AI in Case Management and Judgment Drafting: German courts are utilizing AI systems like OLGA (Oberlandesgerichtsassistent) at the Stuttgart Higher Regional Court and Frauke at the Frankfurt District Court to manage case backlogs and expedite judgment in specific types of cases. OLGA assists in case categorization and information retrieval, while Frauke extracts case-specific data and uses pre-written text to make judgments. These tools enhance efficiency.

China’s “Smart Courts”: China has invested significantly in “smart court” systems that utilize AI for tasks like case filing, legal research and even suggesting sentences based on precedents. These systems aim to improve efficiency and consistency but still concerns exist regarding the transparency and the extent to which they might influence judicial discretion.

Estonia’s Robot Judge for Small Claims: Estonia has experimented with an AI-powered “robot judge” to resolve small claims disputes amounting to €7,000, i.e. approximately 6.58 lakh Indian rupees. It aims to reduce the burden of judges and provide speedy resolution for petty cases.

COMPAS in the United States: The COMPAS Algorithm was developed by the private company “Northpointe”. The court houses of the USA adopted widely use of AI tool “COMPAS”. This tool works as a management and decision-supporting tool. The COMPAS tool was used in Loomis vs. Wisconsin and Kansas vs. Walls. ProPublica analyzed the outcome of the COMPAS Algorithm. Advanced technologies like COMPAS are employed to evaluate an individual’s risk in criminal justice proceedings. This tool relies on statistical data to inform decisions. However, some researchers have criticized COMPAS, arguing that its algorithm perpetuates unfairness and bias, including issues of discrimination and racism. These flaws can negatively impact a wide range of individuals’ rights.³⁹

India’s SUPACE and SUVAS: The Supreme Court of India has adopted AI-driven tools such as SUPACE (Supreme Court Portal for Assistance in Court’s Efficiency) to support legal research, case analysis, and drafting, and SUVAS (Supreme Court Vidhik Anuvaad Software) for translating legal documents. These tools aim to enhance the efficiency of legal research and improve access to justice by addressing language barriers. The e-Courts initiative in India further leverages AI to streamline case management. While these efforts have boosted operational efficiency, commentators have asked whether they include adequate safeguards for data privacy and against algorithmic bias. Former Chief Justice of India, D.Y. Chandrachud, cautioned that AI could deepen disparities in access to justice, potentially creating a system where only the affluent benefit from high-quality legal assistance. Conversely, he acknowledged AI’s potential to make justice more accessible to the broader population.

3.1 Case Study 1: Colombia – Guardianship Action Involving AI Assistance

In 2023, a Colombian judge used ChatGPT to prepare a ruling in a guardianship case filed against a health service provider on behalf of a minor suffering from autism. While reviewing the case, the use of AI by the judges raised concerns about the due process and judicial independence in the Colombian Constitutional Court. This incident highlights the potential risks of relying on AI in legal contexts.⁴⁰

3.2 Case Study 2: The Netherlands – Property Dispute and AI-Generated Facts

In June 2024, a Dutch lower court judge used ChatGPT to determine factual details in a property dispute case.⁴¹ The judge’s reliance on ChatGPT was criticised and raised questions about the suitability of AI in legal contexts and risks of relying on unverified information. This case highlighted the necessity for strict guidelines and oversight when integrating AI into judicial processes.

3.3 Difference of Opinion Among Indian Courts

Punjab & Haryana High Court’s view: Jaswinder Singh, accused of assault resulting in death, was refused bail by the Punjab & Haryana High Court (Anoop Chitkara J.). In the order, the judge also put to ChatGPT a general question on bail jurisprudence where an assault involves cruelty, and recorded its response. The court explained that ChatGPT’s contribution was for a more general legal framework rather than a case-specific viewpoint. The High Court’s use of AI for legal research to support judicial thinking is demonstrated in this case.⁴²

Delhi High Court’s view: In a trademark lawsuit, Delhi High Court Justice Pratibha M. Singh decided in favour of Christian Louboutin. Louboutin’s legal team used comments generated by ChatGPT to show how the company is known for its “spike shoe style” with a “red sole,” which Louboutin contended Shutiq had imitated. However, Justice Singh disapproved of ChatGPT’s use to decide facts or legal matters in court, citing worries over possible errors, fictitious case law, and creative data produced by AI chatbots. This case highlights the value of human judgment over AI-generated content in court rulings and demonstrates the Delhi High Court’s cautious approach to integrating AI in legal processes.⁴³ Still in India there are no exact guidelines as to the use of AI in judiciary though some tools are being used here but it is not used in making decision but only used in taking help.

3.4 Constitutional Framework for AI in Judiciary

To tackle the constitutional disputes posed by AI, it requires a multi-dimensional approach. For which the following framework is proposed:

3.4.1 Developing Legal and Regulatory Frameworks

Existing legal frameworks are not adequate to tackle the exceptional challenges of AI; hence, there is a need for new legislation and judicial interpretations to govern the implementation of AI in judicial context. This includes establishing standards for accountability as well as transparency in AI system. To ensure the compliance with constitutional principles regulatory bodies must necessarily oversee the advancement as well as implementation of AI in judicial applications.

3.4.2 Transparency Frameworks

The constitutional principle is that legal decision must be well reasoned. Transparency and fairness are the important elements of democratic state. People in democratic state must know how decisions are made and on what grounds they are made. Conversely, many AI systems suffer from a lack of interpretability. Steps should be taken to create AI systems that are more transparent, understandable, and user-friendly.

3.4.3 Moderate Algorithmic Bias

Regular updating is essential to minimize bias and uphold right to equal protection. Special measures are necessary to identify bias in the data used by AI systems. This includes employing techniques for bias detection and correction. Legal and ethical guidelines should emphasize the responsibility of developers and users of AI to address bias.

3.4.4 Upholding Human Oversight and Judicial Discretion

AI offers useful insights and suggestions, but it cannot substitute for human judgment and discernment. Judges must maintain ultimate authority and discretion in rendering judicial decisions by retaining ultimate responsibility for making decisions with AI serving only as an assistive tool.

3.4.5 Public Accountability and Legal Redress

The AI systems used in courts should include input from judges, lawyers, civil society and the public to ensure legitimacy and inclusivity. Stakeholders, including defendants, attorneys and the public should have access to information fed in it. Litigants must have the right to challenge AI-assisted decisions, including the logic behind the decisions.

3.4.6 Ethical Governance and Interdisciplinary Collaboration

Addressing the complex constitutional and ethical challenges of AI in the judiciary requires harmonisation among legal experts, computer experts, ethicists and policymakers. Tackling the legal and moral issues in use of AI in courts demands teamwork. Open discussions are essential to create balanced solutions that hold technological progress while upholding the values of justice.

3.4.7 Data Privacy and Security

Strong legal and technical safeguards are essential to protect the privacy and security of data used by AI in the judicial context. This includes implementing strict data protection protocols. Effective mechanisms and privacy regulations are required to be framed to redress the case of data breaches or misuse by AI.

4 Need of Balancing Constitutional Rights and Efficiency of AI

Artificial intelligence (AI) is transforming industries and enhancing judicial decision-making. However, its rapid adoption raises critical concerns about safeguarding constitutional rights. Balancing AI’s efficiency with individual rights and liberties requires careful consideration. Without strict control over AI it may collect and analyse personal data excessively. To address this, governments and organizations must enforce transparent policies and require warrants for sensitive AI-driven surveillance in legal field. Human control over AI must remain as an essential component to prevent overreach of AI in judicial-discretion. AI used in criminal justice such as risk assessment tools for sentencing can perpetuate biases if trained on fake, false or defective data. Strict focus on equal protection principles is necessary to ensure that AI supports justice. A deep scrutiny of work process of AI in judiciary can establish standards for accountability, ensuring it to respect constitutional boundaries. Citizens and advocates too, must stay informed for their rights in an AI-driven world. Ultimately, AI’s efficiency should increase and not decrease constitutional protections. Following these norms society can harness AI’s potential while upholding the principles that safeguard individual freedoms.

4.1 Statistical Data on Use of AI in Indian Judiciary

More than 36,000 rulings were translated into Hindi and 17,000 into regional languages by the Supreme Court’s SUVAS project, significantly improving access for residents who do not know English.⁴⁴ Regarding judicial applications, the Digital Personal Data Protection Act (2023) contains conspicuous gaps. It doesn’t outline detailed protocols for gathering biometric data in courtrooms, grant third-party vendors access to private case information, or specify how long AI training data should be stored.⁴⁵

Xu reports that an algorithm based on Supreme Court data from 1791 to 2015 predicted the Court’s decisions and the Justices’ votes with an accuracy surpassing the 66 percent predictive accuracy of jurists.⁴⁶

A key role of AI in court systems is “prediction.” In judicial proceedings, AI can forecast case outcomes, a practice termed “predictive justice” in the U.S. and British legal systems. The main goal of AI implementation is to minimize risks. As cases grow more complex with additional issues and information, risk factors increase. AI helps mitigate these risks in such scenarios. For instance, several predictive tools are commercially utilized in the U.S. A team of American academics developed a machine learning tool that reportedly predicts U.S. Supreme Court (SCOTUS) case outcomes with 70.2% accuracy and the voting behavior of individual justices with 71.9% accuracy.⁴⁷

4.2 Instances of the Recent Expansion of Artificial Intelligence within India’s Judicial Framework

1. Supreme Court Vidhik Anuvaad Software (SUVAS): The Supreme Court of India introduced an AI-powered application designed to leverage machine-assisted translation technology. This tool focuses on converting English legal documents and court orders into nine regional languages, marking a pioneering step in integrating AI into India’s judicial system.

2. Supreme Court Portal for Assistance in Court’s Efficiency (SUPACE): Recently launched by the Supreme Court of India, this AI-driven tool gathers pertinent legal facts and laws, providing judges with streamlined access to critical information to enhance judicial efficiency. It will yield outcomes customized to the specific requirement of the case and the way the judge thinks.⁴⁸

3. e-Courts Initiative: A significant advancement in judicial reforms, this platform has greatly enhanced efficiency by saving time, effort, and costs, providing free online access to critical data such as judgments and court orders for users.⁴⁹

4.3 Benefits and Challenges of Artificial Intelligence in the Legal Sector

The primary benefit of artificial intelligence (AI) in the legal domain is its ability to significantly reduce time spent on tasks. AI can swiftly and accurately address complex legal issues, achieving results in moments that would take humans far longer. Unlike humans, who are prone to errors, AI systems handle intricate legal tasks with precision and process vast amounts of data in seconds. This efficiency reduces costs by minimizing the number of lawyers needed for a case. AI systems are equipped with predictive algorithms and historical case law data, enabling them to identify and assess potential risks in similar legal disputes. This helps lawyers mitigate risks in court and provide more precise advice to clients regarding legal strategies. Additionally, AI operates tirelessly without requiring breaks, unlike humans who need rest to maintain organized and systematic work. As a result, AI enhances productivity and delivers higher-quality outcomes. In a client-centric legal industry, AI’s ability to automate repetitive tasks frees up lawyers’ time, allowing them to focus on strengthening client relationships and meeting their evolving demands effectively.⁵⁰ But like every system, this one also has its own set of disadvantages.

A key drawback of artificial intelligence (AI) in the legal field is its inability to exhibit originality, creativity, or innovative thinking. Unlike humans, machines cannot fully replicate human judgment and will repetitively execute tasks unless provided with new directives. This makes it challenging for AI to adapt to the ever-changing legal landscape without human guidance. If AI were to replace judges in delivering verdicts, it would rely solely on pre-existing case data, which may not be suitable for every situation, as each case has unique nuances. The integration of AI into the legal sector remains a nascent concept, particularly in developing nations. Many traditional legal professionals in these regions are hesitant to adopt AI, often due to unfamiliarity with the technology. Additionally, in developing countries, technological infrastructure and equipment often lag behind those in developed nations, rendering them less reliable. A significant concern for lawyers is the potential for AI to cause widespread unemployment. As AI becomes more prevalent, its efficiency in handling repetitive, low-skill tasks may lead industries to invest in machines rather than human labor, potentially displacing workers such as paralegals who perform routine tasks. Many of their tasks can now be performed by machines, which poses a threat to human employment. Another significant drawback is the high cost associated with installing, maintaining, and repairing these complex machines—expenses that only a limited number of organizations can bear. Additionally, the ever-evolving technological environment necessitates frequent software upgrades, which demand substantial financial investment. The legal status of artificial intelligence remains ambiguous, as current laws do not clearly define whether AI and machines fall under the scope of common legal frameworks. This ambiguity leads to confusion in legal disputes involving the rights or responsibilities of robots and machines. Furthermore, one of the foremost concerns related to machines is data security.⁵¹ Though employing AI has many advantages it has some disadvantages too but for taking benefit of AI tools they must be used with great caution and concern.

4.4 The Evolving Role of AI in Judicial Processes

The evolving role of AI in the judiciary lies in developing systems that are transparent, explainable, morally and ethically aligned with judicial values. Rather than full automation, the emphasis will likely be on ‘human-in-the-loop’ models, where AI assists judges but final decisions rest with humans. Judicial training, public oversight and interdisciplinary collaboration between technology and law are essential. Legal frameworks must evolve to oversee the advancement and implementation of AI systems within courtrooms. Additionally, AI has the potential to enhance access to legal services through innovative solutions, such as the use of AI-based legal advisors to help citizens to understand their rights and legal procedures.

There are many challenges to address in the context of AI. One challenge suggests that AI technology is changing and it can be hard for laws to maintain balance. Another challenge is that judges and lawyers are unlikely to always possess the necessary technical knowledge to comprehend how AI systems work. Despite these challenges, it’s clear that the legal system is likely to see a growing influence of artificial intelligence. It is therefore necessary to develop effective approaches to implement judicial discretion in AI and by doing so, we can promote the responsible and ethical application of AI which will be consistent with the principles of justice.

4.5 Moral and Legal Issues in Adopting AI within the Judicial System

Over 25 documents outline ethical guidelines for the use of artificial intelligence (AI). The European Commission for the Efficiency of Justice (CEPEJ), an expert body of the Council of Europe, has addressed the role of AI in judicial systems, recognizing its growing significance. The CEPEJ established five core principles to guide the ethical application of AI in the judiciary, which are as follows: “First is the principle of respect for fundamental rights; second is principle of non-discrimination, third is the principle of quality and security; fourth is principle of transparency, impartiality, and fairness, and last is principle under user control”.⁵² AI systems can serve dual purposes in courtrooms. Firstly, as “AI assistants,” they aid judges by forecasting outcomes and drafting judicial decisions. Secondly, as “robot judges,” they could potentially take over from human judges, independently resolving cases in fully automated judicial processes.⁵³

The role of artificial intelligence in judicial frameworks promises efficiency, consistency and data-driven decision-making. Use of AI introduces significant ethical and legal challenges in judicial system in India and to cope with it, the judiciary must collaborate with bodies like NITI Aayog to develop AI frameworks specially developed for India to ensure compliance with the Constitutional principles. The challenges for implementing AI in judiciary are:

Bias and Fairness: AI systems utilize past data to forecast results and assist in decision-making, sentencing and bail determination. If this data reflects any form of bias it can increase inequalities and injustice. Algorithm like COMPAS has faced criticism for superfluously flagging minority defendants as higher-risk of being criminals. This violates principles of fairness and equality. It may risk breaking of anti-discrimination laws guaranteed under the Indian constitution. Reducing bias needs frequent checks and balances but perfect fairness is hard to achieve.

Clarity and Interpretability: The opaque nature of AI systems makes decision processes opaque and poses ethical challenges in a legal system that demands transparency. This opacity can erode public trust and violate legal standards. Ethically stakeholders must balance the efficiency of AI with the right of public to understand the rulings. Developing interpretable models and mandating disclosure of the role of AI in making decisions can be a crucial step to achieve transparency.

Accountability and Judicial Autonomy: It becomes hard to determine judicial accountability when AI influences judicial decisions. There is no clarity that if an error occurs who will be liable, developers, courts or judges? Letting machines make decisions can weaken the human discretion in judgments that are key to a fair justice system. Over-reliance on AI could infringe the constitutional principles. Courts must ensure that judges retain ultimate authority in decision-making.

Privacy and Data Security: AI systems require vast data including sensitive and personal information of persons. Mishandling of such data can breach the trust in this system. It may violate data protection and privacy laws, which makes strict consent and security protocols mandatory. Strong encryption must be used to hide personal details. Following global rules can be the key to manage these risks.

Fair Trial: The use of automated decision-making must not compromise with an individual’s right to a fair trial, legal representation or of appeal. AI can be used to support but not to replace the human discretion in serious legal matters.

4.6 Influence of AI Technologies on Essential Judicial Principles

Artificial intelligence can both enhance and challenge core judicial principles. These principles, which include open justice, impartiality, equality under the law, accountability, accessibility to justice, and efficiency, are wide-ranging and subject to varied interpretations. They often intersect and influence one another.⁵⁴

4.7 Barriers to Open Justice Posed by AI Tools

The deployment of AI tools in judicial systems introduces significant challenges to the principle of open justice, which emphasizes transparence and public trust in legal processes. These barriers are outlined as follows:

1. Opacity in AI Functionality: AI systems often conceal details about their operations due to factors such as operational confidentiality, protection of proprietary data, and concerns over personal privacy. For example, the creators of the COMPAS risk assessment and sentencing tool have not fully revealed its methodologies or underlying datasets. This lack of transparency was a central issue in the State v. Loomis case (2016), where the court highlighted that the inability to understand the tool’s processes posed a major concern. When AI systems remain non-transparent, individuals subject to their decisions are left without clear explanations for court rulings, which undermines open justice and erodes trust in algorithmic outcomes.

2. Inaccessibility to General Audiences: Another significant hurdle is the public’s and legal professionals’ limited understanding of how AI systems function. Complex source code is often incomprehensible to those without technical training, creating a gap in accessibility. In judicial contexts, providing clear reasoning for decisions is essential for effective communication between courts and parties. The COMPAS algorithm exemplifies this challenge, as its technical nature creates a disconnect between technology and legal language, making it difficult for lawyers to interpret. Without clear insight into how AI-driven decisions are made, parties are left exposed, and the delivery of justice is hindered.

3. Systemic Risks to Accountability: A third obstacle to open justice stems from the broader implications of AI integration, such as the potential for unchecked biases and lack of accountability. Without transparent and auditable AI processes, judicial decisions risk becoming difficult to scrutinize or challenge, further distancing the legal system from public oversight and understanding. Certain AI systems are intricate, and process-based explanations often fail to clarify their outputs effectively.⁵⁵

Issues related to data protection, privacy, human rights, and ethical considerations will persist and become more pressing as AI technology advances, posing new problems and necessitating strict self-regulation on the part of technology creators. The risk of covert ceding of decision-making authority exists when an AI assistant and a human judge coexist. It is crucial to talk about how AI’s recommendations could be implemented in the judiciary as soon as feasible to avoid this. Whether to permit AI to assist or replace human judges to collaborate or to institutionally separate them is the key question in this discussion. One of the main problems in this regard is making it possible for human and AI communications to function. The idea that Judge AI will always be unique from other judges may also not be helpful. As was already mentioned, many technologists predict that humans are unlikely to be completely replaced by AI.⁵⁶ Excessive reliance over AI tools in courtrooms can be a major threat to open justice. Advancement in AI system and feeding of multidimensional data in the software can help courts to achieve more accurate results and core judicial values.

4.8 Suggestions for Using Artificial Intelligence in Legal Field

AI where on one hand can be a boon for society, on other hand can be a curse if not used with care and caution, especially in legal field. Every case has different facts and issues and it is difficult for AI to provide accurate decisions and hence certain data must be fed so that it can evaluate each and every scenario in a very detailed manner before coming to any decision. Other than this to use AI efficiently a huge amount of personal data is fed into its memory so in case of any kind of cyber-attack on the system there is a huge risk of data leak of both accused and victims and it can affect society adversely, so certain strict encryption methods and security protocols must be developed to protect data. Other guidelines can be as follows:

Ensure that AI services and tools are designed and implemented with a focus on upholding fundamental rights, aligning with core principles such as privacy, equality, and the right to a fair trial. This requires integrating safeguards for these rights at every stage of AI development and deployment.⁵⁷

Promote equitable treatment by preventing discrimination against individuals and groups in algorithmic applications. Bias can originate from the data used by algorithms or be unintentionally embedded within the algorithms themselves. Addressing both data and algorithmic biases is essential to achieve fair and impartial results.⁵⁸

Transparent Data Processing for Legal Protection: To ensure accountability and compliance with legal precedents, data processing methods should be transparent, auditable, and accessible. Algorithm users must disclose their methodologies, data sources, and assumptions in a timely and comprehensive manner. This transparency enables third-party assessment of decisions, data, and reasoning, thereby providing robust legal protection and facilitating judicial review.⁵⁹

The integration of AI tools in the judicial system can compromise accountability due to their opaque nature, which can hinder the effectiveness of traditional checks and balances. The right to appeal, review, and the requirement for judges to provide reasoned decisions may be undermined when AI-driven processes lack transparency, making it challenging to ensure accountability and fairness in judicial decision-making.⁶⁰

To mitigate the risks associated with AI, systems should be designed to promote fairness and equity, protecting marginalized communities from discriminatory outcomes. The use of transparent and explainable AI tools is crucial in ensuring accountability. Addressing human rights concerns related to AI requires a multifaceted approach, including the establishment of a robust legal framework that guides public authorities in the use of AI. Governments should conduct thorough impact assessments and review existing legislative frameworks, policies, and practices governing algorithmic systems to ensure they align with human rights standards.⁶¹

5 Conclusion

The application of artificial intelligence into the Indian judicial system presents both substantial opportunities and constitutional challenges. AI-assisted technologies can contribute to the effective management of judicial workloads by facilitating case-file analysis, legal research, evidence management and identification of material information. Such applications may assist courts in addressing delays and case pendency and may subsequently contribute in facilitating access to justice. The ongoing digital transformation of the Indian judiciary illustrates the developing use of technological tools in strengthening the administration of justice.

However, the adoption of AI in judicial decision-making cannot be examined only on the basis of efficiency, accuracy, or the technological capability. In India the judicial system is constitutionally placed within such a framework that protects equality, liberty, dignity, privacy, and procedural fairness. Therefore it becomes more crucial that the use of algorithmic systems must remain consistent with the constitutional protections. Algorithmic bias and discriminatory outcomes may raise concerns regarding these protections and the opaque decision-making might create hardships in ensuring procedural fairness and the judicial accountability. Correspondingly, the collection and processing of personal and sensitive information through an AI system might contravene the constitutional privacy.

These concerns become more serious when AI is employed in the criminal justice system, which involves the exercise of extensive coercive powers of the state that include arrest, detention, search, seizure, surveillance and restriction on individual’s personal liberty. Following the lessons from COMPAS there is a need to examine the possibility of discriminatory outcomes and the challenges associated with the opaque algorithmic assessments.

Considering the Indian constitutional framework, it is important that the AI consequently remains an assistive tool rather than an autonomous substitute for judicial reasoning. The highest and final responsibility for adjudication must remain with the judges, who shall critically evaluate AI-generated outputs rather than treating them as conclusive. Human supervision is particularly important where AI-assisted recommendations might influence the outcomes of the judicial proceedings. Transparency, comprehensibility, accountability, data protection, and mechanisms for reviewing misguided or discriminatory outcomes are therefore essential to the constitutionally legitimate use of AI in the judicial decision making.

Therefore it becomes imperative that the future of AI-assisted adjudication in India should be guided accordingly by the principle that the technological advancement must operate within and not outside the constitutional structure. Further, AI can strengthen judicial efficiency, consistency and accessibility only when its deployment upholds constitutional supremacy, judicial independence and other constitutional principles. The object therefore is not the replacement of human adjudication with automated decision-making but the responsible incorporation of AI into judicial decision making in a manner that promises the constitutional promise of justice.