Author: Gautam Sharma, Dharmashastra National Law University, Jabalpur

Artificial intelligence impacts many sectors and there is a need to discuss its consequences in data privacy and start a broader discussion around data privacy and AI technology. This paper will try to analyze its implications in the privacy data and start a discussion around the field. It is true the progress in AI technology is changing the way businesses operate, improving productivity, efficiency and services. The advanced technology raises concerns of privacy due to the vast amount of sensitive data that AI systems depend upon. The debate regarding the technology that violates individual rights is what drives the concern associated with AI. The purpose of this research is to analyze the ethical and legal concerns with AI data practices erosion of consent, opaque algorithms, data minimization and the risks of surveillance and biased decision making. The research examined to what extent legal prose, particularly provisions of the EU’s General Data Protection Regulation (GDPR) and India’s Digital Personal Data Protection Act, 2023 are designed to expose the balance struck in protecting the privacy of individuals against technology. The paper seeks to address the demand for ethical and flexible legal frameworks to protect the privacy rights of individuals in the face of AI technology and ensure the advancement of technology is done responsibly.

Keywords

Artificial Intelligence, Data Privacy, Data Protection, Ethical and Legal Challenges, Digital Personal Data Protection Act, 2023, Algorithmic Accountability, Automated Decision Making 

As correctly articulated by Facebook's head AI researcher Yann LeCun "Our intelligence is what makes us human, and AI is an extension of that quality." In the era of digitalization, Artificial Intelligence has taken over the market as AI has emerged as one of the most transformative technological developments of the twenty-first century, reshaping the ways in which information is created, processed, and utilized. The deployment of AI systems in areas such as finance, healthcare, law enforcement and social media has resulted in unprecedented access to and analysis of personal data. For artificial intelligence to work well it needs huge datasets that often contain private or sensitive data and as we know that how much this data is important for one individual or a person in case of its breach it lead to the leak of sensitive information at the world level which will directly or indirectly harm the right to privacy of such person or an individual. This blog engagement within the context of data privacy and AI in India addresses relevant issues such as the law's protection, ethics inequity, transparency and the ideals of data processing companies. The goal of this research is to evaluate the ethical and legal implications of the processing of personal data to determine whether the impact of AI and the processing of personal data are within the parameters of reasonable legal provisions and oversight and to identify ways in which technological innovation can be effectively integrated with privacy safeguards.

There is outstanding discussion regarding the intersectionality of Technology, Ethics and Regulation on the subjects of AI and Data Privacy. Solove and Nissenbaum’s earlier publications on Infonomics and Privacy, respectively, have propelled other scholars toward the problem of Digital Privacy and Artificial Intelligence. Zuboff’s Surveillance Capitalism has been crucial in providing these scholars with the essential components and structure of AI’s Systems that Profile. The imbalance in the autonomy of these AI systems has convinced scholars that policy and legislation on privacy need to be re-thought. The discussion on the impact of policy advocacy has been on the limitations, especially of the GDPR. Although many scholars have pointed to the GDPR as the gold standard, the literature indicates that it is extremely slow in regulating AI technologies.

In India, literature discussing the IT Act 2000 and the Personal Data Protection Bill 2019 illustrates a lack of coverage of the risks of algorithms, the duties of transparency, and the mechanisms for redress. Institutional reports such as those by OECD, NITI Aayog and the European Commission all call for the need of regulatory approaches that integrate the level of protection of fundamental rights and the protection of technological innovation. The existing literature thus reveals extensive engagement with the conceptual, legal and ethical dimensions of AI and data privacy, but also underscores several research gaps. These include the need for clearer accountability mechanisms for automated decisions, stronger oversight institutions and more robust safeguards against algorithmic discrimination. The present study addresses these gaps by critically evaluating the legal and ethical challenges posed by AI particularly within the developing Indian data protection landscape.

1. To examine the connections between Artificial Intelligence and data privacy regarding issues such as consent, transparency, algorithmic bias, automated decision making, etc.

2. To evaluate the adequacy and efficacy of the domestic and international legal frameworks regarding the privacy of data and the use of Artificial Intelligence. 

3. To evaluate the international legal instruments and Artificial Intelligence and data protection particularly the General Data Protection Regulation (GDPR) and the domestic legal developments regarding the Digital Personal Data Protection Act, 2023.

4. To evaluate the ethical challenges around the use of Artificial Intelligence and the Data Practices especially concerning Surveillance, Discrimination and Accountability. 

5. To assess the regulatory paradoxes and offer a set of reasonable recommendations on the governance of AI and privacy protection.

This study adopts a doctrinal and analytical methodology. Doctrinal research is employed to examine statutory provisions, judicial precedents, regulatory reports and academic commentary relevant to AI and data privacy. The analytical approach facilitates an evaluation of the effectiveness, strengths and limitations of both domestic and international legal frameworks. Primary sources include the Digital Personal Data Protection Act, 2023 the General Data Protection Regulation, relevant case law and policy recommendations issued by bodies such as the OECD and NITI Aayog. Secondary sources comprise scholarly articles, books, institutional papers and expert analyses. The study does not engage in empirical or statistical inquiry but relies on qualitative assessment to draw conclusions based on normative and legal reasoning. This methodological framework ensures transparency, intellectual rigor and a comprehensive understanding of the ethical and legal issues surrounding AI-driven data governance.

1. AI, Datafication, and the Changing Meaning of Privacy: - K.S. Puttaswamy v. Union of India has widespread implications, and so does the use of AI and data thanks to the way the Constitution informs the role of the Supreme Court. It is the domain of the Supreme Court to pivot the discussion to the constitutional implications of the protection of privacy, data protection and both self-autonomy and self-determination from data breach. AI might not have been the central issue of the case but the courts pivot to the issue of privacy sparked the courts to allow the building of poorly conceived AI systems which extrapolate, mine and granule the data of individuals. New systems of constitutional AI should come as standard with systems to protect civil and data rights as the technologies breach and therefore the systems should breach.

2. Consent Dilution and Transparency Deficits: - Informed and freely given consent forms the bedrock of privacy law. However, consent frameworks are often useless as a result of AI practices. Users are often unaware of data flows and the downstream processing and repurposing of their personal information. AI also exacerbates this problem as machine learning systems identifying patterns from the data opaque to the user that were from the data collection resulting in “function creep” and loss of consent. In addition, algorithmic decision-making is often obscured, making it that much harder for data subjects to understand how those decisions are made. The inadequacy of such systems to provide explanations also affects the ability to exercise rights like access, correction, or objection, and fundamentally introduces issues of fairness, accountability and due process.

3. Algorithmic Bias and Discriminatory Outcomes: - You have collected systems to track bias and the law starting to take shape more as principles systems Subramanian Swamy v. Union of India (2016) case. The Supreme Court of India said with every limitation of the persons rights there must be the satisfying of the proportionality test which is crucial when assessing the discrimination of AI. The Court of Appeal in the United Kingdom is to be considered the first case on algorithmic bias with respect to Bridges v. South Wales Police (2020) which ruled there is the use of live facial recognition. There is a breach of the right to privacy and right to equality deficit of which there are the protections to address the discrimination aimed at the underprivileged. These outcomes demonstrate the court's recognition that automated systems have the ability to achieve inequitable and unjust consequences and therefore strong accountability is required.

4. Legal Framework: Gaps in Indian Law Related to AI in Data Privacy: - There is a significant gap in existing Indian legislation. Like in Digital Personal Data Protection (DPDP) Act, 2023 it emphasizes consent, purpose limitation and data minimization, these principles may not be sufficient to address complex AI-driven privacy challenges. The other one is the Information Technology (IT) Act, 2000 it provides a framework for cybersecurity and data protection. However it lacks specific provisions for AI limiting its efficacy in addressing privacy issues arising from AI applications.

5. Global Regulatory Approaches and Comparative Insights: - Employing innovations from abroad in regulation to devise governance for AI systems. First comes the GDPR which has provided the first comprehensive legal foundation for data protection and data portability, protection from automated decision making and legal transparency and accountability. The proposed EU Artificial Intelligence Act builds on that with a risk based framework and imposes more obligations on AI that is considered to be high risk. The OECD Guidelines as well as the AI Ethics Recommendations of UNESCO speak to the protection and promotion of Human Rights with accountability and oversight as well as the imbalanced mitigation and control of protection and promotion of Human Rights. Global concerns relating to technological advancements and privacy like in Carpenter v. United States are being addressed. Here the Supreme Court provided the first cell phone location data privacy protection and stated that such data may only be accessed through proper legal procedures. In Schrems I (2015) and Schrems II (2020) The CJEU repeatedly cautioned about the need to exercise care and fully contemplate the need for effective safeguards when dealing with the cross border flows of personal data, particularly because of the borderless character of data sets training artificial intelligence. This set of decisions concerns the opacity of the algorithms used in government surveillance and the fragility of personal data in the digital ecosystem. 

1. AI-informed data captures the meaning of privacy. In the scope of surveillance inferential analysis has become more intrusive. This has eroded protections based on prior consent. In cases such as K.S. Puttaswamy v. Union of India the Persistence of the Informational Autonomy Constitution has been documented. However the State of the State Regulatory Framework remains on these Principles of Safeguarding Algorithmic Ecosystems.

2. The research shows that the foundations of data protection pertaining to disclosure and consent are ineffective when it comes to understandings of AI structural complexities of data flows and secondary uses, and the discretionary flows during the AI processes. Regarding law and protest, such as Schrems II it advocates for the imposition of accountability that extends beyond mere obligatory consent. Third, the concern of algorithmic discrimination is a prominent concern considering judgements like Bridges v. South Wales Police and the fact that without due concern, automated processes could discrimination for example South Wales Police. South Wales automated discrimination concerns affect the legal principles of proportionality from the supreme court of India which describe the legal standards to be applied to such discrimination.

3. The absence of regulation is still a problem that is damaging the data governance model of the country. The Digital Personal Data Protection Act, 2023 has been passed but still does not account for some issues pertaining to automated decision-making, profiling and accountability of algorithms.

4. The study argues that ethical issues of fairness, transparency and human oversight cannot be resolved through compliance alone. The courts in India and elsewhere have come to the conclusion that technological governance cannot be exercised without the protection of fundamental rights. The findings therefore indicate that India should be working towards dynamic models of protection that are rights-respecting and can deal with the challenges of regulating the data-driven ecosystem that is supported by Artificial Intelligence.

Introduction of the AI technologies have started dealing with data in ways that challenge the very foundations of traditional data privacy and trigger new questions of consent, surveillance, discrimination and informational autonomy. The Puttaswamy Judgments expand constitutionally the frontiers of digital privacy in India. Yet we still have much to achieve in terms of legislating the complexities of artificial intelligence. The World has started maintaining Schrems II, Carpenter and Bridges predictive of the need for transparency and accountability in automated systems.

The enactment of the DPDP Act, 2023 is a step in the right direction for India but still falls short of providing adequate protections, an automated decision-making system and algorithms that profile and discriminate in the absence of safeguards. We cannot stop at satisfying the procedural aspects of the law when we claim to govern artificial intelligence. Building AI systems that do not infringe fundamental rights is important to ensure that dignity, autonomy and equality are preserved in the digital age.

1. India should provide specific legal guidance on the automation of decision making and algorithmic profiling. The DPDP Act, 2023 must include rights such as the right to explanation protections against fully automated decision making and the requirement of impact assessments for high-risk AI systems which would begin to align Indian law with the global norm of the GDPR and the EU AI Act.

2. There should be a specialized adequately empowered regulatory body. Either the Data Protection Board should be expanded, or an Artificial Intelligence Oversight Commission should be established to facilitate enforcement and transparency and to provide for the thorough inspection of AI systems of public and private actors.

3. There should be strict boundaries on how the State can use AI-facilitated surveillance. Based on Puttaswamy there should be legal protection that demonstrates strict necessity, prior adjudication and independent oversight regarding the State's use of facial recognition and analogous technology.

4. To ensure that ethical principles such as responsibility, no discrimination, inclusion and human oversight can be effectively accomplished they should be incorporated into law as mandatory requirements.

5. In order to ensure that AI will be trained on robust datasets that meet constitutional and rights protecting criteria India must enhance cross-border data flow management by partnering with other nations and adopting AI and privacy regulations.

Cases: -

1. K.S. Puttaswamy v. Union of India, (2017) 10 S.C.C. 1 (India).

2. Subramanian Swamy v. Union of India, (2016) 7 S.C.C. 221 (India).

3. Bridges v. South Wales Police, [2020] EWCA Civ 1058 (U.K.).

4. Carpenter v. United States, 138 S. Ct. 2206 (2018) (U.S.).

5. Schrems v. Data Protection Commissioner (Schrems I), Case C-362/14, EU:C:2015:650.

6. Data Protection Commissioner v. Facebook Ireland Ltd. (Schrems II), Case C-311/18, EU:C:2020:559.

Statutes / Regulations / Bills: -

7. Digital Personal Data Protection Act, 2023, No. 22, Acts of Parliament, 2023 (India).

8. Information Technology Act, 2000, No. 21, Acts of Parliament, 2000 (India).

9. General Data Protection Regulation, Regulation (EU) 2016/679.

10. Personal Data Protection Bill, 2019 (India) (lapsed).

International Instruments & Institutional Documents:-

11. Proposal for the Artificial Intelligence Act, COM (2021) 206 final (European Commission).

12. Organisation for Economic Co-operation and Development (OECD), Recommendation of the Council on Artificial Intelligence, OECD/LEGAL/0449 (2019).

13. UNESCO, Recommendation on the Ethics of Artificial Intelligence (2021).

14. NITI Aayog, National Strategy for Artificial Intelligence (2018).

Books & Scholarly Works:-

15. Daniel J. Solove, A Taxonomy of Privacy, 154 U. Pa. L. Rev. 477 (2006).

16. Helen Nissenbaum, Privacy in Context: Technology, Policy, and the Integrity of Social Life (2010).

17. Shoshana Zuboff, The Age of Surveillance Capitalism (2019).

18. Justice B.N. Srikrishna Committee, Report of the Committee of Experts on Data Protection (2018).