KG LEGAL \ INFO
BLOG

A Human in the Loop, Data under Control. The GDPR and the AI Act in Implementation Practice

Publication date: October 05, 2026

Implementing artificial intelligence in an organisation does not begin with choosing a tool, but with answering the question of who we are in the AI value chain and what we are responsible for. This article sets out the relationship between the GDPR and the AI Act: from the roles of provider, deployer and controller, through algorithmic risks, the limits of lawfulness in the workplace and the rights of individuals, to the DPIA, the FRIA and the AI Governance model. It reflects the legal position following the entry into force of the Polish Act on Artificial Intelligence Systems and the postponement of the AI Act application dates by Regulation (EU) 2026/1744.

Introduction

Rapid technological progress and the widespread deployment of systems based on artificial intelligence are permanently reshaping the landscape of modern economic and social life. The use of machine learning algorithms and large language models in professional processes opens up unprecedented opportunities for optimisation, allowing the automation of tasks that previously required prolonged human effort. This technological efficiency, however, brings significant ethical and legal challenges and calls into question existing standards of privacy protection. In an era of such profound digital transformation, the key task is to develop a regulatory framework that balances the drive for innovation against the imperative of protecting fundamental rights. It must be remembered that AI systems, despite their apparent infallibility, operate on data that may be erroneous, biased or out of date. As a result, algorithm-based decision-making processes directly affect the legal and personal situation of natural persons, which creates a need to define precisely who is responsible for the outcome of the technology.

Regardless of how advanced AI tools become, the paradigm of human oversight must remain the fundamental principle of safe implementation. A human being cannot be reduced to a mere recipient of output data; they must perform an active supervisory function and act as a guarantor of reliability and safety when the algorithm fails. The AI Act[1] establishes rigorous procedures and imposes specific restrictions on providers, deployers and other links in the artificial intelligence value chain. In relation to the GDPR[2], the AI Act is complementary: it does not repeal existing data protection principles but refines them in a technological context. The AI Act adds an additional layer of product-related and ethical obligations, ensuring that algorithms are used with respect for the rights and freedoms of the individual. Much of this framework already applies: the prohibitions in Article 5 AI Act have applied since 2 February 2025 and the transparency obligations in Article 50 AI Act since 2 August 2026, while Regulation (EU) 2026/1744 (the so-called Digital Omnibus on AI) has postponed the application of the obligations for stand-alone high-risk systems listed in Annex III to 2 December 2027. In Poland, the national framework is set out in the Act of 3 July 2026 on Artificial Intelligence Systems (Journal of Laws, item 1003), which established the Commission for the Development and Safety of Artificial Intelligence (KRiBSI) as the market surveillance authority.

1st         Who Is Who in the AI Ecosystem: Provider, Deployer, Controller

a)    Provider or Deployer? The Answer Determines the Scope of Liability

Defining an organisation’s role in the artificial intelligence ecosystem is of key importance for determining its legal obligations precisely. It must be established whether a given entity acts as the provider of an AI system or is merely a deployer of a ready-made technological solution. Under Article 3(3) AI Act, a provider is a natural or legal person, public authority, agency or other body that develops an AI system (or a general-purpose AI model) or has it developed, and places it on the market or puts it into service under its own name or trademark, whether for payment or free of charge. The provider’s principal task is therefore to create the technology and ensure its compliance with rigorous safety and data quality requirements before the commercialisation stage. Most organisations using AI systems will not act as providers but as deployers. This distinction is fundamental to risk analysis: while the provider is responsible for the architecture and the training process (compliance of the “product”), the deployer is responsible for how the system is operated, for instructions to employees and for the lawfulness of the data entered into the model during the inference phase.

Under Article 3(4) AI Act, a deployer may be a natural or legal person, public authority, agency or other body that uses an AI system under its authority. These features clearly distinguish the provider of an AI system from an entity that merely uses the supplied system and exercises authority over it. In addition, an entity using artificial intelligence systems may acquire the status of a controller of personal data once it meets the definitional criteria set out in Article 4(7) GDPR. A controller is therefore a natural or legal person, public authority or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data. It should be stressed that an AI system itself has no legal personality, and responsibility for its actions rests with the entity that decides on its deployment and operating parameters. Data processing in this context takes place through operations or sets of operations performed on personal data by automated means, which falls within the broad definition in Article 4(2) GDPR. These processes include, in particular, the collection, recording, adaptation, consultation and use of data by algorithms in order to generate results (output). In the case of AI systems, this processing is highly automated and is usually aimed at optimising and increasing the operational efficiency of information processes, which places a particular duty on the controller to keep control over every stage of the data life cycle.

b)    When an AI System Becomes a High-Risk System

Under the system adopted in the AI Act, the correct classification of artificial intelligence systems is of key importance for organisations, since solutions classified as high-risk systems are subject to the most stringent legal regime. An AI system acquires that status if it meets the conditions set out in Article 6(1) AI Act (this concerns systems that are products, or safety components of products, subject to EU certification) or if it is expressly listed in Annex III to the Regulation. That catalogue covers areas of critical importance for fundamental rights, such as biometrics and the categorisation of persons, the management of critical infrastructure, and education and vocational training. Particular attention should be paid to employment and workers management, access to essential public and commercial services, and systems used in law enforcement, migration management and the administration of justice. Classifying a technology in this category obliges the controller not only to ensure high-quality data and human oversight in accordance with the AI Act, but also to comply strictly with the GDPR, which in most cases will include carrying out a full data protection impact assessment (DPIA).

c)     Transparency, Instructions for Use and Human Oversight: Obligations in Practice

Providers must ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the persons concerned are informed that they are interacting with an AI system (Article 50(1) AI Act). High-risk AI systems must also be designed and developed in accordance with the principle of transparency of their operation (Article 13 AI Act), which enables deployers to interpret the system’s output and use it appropriately. It is also essential to attach instructions for use in an appropriate format, containing complete and comprehensible information for deployers. While AI systems are in use, it must be possible for natural persons to oversee the system (Article 14 AI Act) in order to prevent risks to health, safety or fundamental rights.

Deployers of high-risk AI systems must implement appropriate technical and organisational measures to ensure that they use such systems in accordance with the instructions for use supplied by the provider (Article 26(1) AI Act). Under Article 26(11) AI Act, deployers of the high-risk AI systems referred to in Annex III that make decisions, or assist in making decisions, related to natural persons must fulfil their information obligation before actually using the high-risk AI system, by clearly informing those persons that such technology is being used in relation to them.

In order to give full effect to the requirements of the AI Act, deployers are also obliged to assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary organisational support. Under Article 14(2) of the Regulation, “human oversight shall aim to prevent or minimise the risks to health, safety or fundamental rights (…)”. Article 14(4), in turn, requires providers to deliver a high-risk AI system to the deployer in such a way as to enable the natural persons to whom human oversight is assigned, among other things, “to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias), in particular for high-risk AI systems used to provide information or recommendations for decisions to be taken by natural persons”.

d)    Training and Inference: Where the Data Risk Hides in Language Models

The foundation of large language models (LLMs) is the processing of vast quantities of data, enabling the generation of text and the performance of advanced natural-language tasks. As tools belonging to generative artificial intelligence, these models operate on the basis of the statistical prediction of successive tokens (words or parts of words) in response to instructions entered by the user, known as prompts. In market and corporate practice, the most common are the GPT series models (OpenAI), implemented in solutions such as ChatGPT and Microsoft Copilot, as well as the Llama family of models (Meta), DeepSeek and the Polish projects Bielik and PLLuM. From a data protection perspective, the key distinction is between the two fundamental phases of a model’s operation: training and inference.

Training phase

In this phase, the substantive “intelligence” of the model is created. The process is based on the analysis of enormous data sets (Big Data) in order to detect statistical correlations between tokens. It should be emphasised that an LLM does not interpret content cognitively (in the human sense) but calculates the probability of the next element in a sequence. The data is often obtained through the mass collection of content from the open internet (web scraping). From the GDPR perspective, this is the most controversial stage, because the controller of personal data (usually the provider of the system) must demonstrate a specific legal basis and purpose for the processing (Article 6 GDPR). At this stage the provider becomes responsible for the quality of the data sets. Under Article 10 AI Act, training data sets must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the intended purpose of the system.

Inference phase

The inference phase is the operational stage in which the trained model generates answers to specific user queries (prompts), using the statistical weights developed during training. In this phase the model does not, as a rule, “learn” in real time but processes the information supplied to it at a given moment. From the perspective of an organisation deploying an AI system, this is the area of the highest legal risk. The key problem is employees entering customer data, personal information or trade secrets into queries sent to an external model. Such conduct constitutes a transfer of data to the provider’s servers, which often involves a transfer outside the European Economic Area (EEA). In this scenario the organisation acquires the status of a deployer within the meaning of Article 3(4) AI Act. Under Article 4(7) GDPR, the organisation, as the controller of its employees’ and customers’ data, bears full responsibility for the content of the information that reaches the AI system. So while the training phase remains the domain of large technology corporations (providers), the inference phase is the area in which every business bears direct responsibility for the security and lawfulness of the prompts it generates.

2nd    Bias, Hidden Scoring and Explainability: Algorithmic Risks under Article 5 GDPR and Article 10 AI Act

The practical implementation of artificial intelligence systems in socially sensitive areas, such as the recruitment of employees or the assessment of creditworthiness, requires the systematic management of the risk of algorithmic error. The key challenge is to ensure the explainability of AI and to combat effectively the phenomenon of bias, which can lead to the discrimination of entire social groups. The foundation for counteracting these risks under data protection law is the set of principles expressed in Article 5(1) GDPR. Under the principle of transparency (point (a)), the organisation must be able to explain to the data subject the logic behind an autonomous decision of the algorithm. The principle of data minimisation (point (c)), in turn, requires the information collected to be limited to what is necessary for the purpose, which prevents models from being fed with redundant data or data that has no objective bearing on the result of the analysis. In this context, the prohibition of so-called hidden scoring takes on particular significance; this means an AI system covertly assessing, for example, job candidates by assigning them points without ensuring adequate transparency of the process. The controller, as the employer, is obliged under Article 13(2)(f) GDPR to provide the data subject with information about the existence of automated decision-making, including profiling (referred to in Article 22(1) and (4)), and meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject.

AI systems frequently rely on profiling mechanisms, that is, the automated processing of personal data consisting of the use of that data to evaluate certain aspects of a natural person, in particular their competence, expected work performance or other characteristics relevant to the purpose for which the AI tool is used. Under Article 22(1) GDPR, “the data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her”, unless the decision is necessary for entering into or performing a contract, is authorised by Union or Member State law, or is based on the data subject’s explicit consent (Article 22(2) GDPR). Profiling as such is not prohibited in principle, but its use is subject to significant restrictions, particularly where it leads to decisions with significant effects on the person being profiled. In practice, this means that special care must be taken in designing and using AI tools, particularly with regard to the risk of bias. These systems learn from historical data, which may lead to the replication of earlier decision patterns and to unintended discrimination, for example on grounds of age, state of health or other personal characteristics. It is therefore essential to ensure transparency of the systems’ operation, adequate human oversight and the ability to verify and challenge the results generated by AI.

Relying solely on the rules contained in the GDPR is not enough, however, when confronted with deep neural networks, which is why a key role is played by Article 10 AI Act, which governs data and data governance in high-risk AI systems. That provision imposes rigorous obligations on providers to examine and verify training, validation and testing data. These data sets must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the specific geographical, behavioural or functional setting in which the system is to operate. This requirement directly compels organisations to apply procedures for detecting and neutralising hidden bias (bias mitigation). For example, a recruitment algorithm trained on the historical data of a company in which mainly men were promoted may classify the female gender as a negative feature; Article 10 AI Act requires such anomalies to be actively monitored and corrected at the design stage and through continuous oversight of the system’s operation.

3rd      Can an Employer Require the Use of AI? The Limits of Lawfulness in the Workplace

Assessing the lawfulness of deploying artificial intelligence systems in the workplace requires a precise delineation of the boundary between the employer’s managerial prerogatives and the fundamental rights of employees. Whether an employee can be required to use AI tools depends on the nature of the tool and the extent of its intrusion into the employee’s privacy. While issuing a work instruction to use AI as simple support in daily duties falls within the law, the deployment of monitoring, evaluation or biometric systems encounters strict legal barriers. The legal basis for deploying standard algorithms that optimise work processes is most often Article 6(1)(f) GDPR, that is, the legitimate interest of the controller (the employer). Its application is conditional, however, on passing the so-called balancing test, which must show that the employer’s economic or organisational interests do not override the autonomy, rights and freedoms of the persons employed. In the case of high-risk AI tools, the balancing test will generally be harder to pass, and legitimate interest alone may prove insufficient, particularly where the processing leads to decisions of the kind referred to in Article 22 GDPR.

The situation becomes radically more complicated where an AI system requires the processing of biometric data in order to operate (for example, facial recognition systems used to record working time or to analyse an employee’s emotions). Biometric data belong to the special categories of data whose processing is, as a rule, prohibited under Article 9(1) GDPR. For an employer to use such data lawfully, one of the exceptional conditions in Article 9(2) GDPR must be met; in the reality of Polish labour law this most often requires the employee’s explicit and fully voluntary consent (subject to the requirements of Article 22¹ᵇ of the Labour Code) or a specific legal provision imposing such an obligation on safety grounds. The employee’s lack of genuine freedom of choice in the relationship with the employer makes such consent extremely difficult to defend before the supervisory authorities.

The ultimate barrier to the implementation of high-risk algorithmic AI systems is Article 5 AI Act, which introduces a categorical list of practices prohibited in the European Union. On that basis it is unlawful to deploy AI systems used for so-called social scoring (the point-based classification of citizens or employees on the basis of their social behaviour), as well as systems that carry out biometric categorisation in order to infer sensitive characteristics (for example political opinions or sexual orientation). Moreover, in the context of employment relationships, the AI Act prohibits the use of emotion recognition systems in the workplace (Article 5(1)(f) AI Act), except for systems put in place for medical or safety reasons, which in practice closes the door to employers algorithmically testing the mood or stress levels of their staff. These prohibitions have applied since 2 February 2025, and Regulation (EU) 2026/1744 extends their catalogue from 2 December 2026.

4th       Individuals versus the Algorithm: Erasure, Human Intervention, Transparency

The informational and technological asymmetry between the individual and the entities deploying artificial intelligence systems requires effective instruments of legal protection. The greatest controversy in this area currently surrounds the exercise of rights of an erasure nature. Enforcing the right to be forgotten under Article 17 GDPR encounters a fundamental technical barrier in the form of the so-called “memory” of generative models and neural networks. In traditional databases, deleting information means erasing a specific record. In the case of AI models, input data becomes irreversibly integrated into the structure of the model’s mathematical weights during the training process. Reversing that state, described in the literature as machine unlearning, is a complex and costly process and often simply impossible without retraining the model in its entirety. This creates a deep conflict between the individual’s right to demand the erasure of their data and the architecture of modern technologies.

Another key safeguard is Article 22 GDPR, which lays down a general prohibition on subjecting natural persons to decisions based solely on automated processing, including profiling, which produce legal effects concerning them or similarly significantly affect them. This provision gives the individual the right to obtain human intervention (human-in-the-loop), to express their own point of view and to contest a decision taken by an algorithm (for example, an automated refusal of credit or rejection of a job application). Supervisory authorities’ guidelines and the case law of the CJEU (including the judgment of 7 December 2023 in Case C-634/21 SCHUFA Holding) indicate that human involvement in the decision-making process may not be fictitious or token; a manager or analyst must have a real ability to change the verdict generated by the AI. These guarantees are complemented, at the level of direct interaction with the technology, by the information obligations arising from Article 50 AI Act, applicable since 2 August 2026. That provision imposes strict operational transparency requirements on providers and deployers of artificial intelligence systems. Under it, every natural person interacting with an AI system, such as a chatbot or voice assistant, must be clearly and promptly informed of that fact, unless this is obvious from the context of use. Equally strict obligations apply to the labelling of synthetically generated content, including so-called deepfakes. Deployers of AI systems that generate or manipulate images, audio or video content in such a way that it closely resembles authentic persons or events are legally required to disclose that the content has been artificially generated. The purpose of this regulation is to protect the cognitive autonomy of the individual and to counter mass disinformation.

5th       Four Risk Tiers and a New Supervisory Map: KRiBSI alongside the President of the Personal Data Protection Office

A key element of the EU strategy for regulating artificial intelligence is the risk-based approach, which directly determines the scope of the legal obligations imposed on organisations. The Artificial Intelligence Act introduces a rigid division of systems into four categories: prohibited practices (posing an unacceptable risk to fundamental rights), high-risk systems, limited-risk systems (mainly subject to information obligations) and minimal-risk systems. From the operational perspective of businesses and public institutions, the most important category is that of high-risk systems, whose definition and classification mechanism are governed by Article 6 AI Act. That provision refers directly to Annex III to the AI Act, which contains a closed catalogue of critical areas, including workers management and access to employment (HR), education and vocational training, as well as law enforcement and the management of critical infrastructure. Implementing AI tools in these sectors triggers a strict legal regime, including the need to implement a risk management system, ensure high-quality data and provide human oversight. Under Regulation (EU) 2026/1744, these obligations will apply to stand-alone systems listed in Annex III from 2 December 2027 and to systems embedded in products covered by Annex I from 2 August 2028; the postponement does not, however, change the scope of the requirements but merely gives organisations time to implement them.

From the perspective of mapping risk within an organisation, this classification has profound consequences under data protection law. The Act of 3 July 2026 on Artificial Intelligence Systems entrusted market surveillance to the Commission for the Development and Safety of Artificial Intelligence (KRiBSI), which from 28 October 2026 acquires full powers of inspection and the power to impose the fines provided for in the AI Act. This does not, however, exclude the President of the Personal Data Protection Office (PUODO). Under Article 74(8) AI Act, in respect of the high-risk AI systems listed in Annex III, point 1 (biometrics, in so far as used for law enforcement, migration and the administration of justice), point 6 (law enforcement), point 7 (migration and asylum) and point 8 (administration of justice and democratic processes), market surveillance is linked to the data protection authorities, and the PUODO consistently emphasised, already in its comments on the draft act, that any processing of personal data in AI systems remains within its competence under the GDPR. This means that a compliance audit cannot be limited to a standard data protection impact assessment (DPIA). The organisation must take account of a dual supervision model in which KRiBSI verifies the system’s compliance with the AI Act, while the data protection authority retains full investigative and corrective powers in respect of the processing of personal data, including the data used to build the model. Ignoring this dual competence at the system design stage (privacy and compliance by design) exposes controllers to a serious risk of financial penalties and administrative sanctions in the form of an order to shut down the algorithm immediately.

6th       DPIA and FRIA: from a Document in a Drawer to a Dialogue with the Regulator

Because of their systemic and often unpredictable impact on privacy, the deployment of advanced artificial intelligence systems in most cases automatically triggers an obligation to carry out a data protection impact assessment. Under Article 35 GDPR, that assessment (Data Protection Impact Assessment, DPIA) is mandatory whenever processing using new technologies is likely to result in a high risk to the rights or freedoms of natural persons, which, in the context of recruitment or credit-assessment algorithms (mentioned above in relation to the principles of minimisation and transparency in Article 5 GDPR), is an operational standard. The contemporary AI legal regime, however, dramatically extends that obligation. Under Article 27 AI Act, public bodies and private entities providing public services, as well as deployers of the high-risk AI systems referred to in Annex III, point 5(b) and (c), are required to carry out a rigorous fundamental rights impact assessment (FRIA). The entities referred to in Annex III, point 5(b) and (c) AI Act are those using high-risk AI systems to evaluate the creditworthiness of natural persons or establish their credit score, and systems used for risk assessment and pricing in relation to natural persons in the case of life and health insurance. This mechanism cannot operate in isolation from the GDPR; in practice, the FRIA becomes an integral, deeper extension of the classic DPIA, requiring the organisation to examine the impact of the algorithm not only on privacy but also on human dignity, non-discrimination and the right to a fair trial, that is, the impact on the fundamental rights of the persons in relation to whom the AI systems are used.

The traditional approach, in which the DPIA was merely an internal, static corporate document filed away in a drawer, is becoming a thing of the past when confronted with Article 57(10) AI Act, which governs so-called regulatory sandboxes. These sandboxes, which are controlled testing environments for innovative AI solutions, redefine the relationship between the controller and the supervisory authority. An organisation that decides to participate in a regulatory sandbox is legally obliged to cooperate closely and transparently with the authority operating the sandbox (in Poland, KRiBSI) and, in so far as the system under test processes personal data, also with the President of the Personal Data Protection Office (Article 57(10) AI Act). In this new model of cooperation, the DPIA process evolves into a tool for a dynamic, multilateral dialogue with the state authority. The results of the risk analysis are consulted on an ongoing basis, which makes it possible to eliminate algorithmic bias under the regulator’s eye before the system is commercialised. Such a compliance model not only minimises the risk of severe financial penalties but also makes it possible to build AI systems that are safe and transparent from the stage of their technological incubation.

7th       AI Governance: How to Implement AI in an Organisation and Be Able to Prove It

Effective management of artificial intelligence systems within an organisation requires a multi-level oversight model (AI Governance[3]) that combines technical protective measures with a systematic assessment of legal risk. The foundation for designing safe algorithms is formed by the principles of data protection by design and by default, codified in Article 25 GDPR. They oblige the organisation to take privacy protection and data minimisation into account at the earliest stage of creating the architecture of the AI model, and also when selecting training data and model parameters. Article 32 GDPR, in turn, which lays down the principle of security of processing, takes on an entirely new technological dimension in the world of autonomous systems. Classic IT safeguards (such as encryption or access control) are no longer sufficient against the specific vulnerabilities of neural networks. Ensuring the integrity and confidentiality of data requires controllers to actively harden models against a new generation of cyberattacks, including adversarial attacks[4], attempts to poison training data sets (data poisoning) and model inversion techniques aimed at extracting the personal data used to build the model.

The formal tool for verifying these safeguards and estimating the risks to natural persons remains the data protection impact assessment (DPIA) carried out under Article 35 GDPR. Because of the informational asymmetry and intrusiveness of AI models, however, this assessment ceases to be merely an internal, static document and becomes a platform for continuous monitoring. For artificial intelligence systems classified as high-risk solutions, the traditional DPIA must be integrated with the new, strict obligation under Article 27 AI Act: the fundamental rights impact assessment (FRIA). Only the synergy of the technical resilience of the model with a transparently conducted DPIA/FRIA procedure enables an organisation to demonstrate compliance and lawfully keep an AI system in commercial use.

Summary and Conclusions

The analysis shows that the implementation of artificial intelligence systems, including advanced large language models (LLMs), redefines existing standards of privacy protection and requires organisations to build entirely new governance structures. The key to ensuring the lawfulness of data processing is the precise identification of the entity’s role in the AI ecosystem. The distinction between the provider, responsible for the training phase and the quality of the product data, and the deployer, which as controller bears full responsibility for the inference phase and the content of the prompts entered, is the starting point for effectively mapping regulatory risk under the GDPR and the Artificial Intelligence Act.

The main challenge for modern organisations is the management of specific algorithmic risks, such as the lack of explainability of machine decisions and hidden bias, which in sensitive areas such as recruitment or credit assessment may lead to the replication of discriminatory social patterns. Traditional data protection principles, in particular the principles of transparency and minimisation, gain strong support from Article 10 AI Act. That provision revolutionises compliance processes by imposing strict requirements for the verification of training data, with the aim of actively neutralising bias at the system design stage. Furthermore, the right to be forgotten (Article 17 GDPR) encounters a technological barrier in the form of the mathematical memory of neural networks, which means that the exercise of individuals’ rights against the algorithm requires advanced procedures such as machine unlearning, and the unconditional guarantee of real rather than merely token human oversight (human-in-the-loop).

Equally important is the categorisation of systems by level of risk. Classifying AI tools as high-risk (for example in the employment sector or financial scoring) triggers strict information obligations and absolute prohibitions of impermissible practices, which include emotion recognition systems in the workplace and biometric profiling to infer sensitive characteristics. In this context, the classic data protection impact assessment (DPIA) is evolving: for high-risk systems it must be integrated with the fundamental rights impact assessment (FRIA) under Article 27 AI Act. In addition, the analysis of institutional roles points to a model of supervision shared between the newly established Commission for the Development and Safety of Artificial Intelligence (KRiBSI) and the President of the Personal Data Protection Office (PUODO), which retains full competence wherever an AI system processes personal data, while risk assessment processes cease to be static documents and become part of a dynamic dialogue with the regulator, including within regulatory sandboxes.

Consequently, the lawful and ethical use of artificial intelligence in modern business is not possible without a comprehensive oversight model: AI Governance. Traditional IT security measures must be extended to include mechanisms that harden models against a new generation of cyberattacks, such as adversarial attacks and data poisoning. The synergy of the principles of privacy by design and privacy by default with the rigorous procedures of the AI Act and the GDPR leads to the conclusion that the EU legal framework does not block innovation but civilises the process of digital transformation. It guarantees that technological development takes place within the limits of the law, with respect for the personhood and cognitive autonomy of the human being.

Bibliography

Legislation

  1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1, as amended) – hereinafter: GDPR
  2. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance) (OJ L, 2024/1689, 12.7.2024) – hereinafter: AI Act
  3. Act of 26 June 1974 – Labour Code (consolidated text: Journal of Laws of 2025, item 277, as amended).

Online Sources

  • Letter of [date] issued by the President of the Personal Data Protection Office, ref. DOL.401.354.2024, Comments of the PUODO on the draft Act on Artificial Intelligence Systems, www.uodo.gov.pl.
  • Wolters Kluwer: “Przetwarzanie danych osobowych na podstawie prawnie uzasadnionego interesu administratora” [Processing of personal data on the basis of the controller’s legitimate interest], 12.12.2025, available online: https://www.lex.pl/test-rownowagi-rodo,43864.html
  • European Parliamentary Research Service (EPRS): Artificial Intelligence Act: State of play and implementation challenges, European Parliamentary Research Service (EPRS) At a Glance Briefing, Brussels 2026, [online], available at: https://www.europarl.europa.eu/RegData/etudes/ATAG/2026/785673/EPRS_ATA(2026)785673_EN.pdf (accessed: 29.05.2026).
  • Biznes Myśli: Fine-tuning LLM – fakty i mity. Jak skutecznie dotrenować duży model językowy [Fine-tuning LLMs – facts and myths. How to fine-tune a large language model effectively], [online], available at: https://biznesmysli.pl/fine-tuning-llm-fakty-i-mity/ (accessed: 29.05.2026).
  • LegalGeek: AI Act vs RODO. Jak sztuczna inteligencja wpływa na ochronę danych osobowych [AI Act vs GDPR. How artificial intelligence affects personal data protection], Legal Knowledge Portal for Business, [online], available at: https://legalgeek.pl/blog/ai-act-vs-rodo/ (accessed: 29.05.2026).
  • Wikipedia (the free encyclopedia): Duży model językowy [Large language model], [online], available at: https://pl.wikipedia.org/wiki/Du%C5%BCy_model_j%C4%99zykowy (accessed: 29.05.2026).
  • Data Science Robię: AI Governance – co to takiego i dlaczego jest teraz kluczowe? [AI Governance – what is it and why is it now key?], [online], available at: https://www.datasciencerobie.pl/ai-governance-co-to-takiego/ (accessed: 02.06.2026).

[1] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance) (OJ L, 2024/1689, 12.7.2024) – hereinafter: AI Act

[2] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1, as amended) – hereinafter: GDPR

[3] Data Science Robię: AI Governance – co to takiego i dlaczego jest teraz kluczowe? [AI Governance – what is it and why is it now key?], [online], available at: https://www.datasciencerobie.pl/ai-governance-co-to-takiego/ (accessed: 02.06.2026).

[4] adversarial attacks – the deliberate manipulation of input data in order to deceive the algorithm.

 

UP