The loss or corruption of data in a business is rarely “just” a technical problem. In practice it is a serious legal and financial crisis: production downtime, loss of trust among business partners and, in extreme cases, administrative fines running into millions. The scope of liability, however, depends above all on what kind of data has been lost or corrupted. The law treats the loss of source code or technical documentation quite differently from a leak of employee records or a customer database. The first step in assessing liability for a breach of the integrity or availability of data is therefore its unambiguous legal classification.

Personal Data and Non-Personal Data

It is worth starting with the category that, in recent years, has come under particular legal protection as a result of stringent EU rules: personal data, the protection of which directly concerns the privacy and rights of natural persons. Personal data protection is governed by Regulation (EU) 2016/679 of the European Parliament and of the Council on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (GDPR). Under Article 4(1) GDPR, personal data means any information relating to an identified or identifiable natural person, i.e. a person who can be identified, directly or indirectly, in particular by reference to an identifier such as a name or a national identification number (in Poland, the PESEL number).

The rapid growth of the data-driven economy means that non-personal data is gaining ever greater economic and legal significance. Non-personal data is data that does not relate to an identified or identifiable natural person and therefore falls outside the scope of the GDPR. As the recitals of Regulation (EU) 2018/1807 of the European Parliament and of the Council on a framework for the free flow of non-personal data point out, information and communication technologies are no longer a separate sector but the foundation of all modern economic systems, with electronic data at their core. The statutory definition of non-personal data is contained in Article 3(1) of Regulation 2018/1807 and Article 2(4) of Regulation 2022/868 (the Data Governance Act): it is data other than personal data within the meaning of the GDPR. This category includes in particular:

• technical, operational, industrial and telemetry data,
• financial and production data,
• source code and design documentation,
• artificial intelligence models together with their training data,
• data on the operation of IT systems.

Non-personal data includes both data that has never contained any information about natural persons (e.g. weather data or share prices) and data that was originally personal but has subsequently been effectively anonymised, i.e. to a degree that permanently prevents the identification of a natural person. It should be borne in mind that data which has merely been pseudonymised remains personal data (recital 26 GDPR). Non-personal data may be a key asset of a business, often worth more than its traditional tangible assets; in the case of artificial intelligence models, the quality, volume and diversity of the data directly determine the value and effectiveness of the model itself.

A Different Centre of Gravity of Protection

The fundamental distinction between personal and non-personal data is not merely a technical or definitional matter. It determines where the centre of gravity of legal protection lies:

• for personal data, the central protected value is the rights and freedoms of natural persons, including the right to privacy and the right to informational self-determination; the GDPR regime is therefore constructed primarily from the perspective of the data subject as the weaker party in need of protection;
• for non-personal data, the centre of gravity shifts dramatically: what matters is the economic interest of the business, its operational continuity, organisational security and the economic value of the data as such.

The loss or corruption of non-personal data does not infringe the rights of any natural person within the meaning of the GDPR, but it may lead to serious financial losses, loss of competitive advantage, disruption of production processes or the permanent destruction of a resource that cannot be recreated at all, or only at disproportionate cost. This distinction translates into different bases of liability, because non-personal data is not subject to the sanctions provided for in the GDPR.

Mixed Data Sets

A separate issue is that of mixed data sets, i.e. data sets composed of both personal and non-personal data (e.g. the ERP and CRM systems of online shops). Under Article 2(2) of Regulation 2018/1807, where personal and non-personal data are inextricably linked, the GDPR applies to the whole data set. In other cases the GDPR applies only to the personal data part of the set, while non-personal data is governed primarily by the rules of contractual liability arising from the agreement and by the provisions of civil law.

Liability for Loss or Corruption of Personal Data

To assess the consequences of the loss or corruption of personal data properly, three distinct concepts must be kept apart:

• a personal data breach (the security incident),
• an infringement of the GDPR by the controller or processor,
• damage suffered by the data subject.

The occurrence of an incident does not automatically mean that the law has been infringed, and an infringement of the law does not in itself give rise to a right to compensation.

Personal Data Breach

Under Article 4(12) GDPR, a personal data breach is a breach of security leading to the accidental or unlawful destruction, loss, alteration, unauthorised disclosure of, or access to, personal data transmitted, stored or otherwise processed. Both the loss and the corruption of data therefore fall within the broad scope of this concept and may give rise to legal liability. A personal data breach may concern:

• confidentiality – unauthorised disclosure of, or access to, the data;
• integrity – unauthorised alteration of the data, i.e. any change made by an unauthorised person or an incorrect change made by an authorised person;
• availability – unauthorised loss of data (a situation in which the data cannot be used, temporarily or permanently, although it can be recovered or recreated) or unauthorised destruction of data (the data is lost irretrievably because the controller has no means of recreating it).

A breach may occur by accident or as a result of deliberate and unlawful action. The most common causes include:

• human error, e.g. a mistake, a lost storage device or a failure to recognise that a breach has occurred;
• inadequate safeguards or procedures;
• cybercrime, e.g. phishing (impersonating a trusted person in order to extract sensitive data) or ransomware (an attack that blocks access to a system or renders data unreadable, combined with a demand for payment to restore the original state);
• physical or environmental factors, such as natural disasters and failures of technical infrastructure.

A personal data breach occurs regardless of whether any adverse consequences actually materialise. A failure to respond appropriately and promptly, however, may result in adverse consequences for the data subject, such as physical harm, material or non-material damage, or identity theft or fraud (recital 85 GDPR).

Infringement of the GDPR

Article 5(1)(f) GDPR sets out the principle of integrity and confidentiality: personal data must be processed in a manner that ensures appropriate security, including protection against unauthorised or unlawful processing and against accidental loss, destruction or damage, using appropriate technical or organisational measures. Article 5(2) GDPR introduces the principle of accountability, under which the controller must be able to demonstrate compliance with these principles.

The GDPR imposes on controllers a range of obligations relating to secure processing and to responding to breaches. These obligations are built on a risk-based approach: protective measures must be tailored to the risk that the processing poses to the rights and freedoms of natural persons. The requirements therefore differ from controller to controller depending on the specifics of the processing, and controllers must assess the associated risk themselves. The key obligations are:

• Article 24(1) GDPR – the obligation to implement appropriate technical and organisational measures and to assess risk;
• Article 25(1) GDPR – the obligation to take data protection into account already at the design stage of the processing and throughout the processing itself (privacy by design);
• Article 32(1) GDPR – the obligation to implement measures ensuring a level of security appropriate to the risk, e.g. authentication procedures and appropriate infrastructure and safeguards;
• Articles 33 and 34 GDPR – the obligation to notify a breach to the supervisory authority (as a rule within 72 hours) and, where the breach is likely to result in a high risk to the rights and freedoms of natural persons, also to communicate it to the data subject;
• Article 35 GDPR – the obligation to carry out a formal data protection impact assessment (DPIA) where a type of processing is likely to result in a high risk (e.g. large-scale processing of special categories of data or of data relating to criminal convictions); this obligation is further specified by the list of processing operations published by the President of the Polish Personal Data Protection Office (PUODO).

Risk assessment consists in estimating the severity of the potential consequences of a breach and the likelihood of their occurrence, taking into account, among other things, the type of breach, the sensitivity of the data and the seriousness of the consequences for the data subjects.

An infringement of the GDPR is therefore conduct that fails to meet the above requirements. It is legal in nature, as opposed to the incident itself, which is a matter of fact. Failure to comply with the obligations listed in Articles 8, 11, 25 to 39, 42 and 43 GDPR is punishable under Article 83(4) GDPR by an administrative fine of up to EUR 10,000,000 or, in the case of an undertaking, up to 2% of its total worldwide annual turnover of the preceding financial year, whichever is higher.

The Regulation also grants the data subject a number of rights, the exercise of which may become difficult or impossible once data has been lost or corrupted:

• the right of access (Article 15 GDPR) – the right to obtain confirmation as to whether the controller is processing personal data and, if so, access to that data; after loss or corruption, the controller may be unable to give effect to this right, and if it fails to act on the request within one month (Article 12(3) and (4) GDPR), the data subject may lodge a complaint with the supervisory authority and seek a judicial remedy;
• the right to rectification (Article 16 GDPR) – of particular importance where data has been corrupted; the difficulty may be that the controller does not know which data has been corrupted or to what state it should be restored, e.g. where the corruption has also affected the backup copies;
• the right to erasure (Article 17 GDPR) – inter alia where the data is no longer necessary for the purposes for which it was collected, the data subject has withdrawn consent and there is no other legal basis, or the data has been processed unlawfully; where data has been lost (as opposed to destroyed) it still exists and can be restored, and a controller that cannot locate it will be unable to comply with an erasure request, which may in itself constitute an infringement of the GDPR;
• the right to restriction of processing (Article 18(1)(a) GDPR) – where the data subject contests the accuracy of the data, which is precisely the case when data has been corrupted.

Infringement of the provisions governing data subjects’ rights (Articles 12 to 22 GDPR) and of the basic principles of processing (Articles 5 to 7 and 9 GDPR) is punishable under Article 83(5) GDPR by an administrative fine of up to EUR 20,000,000 or, in the case of an undertaking, up to 4% of its total worldwide annual turnover of the preceding financial year, whichever is higher.

Damage and Liability for Compensation

Even where the GDPR has been infringed, the injured party does not automatically acquire a right to compensation. Under Article 82 GDPR, three conditions must be met cumulatively: an infringement of the Regulation, material or non-material damage, and a causal link between the two. The controller is liable; the processor is liable only where it has failed to comply with obligations specifically directed to processors or has acted outside or contrary to the lawful instructions of the controller (Article 82(2)). Exemption from liability is possible only on proof that the entity concerned is not in any way responsible for the event giving rise to the damage (Article 82(3)). Where more than one controller or processor is responsible for the damage, they are jointly and severally liable (Article 82(4)).

The case law of the CJEU and of the Polish courts on Article 82 GDPR has so far developed mainly in cases concerning the unauthorised disclosure of data to third parties; there are no decisions dealing directly with the loss or corruption of data. The principles developed by the Court are, however, general in nature and apply to all types of breach listed in Article 4(12) GDPR, and thus also to breaches of the integrity and availability of data. The most important of them are:

• C-300/21 (Österreichische Post) – a mere infringement is not sufficient for an award of compensation; the claimant must show damage (material or non-material) and a causal link; at the same time, there is no minimum threshold of seriousness of the damage;
• C-687/21 (BL v MediaMarktSaturn Hagen-Iserlohn) – the mere fear or apprehension of possible misuse of data by third parties may constitute non-material damage, but the claimant must prove that it actually occurred; such apprehension does not constitute damage where there was no possibility of a third party becoming aware of the data, which is typically the case where data has been destroyed or lost without being disclosed; compensation is purely compensatory and not punitive in nature;
• judgment of the Warsaw Court of Appeal of 22 June 2023, case no. I ACa 352/23 – the court confirmed that non-material damage within the meaning of Article 82 GDPR also covers distress and the risk of reputational harm.

The CJEU has also pointed out that Article 82 GDPR covers only infringements of the Regulation itself. If the loss or corruption of data additionally involves an infringement of national law, including of personality rights, any extension of liability can be considered only on the basis of national provisions; in Poland, the injured party may then claim separate compensation for non-material harm under Articles 23 and 24 in conjunction with Article 448 of the Civil Code.

Liability for Loss or Corruption of Non-Personal Data

Contractual Liability

The basis for claims against external providers whose improper performance has contributed to the loss or corruption of data is Article 471 of the Polish Civil Code. This applies in particular to software vendors, cloud service operators, IT outsourcing companies and entities responsible for backup and disaster recovery. Under that provision, the debtor is obliged to remedy the damage resulting from non-performance or improper performance of an obligation, unless it proves that this is the consequence of circumstances for which it is not responsible.

In commercial practice the key instrument governing the scope of that liability is the agreement concluded with the IT service provider (e.g. a maintenance or implementation agreement) and, within it, the Service Level Agreement (SLA). It is in the SLA that the parties define the scope of the service, the required availability of systems, the permitted response time in the event of a failure and the consequences of non-compliance, including contractual penalties and grounds for termination. From the perspective of liability for data loss, two SLA parameters are of particular importance:

• Recovery Point Objective (RPO) – the acceptable amount of data an organisation can afford to lose in the event of an incident; this parameter determines the required frequency of backups;
• Recovery Time Objective (RTO) – the maximum acceptable time for restoring operational processes after a disruption.

Both parameters form part of disaster recovery, i.e. the set of policies, procedures and tools designed to minimise the effects of unforeseen events such as cyberattacks, infrastructure failures or natural disasters. Disaster recovery comprises three layers:

• prevention (backups, protective systems, maintenance),
• detection (monitoring, security audits),
• remediation (data restoration, activation of the business continuity plan).

In the case of cloud services, which consist in providing storage space, computing power or ready-made applications on infrastructure managed by the provider, liability for the loss or corruption of data rests as a rule with the operator that has taken control of the infrastructure. The customer should pay particular attention to data sovereignty clauses: providers subject to US law may be required to disclose data to law enforcement authorities under the CLOUD Act, even if the data is stored outside the territory of the United States.

Contractual Penalties, Limitation of Liability and Force Majeure

A breach of the agreement with the provider gives rise to contractual liability. Where the agreement provides for a contractual penalty, improper performance obliges the provider to pay it. Where the breach has caused damage and no contractual penalty has been stipulated, or the right to claim damages exceeding the penalty has not been excluded, the customer may claim damages under Article 471 of the Civil Code. In the cases described here, improper performance will most often consist in exceeding the RTO or failing to meet other SLA parameters.

The parties may contractually modify the scope of the provider’s liability, in particular by:

• limiting liability to actual loss, excluding lost profits,
• introducing a monetary cap on damages,
• excluding the statutory warranty for defects.

In practice, customers often stipulate that the contractor is liable without limit for the most serious breaches of the agreement, which include precisely the loss of data, breach of confidentiality obligations and infringement of intellectual property rights. Force majeure may relieve the provider of liability, but this requires proof that the event was external, unforeseeable and unavoidable even with the use of all available means. In the case of known methods of cyberattack or foreseeable infrastructure failures, such a defence may be difficult to sustain, which is why the parties very often define in their agreements the catalogue of circumstances treated as force majeure.

The contractual liability regime described above applies to both personal and non-personal data, but its role differs fundamentally depending on the type of data. For personal data, liability under Article 471 of the Civil Code supplements the GDPR regime: the controller may bring recourse claims against a provider whose negligence contributed to a breach for which the controller has been held liable towards data subjects or the supervisory authority. For non-personal data, the agreement and the contractual liability arising from it are often the only available source of legal protection, given the absence of specific statutory rules dedicated to this type of data. This means that the SLA provisions, the precision of the RPO and RTO parameters, the scope of the liability clauses and the contractual penalties determine whether the provider will be held liable towards the customer.

Other Bases of Liability

Criminal Liability

Under Article 268a of the Polish Criminal Code, criminal liability is incurred by anyone who destroys, damages, deletes, alters or hinders access to computer data, or significantly disrupts or prevents the automatic processing, collection or transmission of such data. The offence is prosecuted on the motion of the injured party, meaning that the injured entity must file a request for prosecution. Criminal liability therefore extends, for example, to a person who deletes or corrupts data in a company’s computer system. The type of data (personal or non-personal) is irrelevant to the existence of the offence. Under Article 269 of the Criminal Code, where the data is of particular importance for national defence, transport safety, the functioning of government administration, another state authority or state institution, or local government, the offence carries a heavier penalty.

Liability of Members of Company Bodies

Liability of management board members may also arise under Article 293 of the Polish Commercial Companies Code (limited liability company) and Article 483 of that Code (joint-stock company) for damage caused to the company by an act or omission contrary to the law or the articles of association. Such damage may include an administrative fine imposed on the company as a result of a breach caused, for example, by a failure to implement appropriate safeguards or a data protection policy, or by a failure to carry out a DPIA, as well as the need to pay compensation to data subjects.

Loss or Corruption of Data as a Cybersecurity Incident

The loss or corruption of data in a business is a serious cybersecurity incident. In view of the reliance of the modern economy on data and the growth of digital threats, the EU legislator places great emphasis on protecting the availability and integrity of all of a company’s digital assets, attaching severe public-law liability to their loss. The legal framework is currently set by three key instruments: the DORA Regulation (dedicated to the financial sector), the NIS2 Directive and the Polish Act on the National Cybersecurity System (UKSC) implementing it. These rules impose stringent obligations on organisations in three main areas:

• ICT risk management – entities covered by the rules are required to implement security management systems, continuously assess risk and monitor their systems for threats. Under Article 18(1)(d) DORA, data loss is one of the direct criteria for classifying an incident as major. Under both DORA and the UKSC, incidents must be reported to the competent authorities and, in the case of more serious incidents, also to clients.
• Digital resilience and business continuity – organisations must have genuine mechanisms ensuring continuity of operations in a crisis, including backup procedures, emergency data restoration and recovery plans (for both personal and non-personal data) and systematic testing of protective systems.
• Oversight of third-party providers – the rules (DORA in particular) significantly restrict cooperation with IT service providers that do not meet security standards. Agreements between financial entities and third-party providers must, among other things, guarantee the right to terminate where weaknesses in data protection are identified and must contain provisions on that protection.

Failures in these areas resulting in the loss or corruption of data are no longer merely an operational problem. The amended UKSC provides for severe financial penalties for non-compliance, ranging from PLN 20,000 up to EUR 10,000,000.

Conclusion

The loss or corruption of data is never a legally neutral event. The scope and nature of liability, however, depend above all on the type of data affected by the incident and on whether the entity responsible for the data has complied with its obligations:

• personal data – administrative liability arises from an infringement of the GDPR, while liability for compensation depends on the existence of damage and a causal link;
• non-personal data – the burden of protection rests on civil law and on the contract; the precision of the SLA provisions, the RPO and RTO parameters and the liability clauses determines whether the business can successfully pursue its claims, and in the absence of dedicated statutory rules the contract often remains the only instrument of protection;
• regardless of the type of data – the loss or corruption of data may give rise, in parallel, to the criminal liability of the perpetrator, the liability of management board members towards the company and, for entities subject to sectoral regulation, public-law sanctions under DORA and the UKSC.

An appropriate response to the incident, including timely notification of the breach to the competent authorities, may significantly reduce the scope of that liability.

Artykuł Liability for Loss and Corruption of Data pochodzi z serwisu KIELTYKA GLADKOWSKI LEGAL | CROSS BORDER POLISH LAW FIRM RANKED IN THE LEGAL 500 EMEA SINCE 2019.

Implementing artificial intelligence in an organisation does not begin with choosing a tool, but with answering the question of who we are in the AI value chain and what we are responsible for. This article sets out the relationship between the GDPR and the AI Act: from the roles of provider, deployer and controller, through algorithmic risks, the limits of lawfulness in the workplace and the rights of individuals, to the DPIA, the FRIA and the AI Governance model. It reflects the legal position following the entry into force of the Polish Act on Artificial Intelligence Systems and the postponement of the AI Act application dates by Regulation (EU) 2026/1744.

Introduction

Rapid technological progress and the widespread deployment of systems based on artificial intelligence are permanently reshaping the landscape of modern economic and social life. The use of machine learning algorithms and large language models in professional processes opens up unprecedented opportunities for optimisation, allowing the automation of tasks that previously required prolonged human effort. This technological efficiency, however, brings significant ethical and legal challenges and calls into question existing standards of privacy protection. In an era of such profound digital transformation, the key task is to develop a regulatory framework that balances the drive for innovation against the imperative of protecting fundamental rights. It must be remembered that AI systems, despite their apparent infallibility, operate on data that may be erroneous, biased or out of date. As a result, algorithm-based decision-making processes directly affect the legal and personal situation of natural persons, which creates a need to define precisely who is responsible for the outcome of the technology.

Regardless of how advanced AI tools become, the paradigm of human oversight must remain the fundamental principle of safe implementation. A human being cannot be reduced to a mere recipient of output data; they must perform an active supervisory function and act as a guarantor of reliability and safety when the algorithm fails. The AI Act[1] establishes rigorous procedures and imposes specific restrictions on providers, deployers and other links in the artificial intelligence value chain. In relation to the GDPR[2], the AI Act is complementary: it does not repeal existing data protection principles but refines them in a technological context. The AI Act adds an additional layer of product-related and ethical obligations, ensuring that algorithms are used with respect for the rights and freedoms of the individual. Much of this framework already applies: the prohibitions in Article 5 AI Act have applied since 2 February 2025 and the transparency obligations in Article 50 AI Act since 2 August 2026, while Regulation (EU) 2026/1744 (the so-called Digital Omnibus on AI) has postponed the application of the obligations for stand-alone high-risk systems listed in Annex III to 2 December 2027. In Poland, the national framework is set out in the Act of 3 July 2026 on Artificial Intelligence Systems (Journal of Laws, item 1003), which established the Commission for the Development and Safety of Artificial Intelligence (KRiBSI) as the market surveillance authority.

1st         Who Is Who in the AI Ecosystem: Provider, Deployer, Controller

a)    Provider or Deployer? The Answer Determines the Scope of Liability

Defining an organisation’s role in the artificial intelligence ecosystem is of key importance for determining its legal obligations precisely. It must be established whether a given entity acts as the provider of an AI system or is merely a deployer of a ready-made technological solution. Under Article 3(3) AI Act, a provider is a natural or legal person, public authority, agency or other body that develops an AI system (or a general-purpose AI model) or has it developed, and places it on the market or puts it into service under its own name or trademark, whether for payment or free of charge. The provider’s principal task is therefore to create the technology and ensure its compliance with rigorous safety and data quality requirements before the commercialisation stage. Most organisations using AI systems will not act as providers but as deployers. This distinction is fundamental to risk analysis: while the provider is responsible for the architecture and the training process (compliance of the “product”), the deployer is responsible for how the system is operated, for instructions to employees and for the lawfulness of the data entered into the model during the inference phase.

Under Article 3(4) AI Act, a deployer may be a natural or legal person, public authority, agency or other body that uses an AI system under its authority. These features clearly distinguish the provider of an AI system from an entity that merely uses the supplied system and exercises authority over it. In addition, an entity using artificial intelligence systems may acquire the status of a controller of personal data once it meets the definitional criteria set out in Article 4(7) GDPR. A controller is therefore a natural or legal person, public authority or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data. It should be stressed that an AI system itself has no legal personality, and responsibility for its actions rests with the entity that decides on its deployment and operating parameters. Data processing in this context takes place through operations or sets of operations performed on personal data by automated means, which falls within the broad definition in Article 4(2) GDPR. These processes include, in particular, the collection, recording, adaptation, consultation and use of data by algorithms in order to generate results (output). In the case of AI systems, this processing is highly automated and is usually aimed at optimising and increasing the operational efficiency of information processes, which places a particular duty on the controller to keep control over every stage of the data life cycle.

b)    When an AI System Becomes a High-Risk System

Under the system adopted in the AI Act, the correct classification of artificial intelligence systems is of key importance for organisations, since solutions classified as high-risk systems are subject to the most stringent legal regime. An AI system acquires that status if it meets the conditions set out in Article 6(1) AI Act (this concerns systems that are products, or safety components of products, subject to EU certification) or if it is expressly listed in Annex III to the Regulation. That catalogue covers areas of critical importance for fundamental rights, such as biometrics and the categorisation of persons, the management of critical infrastructure, and education and vocational training. Particular attention should be paid to employment and workers management, access to essential public and commercial services, and systems used in law enforcement, migration management and the administration of justice. Classifying a technology in this category obliges the controller not only to ensure high-quality data and human oversight in accordance with the AI Act, but also to comply strictly with the GDPR, which in most cases will include carrying out a full data protection impact assessment (DPIA).

c)     Transparency, Instructions for Use and Human Oversight: Obligations in Practice

Providers must ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the persons concerned are informed that they are interacting with an AI system (Article 50(1) AI Act). High-risk AI systems must also be designed and developed in accordance with the principle of transparency of their operation (Article 13 AI Act), which enables deployers to interpret the system’s output and use it appropriately. It is also essential to attach instructions for use in an appropriate format, containing complete and comprehensible information for deployers. While AI systems are in use, it must be possible for natural persons to oversee the system (Article 14 AI Act) in order to prevent risks to health, safety or fundamental rights.

Deployers of high-risk AI systems must implement appropriate technical and organisational measures to ensure that they use such systems in accordance with the instructions for use supplied by the provider (Article 26(1) AI Act). Under Article 26(11) AI Act, deployers of the high-risk AI systems referred to in Annex III that make decisions, or assist in making decisions, related to natural persons must fulfil their information obligation before actually using the high-risk AI system, by clearly informing those persons that such technology is being used in relation to them.

In order to give full effect to the requirements of the AI Act, deployers are also obliged to assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary organisational support. Under Article 14(2) of the Regulation, “human oversight shall aim to prevent or minimise the risks to health, safety or fundamental rights (…)”. Article 14(4), in turn, requires providers to deliver a high-risk AI system to the deployer in such a way as to enable the natural persons to whom human oversight is assigned, among other things, “to remain aware of the possible tendency of automatically relying or over-relying on the output produced by a high-risk AI system (automation bias), in particular for high-risk AI systems used to provide information or recommendations for decisions to be taken by natural persons”.

d)    Training and Inference: Where the Data Risk Hides in Language Models

The foundation of large language models (LLMs) is the processing of vast quantities of data, enabling the generation of text and the performance of advanced natural-language tasks. As tools belonging to generative artificial intelligence, these models operate on the basis of the statistical prediction of successive tokens (words or parts of words) in response to instructions entered by the user, known as prompts. In market and corporate practice, the most common are the GPT series models (OpenAI), implemented in solutions such as ChatGPT and Microsoft Copilot, as well as the Llama family of models (Meta), DeepSeek and the Polish projects Bielik and PLLuM. From a data protection perspective, the key distinction is between the two fundamental phases of a model’s operation: training and inference.

Training phase

In this phase, the substantive “intelligence” of the model is created. The process is based on the analysis of enormous data sets (Big Data) in order to detect statistical correlations between tokens. It should be emphasised that an LLM does not interpret content cognitively (in the human sense) but calculates the probability of the next element in a sequence. The data is often obtained through the mass collection of content from the open internet (web scraping). From the GDPR perspective, this is the most controversial stage, because the controller of personal data (usually the provider of the system) must demonstrate a specific legal basis and purpose for the processing (Article 6 GDPR). At this stage the provider becomes responsible for the quality of the data sets. Under Article 10 AI Act, training data sets must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the intended purpose of the system.

Inference phase

The inference phase is the operational stage in which the trained model generates answers to specific user queries (prompts), using the statistical weights developed during training. In this phase the model does not, as a rule, “learn” in real time but processes the information supplied to it at a given moment. From the perspective of an organisation deploying an AI system, this is the area of the highest legal risk. The key problem is employees entering customer data, personal information or trade secrets into queries sent to an external model. Such conduct constitutes a transfer of data to the provider’s servers, which often involves a transfer outside the European Economic Area (EEA). In this scenario the organisation acquires the status of a deployer within the meaning of Article 3(4) AI Act. Under Article 4(7) GDPR, the organisation, as the controller of its employees’ and customers’ data, bears full responsibility for the content of the information that reaches the AI system. So while the training phase remains the domain of large technology corporations (providers), the inference phase is the area in which every business bears direct responsibility for the security and lawfulness of the prompts it generates.

2nd    Bias, Hidden Scoring and Explainability: Algorithmic Risks under Article 5 GDPR and Article 10 AI Act

The practical implementation of artificial intelligence systems in socially sensitive areas, such as the recruitment of employees or the assessment of creditworthiness, requires the systematic management of the risk of algorithmic error. The key challenge is to ensure the explainability of AI and to combat effectively the phenomenon of bias, which can lead to the discrimination of entire social groups. The foundation for counteracting these risks under data protection law is the set of principles expressed in Article 5(1) GDPR. Under the principle of transparency (point (a)), the organisation must be able to explain to the data subject the logic behind an autonomous decision of the algorithm. The principle of data minimisation (point (c)), in turn, requires the information collected to be limited to what is necessary for the purpose, which prevents models from being fed with redundant data or data that has no objective bearing on the result of the analysis. In this context, the prohibition of so-called hidden scoring takes on particular significance; this means an AI system covertly assessing, for example, job candidates by assigning them points without ensuring adequate transparency of the process. The controller, as the employer, is obliged under Article 13(2)(f) GDPR to provide the data subject with information about the existence of automated decision-making, including profiling (referred to in Article 22(1) and (4)), and meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject.

AI systems frequently rely on profiling mechanisms, that is, the automated processing of personal data consisting of the use of that data to evaluate certain aspects of a natural person, in particular their competence, expected work performance or other characteristics relevant to the purpose for which the AI tool is used. Under Article 22(1) GDPR, “the data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her”, unless the decision is necessary for entering into or performing a contract, is authorised by Union or Member State law, or is based on the data subject’s explicit consent (Article 22(2) GDPR). Profiling as such is not prohibited in principle, but its use is subject to significant restrictions, particularly where it leads to decisions with significant effects on the person being profiled. In practice, this means that special care must be taken in designing and using AI tools, particularly with regard to the risk of bias. These systems learn from historical data, which may lead to the replication of earlier decision patterns and to unintended discrimination, for example on grounds of age, state of health or other personal characteristics. It is therefore essential to ensure transparency of the systems’ operation, adequate human oversight and the ability to verify and challenge the results generated by AI.

Relying solely on the rules contained in the GDPR is not enough, however, when confronted with deep neural networks, which is why a key role is played by Article 10 AI Act, which governs data and data governance in high-risk AI systems. That provision imposes rigorous obligations on providers to examine and verify training, validation and testing data. These data sets must be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the specific geographical, behavioural or functional setting in which the system is to operate. This requirement directly compels organisations to apply procedures for detecting and neutralising hidden bias (bias mitigation). For example, a recruitment algorithm trained on the historical data of a company in which mainly men were promoted may classify the female gender as a negative feature; Article 10 AI Act requires such anomalies to be actively monitored and corrected at the design stage and through continuous oversight of the system’s operation.

3rd      Can an Employer Require the Use of AI? The Limits of Lawfulness in the Workplace

Assessing the lawfulness of deploying artificial intelligence systems in the workplace requires a precise delineation of the boundary between the employer’s managerial prerogatives and the fundamental rights of employees. Whether an employee can be required to use AI tools depends on the nature of the tool and the extent of its intrusion into the employee’s privacy. While issuing a work instruction to use AI as simple support in daily duties falls within the law, the deployment of monitoring, evaluation or biometric systems encounters strict legal barriers. The legal basis for deploying standard algorithms that optimise work processes is most often Article 6(1)(f) GDPR, that is, the legitimate interest of the controller (the employer). Its application is conditional, however, on passing the so-called balancing test, which must show that the employer’s economic or organisational interests do not override the autonomy, rights and freedoms of the persons employed. In the case of high-risk AI tools, the balancing test will generally be harder to pass, and legitimate interest alone may prove insufficient, particularly where the processing leads to decisions of the kind referred to in Article 22 GDPR.

The situation becomes radically more complicated where an AI system requires the processing of biometric data in order to operate (for example, facial recognition systems used to record working time or to analyse an employee’s emotions). Biometric data belong to the special categories of data whose processing is, as a rule, prohibited under Article 9(1) GDPR. For an employer to use such data lawfully, one of the exceptional conditions in Article 9(2) GDPR must be met; in the reality of Polish labour law this most often requires the employee’s explicit and fully voluntary consent (subject to the requirements of Article 22¹ᵇ of the Labour Code) or a specific legal provision imposing such an obligation on safety grounds. The employee’s lack of genuine freedom of choice in the relationship with the employer makes such consent extremely difficult to defend before the supervisory authorities.

The ultimate barrier to the implementation of high-risk algorithmic AI systems is Article 5 AI Act, which introduces a categorical list of practices prohibited in the European Union. On that basis it is unlawful to deploy AI systems used for so-called social scoring (the point-based classification of citizens or employees on the basis of their social behaviour), as well as systems that carry out biometric categorisation in order to infer sensitive characteristics (for example political opinions or sexual orientation). Moreover, in the context of employment relationships, the AI Act prohibits the use of emotion recognition systems in the workplace (Article 5(1)(f) AI Act), except for systems put in place for medical or safety reasons, which in practice closes the door to employers algorithmically testing the mood or stress levels of their staff. These prohibitions have applied since 2 February 2025, and Regulation (EU) 2026/1744 extends their catalogue from 2 December 2026.

4th       Individuals versus the Algorithm: Erasure, Human Intervention, Transparency

The informational and technological asymmetry between the individual and the entities deploying artificial intelligence systems requires effective instruments of legal protection. The greatest controversy in this area currently surrounds the exercise of rights of an erasure nature. Enforcing the right to be forgotten under Article 17 GDPR encounters a fundamental technical barrier in the form of the so-called “memory” of generative models and neural networks. In traditional databases, deleting information means erasing a specific record. In the case of AI models, input data becomes irreversibly integrated into the structure of the model’s mathematical weights during the training process. Reversing that state, described in the literature as machine unlearning, is a complex and costly process and often simply impossible without retraining the model in its entirety. This creates a deep conflict between the individual’s right to demand the erasure of their data and the architecture of modern technologies.

Another key safeguard is Article 22 GDPR, which lays down a general prohibition on subjecting natural persons to decisions based solely on automated processing, including profiling, which produce legal effects concerning them or similarly significantly affect them. This provision gives the individual the right to obtain human intervention (human-in-the-loop), to express their own point of view and to contest a decision taken by an algorithm (for example, an automated refusal of credit or rejection of a job application). Supervisory authorities’ guidelines and the case law of the CJEU (including the judgment of 7 December 2023 in Case C-634/21 SCHUFA Holding) indicate that human involvement in the decision-making process may not be fictitious or token; a manager or analyst must have a real ability to change the verdict generated by the AI. These guarantees are complemented, at the level of direct interaction with the technology, by the information obligations arising from Article 50 AI Act, applicable since 2 August 2026. That provision imposes strict operational transparency requirements on providers and deployers of artificial intelligence systems. Under it, every natural person interacting with an AI system, such as a chatbot or voice assistant, must be clearly and promptly informed of that fact, unless this is obvious from the context of use. Equally strict obligations apply to the labelling of synthetically generated content, including so-called deepfakes. Deployers of AI systems that generate or manipulate images, audio or video content in such a way that it closely resembles authentic persons or events are legally required to disclose that the content has been artificially generated. The purpose of this regulation is to protect the cognitive autonomy of the individual and to counter mass disinformation.

5th       Four Risk Tiers and a New Supervisory Map: KRiBSI alongside the President of the Personal Data Protection Office

A key element of the EU strategy for regulating artificial intelligence is the risk-based approach, which directly determines the scope of the legal obligations imposed on organisations. The Artificial Intelligence Act introduces a rigid division of systems into four categories: prohibited practices (posing an unacceptable risk to fundamental rights), high-risk systems, limited-risk systems (mainly subject to information obligations) and minimal-risk systems. From the operational perspective of businesses and public institutions, the most important category is that of high-risk systems, whose definition and classification mechanism are governed by Article 6 AI Act. That provision refers directly to Annex III to the AI Act, which contains a closed catalogue of critical areas, including workers management and access to employment (HR), education and vocational training, as well as law enforcement and the management of critical infrastructure. Implementing AI tools in these sectors triggers a strict legal regime, including the need to implement a risk management system, ensure high-quality data and provide human oversight. Under Regulation (EU) 2026/1744, these obligations will apply to stand-alone systems listed in Annex III from 2 December 2027 and to systems embedded in products covered by Annex I from 2 August 2028; the postponement does not, however, change the scope of the requirements but merely gives organisations time to implement them.

From the perspective of mapping risk within an organisation, this classification has profound consequences under data protection law. The Act of 3 July 2026 on Artificial Intelligence Systems entrusted market surveillance to the Commission for the Development and Safety of Artificial Intelligence (KRiBSI), which from 28 October 2026 acquires full powers of inspection and the power to impose the fines provided for in the AI Act. This does not, however, exclude the President of the Personal Data Protection Office (PUODO). Under Article 74(8) AI Act, in respect of the high-risk AI systems listed in Annex III, point 1 (biometrics, in so far as used for law enforcement, migration and the administration of justice), point 6 (law enforcement), point 7 (migration and asylum) and point 8 (administration of justice and democratic processes), market surveillance is linked to the data protection authorities, and the PUODO consistently emphasised, already in its comments on the draft act, that any processing of personal data in AI systems remains within its competence under the GDPR. This means that a compliance audit cannot be limited to a standard data protection impact assessment (DPIA). The organisation must take account of a dual supervision model in which KRiBSI verifies the system’s compliance with the AI Act, while the data protection authority retains full investigative and corrective powers in respect of the processing of personal data, including the data used to build the model. Ignoring this dual competence at the system design stage (privacy and compliance by design) exposes controllers to a serious risk of financial penalties and administrative sanctions in the form of an order to shut down the algorithm immediately.

6th       DPIA and FRIA: from a Document in a Drawer to a Dialogue with the Regulator

Because of their systemic and often unpredictable impact on privacy, the deployment of advanced artificial intelligence systems in most cases automatically triggers an obligation to carry out a data protection impact assessment. Under Article 35 GDPR, that assessment (Data Protection Impact Assessment, DPIA) is mandatory whenever processing using new technologies is likely to result in a high risk to the rights or freedoms of natural persons, which, in the context of recruitment or credit-assessment algorithms (mentioned above in relation to the principles of minimisation and transparency in Article 5 GDPR), is an operational standard. The contemporary AI legal regime, however, dramatically extends that obligation. Under Article 27 AI Act, public bodies and private entities providing public services, as well as deployers of the high-risk AI systems referred to in Annex III, point 5(b) and (c), are required to carry out a rigorous fundamental rights impact assessment (FRIA). The entities referred to in Annex III, point 5(b) and (c) AI Act are those using high-risk AI systems to evaluate the creditworthiness of natural persons or establish their credit score, and systems used for risk assessment and pricing in relation to natural persons in the case of life and health insurance. This mechanism cannot operate in isolation from the GDPR; in practice, the FRIA becomes an integral, deeper extension of the classic DPIA, requiring the organisation to examine the impact of the algorithm not only on privacy but also on human dignity, non-discrimination and the right to a fair trial, that is, the impact on the fundamental rights of the persons in relation to whom the AI systems are used.

The traditional approach, in which the DPIA was merely an internal, static corporate document filed away in a drawer, is becoming a thing of the past when confronted with Article 57(10) AI Act, which governs so-called regulatory sandboxes. These sandboxes, which are controlled testing environments for innovative AI solutions, redefine the relationship between the controller and the supervisory authority. An organisation that decides to participate in a regulatory sandbox is legally obliged to cooperate closely and transparently with the authority operating the sandbox (in Poland, KRiBSI) and, in so far as the system under test processes personal data, also with the President of the Personal Data Protection Office (Article 57(10) AI Act). In this new model of cooperation, the DPIA process evolves into a tool for a dynamic, multilateral dialogue with the state authority. The results of the risk analysis are consulted on an ongoing basis, which makes it possible to eliminate algorithmic bias under the regulator’s eye before the system is commercialised. Such a compliance model not only minimises the risk of severe financial penalties but also makes it possible to build AI systems that are safe and transparent from the stage of their technological incubation.

7th       AI Governance: How to Implement AI in an Organisation and Be Able to Prove It

Effective management of artificial intelligence systems within an organisation requires a multi-level oversight model (AI Governance[3]) that combines technical protective measures with a systematic assessment of legal risk. The foundation for designing safe algorithms is formed by the principles of data protection by design and by default, codified in Article 25 GDPR. They oblige the organisation to take privacy protection and data minimisation into account at the earliest stage of creating the architecture of the AI model, and also when selecting training data and model parameters. Article 32 GDPR, in turn, which lays down the principle of security of processing, takes on an entirely new technological dimension in the world of autonomous systems. Classic IT safeguards (such as encryption or access control) are no longer sufficient against the specific vulnerabilities of neural networks. Ensuring the integrity and confidentiality of data requires controllers to actively harden models against a new generation of cyberattacks, including adversarial attacks[4], attempts to poison training data sets (data poisoning) and model inversion techniques aimed at extracting the personal data used to build the model.

The formal tool for verifying these safeguards and estimating the risks to natural persons remains the data protection impact assessment (DPIA) carried out under Article 35 GDPR. Because of the informational asymmetry and intrusiveness of AI models, however, this assessment ceases to be merely an internal, static document and becomes a platform for continuous monitoring. For artificial intelligence systems classified as high-risk solutions, the traditional DPIA must be integrated with the new, strict obligation under Article 27 AI Act: the fundamental rights impact assessment (FRIA). Only the synergy of the technical resilience of the model with a transparently conducted DPIA/FRIA procedure enables an organisation to demonstrate compliance and lawfully keep an AI system in commercial use.

Summary and Conclusions

The analysis shows that the implementation of artificial intelligence systems, including advanced large language models (LLMs), redefines existing standards of privacy protection and requires organisations to build entirely new governance structures. The key to ensuring the lawfulness of data processing is the precise identification of the entity’s role in the AI ecosystem. The distinction between the provider, responsible for the training phase and the quality of the product data, and the deployer, which as controller bears full responsibility for the inference phase and the content of the prompts entered, is the starting point for effectively mapping regulatory risk under the GDPR and the Artificial Intelligence Act.

The main challenge for modern organisations is the management of specific algorithmic risks, such as the lack of explainability of machine decisions and hidden bias, which in sensitive areas such as recruitment or credit assessment may lead to the replication of discriminatory social patterns. Traditional data protection principles, in particular the principles of transparency and minimisation, gain strong support from Article 10 AI Act. That provision revolutionises compliance processes by imposing strict requirements for the verification of training data, with the aim of actively neutralising bias at the system design stage. Furthermore, the right to be forgotten (Article 17 GDPR) encounters a technological barrier in the form of the mathematical memory of neural networks, which means that the exercise of individuals’ rights against the algorithm requires advanced procedures such as machine unlearning, and the unconditional guarantee of real rather than merely token human oversight (human-in-the-loop).

Equally important is the categorisation of systems by level of risk. Classifying AI tools as high-risk (for example in the employment sector or financial scoring) triggers strict information obligations and absolute prohibitions of impermissible practices, which include emotion recognition systems in the workplace and biometric profiling to infer sensitive characteristics. In this context, the classic data protection impact assessment (DPIA) is evolving: for high-risk systems it must be integrated with the fundamental rights impact assessment (FRIA) under Article 27 AI Act. In addition, the analysis of institutional roles points to a model of supervision shared between the newly established Commission for the Development and Safety of Artificial Intelligence (KRiBSI) and the President of the Personal Data Protection Office (PUODO), which retains full competence wherever an AI system processes personal data, while risk assessment processes cease to be static documents and become part of a dynamic dialogue with the regulator, including within regulatory sandboxes.

Consequently, the lawful and ethical use of artificial intelligence in modern business is not possible without a comprehensive oversight model: AI Governance. Traditional IT security measures must be extended to include mechanisms that harden models against a new generation of cyberattacks, such as adversarial attacks and data poisoning. The synergy of the principles of privacy by design and privacy by default with the rigorous procedures of the AI Act and the GDPR leads to the conclusion that the EU legal framework does not block innovation but civilises the process of digital transformation. It guarantees that technological development takes place within the limits of the law, with respect for the personhood and cognitive autonomy of the human being.

Bibliography

Legislation

1. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1, as amended) – hereinafter: GDPR
2. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance) (OJ L, 2024/1689, 12.7.2024) – hereinafter: AI Act
3. Act of 26 June 1974 – Labour Code (consolidated text: Journal of Laws of 2025, item 277, as amended).

Online Sources

• Letter of [date] issued by the President of the Personal Data Protection Office, ref. DOL.401.354.2024, Comments of the PUODO on the draft Act on Artificial Intelligence Systems, www.uodo.gov.pl.
• Wolters Kluwer: “Przetwarzanie danych osobowych na podstawie prawnie uzasadnionego interesu administratora” [Processing of personal data on the basis of the controller’s legitimate interest], 12.12.2025, available online: https://www.lex.pl/test-rownowagi-rodo,43864.html
• European Parliamentary Research Service (EPRS): Artificial Intelligence Act: State of play and implementation challenges, European Parliamentary Research Service (EPRS) At a Glance Briefing, Brussels 2026, [online], available at: https://www.europarl.europa.eu/RegData/etudes/ATAG/2026/785673/EPRS_ATA(2026)785673_EN.pdf (accessed: 29.05.2026).
• Biznes Myśli: Fine-tuning LLM – fakty i mity. Jak skutecznie dotrenować duży model językowy [Fine-tuning LLMs – facts and myths. How to fine-tune a large language model effectively], [online], available at: https://biznesmysli.pl/fine-tuning-llm-fakty-i-mity/ (accessed: 29.05.2026).
• LegalGeek: AI Act vs RODO. Jak sztuczna inteligencja wpływa na ochronę danych osobowych [AI Act vs GDPR. How artificial intelligence affects personal data protection], Legal Knowledge Portal for Business, [online], available at: https://legalgeek.pl/blog/ai-act-vs-rodo/ (accessed: 29.05.2026).
• Wikipedia (the free encyclopedia): Duży model językowy [Large language model], [online], available at: https://pl.wikipedia.org/wiki/Du%C5%BCy_model_j%C4%99zykowy (accessed: 29.05.2026).
• Data Science Robię: AI Governance – co to takiego i dlaczego jest teraz kluczowe? [AI Governance – what is it and why is it now key?], [online], available at: https://www.datasciencerobie.pl/ai-governance-co-to-takiego/ (accessed: 02.06.2026).

[1] Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance) (OJ L, 2024/1689, 12.7.2024) – hereinafter: AI Act

[2] Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) (OJ L 119, 4.5.2016, p. 1, as amended) – hereinafter: GDPR

[3] Data Science Robię: AI Governance – co to takiego i dlaczego jest teraz kluczowe? [AI Governance – what is it and why is it now key?], [online], available at: https://www.datasciencerobie.pl/ai-governance-co-to-takiego/ (accessed: 02.06.2026).

[4] adversarial attacks – the deliberate manipulation of input data in order to deceive the algorithm.

Artykuł A Human in the Loop, Data under Control. The GDPR and the AI Act in Implementation Practice pochodzi z serwisu KIELTYKA GLADKOWSKI LEGAL | CROSS BORDER POLISH LAW FIRM RANKED IN THE LEGAL 500 EMEA SINCE 2019.

Artificial intelligence (AI) is currently finding widespread use in healthcare. A prime example is the Polish National Health Fund (NFZ) initiative, which utilizes AI to analyze patient data stored in the Fund’s databases. This data is then analyzed with the support of machine learning tools to make strategic decisions regarding the health of Poles. This approach will certainly simplify the work of doctors by searching for and analyzing the desired information, undoubtedly reducing their workload. However, such a solution may raise several issues and legal requirements related to regulations regarding the protection and processing of personal data.

What is personal data?

The most common definition of personal data is contained in the EU Regulation 2016/679 (GDPR), according to which personal data is any information about an identified or identifiable natural person. This includes direct identification (e.g., name and surname) or certain factors allowing indirect identification (e.g., job description or nationality). This concept is expanded by the Polish Act on the Protection of Personal Data Processed in Connection with the Prevention and Combating of Crime of December 14, 2018, by applying it directly to health. Health data here means personal data relating to the physical or mental health of an individual, including data on the use of healthcare services that reveal information about their health.

Patients’ rights

A number of patient rights are listed in the EU Regulation 2025/327 (Regulation on the European Health Data Space). The fundamental right is the right of individuals to access their electronically collected data (especially “priority data,” e.g., electronic prescriptions or imaging test results), which should be granted immediately after data is registered in the system. Individuals can also add their own information to their data already visible in the system and correct it. Furthermore, patients can grant access or request the transfer of their data to another provider. Access to healthcare professionals can also be restricted (however, in such cases, the patient should also be informed of the potential impact of such action on the quality of care provided). In this case, institutions collecting patient data should be aware of these rights, because if they are not respected, the patient could file a complaint (provided, however, that the rights or interests of the individual are adversely affected) and demand appropriate compensation.

The EU GDPR also provides similar rights, which additionally provides for one crucial privilege: the right to object. According to this regulation, an individual may object at any time to the processing of their data, including in connection with the performance of healthcare tasks, for reasons relating to their particular situation. In such a case, the data may no longer be processed unless the controller demonstrates compelling and legitimate grounds for further processing. Under the regulation, a patient could also request the deletion of their personal data if, for example, they are no longer necessary for the purpose for which they were collected or if they were processed unlawfully. Furthermore, the regulation also provides for the possibility of imposing an administrative fine of up to €20 million for a controller’s violation of guaranteed rights. Furthermore, Article 79 of the GDPR grants the right to an effective judicial remedy if the individual (patient) believes that the processing of their personal data violated the law.

Obligations of entities storing and processing data

Pursuant to Article 24 of the GDPR, the data controller is obligated to implement appropriate technical and organizational measures to ensure data processing is carried out in compliance with legal provisions and the rights and freedoms of others. The controller must also review and update these measures as necessary. In the case of AI-based patient data processing, the obligation specified in this article to design the measures described above is also crucial, ensuring that only information necessary to protect the patient’s life and health is processed by default. In the event of a personal data breach, the controller should (within 72 hours of becoming aware of the breach) notify the relevant supervisory authority of the personal data breach. However, the controller is not obligated to do so if the likelihood of a breach affecting the rights and freedoms of natural persons is low. If the risk of a breach is high, the controller should also notify the affected individual. Furthermore, before processing begins, even using new technologies (including AI), if it may result in a high future risk to the rights and freedoms of natural persons, it will be necessary to assess the impact of the planned processing on personal data protection. If such an assessment indeed reveals a high risk, and if the controller fails to implement any measures to mitigate it, the controller must contact the relevant supervisory authority (in Poland, the President of the Personal Data Protection Office [President of the UODO]), which then provides the controller with a written recommendation and may also temporarily restrict or prohibit processing or issue a warning to the controller. General obligations, according to which personal data must be processed lawfully and fairly, in a transparent manner, and limited to what is necessary for the purposes for which they are processed, are also important.

In this situation, Regulation 2024/1689 (“AI Act”) also provides an interesting requirement. According to Article 4 thereof, healthcare entities using AI systems to make strategic decisions about patients are responsible for maintaining an appropriate level of AI competence among their staff, taking into account the purpose of using the system and the persons for whom the systems are to be used.

Requirements for the AI systems themselves

The basic requirements that AI systems used for data processing would have to meet are set out in the aforementioned AI Act. This document explicitly classifies AI systems as “high-risk systems”, and therefore, the requirements set out in the act apply to them. Primarily, this requires maintaining appropriate documentation for the system: technical documentation regarding the quality management system (including data acquisition, collection, analysis, and labeling) and an EU declaration of conformity confirming the system’s compliance with the requirements set out in the regulation. Furthermore, this documentation should be kept at the disposal of the competent national authorities for 10 years after the system’s commissioning. Such systems should also meet transparency requirements, meaning they should be designed to facilitate proper use and interpretation of their actions (they should also have clear operating instructions). They must also have an appropriate oversight system that allows for human oversight of the AI if necessary, especially if its operation were to get out of control and harm others. Finally, AI systems are also subject to certain formal requirements, such as undergoing a pre-market conformity assessment and registering in a dedicated EU database for high-risk AI systems. It is also important to remember that high-risk AI systems are subject to general CE marking regulations. Once placed on the market, suppliers are required to establish a post-market monitoring system for AI systems to ensure their legal compliance.

The issue of non-personal data

It is also worth raising the issue of non-personal data, for example, in the context of a situation where a hospital, in order to decide on the appropriate medication for a patient, requests information about certain medications from a pharmacy. A public sector body may request such information only to the extent that the lack of this data would prevent it from performing its public interest tasks or when the body has no other available means of obtaining such data. A request for this purpose should be submitted (specifying, in particular, the purpose for which the information is requested). However, the data subject who received it may refuse to provide it if they have no control over the requested information or if a similar request for the same purpose has already been submitted by another public sector body. Once the requested information is in the possession of the requester, they must not use it in a manner inconsistent with the purpose for which the data was provided. They must ensure measures to protect its confidentiality or integrity, and they must delete the data as soon as it is no longer needed for the specified purpose. They are also prohibited from using the information obtained to improve a competitive product or from disclosing any information in this regard to third parties. It is also important to bear in mind the right of a public sector body to share the data received with individuals or organisations for the purposes of scientific research or analyses consistent with the purpose for which the data was requested, or with national statistical offices (e.g. the Central Statistical Office). It is important here that these organisations do not have a commercial nature or are not related to entities that do.

The status in Poland

As mentioned above, Poland has established a special supervisory authority for personal data protection, the President of the Personal Data Protection Office (UODO), acting with the assistance of the Office for Personal Data Protection. Among other things, this authority is responsible for consultations on data processing that poses a significant risk of violating the rights of others. It also conducts proceedings in cases of violations of personal data protection regulations and establishes a plan for monitoring compliance with these regulations.

It is also worth remembering the regulations of the Polish Act on Patients’ Rights and the Patient Ombudsman. It stipulates that patients have the right to access medical records concerning their health and the services provided to them. The entity storing this documentation is obligated to disclose the data contained therein only to the patient themselves or an authorized person (or, for example, to a university or research institute for scientific purposes, but without any data allowing for the identification of the individual). Furthermore, the entity providing services is obligated to retain medical records only for a specified period (generally 20 years), after which they should be destroyed in a way that prevents the identification of the patient to whom they pertained. The Act on the Healthcare Information System also limits access to these records to medical professionals and physicians.

Also important are the provisions of the Act on the computerization of the activities of entities carrying out public activities, under which an entity maintaining a public register (i.e. any type of records used to carry out public tasks based on the relevant provisions) should provide another public entity with access to the data in its possession to the extent necessary to carry out public tasks.

It is also important to remember the Polish Code of Medical Ethics, which, in Article 14, requires physicians to inform patients about the benefits and risks associated with proposed diagnostic procedures and, where appropriate, about the possibility of using other methods. Furthermore, according to Article 12, the use of AI in treatment may only occur after the following conditions are met: informing the patient that artificial intelligence will be used in the diagnosis or therapeutic process; obtaining the patient’s informed consent to the use of artificial intelligence in the diagnostic or therapeutic process; and using AI algorithms that are approved for medical use and have the appropriate certifications. However, the final decision always rests with the physician.

A government draft legislation is currently being prepared, which will be designed to adapt the national legal system to the requirements imposed by the AI Act. The government’s proposals primarily envisage the establishment of the Artificial Intelligence Development and Security Commission, which will oversee the AI market within the scope specified in Article 2 of Regulation 2024/1689. The second main body will be the President of the Personal Data Protection Office (UODO) that will oversee high-risk AI systems, including those related to healthcare.

Summary

Processing personal data for healthcare purposes, additionally supported by artificial intelligence, is undoubtedly a convenient and practical solution, but it is associated with a number of legal obligations intended to ensure the security of the data used (e.g., using the acquired data only for a strictly defined purpose), the security of patients themselves (e.g., the obligation to inform the patient of the intention to use artificial intelligence in the treatment process), or simply related to formalities (e.g., the requirement to register the artificial intelligence system in an EU database). Currently, EU regulations are much more detailed in this matter.

Artykuł National Healthcare and the processing of personal data by means of AI pochodzi z serwisu KIELTYKA GLADKOWSKI LEGAL | CROSS BORDER POLISH LAW FIRM RANKED IN THE LEGAL 500 EMEA SINCE 2019.

Siri, Cortana, Google and other applications use human voice to do a variety of things, e.g. searching for information, sending emails, calling somebody. Voice-based technologies are increasingly applied in legal environment and legal services, for example in legal advice rendered online and in legal translations. At the same time, new applications of innovative technologies caused the necessity to define the approach to privacy issues anew. The cases of Edward Snowden and Julian Assange showed us how meaningful privacy and its protection is and made us realize the excessive amount of personal data processed and stored daily. This is why privacy and its protection will soon become one of the most important personal rights. The issue of voice protection comes to the fore in this context. Voice is, obviously, a personal right. What is more, voice is becoming a tool used by most applications both for mundane activities as well as more complex ones, like ROSS AI operating on IBM’s Watson, which can do legal research and is learning to understand law with every research conducted by it. What if it was possible for such applications as Watson to use the voice of a specific lawyer and, with the use of voice sample, produce speech of a different content, for example in the form of legal advice? Well, practically it is possible, since last November Adobe presented Adobe VoCo to the world, which (when having a voice sample) is able to read various content differing from the conent sampled. The present article will try to shed some light to the issue of the risk involved with voice cloning technology in legal environment and will analyse whether law can adequately protect human voice as a personal right.

Background of Voice Cloning Technology

Voice cloning technology is based on copying and reusing of recorded speech. In the future, such software will be able to record voice samples and, afterwards, produce an infinite number of combined syllables leading to an unlimited number of sentences, without the participation of the human being that provided the voice sample. In reference to the latest developments, we may be first to witness the creation of such software for commercial use. The first project worth mentioning is Google Deep Mind’s WaveNet. It is a deep neural network for generating raw audio waveforms, including speech and music. WaveNet has outperformed other text-to-speech systems, but this product has not yet been declared to be available for consumers.[i] From this point of view, it is of importance to mention Adobe Project VoCo, presented during the Adobe MAX 2016 Sneak Peeks. It is a software which is able to create a voice model of the speaker from an earlier given voice sample of 20 minutes duration by said speaker.[ii] VoCo can construct new words and sentences which did not occur in the provided recordings.[iii] Such potential of said software, with the plans to release VoCo to the consumer market, raises considerable concerns, also legal ones, in respect of data and privacy protection.

Voice as a Personal Right and its Protection in Polish Jurisdiction

In order to specify if the European or the Polish law can adequately protect the use of voice technology-based applications and word-building software, we need to indicate the legal status of the human voice at first. From the legal point of view, the human voice should generally be classified as a personal right and, more specific, as a non-pecuniary property of every human being connected with his individual existence, which is effective against everyone, inalienable and not inherited. Polish provisions (art. 23–24 of the  Polish Civil Code) include a sample and open catalogue of personal rights and their protection, irrespective of other regulations. Polish jurisprudence and the majority of law practitioners[iv] express approval for the most essential judgement in this regard, which has been delivered by the Polish Court of Appeals in Gdańsk on 21 June 1991 (case citation: I ACr 127/91, LEX), where the Court acknowledged that the voice shall be regarded as a personal right (as defined in art. 23 of the Poish Civil Code)[v] and protected pursuant to art. 24 of the Polish Civil Code.[vi] Voice serves the same purpose as a human image, namely: identification. It is an element of appearance, given that it relates to individual voice alteration, pitch, sound and the ways someone speaks, i.e. intonation and characteristic words. The violation of this right could occur, e.g., by duplication of voice records or their modification and, what is more, by imitation of distinctive voices, if it could be demonstrated that the above mentioned use was intended to deceive listeners in regard to the identity of the person speaking.[vii] In case of acknowledging that recognizing a person within the sphere of a sound is possible just as well as through an external image, principles related to images apply analogically to voices, provided that the voice is protected as a separate personal right.[viii]

In accordance with the latter provision, the one whose personal right is threatened by the activity of third party may demand this activity to cease, unless it is legitimate (art. 24 par. 1 of the Polish Civil Code). Nevertheless, there are also legal experts who intend to recognize the voice not as a separate personal right but rather as a part of human image or as «audio-image» / «sound-image» that makes it possible to identify a person by sense of hearing. If this view is adopted, then the voice is protected not only on the basis of the Polish Civil Code but also within the framework of copyright law (art. 24 par. 3 of the Polish Civil Code). 

The protection system of the personal rights should be deeply analysed in regard to new technologies based on the use of human voice, since new ways of using (i.e. for online legal advice) or modifying it (i.e. in order to circumvent voice recognition technologies used by banks while making payment orders) could not be protected adequately enough.

Under the Polish Civil Code, the conditions for legal protection of the voice as a personal right are to be viewn as a breach or threat of a breach of personal rights and unlawfulness of such breach or threat. The person who provides his or her voice may therefore demand, amongst others, that the consequences of said breach are removed and that monetary compensation is paid under this title. In this context, the controversy arises, whether – given the situation that a person voluntarily and in consent provides a voice sample – the element of unlawfulness can be demonstrated when the specific software clones the voice in an unintended manner. Accordingly, the open question is whether the means mentioned above provide sufficient protection in this respect. It appears, that nowadays new technologies use subjects of personal rights (protected by given legal methods) in pioneer ways, so that the effects of those activities require new concepts, i.e. applications editing an attorney’s voice (such as VoCo by Adobe) could be used for providing unfounded legal advice and therefore we not only deal with a breach of the personal right related to the voice but also related to the image, scientific activity, freedom of conscience or other implied legal consequences. On the other hand, VoicePass technology, constructed by the Polish University of Science and Technology in Cracow, which is able to identify our voice and allows to verify our identity i.e. in banks, insurance offices or authority bodies[ix] is not only a great invention and simplifying various official procedures but also a potential risk of violating our personal data.

Voice Cloning in the Light of Penal Liability (in the Polish Copyright Law)

It has to be considered that manufacturers of computer programs which allow voice cloning will provide adequate protection in the form of tags, digital watermarks or any other forms, so that it can be showed that somebody’s voice being used in bad faith has been created by the program. But what if somebody circumvents effective technical devices applied to protect the software in order to remove digital watermarks and to use somebody’s voice unlawfully? This has to be viewed as cracking and the accountable person can be treated as cracker or hacker. Polish Copyright Law indicates penalties in its art. 118 para. 1, stating that «anyone who produces devices or components of devices for the purpose of unauthorised removal or circumvention of effective technical devices applied to protect a work or the subject matter of related rights against replaying, copying or reproduction or trades in such devices or components of such devices, or advertises their sale or rental, is liable to a fine, restriction of personal liberty or imprisonment for up to 3 years». In turn, para. 2 of said article sets forth that «anyone who owns, stores or uses devices or components of devices as referred to in paragraph 1, is liable to a fine, restriction of personal liberty or imprisonment for up to a year».

First of all, the term «effective technical devices» must be clarified: it means that the introduced technical security is objectively capable of fulfilling its function and – without it being removed or bypassed – replaying, copying or reproducing are impossible.[x] The problem arising from the wording of the quoted legal provision concerns computer programs and whether this provision also applies to computer programs striving for illegal neutralization of security. It is worth pointing out that computer programs are not devices, since, according to the Polish Languages Dictionary, a device is a mechanism or a set of mechanisms performing specific actions,[xi] meaning that devices must be material and, apart from that, computer programs constitute intangible rights. In the literature, it is proposed that computer programs may be, at most, treated as components of devices.[xii] This is a significant issue, since the removal or circumvention of the effective technical devices applied (in this case, a voice cloning computer program) is usually performed by special computer programs and the appropriate interpretation will decide whether art. 118 para. 1 applies in this regard.

Voice Cloning Software and its Risks

It seems that the main anxiety in this area is connected with the use of audio recordings as an evidence in court. Obviously, audio recording could be used as a valid evidence in the course of litigation under the Polish jurisdiction.[xiii] Some restrictions apply to recordings acquired illegally but a general rule states that such recordings are also admitted in court if they support reaching a fair ruling.[xiv] The software enabling the creation of statements which sound, for example, like the defendant, can cause a considerable risk for the fairness of the trial. Accordingly, one of the ideas to provide protection against fake statements generated by means of voice cloning technology is adding audio watermarks to every output of such software. Digital watermarking is the process of imperceptibly embedding watermarks into digital media as a permanent sign to assure its authenticity.[xv]

Data Protection in the Light of Voice Cloning

Voice cloning technology requires the obtained data to be saved, therefore it is necessary to also look at this new technology from a personal data protection point of view. In the European Union, this issue is regulated by a number of directives, i.e. the Data Protection Directive (95/46/EC), the Telecommunications Act of 16 July 2004 (unified text of 2016, item 1489 as amended) or the Electronic Communications Data Protection Directive (2002/58/EC). In spite of sealing personal data protection (not only internationally but also on a national level), multiple problems occur in practice, e.g., when the data controller entrusts data to countries with insufficient data protection standards. Sufficient data protection standards shall be assessed in the light of all circumstances surrounding a data transfer operation, in particular, the nature of data, its purpose and the duration of the proposed processing operation. According to the Polish Data Protection Act of 29 August 1998 (unified text Journal of Laws of 2016, item 922), the above stated doubts arise whenever personal data is transferred to a country not belonging to the European Economic Area. However, the issue of voice filing as personal data requires a more extensive description and exceeds the scope of this paper.

Voice Biometrics vs Voice Cloning

Voice cloning technology differs from voice biometrics technology, since the latter is a technology used to identify people by their voices. Biometrics refers to metrics related to human characteristics. Nowadays, this technology is being used increasingly, especially in matters of security (e.g., at the airport, where one can choose the facial recognition system instead of the traditional way of checking in).

The human voice is as unique as fingerprints. Moreover, everyone articulates sentences in an original way: one puts emphasis differently, the rate of speech and the intonation are varying. The system records and picks out all the differences while taking into consideration details such as the size and the shape of throat, mouth cavity, nasal cavity, length and tension of vocal cords. Every recorded voice print is stored as a mathematical model. To avoid mistakes during voice recording, the commands are dictated by a speech synthesizer. The verification process consists of comparing samples of recordings to previous recordings. The said systems are currently equipped with technologies removing ambient noise, and can therefore recognize voices in most cases.

Companies using voiceprint checks to verify their customers are at risk of voice cloning technologies, too. It is said that biometric systems would not be tricked by this, as the inspected items differ from what humans look for when identifying people.[xvi] The authors of the VoicePIN, a new startup from Poland, claim that their product based on voice authentication is resistant to spoofing and can detect whether the sample is original or re-played.[xvii] If such assumption is correct, it seems reasonable that the biometric system could likewise be protected from voice cloning software. The final answer will be known only after specific tests and experiments.

The above remarks lead to the conclusion that voice cloning technology may cause new types of legal liability, both civil and penal, of the entities applying this technology. Consequently, voice cloning, like any other new technology, will involve the need to amend and adjust the existing legal provision. Nevertheless, these should not restrict the application of this technology in areas like providing services, e.g., legal advice and legal translation. The said changes are particularly required in the area of administrative law when defining the authority and supervisory competence of entities protecting personal data. Moreover, an important postulate would be to precisely define human voice as a specific personal interest. Furthermore, an unauthorised modification of such voice by means of computer devices should be classified as a specific type of infringement of human voice as a personal interest. Nevertheless, despite the existence of potential risks, when properly safeguarded by legal provisions, voice cloning software can indeed influence the effectiveness and cost-efficiency of legal services positively.

[i] Aaron van den Oord / Karen Simonyan / Nal Kalchbrenner / Sander Dieleman / Oriol Vinyals / Andrew Senior / Heiga Zen / Alex Graves / Koray Kavukcuoglu, WaveNet: A Generative Model for Raw Audio, 19 September 2016, www.arxiv.org/pdf/1609.03499.pdf (all internet addresses last accessed 18 April 2017).

[ii] Official live presentation during the Adobe MAX 2016 Sneak Peeks, co-hosted by Jordan Peele, www.youtube.com/watch?v=I3l4XLZ59iw.

[iii] Sebastian Anthony, Adobe demos «photoshop for audio,» lets you edit speech as easily as text, arsTECHNICA, 11 July 2016, www.arstechnica.com/information-technology/2016/11/adobe-voco-photoshop-for-audio-speech-editing.

[iv] Janusz Barta / Ryszard Markiewicz / Andrzej Matlak, Media Law, LexisNexis, Warsaw 2005; Justyna Balcarczyk, The right to image and its commercialization, Oficyna Wolter Kluwer Business, Warsaw 2009, pp. 52–54; Justyna Balcarczyk, Voice right – outline of basis issues, Zeszyty Naukowe Uniwersytetu Jagiellońskiego 2010/2/115–126, LEX; Maksymilian Pazdan, Commentary on Article 23 of the Civil Code, in: Krzysztof Pietrzkowski (ed.), Civil Code. Commentary on Articles 1–449[10], Volume 1, Legalis.

[v] Art. 23 of the Polish Civil Code dated on 23 April 1964, Journal of Laws No 16.94 as amended.

[vi] Art. 24 of the Polish Civil Code dated on 23 April 1964, Journal of Laws No 16.94 as amended.

[vii] Małgorzata Pyziak-Szafnicka / Paweł Księżak, Civil Code – Comment. General Part. Edition II. LEX, 2014.

[viii] Justyna Balcarczyk,  The right to image and its commercialization, Oficyna Wolter Kluwer Business, Warsaw 2009, pp. 52–54.

[ix] Polish Press Agency, You know your neighbour by his voice, 31 March 2014, http://naukawpolsce.pap.pl/aktualnosci/news,399802,poznasz-blizniego-po-glosie-jego.html.

[x] Zbigniew Ćwiąkalski, Commentary on Article 118(1) of the Copyright Law, in: Barta Janusz / Markiewicz Ryszard (eds.), Copyright Law. Commentary, Volume 5, LEX no. 8545, 2011.

[xi] Polish Language Dictionary, http://sjp.pwn.pl/sjp/urzadzenie;2533403.html.

[xii] Janusz Raglewski, Commentary on Article 118(1) of the Copyright Law, in: Damian Flisak (ed.), Copyright Law. Commentary, LEX no. 9083, 2015.

[xiii] Article 308 §1 of Polish Code of Civil Procedure of 17 November 1964, Journal of Laws 2016.1822 as amended.

[xiv] Resolution of the Supreme Court of 22 April 2016, ref. no. II CSK 478/15.

[xv] Yiqing Lin / Waleed H. Abdulla, Audio Watermark: A Comprehensive Foundation Using MATLAB, Springer, 2014, ISBN: 9783319079745.

[xvi] British Broadcasting Corporation, Adobe VoCo «Photoshop-for-voice» causes concern, 7 November 2016, www.bbc.com/news/technology-37899902.

[xvii] Information given by CEO on VoicePIN in the interview for Business Insider, 28 March 2017, www.businessinsider.com.pl/technologie/nowe-technologie/voicepin-zabezpieczenia-biometryczne-thing-big-upc/l20w4f3.

Artykuł Voice Cloning as a Global New Technology and its Challenges for EU and Polish Law pochodzi z serwisu KIELTYKA GLADKOWSKI LEGAL | CROSS BORDER POLISH LAW FIRM RANKED IN THE LEGAL 500 EMEA SINCE 2019.