Risks of Implementing ChatGPT in Businesses: Explaining Examples of Confidential Information Leakage and Countermeasures

The implementation of ChatGPT within companies is gradually gaining traction. While its usefulness is drawing attention, there are several points of caution to be aware of. One such point is the importance of not inputting confidential information into ChatGPT. There have been instances abroad where the input of sensitive data has led to significant corporate data breaches.

In this article, our attorneys will discuss the evolving business use of ChatGPT, focusing on the risk of confidential information leakage. We will provide examples and outline the necessary measures to mitigate this risk.

Why You Shouldn’t Enter Confidential Information into ChatGPT

While ChatGPT is convenient, it is an AI chatbot generated by learning from big data and usage data on the internet. Without proper precautions, there is a lurking risk that any entered confidential information could be leaked.

We will delve into the measures to mitigate the risk of confidential information leakage later, but first, let us explain the risks associated with ChatGPT that go beyond the leakage of confidential information.

Risks for Companies Using ChatGPT Beyond Confidential Information Leakage

ChatGPT is currently in the stage of being tested for implementation by many companies. Therefore, it is necessary to understand the risks thoroughly before deciding whether to utilize it for business purposes.

Beyond the leakage of confidential information (including personal information), the security risks for companies using ChatGPT can be summarized in the following two points:

• Risk of credibility in the information output
• Risk of copyright infringement in input/output information

We will explain each of these in more detail.

Lack of Credibility in the Information Output

The GPT-4, released on March 14, 2023, has become capable of providing up-to-date information due to its enhanced search functionality. However, while ChatGPT outputs information as if it were true, its credibility is not guaranteed. The responses generated by ChatGPT are not based on the accuracy of the information in the training data but are produced as the most probable (most likely) text. Therefore, it is essential to perform a fact-check before using the output. If a company inadvertently disseminates false information, it could damage the company’s credibility.

Legal Risks Such as Copyright Infringement

The assessment of copyright infringement in ChatGPT is divided between the ‘AI development and training stage’ and the ‘generation and utilization stage.’ The use of copyrighted works differs at each stage, and so do the applicable articles of copyright law. Therefore, it is necessary to consider both stages separately.

Reference: Agency for Cultural Affairs | Reiwa 5 (2023) Copyright Seminar “AI and Copyright”[ja]

The revised Copyright Law enacted in January 2019 introduced a new provision for the ‘AI development and training stage’ in Article 30-4, which is a limitation of rights provision (an exception where permission is not required). Uses of copyrighted works for information analysis, such as for AI development, that do not aim to enjoy the thoughts or emotions expressed in the works, are generally permissible without the copyright holder’s consent.

On the other hand, if the output generated by ChatGPT shows similarity or reliance (modification) to copyrighted works, it may constitute copyright infringement. Therefore, before publishing, it is crucial to verify the rights holders of the information referenced by ChatGPT and check for any content similar to what ChatGPT has created. When quoting copyrighted works, it is important to properly attribute the source (limitation of rights provision) or, if reproducing, to obtain the copyright holder’s permission for use.

If a copyright holder points out an infringement, the company may be subject to civil liabilities (such as damages, consolation fees, injunctions against use, and legal measures for reputation restoration) or criminal liabilities (prosecutable offenses).

Incidents Arising from Inputting Confidential Information into ChatGPT

On March 30, 2023, the Korean media outlet ‘EConomist’ reported that three incidents occurred at Samsung Electronics’ semiconductor division after the use of ChatGPT was authorized, where confidential information was inadvertently entered into the system.

Despite Samsung Electronics having issued warnings about information security within the company, there were cases where employees sent source code to request program modifications (two incidents) and transmitted meeting details for the purpose of creating minutes.

Following these incidents, the company took emergency measures to limit the upload capacity for each question to ChatGPT. They also stated that if similar incidents recur, they might consider cutting off access to the service.

Furthermore, Walmart and Amazon have been warning their employees not to share confidential information with chatbots. An attorney for Amazon has mentioned that there have already been instances where ChatGPT’s responses resembled Amazon’s internal data, suggesting the possibility that the data might have been used for learning purposes.

Measures to Prevent the Leakage of Confidential Information When Using ChatGPT

OpenAI explains in its terms of use and other documents that data entered for system improvement may be used for training purposes, and they request that users refrain from transmitting sensitive information.

In this section, we will introduce four measures, both hardware and software, to prevent the leakage of confidential information when using ChatGPT.

Developing Guidelines for Internal Use

When implementing ChatGPT within a company, it is essential not only to enhance individual information security literacy and reskill the workforce but also to establish your own ChatGPT usage guidelines.

On May 1, 2023, the Japan Deep Learning Association (JDLA) compiled the ethical, legal, and social issues (ELSI) related to ChatGPT and released “Guidelines for the Use of Generative AI.” Various sectors, including industry, academia, and government, are also beginning to consider the development of their own guidelines.

By referencing these and formulating clear guidelines for the use of ChatGPT within your company, you can expect to mitigate certain risks.

Reference: Japan Deep Learning Association (JDLA) | Guidelines for the Use of Generative AI[ja]

Incorporating Technology to Prevent Leakage of Confidential Information

To prevent human errors that could lead to the leakage of confidential information, the implementation of a system known as DLP (Data Loss Prevention) can help prevent the transmission and copying of sensitive data. This system is designed to safeguard against the unauthorized dissemination of specific data.

DLP continuously monitors input data, automatically identifying and protecting confidential and critical information. By using DLP, it is possible to receive alerts or block actions when sensitive information is detected, thus effectively preventing internal data breaches while keeping management costs low. However, a sophisticated understanding of security systems is required, and smooth implementation may be challenging for companies without a technical department.

Considering the Implementation of Dedicated Tools

Since March 2023, ChatGPT has enabled the prevention of data leaks by utilizing its API (an abbreviation for “Application Programming Interface,” which connects software, programs, and web services). Data sent via the API is not used for learning or improvement, but it is stored for 30 days for “monitoring to prevent misuse or abuse” before being deleted, according to the updated storage policy. It should be noted that the data retention period may be extended in response to “legal demands.”

Even with settings that prevent the use of ChatGPT for learning or improvement, the data is still stored on the server side for a certain period, which theoretically poses a risk of information leakage. Therefore, when entering confidential or personal information, it is crucial to exercise caution.

However, OpenAI prioritizes user privacy and data security, implementing stringent security measures. For those who want to use the service more securely, it is recommended to implement the “Azure OpenAI Service,” a tool capable of advanced security measures.

The “Azure OpenAI Service,” a tool specialized for businesses, ensures that data entered into ChatGPT via its API is not collected. Furthermore, by applying for and passing an opt-out review, it is possible to refuse the standard 30-day data retention and monitoring, thus avoiding the risk of information leakage.

How to Configure ChatGPT to Not Learn Sensitive Information Entered

As mentioned above, since ChatGPT is designed to learn from all opt-in content, a feature to set opt-out preferences in advance has been available since April 25, 2023.

As a direct preventative measure, if you wish to refuse the use of data entered into ChatGPT for learning and improvement purposes, it is necessary to submit an ‘opt-out’ request. ChatGPT provides an ‘opt-out’ Google Form, and it is advisable to complete this procedure. (You will need to enter and submit your email address, organization ID, and organization name.)

However, even in this case, the input data will still be monitored by OpenAI and stored on the server side for a certain period (typically 30 days).

ChatGPT Terms of Service

3. Content

(c) Use of Content to Improve Services

We do not use Content that you provide to or receive from our API (“API Content”) to develop or improve our Services.

Our company does not use the content you provide to or receive from our API (“API Content”) to develop or improve our services.

We may use Content from Services other than our API (“Non-API Content”) to help develop and improve our Services.

Our company may use content from services other than our API (“Non-API Content”) to help develop and improve our services.

If you do not want your Non-API Content used to improve Services, you can opt out by filling out this form[ja]. Please note that in some cases this may limit the ability of our Services to better address your specific use case.

If you do not wish to have your Non-API Content used for the improvement of services, you can opt out by filling in this form. Please be aware that this may limit the capability of our services to adequately address your specific use case in some instances.

Source: OpenAI Official Website | ChatGPT Terms of Service https://openai.com/policies/terms-of-use[ja]

Summary: Essential Measures for Handling Confidential Information in Business Use of ChatGPT

We have explained the risks of confidential information leakage and the necessary countermeasures in the business use of ChatGPT, based on actual examples.

In the rapidly evolving field of AI business, such as ChatGPT, it is indispensable to collaborate with experts to establish internal usage guidelines, assess the legality of business models, create contracts and terms of use, protect intellectual property rights, and address privacy concerns.

Related article: What is Web3 Law? Explaining Key Points for Businesses Entering the Space[ja]

Guidance on Measures by Our Firm

Monolith Law Office is a law firm with extensive experience in both IT, particularly the internet, and legal matters. The AI business is fraught with numerous legal risks, and the support of attorneys well-versed in AI-related legal issues is essential.

Our firm provides sophisticated legal support for AI businesses, including those involving ChatGPT, through a team of AI-knowledgeable attorneys and engineers. Our services include contract drafting, legality reviews of business models, protection of intellectual property rights, and privacy compliance. Details are provided in the article below.

Areas of practice at Monolith Law Office: AI (including ChatGPT) Legal Services[ja]