Text Privacy in the age of Large Language Models

Abstract

Large language models (LLMs) are trained on large text corpora that often contain sensitive information such as medical records, political opinions, and demographic details. This information can be exposed directly from text or indirectly through trained models, creating a need for methods that reduce privacy risks while retaining utility.

This thesis investigates redaction as an approach to sanitising text data and compares it with existing privacy-preserving methods. We propose viewing a text corpora as a probability distribution over sequences of words and redacting words changes the probability distribution. We use the Renyi-divergence divergence as a measure of the distance between two redacted datasets. We show that if enough words are redacted then sensitive redacted text can be made be statistically indistinguishable from non-sensitive redacted text. This provides a basis for developing redaction strategies that minimise the number of words masked while achieving a given privacy target.

The work then evaluates redaction as an alternative to differentially private stochastic gradient descent (DP-SGD). While DP-SGD adds noise to gradients during training, it often leads to a poor privacy and utility trade-off. Experiments show that models trained on redacted datasets achieve improved utility while still protecting sensitive information.

A new redaction methodology is also proposed that uses a non-linear ranker trained with a KL-divergence loss to select words for masking. This technique improves privacy guarantees while requiring lower redaction levels.

Finally, the thesis examines whether LLMs develop internal representations of training data that may themselves encode sensitive information. Through graph reconstruction experiments under in-context and zero-shot learning conditions, it is shown that LLMs can capture graph structure only when explicit context is provided, with no evidence of robust internal graph representations in the zero-shot setting.