Controllable Language Generation at Every Scale: Balancing Precision and Resources

Controllable generation tasks, which aim to direct a model’s output to meet specific criteria, are increasingly important in applications requiring privacy, style adaptation, and specific task criteria. Authorship obfuscation is one such task, where the goal is to control text generation to produce fluent, semantically consistent content in a style distinct from the original author. In this talk, I will present two approaches to controllable language generation for authorship obfuscation, each requiring different amounts of computation to implement. First, in a resource-limited scenario, I introduce JAMDEC, an authorship obfuscation method for small language models that employs an unsupervised, inference-time approach with constrained decoding. Next, in a more flexible setting with access to medium-sized open-source models, I present StyleRemix, which uses Low-Rank Adaptation (LoRA) modules to provide fine-grained stylistic control and achieve interpretable obfuscation. Finally, I discuss ongoing work involving large, closed language models, leveraging knowledge distillation to expand controllability for more broad reasoning generation tasks. Together, these methods provide a scalable framework for authorship obfuscation and other controllable generation applications across varying computational resources.