Detecting AI fingerprints: A guide to watermarking and beyond

Over the last year, generative AI tools have made the jump from research prototype to commercial product. Generative AI models like OpenAI’s ChatGPT and Google’s Gemini can now generate realistic text and images that are often indistinguishable from human-authored content, with generative AI for audio and video not far behind. Given these advances, it’s no longer surprising to see AI-generated images of public figures go viral or AI-generated reviews and comments on digital platforms. As such, generative AI models are raising concerns about the credibility of digital content and the ease of producing harmful content going forward.

Against the backdrop of such technological advances, civil society and policymakers have taken increasing interest in ways to distinguish AI-generated content from human-authored content. The EU AI Act contains provisions that require users of AI systems in certain contexts to disclose and label their AI-generated content, as well as provisions that require people to be informed when interacting with AI systems. The National Defense Authorization Act (NDAA) for Fiscal Year 2024 has provisions for a prize competition “to evaluate technology…for the detection and watermarking of generative artificial intelligence.” It also has provisions for the Department of Defense to study and pilot an implementation of “industry open technical standards” for embedding content provenance information in the metadata of publicly released official audio/video. Last fall, Senator Ricketts introduced a bill requiring all AI models to watermark their outputs. Perhaps most prominently, the White House announced last summer that it had secured voluntary commitments from major AI companies to develop “robust technical mechanisms to ensure that users know when content is AI generated,” such as watermarking or content provenance for audio/visual media. The subsequent White House Executive Order on AI directs the Department of Commerce to identify and further develop standards for labeling AI-generated content, and at the UK AI Safety Summit, seven leading AI companies announced company policies on “identifiers of AI-generated material.”

The motivations for such interest are diverse, including limiting the proliferation of spam, targeted deception, and scams; suppressing non-consensual pornography and targeted harassment; limiting misinformation/disinformation; safeguarding the future of education, admissions, and hiring; confirming authenticity of content for legal proceedings and public comment; maintaining public trust in media; and even maintaining the ability to improve future generative AI models by ensuring that they are not trained on AI-generated content since this can hurt their performance.¹ To be sure, the usefulness of AI detection tools will vary with the domain. In particular, some scholars have argued that for concerns like disinformation or spam, content distribution might be more of a bottleneck than content generation; after all, tools to generate synthetic audio/visual media (like Photoshop or Instagram filters) have been around for a while. Nevertheless, generative AI does significantly reduce barriers to generating large volumes of synthetic media across many domains. Thus, AI detection tools can serve as one part of a larger toolbox to help platforms track and limit the distribution of spurious AI-generated content and individuals assess the credibility of digital content.

There are several ideas for how to tell whether a given piece of content—be it text, image, audio, or video—originates from a machine or a human. This report explores what makes for a good AI detection tool, how the oft-touted approach of “watermarking” fares on various technical and policy-relevant criteria, governance of watermarking protocols, what policy objectives need to be met to promote watermark-based AI detection, and how watermarking stacks up against other suggested approaches like content prove