Deepfakes and AI Disinformation — How Generative Models Threaten the Information Ecosystem

Cos'è: An analysis of concrete cases of AI-generated disinformation in the 2024 elections — the largest electoral cycle in history with 4 billion people called to vote — and the regulatory and technological responses deployed, with an honest assessment of what works and what does not.

Biden Robocall in the New Hampshire Primary: The First Political Case at Scale

On 21 January 2024, two days before the New Hampshire Democratic primary, tens of thousands of Democratic voters received an automated phone call with Joe Biden's cloned voice. The message, convincingly constructed with the president's characteristic inflections and linguistic patterns, told listeners not to participate in the primary: 'Saving your vote for the November election is more important. Don't vote in Tuesday's primary.' The operation had multiple objectives: to suppress turnout in the primary among those who might support Biden, and to test the vulnerability of the democratic information system to voice cloning. The person responsible for the operation was identified months later as Steve Kramer, a political consultant working for candidate Dean Phillips' campaign; the technical provider was an AI voice cloning startup. The FCC subsequently established that AI-generated robocalls are illegal under the Telephone Consumer Protection Act, in a decision with significant regulatory implications for all uses of synthetic voices in non-consensual communications. The technical cost of the operation had been trivial: cloning a public voice like Biden's requires a few minutes with commercial tools available for less than $50 per month.

The Fake Taylor Swift Endorsement: 45 Million Views and the Viral Scale Problem

In January 2024, deepfake images of Taylor Swift — some of an explicitly sexual nature, others portraying her expressing a political endorsement for Donald Trump — circulated massively on X (formerly Twitter), reaching an estimated 45 million views before being removed. The speed of dissemination far exceeded the platform's moderation capacity: despite X having policies against non-consensual deepfakes, the images remained accessible for hours after the first reports. The case had immediate political consequences: the White House pronounced explicitly asking Congress to legislate against deepfakes, Taylor Swift subsequently published an authentic endorsement for Kamala Harris in September 2024, and the case accelerated the urgency of the DEFIANCE Act (Disrupt Explicit Forged Images and Non-Consensual Edits), passed in the US Senate in July 2024, which criminalizes the creation and distribution of non-consensual sexual deepfakes. The pattern is systematically the same in every viral disinformation case: the false content reaches its audience before any fact-checking or removal mechanism can intervene. The speed of deepfake production and distribution structurally exceeds the speed of verification.

EU AI Act Art. 50 and the Mandatory Watermarking Problem

The European Union incorporated into the AI Act, formally entering into force in August 2024, specific obligations for AI-generated content. Article 50 establishes that AI systems generating synthetic content — audio, video, images, text — must label the produced material as artificially generated in a machine-readable way, so that it is detectable by automated tools. Providers of general-purpose AI systems with content generation capabilities must implement technical watermarking and require deployers to disclose to end users when content is AI-generated. However, digital watermarking for audiovisual content is technically circumventable: aggressive JPEG compression, low-quality video re-encoding, or a simple smartphone screenshot remove most invisible watermarks based on steganography. Academic research (Stanford, MIT) shows that watermarks in synthetic content rarely survive common transformation in the social media sharing cycle. The regulatory obligation creates a compliance signal for responsible providers, but is not a technically reliable mechanism for authenticity verification.

C2PA and the Coalition for Content Provenance: The Technical Standard for Media Provenance

A more robust approach to watermarking is the Coalition for Content Provenance and Authenticity (C2PA), a consortium founded in 2021 by Adobe, Microsoft, Intel, BBC, CBC and Truepic, today with over 100 members including camera manufacturers like Canon and Nikon. C2PA defines an open standard for embedding cryptographically signed metadata in media files: every image, video or document can carry a signed manifest declaring the origin (camera, editing software, AI model) and the chain of modifications. Unlike steganographic watermarking, the C2PA cryptographic signature is independently verifiable by anyone with access to the unaltered original file — but like any cryptographic system, it guarantees file integrity from origin to the verification point, not after lossy conversions. Adobe has integrated Content Credentials (the C2PA implementation) into Photoshop, Firefly and Premiere Pro. Canon and Nikon are integrating signing into the cameras themselves, so every RAW shot carries a signature attesting 'this file came from this sensor at this moment'. The limitation remains the distribution chain: most social platforms do not preserve EXIF metadata or C2PA manifests during the compression and re-encoding process.

The Paradox: AI Used Both to Create and Detect Deepfakes. What Really Works?

2024 confirmed a structural paradox: the same advances that make deepfakes more convincing also feed the detectors. Microsoft, Google, Meta and Anthropic have all invested in detection systems that search for visual artifacts (anomalies in eye-mouth transitions, inconsistencies in light reflected in the cornea, anomalous compression patterns) or acoustic ones (unnatural frequency spectrum, absence of vocal micro-jitter) typical of synthetic content. Academic benchmarks show detection accuracy above 85-90% in controlled conditions. In real conditions — compressed content, shared on social media, with background noise — accuracy typically drops below 70%, with a false positive rate that would make any automatic flagging system unusable on platforms with billions of users. The arms race between generation and detection is fundamentally asymmetric: generating a convincing deepfake becomes progressively cheaper and more accessible; detecting it reliably requires access to the high-quality file and conditions that rarely occur in viral distribution. Empirical evidence from the 2024 elections — India (the world's largest democracy with 970 million voters), UK, USA, European elections — suggests that the most effective response is not technological but behavioral: media literacy programs that teach users to verify primary sources before sharing, to recognize patterns of coordinated disinformation, and to develop a healthy suspension of judgment faced with emotionally charged content. Not because technology is irrelevant, but because the critical threshold of viral spread is reached before any technical system can intervene systematically.

---