What Is a Deepfake? (Definition, How to Spot One) | Built In (2024)

While the idea of swapping faces, or creating synthetic media, with photo-editing software is not a new one, what makes the technique behind deepfakes so important is that it allows almost anyone to produce the same thing without learning any artistry, and on a much larger scale.

Related Reading From Built In ExpertsHow to Spot Deepfake Technology

How Does a Deepfake Work?

Deepfakes use advanced deep learning techniques to first encode features, then reconstruct images from the encoded features. Autoencoders, a type of neural network, are the most commonly used deep learning architecture for creating deepfakes. Let’s explore this through an example of using the deepfake technique to swap faces of two people. This is shown in detail in the figure below.

The first step to producing a deepfake is transforming the face images into smaller feature-based representations using an encoder. This more information-rich representation is often referred to as the latent face. The latent face will contain representations for features such as the nose shape, skin tone and eye color. We use the same encoder for each person so the representations produced have the same meaning.

We then transform the latent face back into an image using a decoder. Depending on which images we use to train the decoder, the output face image will vary. The key part for face swapping is that the decoder for person A is applied on the latent face of person B, and vice versa. In this way, the output face will have the expression and structure of person A, but the style and look of person B.

This figure shows the deepfake process through training (top two rows) and the creation of a deepfake (bottom row). In the training phase, the encoder (blue) is shared between different images to create the latent faces (middle portraits) from the input images (left portraits).

Individual decoders are trained for each person (green and pink). In the creation phase, the same encoder is used, but the decoder for a different person is used which leads to the creation of an image which has the pose and expression of the input image but the style of the person used to create the decoder.

Are Deepfakes Only Videos?

While deepfakes are most often presented in videos, they can be found in photos and even audio.

The key applications of deepfake technology include:

• Face swap: Face swap is the most common and obvious technique, where the faces of two people are swapped. This is generally to produce an image of a celebrity in a scene they weren’t in.
• Face synthesis: Face synthesis stretches reality and generates a face for a person that never existed.
• Facial attributes and expression manipulation: In attribute and expression manipulation a face is altered by either changing specific features such as the eyes, or by altering their expression, for example, turning a frown upside down.

How to Spot a Deepfake

With the wide availability of deepfake generation tools, it’s important that everyone has a basic understanding of how to spot a deepfake. In fact, companies including Google, Amazon and Meta have been actively encouraging the community to analyze and understand what gives a deepfake away. Through their and others’ research, they have discovered a variety of ways to detect a deepfake:

• Unnatural face, environment or lighting: Deepfake images or sections of videos can have unnatural facial expressions, facial feature placement or jagged edges. The environment itself (such as the lighting) can also be unrealistic.
• Unnatural behavior: In deepfake videos there must be continuity between images, but this is difficult to implement. As a result, you might spot unnatural behaviors such as uneven blinking or choppy motion.
• Image artifacts and blurriness: Deepfake images may have weird artifacts such as blurriness around the neck where the body of one person is stitched together with the face of another.
• Audio: When deepfakes are combined with audio, the lips may follow an unexpected motion compared to what you would expect from the audio.

How Do You Make a Deepfake?

Creating deepfakes is surprisingly easy. There exist a wide array of smartphone apps that make deepfake production readily accessible toa majorityof the population. In addition to apps, there exist computer programs that allow a person to run far more advanced reproductions on local CPUs (computer processing unit), or the best reproductions on GPUs (graphics processing unit).

Some of the most popular tools are:

• FaceApp: FaceApp transforms photos to add and remove features, thereby allowing users to craft celebrity-like photos in just a few clicks.
• Wombo: Wombo creates singing and dancing video clips from users’ uploaded photos.
• DeepFaceLab: DeepFaceLab PC software is used in the generation of 95 percent of deepfake videos.
• Deepfakes Web: Deepfakes Web is a cloud-based application that can realistically swap out faces of people in videos.

More on Deepfakes on Built InDeepfake Phishing: Is That Actually Your Boss Calling?

What Are the Risks of Deepfakes?

As with many technological innovations, deepfakes have had both positive and negative effects.

Research has estimated that 98 percent of deepfake videos are created for p*rnography, in which creators produce videos that place celebrity faces onto models. This application has led to critical response, with many countries deciding to ban p*rnographic deepfakes; various websites have stated deepfake p*rnography is against their terms and conditions.

In 2022, Google banned the use of their Colab service for the creation of deepfakes, a previously popular way to train deepfake models. And as of 2024, YouTube requires creators to disclose to viewers when realistic content is made with synthetic media or generative AI, including content “using the likeness of a real person.” Meta will also begin labeling AI-generated images on Facebook, Instagram and Threads as determined by industry-standard indicators starting in 2024. These measures from Google and Meta aim to further reduce the prevalence of these types of deepfakes.

The application of deepfakes in the arena of politics is likely the most controversial. During the 2022 Russian invasion of Ukraine, a deepfake of Russian leader Vladimir Putin showed him surrendering to Ukraine circulated on Twitter. There have also been many deepfakes of U.S. president Donald Trump, including an entire film developed by South Park creators Matt Stone and Trey Parker.

The high quality of deepfakes make it difficult for the general public to discern fact from fiction. Even if a deepfake is later revealed to be fake, the video can already have done significant damage.

Frequently Asked Questions