To combat the proliferation of deepfake videos on the internet, Dr. Khaled Salah, Professor of Electrical and Computer Engineering, with Haya Raed, as part of her MSc thesis work, investigated the use of blockchain and smart contracts for her MEng in Computer and Electrical Engineering.
The paper for this research was published in April 2019 in IEEE Access and is currently ranked within the top 50 most popular papers published by the Institute of Electrical and Electronics Engineers (IEEE), with 4,074 views, increasing exponentially every month.
Deepfakes are manipulated digital media produced using artificial intelligence technique to create fabricated images and sounds that appear to be real . They often combine and superimpose existing media onto source media using machine learning techniques. As artificial intelligence (AI) applications make their way into mainstream use, deepfake video production has become commonplace.
The deepfake video of the former US President Barack Obama that went viral in April 2018 caught Haya Raed’s attention and served as a catalyst to her research. Modified video footage of Obama depicts him mouthing the words contained in a separate audio track and highlights the potential of deepfakes to undermine truth, confuse viewers, and accurately fake reality.
“With the rise of artificial intelligence and deep learning techniques, fake digital content has proliferated in recent years,” said Raed. “Fake footage, images, audio and videos (known as deepfakes) can be a scary and dangerous phenomenon and have the potential to alter truth and erode trust by virtue of false reality.”
Although academic interest related to deepfakes dates back to 1997, deepfake came into the mainstream in 2017, starting with a group of social media users employing AI to swap the faces of celebrities with other film characters. This may have been an innocuous start to the use of such technology, but as artificial intelligence applications have made their way into the hands of the average person, such content quickly took on a far more nefarious quality. In some early examples of deepfakes, a large number of famous political leaders, actresses, and other celebrity figures had their faces weaved into indecent videos. More worryingly, as much of today’s news is circulated on social media, deepfakes can have an enormous impact on public opinion and democratic outcomes.
“With the advent of social networks, proliferation of such content can be unstoppable and can potentially exacerbate problems related to misinformation and conspiracy theories,” added Dr. Salah. “Hence, there is an immense need for a Proof of Authenticity (PoA) system for online digital content to identify trusted published sources and therefore be able to combat deepfake videos, audio, and images.”
There are real-world solutions available to prove the authenticity of physical works. A certificate of authenticity is often provided with the purchase of a piece of art or a luxury item, but it is possible to forge this or to find the item unsigned from a known and trusted authority. Many times, when an item is bought on a secondary market, the onus is on the buyer to prove authenticity, leaving them with substantial manual work and risk. For digital media online, there are no established methods to determine authenticity and subjecting online content to a certificate of authenticity would be unfeasible.
“Deepfake videos are very realistic and make use of deep learning techniques with large samples of video images used to achieve face swapping,” explained Dr. Salah. “The higher the number of samples, the more realistic the outcome. The Obama video was created from more than 56 hours of sample recordings to make it extremely real and believable.”
“It is crucial to have techniques to detect, fight, and combat deepfake digital content that may include fake videos, images, paintings, audio, and so on. Achieving this purpose is not difficult if there is a credible, secure and trusted way to trace the history of digital content. Users should be given access to a trusted data provenance of the digital content, and be able to track back an item in history to prove its originality and authenticity.”
Blockchain technology offers an immutable and tamper-proof ledger of data and transactions as a shared database, validated by a wide community. Each record created forms a block, and as each block is confirmed by the community, it is paired up with the previous entry in the chain, creating a chain of blocks. Blockchain could be used to prove authenticity and originality of digital media in a way that is decentralized, trusted, and secure, with tamper-proof records, logs, and transactions.
“Proof of authenticity of digital media is critical to help eradicate the epidemic of forged content,” explained Raed. “Current solutions lack the ability to provide history tracking and provenance of digital media. Our solution uses Ethereum smart contracts to trace and track digital content to its original source, even if the digital content is copied multiple times.”
The solution designed by Dr. Salah and Raed uses an Ethereum blockchain-based solution that establishes authenticity of digital content by providing credible and secure traceability to a trusted artist or publishing source. The main aim of the solution is to assist a user in tracing back a video with multiple versions to its origin. If a video cannot be traced to its original publisher, then it cannot be trusted.
“The smart contract uses the hashes of the interplanetary file system (IPFS) used to store digital content and its metadata,” said Dr. Salah. “Our solution focuses on video content, but the solution framework is generic enough to be applied to any other form of digital media. Our solution relies on the principle that if the content can be credibly traced to a trusted or reputable source, the content can then be real and authentic.”
The Ethereum platform also allows for the development of a decentralized application which can automate the authenticity process, or integrated within video players or a web browser to indicate authenticity of played or displayed content. Every video in the system is associated with a smart contract that points to its parent video, and every parent video is linked to its child, in a hierarchical fashion. Smart contracts are simple programs that are stored on the blockchain, inheriting the immutable and distributed properties from the blockchain host.
In the solution, the owner of a video first creates a smart contract where other artists can request permission to edit, alter or distribute the content. The secondary artist requests permission, the request is assessed by the original artist, and the result is then announced. Once an artist gets approval, they create a child contract similar to the original contract and update the parent’s information. Hence, both contracts point to each other and the chain between the videos is complete. The original smart contract can handle multiple requests at the same time and multiple different requests by the same artist.
“It is important that all transaction history as well as the provenance data available for the users to track and trace a video is tamper-proof,” said Dr Salah. “Our solution ensures the integrity of all the events and logs by storing it in the immutable blockchain infrastructure. Moreover, a video’s integrity is also maintained by storing it on the IPFS distributed servers and only storing the hash in the smart contracts. Any change to the video will lead to a new hash that will not match the hash in the smart contract. Consequently, the video content on the blockchain is tamper-proof as well as the reputation of the creator.”
Future work focuses on developing front-end decentralized applications for users to automate establishing authenticity of published videos. Dr. Salah and Raed also plan to develop a pluggable application to provide traceability and establish authenticity when playing or displaying videos within a web browser, and is also designing a fully functional and operational decentralized reputation system.
“The full code of the smart contract has been made publicly available at Github,” said Raed. “I wanted to contribute to the scientific community and push science forward. Sharing the code helps accelerate innovation and research.”
Jade Sterling
News and Features Writer
11 December 2019
