Skip to content
post thumbnail
  • Yearender Special

AI supercharges scam production globally in 2024

This is the third consecutive year that VERA Files Fact Check observed the flood of scams in online spheres in the Philippines, which we monitor as part of our third-party fact checking partnership with Meta.

By Celine Isabelle Samson

Dec 28, 2024

9-minute read
ifcn badge

Share This Article

:

There’s always a scam out there. But the tactics used this year by online defrauders have grown to be more intricate and easier to replicate, resulting in a deluge of hoaxes and over a trillion dollars lost to them globally this 2024. At VERA Files Fact Check, scams comprised 81 of 386 fact check articles on online disinformation published from Jan. 1 to Dec. 9 – or about a fifth of the total.

This is the third consecutive year that VERA Files Fact Check observed the flood of scams in online spheres in the Philippines, which we monitor as part of our third-party fact checking partnership with Meta.

In this year-end report, we offer three new findings: First is the rise in the use of artificial intelligence (AI) to facilitate the hustling of hoaxes, second is the tactic of using social media as bait before moving the interaction somewhere else, and third is that this issue is not just a local one but global.

Health product hoaxes and the deepfake dilemma

Of the 81 scam fact checks, 32 were on health fraud scams, which are defined by the United States (U.S.) Food and Drug Administration (FDA) as “products that claim to treat, prevent, or cure diseases or other health conditions, but are not proven safe and effective.”

Health fraud scammers flagged by VERA Files Fact Check advertised fake “Philippine FDA-approved” products that claimed to treat the top killers of Filipinos like heart disease, hypertension (high blood pressure), cancer, and diabetes, among other illnesses. Also popular were pain management medicine for the joints and cures for prostatitis or the inflammation of the prostate gland in males.

VERA Files Fact Check observed that over 80% of these scam posts used deepfake videos of celebrities, social-media-popular physicians, news personalities, and even clerics – who are influential in a country where nearly 80% identify as Catholic. Deepfakes are digital photos, videos and voices of people that have either been synthetically created or manipulated with AI and can be hard to distinguish from the real thing.

For example, fraudulent ads for the products Vitaman and Vitaman Plus – which are neither proven to cure prostatitis nor registered with the Philippine FDA, contrary to their claims – altered existing videos of Dr. Willie Ong, Sen. Manny Pacquiao and broadcast journalist Karen Davila to make it look like they were endorsing the product.

Fraudulent ads for products that claim to cure prostatitis have been using AI to make it look like public figures such as Dr. Willie Ong, Sen. Manny Pacquiao and broadcast journalist Karen Davila are endorsing them.

The deepfakes ranged from low quality fakes where a personality’s mouth movements obviously did not match the audio (such was the case with the Ong and Pacquiao video), to better quality ones where the audio actually sounded like the public figure/s being impersonated and was in sync with their mouth movement.

In August, we flagged a fake ad for a “diabetes cure” with “testimonies” from celebrities Lea Salonga and Sharon Cuneta. There is no cure for diabetes, and they gave no such endorsements. More, scammers used one script for their testimonies, which was an extra red flag because personal experiences cannot be identical.

@verafiles.org

Hindi inindorso nina Lea Salonga at Sharon Cuneta ang isang gamot sa diabetes. Gumamit ang publisher ng artificial intelligence para gayahin ang kanilang mga boses at manipulahin ang kanilang mga video. Music credit: TikTok Creative Center, loop, tap, routine, work, explain by Lill Lilly #factcheck #FactCheckPH #verafilesfactcheck #VERAfied #LeaSalonga #SharonCuneta #fypシ #fyp #scam #ScamAlert

♬ original sound – VERA Files – VERA Files

Another noticeable quality in the deepfakes is that the speech did not sound natural. In the Ong and Pacquiao video, they spoke in a mix of Tagalog and English but the grammar and pronunciations did not make sense in some instances, suggesting that the script they were made to speak was machine-translated from another language (possibly English) and that the video was made with an AI-powered video generator.

Ong: Kumusta sa lahat na nagdurusa sa prostatitis. Manny, sabihin sa amin ang tungkol sa iyong paglalakbay sa paggamot para sa sakit na ito gamit ang Vitaman Plus.

Pacquiao: Maraming mga doktor na sinasabing tumutulong sa iyong kalusugan ay talaga palang nagpapalala nito. Nalaman ko ito ng akoy nagkaprostatitis at lumipat sa ibat ibang eurologist (sic). Akoy nagdusa sa sakit na ito simula noong ako ay trentitisephane at taong gulang at sa loob ng walong taon ay nakibaka ako sa prostatitis at sa mga epekto nito.

Apart from the unnatural phrasing, there were also made-up words such as “trentitisephane”, which was used to refer to Pacquiao’s age. This is a “hallucination” or when AI makes up nonsensical or inaccurate words.

VERA Files Fact Check spoke with Jack Stubbs, chief intelligence officer at Graphika, a U.S.-based social media analytics firm that maps online networks and harms, to talk about the prevalence of scam content in 2024. This month, Graphika launched a report called “Holiday Hoaxes Unwrapped: Fake Trees, Freebies, & Shopping Sprees” which described several types of scams all over the world that sought to defraud people during the November and December holiday seasons.

Stubbs said fake endorsement videos are not a new phenomenon, but the ease and affordability of creating these using AI could have affected the flood of these fake videos now.

“I think the thing to keep in mind about all this stuff is they’re (the scammers) trying to make money. So they’re trying to do the thing as quickly and efficiently as possible. So they will take shortcuts everywhere. And this is what AI is in scams. It’s about scaling and efficiency. How can we do more of this stuff quicker, cheaper, easier?” Stubbs said.

He added: “The capability is not new. It just took more time and was more expensive (in the past). What’s changed is now anyone can do it for, like, two cents and they can do it a hundred times in five minutes.”

Social media as a bait, then takes scam activity somewhere else

All of the scams flagged by VERA Files Fact Check circulated on Meta’s platforms, particularly on Facebook (FB). As of this story’s publishing, a version of the Vitaman Plus ad with yet another deepfake of Dr. Ong is currently live, with 29 copies of the video and caption going around in separate campaigns, according to the Meta Ads Library.

Screencapture shows data from Meta Ads Library featuring posts from a page called Natural Healing, a long caption selling a prostatitis cure called Vitaman Plus, and a video of Dr. Willie Ong.
Twenty-nine ad campaigns for Vitaman Plus using a deepfake of Dr. Willie Ong are live on Meta as of Dec. 28, 2024.

“Advertisers often use the same image or video and text to create ad campaigns with different start dates, locations or budgets,” according to the Ads Library. One particular campaign for Vitaman Plus published on July 31 continues to go around, currently with 4,300 reactions, over a thousand comments and nearly 300 shares.

Those interested to purchase the Vitaman Plus soft gel product are invited to either visit a link to a website (vitamanplus.store) or send a message to the FB page Natural Healing, both of which then present a reader with promotional offers and a form to submit their orders. The forms collect one’s name, full address, the landmark nearest their home, and their mobile number.

These screenshots show the buyer’s form for Vitaman Plus and an automatically generated receipt after clicking “Submit Order”.

After completing the form, a receipt is generated that says the product will be delivered in five to nine days and instructs the customer to prepare P1,499 (~$25) to be paid in cash upon delivery. It is unclear where the information goes, who is behind the scam or if a product actually arrives at one’s doorstep.

According to Graphika’s report: “Consumers who fell prey to the online shopping scams we identified typically failed to receive their goods, received goods not meeting expectations, or were deceived into disclosing their personal information. When scammers managed to obtain personal information, they very likely used it for further targeting.”

Meta removes content and accounts on their platforms that purposefully deceive or exploit others for money. This year alone, they reported taking down over two million accounts linked to scam centers in the Philippines, as well as Myanmar, Laos, Cambodia and the United Arab Emirates.

However, scams observed by VERA Files Fact Check exhibit persistent qualities to evade detection and enforcement from the platform. The Vitaman Plus ad did not include a link to its website in the caption, ostensibly to avoid the link getting flagged. Instead, they displayed the URL throughout the deepfake video. Another tactic observed by VERA Files Fact Check is some FB accounts use link shorteners like Bitly and redirecting links, so that if a platform blocks the URL displayed on the platform, it does not technically block the end website where the URLs lead to.

The “State of Scams in the Philippines 2024” report published by the Global Anti-Scam Alliance (GASA) noted that Filipinos lost nearly P460 billion (~$8.1 billion) to scams this year. It added that Filipinos lost $275 (or about P16,200) each.

Scams are a global issue

As noted in the reports made by Graphika and GASA, the scam problem affects countries worldwide. VERA Files Fact Check first observed an international link to the scams after noticing that some of the products being sold online or the accounts selling them appear to be foreign in nature.

The Vitaman Plus ad with Dr. Ong and Pacquiao, for example, was published by the FB page Ân Sủng của Thiên Nhiên, which is Vietnamese for “Grace of Nature”. It has a +86 contact number listed on the page, which is the area code for mainland China, even though its listed locations for people who manage the page are in the U.S., France and Ukraine.

One Facebook page selling Vitaman Plus appears to have multiple foreign links: Vietnam, mainland China, the United States, France, and Ukraine.

When VERA Files Fact Check asked its colleagues in the international fact checking community about their experiences with scams, 23 organizations across six continents – all except Antarctica – shared that they, too, are dealing with the issue. Many of them have published dozens of fact checks on hoaxes, similar to VERA Files.

Factly, a fact checking organization in India, has also encountered “prostatitis cure” ads that used deepfakes, the main difference being that the public figures impersonated there were prominent Indian personalities.

Fact checkers from Ellinika Hoaxes in Greece, meanwhile, flagged a scam ad for arthritis medication that used the “two testimonies, one script” tactic that we also observed in the Philippines.

In the African continent, PesaCheck – whose fact checking work spans 16 countries – flagged a significant number of fraudulent job ads. Similarly, apart from health fraud scams, other popular types of scams flagged by VERA Files Fact Check concerned fake job advertisements, social benefit scams (e.g. 4Ps, ayuda for PhilSys registrants) and government educational assistance scams (e.g. from DSWD and DepEd). As more Filipinos reported feeling poor this year, the high demand for social aid, educational assistance and job opportunities make it an easy area for scammers to exploit.

Addressing the scam problem needs a multi-pronged solution, says Stubbs.

“You need to be able to come at this from multiple angles. It’s about reducing the volume of scam activity, introducing measures online to help people identify what’s a scam, and ideally make life harder for the scammers. There’s a public awareness education piece so people can operate safely on the internet. And there’s a role for law enforcement to play in terms of identifying and holding people to justice when they’re breaking the law,” he said.

VERA Files has produced resources to help Filipinos avoid getting scammed. You may find them here: Bye victim era, hello scam-free era! Paano hindi ma-fall sa online scams?

Get VERAfied

Receive fresh perspectives and explainers in your inbox every Tuesday and Friday.