How misinformation spreads on social media
Deep fakes, which are manipulated photos and videos, have received a lot of attention in the news media. This Wall Street Journal article talks about deep fakes spreading misinformation and the role of tech startups in fighting them. However, deep fakes are just one facet of misinformation campaigns. Another important facet is how bad actors take a deep fake and distribute it for public consumption. There is one thing in particular that helps them: impersonation of popular news sources such as CNN and WSJ.
Figure 1 shows a breaking news update in the wake of Russian invasion of Ukraine on February 24, 2022. This update was posted by @cnn_live_news account on Instagram. Would you have believed this if you were reading it within minutes of the Russian invasion? Would you believe other news updates from this account claiming to be CNN?
The critical role of impersonation in spreading misinformation has received little attention from news media and tech companies. We see rampant impersonation of mainstream news sources such as WSJ, LA Times, New York Times, CNN on social media platforms such as Instagram. Let’s see how impersonation looks like using data related to WSJ and CNN (current as of March 22, 2022).
Impersonation of news outlets promotes misinformation
You may come across the news updates in Figure 2. Would you believe that Rep. Massie proposed July as White History Month? Would you believe that France surrendered to Russia ahead of possible European war? Would you consider them misinformation and fake news? How would you decide?
It is important to note that many people care about the source of the news. When the news comes from an authoritative source such as WSJ or CNN, people are likely to believe it. They are much more likely to believe deep fakes when they appear to come from a source they trust. Hence, by impersonating authoritative sources, bad actors can spread deep fakes effectively and accelerate the spread of misinformation.
How does one identify a real versus fake news account on social media? Let’s discuss what people are doing today and why it’s not working.
Identifying authentic news accounts is hard
To verify the authenticity of an account on social media, users are advised to look for indicators such the blue checkmark on Instagram and Twitter. Some people may consider the blue checkmark a panacea against impersonation, but the blue checkmark is not that. There are several issues with the blue checkmark.
- Not all social media platforms provide a blue checkmark indicator. LinkedIn for instance does not provide it.
- The criteria set by a platform to get a blue checkmark is subjective and unclear. Authenticity is not the only criterion— uniqueness, popularity etc. also matter.
- Not all businesses can get a blue checkmark; not all news media outlets can get it. It is hard for smaller businesses to get the checkmark because popularity is one of the requirements to get the blue checkmark. So, a national news outlet could get it, but a small local newspaper might not.
- The way social media platforms measure popularity is subjective. We have seen accounts with 10 followers get a blue checkmark, but accounts with 100k followers don’t get one.
- Uniqueness is another requirement to get the checkmark. There can be multiple fan accounts for a brand. By definition fan accounts are not unique, and they can not get the checkmark. However, there are dozens of fan accounts for CNN that share news articles.
Another indicator that’s often used as a proxy for authenticity is the number of followers. However, it is not a reliable indicator of authenticity. We see many instances where a fake account has more number of followers than the real account. Fan accounts can have thousands of followers, but who is monitoring fan accounts for misinformation?
If a high number of followers is supposed to signal trust in an account, then the two CNN accounts in Figure 3 must be believed because they have 98.7k and 23.6k followers. One claims to be the official Uzbekistan account of CNN and the other official USA account of CNN. The latter one also claims to be a fan account. The news articles shared by these two accounts are read by thousands of people, but there is no way for the user to know if they are authentic sources of news.
News outlets are inconsistent about verifying their accounts
We discussed the reasons why it is difficult to get the blue checkmark. However, even when news outlets can get the blue checkmark, they are not consistent in getting it for all their social media accounts. This is a huge problem because it incentivizes people to learn to ignore the checkmark.
CNN and WSJ are national-level news outlets. They could get each of their accounts verified but they are not doing that. Let’s see this with an example. Consider the Chinese, Japanese and Persian accounts of WSJ: @wsjchinese, @wsjjp, and @wsjpersian. Can you guess which one is fake and which one is real?
@wsjchinese is shown in Figure 4. This account claims to be the official Instagram account of the Chinese Wall Street Journal and has 7230 followers. How does an Instagram identify if this is an authentic source of information? Is there a blue checkmark? No.
After much investigation and cross checking with external sources of information, we found this to be an unverified account of the WSJ. However, not every Instagram user would spend that much time or have access to such information to make an informed decision.
Next, see the Japanese WSJ account in Figure 3. The @wsjjp account claims to be the official account of WSJ Japan edition and links to jp.wsj.com. In contrast to the Chinese account, it has a blue checkmark, but only 1772 followers.
An account claiming to be the Persian edition of WSJ is shown in Figure 6. It is unverified and links to wsj.com. With 903 followers, it has about the same number of followers as WSJ Japan.
We know by now that WSJ does not have blue checkmark for all of their accounts. We also know that accounts with few followers can be legitimate accounts. So, is WSJ Persian an authentic account? In the face of inconsistent verification, how would people determine if WSJ Persian is authentic?
What if we told you that the Persian WSJ account was created on March 1, 2022 and was created in response to the Russian invasion of Ukraine? The very first news update posted by this account was about Russia invading Ukraine.
In a span of three weeks, the account has posted 345 news updates, at a rate of 345/22 = 16 posts per day. If this were an impersonation account, it is not just spreading individual pieces of information, but an entire narrative around the Russia-Ukraine war. The followers of this account might be believing this narrative only because it is from the WSJ. Perhaps the information and narrative are correct, but how would a user know?
Figure 7 shows two accounts, @wsjmagazine and @wsjmagchina, that are potential impersonation accounts of @wsjmag, the official account of WSJ Magazine, the Wall Street Journal’s flagship magazine that provides information about luxury consumer products to its readers.
@wsjmagazine has only 108 followers, has only one post since 2017, and is unverified. Based on heuristics such as the number of followers and posts, you might guess that @wsjmagazine is an impersonation account.
On the other hand, @wsjchinese has 7230 followers and 357 posts. Given that there are more followers and more frequent posts, it might be harder for people to know whether @wsjchinese is an impersonation account or not.
Based on our analysis, which includes cross-checking information from sources outside of social media, it appears that @wsjchinese account is an unverified account belonging to WSJ. However, we cannot be sure of that without WSJ’s acknowledgement.
Followers of WSJ are not security experts. They don’t have access to the additional data sources that we did, and they are not going to spend time to understand if @wsjchinese account is an impersonation account.
If @wsjchinese were an impersonation account, WSJ followers might get harmed by it. If it is not an impersonation account, followers will learn to accept unverified accounts as legitimate. Both are undesirable effects that would be mitigated if WSJ consistently verified it’s social media accounts.
Social media features amplify the spread of misinformation
Another key aspect to note is that features such as tagging and commenting can make impersonation accounts look legitimate and amplify the harm caused by impersonation.
Figure 8 shows a post featuring Tom Brady on the cover of WSJ Magazine. The post’s author tagged three accounts: @wsjphotos, @wsjmag and @wallstreet.journal. The first two, @wsjphotos and @wsjmag, are verified WSJ accounts. The third account @wallstreet.journal is unverified and has 1437 followers and 2 posts. Based on our analysis, it is an impersonation account.
By tagging the @wallstreet.journal impersonation account, the author of the post has exposed the author’s followers to the impersonation account. Furthermore, the official @wsjmag account commented on the post with the football emoji 🏈. This comment exposed all 398k followers of @wsjmag to the impersonation account!
The tagging and commenting features of social media will enable the impersonation account @wallstreet.journal to amass more followers. This increase the harm caused to both WSJ brand, its followers and its customers.
Beyond news — misinformation via books and more
Impersonation on social media is not limited to news accounts. We also see widespread impersonation of other products such as WSJ Magazine and WSJ Books. These impersonation accounts are hijacking the WSJ brand for illicit purposes from selling counterfeit consumer goods in Brazil to peddling fake books related to crypto and former US president Donald Trump.
Instagram is replete with impersonation accounts of @wsjmag. Figure 7 showed @wsjmagazine, one of the impersonation accounts. Instagram also has many accounts impersonating @wsjbooks, which is a WSJ account that promotes books. Figure 9 shows @wsjbookstore, an account impersonating @wsjbooks. The impersonation account redirects users to an external website selling fake books.
How can news outlets improve the situation?
Impersonation is not limited to WSJ and CNN. Many other reputable news sources such as NY Times, LA Times, NBC and ABC are also impersonated on social media. Whose responsibility is to address this issue? Is it the users, the platform or the news outlet?
In the US, social media platforms are rarely liable for actions of the users, and this includes any liability for impersonation accounts. Platforms have community guidelines that prohibit creation of impersonation accounts. However, it is not clear to what extent they proactively check for creation of impersonation accounts.
Platforms rely on their users to report impersonation accounts to the platform. Users who come across impersonation accounts are expected to identify them and report them to the platform. This however implies that users are exposed to potential misinformation. Some users might be savvy enough to identify fake accounts, but many won’t. Even savvy users and security experts are sometimes not sure in their judgement.
Furthermore, platforms do not verify user identity at the time of account creation. Without verifying the identity of the users, platforms cannot reliably decide which account is legitimate and which is an impersonation account. As a result, often AI-based algorithms deactivate or delete legitimate accounts instead of the impersonation accounts. To reduce the effort involved in verifying users, platforms can work with fact checkers and companies that can provide this information.
Recently the European Union strengthened it’s Code of Practice for tackling online disinformation across the EU. Given major penalties of up to 6% of global annual turnover for not curtailing spread of disinformation (spreading misinformation with malicious intent), social media platforms have more incentive to address impersonation.
Lastly, we can ask whether news outlets have a responsibility to identify accounts that impersonate their brand. From a reputational standpoint, it would be highly beneficial for news outlets to proactively identify and take down accounts that spread misinformation, mislead their readers and tarnish their brand.
A good start for news outlets is a policy to get all their social media accounts verified consistently. If the outlets do not have the technical capabilities to monitor and identify impersonation accounts on social media, they can work with tech companies that provide such services.
There is no panacea for spread of misinformation, but there are actions that news outlets and social media platforms can take to significantly curtail the spread of misinformation.
Have questions or comments? Contact us at [email protected]. To understand how the Eydle platform can identify and take down impersonation accounts on social media, contact us or visit www.eydle.com.
Impersonating CNN and Wall Street Journal was originally published in Eydle on Medium, where people are continuing the conversation by highlighting and responding to this story.
