Main Topic: Increasing use of AI in manipulative information campaigns online.
Key Points:
1. Mandiant has observed the use of AI-generated content in politically-motivated online influence campaigns since 2019.
2. Generative AI models make it easier to create convincing fake videos, images, text, and code, posing a threat.
3. While the impact of these campaigns has been limited so far, AI's role in digital intrusions is expected to grow in the future.
### Summary
The rise of generative artificial intelligence (AI) is making it difficult for the public to differentiate between real and fake content, raising concerns about deceptive fake political content in the upcoming 2024 presidential race. However, the Content Authenticity Initiative is working on a digital standard to restore trust in online content.
### Facts
- Generative AI is capable of producing hyper-realistic fake content, including text, images, audio, and video.
- Tools using AI have been used to create deceptive political content, such as images of President Joe Biden in a Republican Party ad and a fabricated voice of former President Donald Trump endorsing Florida Gov. Ron DeSantis' White House bid.
- The Content Authenticity Initiative, a coalition of companies, is developing a digital standard to restore trust in online content.
- Truepic, a company involved in the initiative, uses camera technology to add verified content provenance information to images, helping to verify their authenticity.
- The initiative aims to display "content credentials" that provide information about the history of a piece of content, including how it was captured and edited.
- The hope is for widespread adoption of the standard by creators to differentiate authentic content from manipulated content.
- Adobe is having conversations with social media platforms to implement the new content credentials, but no platforms have joined the initiative yet.
- Experts are concerned that generative AI could further erode trust in information ecosystems and potentially impact democratic processes, highlighting the importance of industry-wide change.
- Regulators and lawmakers are engaging in conversations and discussions about addressing the challenges posed by AI-generated fake content.
AI-generated child pornography: A controversial solution or a Pandora's Box?
The emergence of generative AI models that can produce realistic fake images of child sexual abuse has sparked concern and debate among regulators and child safety advocates. On one hand, there is fear that this technology may exacerbate an already abhorrent practice. On the other hand, some experts argue that AI-generated child pornography could offer a less harmful alternative to the existing market for such explicit content. They believe that pedophilia is rooted in biology and that finding a way to redirect pedophilic urges without involving real children could be beneficial.
While psychiatrists strive for a cure, utilizing AI-generated imagery as a temporary solution to replace the demand for real child pornography may have its merits. Currently, law enforcement comb through countless images in their efforts to identify victims, and the introduction of AI-generated images further complicates their task. Additionally, these images often exploit the likenesses of real people, further perpetuating abuse of a different nature. However, AI technology could also play a role in helping distinguish between real and simulated content, aiding law enforcement in targeting actual cases of child sexual abuse.
There are differing opinions on whether satisfying pedophilic urges through AI-generated child pornography can actually prevent harm in the long run. Some argue that exposure to such content might reinforce and legitimize these attractions, potentially leading to more severe offenses. Others suggest that AI-generated images could serve as an outlet for pedophiles who do not wish to harm children, allowing them to find sexual catharsis without real-world implications. By providing a controlled environment for these individuals, AI-generated images could potentially help curb their behavior and encourage them to seek therapeutic treatment.
Concerns about the normalization of child pornography and the potential gateway effect are addressed by experts. They argue that individuals without pedophilic tendencies are unlikely to be enticed by AI-generated child pornography, and the scientific research indicates that the act of viewing alone does not necessarily lead to hands-on offenses. Moreover, redirecting potential viewers to AI-generated images could reduce the circulation of real images, offering some protection to victims.
While the idea of utilizing AI-generated child pornography as a form of harm reduction may be difficult to accept, it parallels the philosophy behind other public health policies aimed at minimizing damage. However, it is crucial to differentiate between controlled psychiatric settings and uncontrolled proliferation on the web. Integrating AI-generated images into therapy and treatment plans, tailored to each individual's needs, could offer a way to diminish risks and prioritize the safety of both victims and potential offenders.
Major media organizations are calling for new laws to protect their content from being used by AI tools without permission, expressing concerns over unauthorized scraping and the potential for AI to produce false or biased information.
Salesforce has released an AI Acceptable Use Policy that outlines the restrictions on the use of its generative AI products, including prohibiting their use for weapons development, adult content, profiling based on protected characteristics, medical or legal advice, and more. The policy emphasizes the need for responsible innovation and sets clear ethical guidelines for the use of AI.
The proliferation of deepfake videos and audio, fueled by the AI arms race, is impacting businesses by increasing the risk of fraud, cyberattacks, and reputational damage, according to a report by KPMG. Scammers are using deepfakes to deceive people, manipulate company representatives, and swindle money from firms, highlighting the need for vigilance and cybersecurity measures in the face of this threat.
AI Algorithms Battle Russian Disinformation Campaigns on Social Media
A mysterious individual known as Nea Paw has developed an AI-powered project called CounterCloud to combat mass-produced AI disinformation. In response to tweets from Russian media outlets and the Chinese embassy that criticized the US, CounterCloud produced tweets, articles, and even journalists and news sites that were entirely generated by AI algorithms. Paw believes that the project highlights the danger of easily accessible generative AI tools being used for state-backed propaganda. While some argue that educating users about manipulative AI-generated content or equipping browsers with AI-detection tools could mitigate the issue, Paw believes that these solutions are not effective or elegant. Disinformation researchers have long warned about the potential of AI language models being used for personalized propaganda campaigns and influencing social media users. Evidence of AI-powered disinformation campaigns has already emerged, with academic researchers uncovering a botnet powered by AI language model ChatGPT. Legitimate political campaigns, such as the Republican National Committee, have also utilized AI-generated content, including fake images. AI-generated text can still be fairly generic, but with human finesse, it becomes highly effective and difficult to detect using automated filters. OpenAI has expressed concern about its technology being utilized to create tailored automated disinformation at a large scale, and while it has updated its policies to restrict political usage, it remains a challenge to block the generation of such material effectively. As AI tools become increasingly accessible, society must become aware of their presence in politics and protect against their misuse.
Generative AI tools are providing harmful content surrounding eating disorders around 41% of the time, raising concerns about the potential exacerbation of symptoms and the need for stricter regulations and ethical safeguards.
Generative AI is being used to create misinformation that is increasingly difficult to distinguish from reality, posing significant threats such as manipulating public opinion, disrupting democratic processes, and eroding trust, with experts advising skepticism, attention to detail, and not sharing potentially AI-generated content to combat this issue.
AI-generated videos are targeting children online, raising concerns about their safety, while there are also worries about AI causing job losses and becoming oppressive bosses; however, AI has the potential to protect critical infrastructure and extend human life.
AI technology is making it easier and cheaper to produce mass-scale propaganda campaigns and disinformation, using generative AI tools to create convincing articles, tweets, and even journalist profiles, raising concerns about the spread of AI-powered fake content and the need for mitigation strategies.
Generative artificial intelligence (AI) tools, such as ChatGPT, have the potential to supercharge disinformation campaigns in the 2024 elections, increasing the quantity, quality, and personalization of false information distributed to voters, but there are limitations to their effectiveness and platforms are working to mitigate the risks.
Attorneys general from all 50 states have called on Congress to establish protective measures against AI-generated child sexual abuse images and expand existing restrictions on such materials. They argue that the government needs to act quickly to prevent the potentially harmful use of AI technology in creating child exploitation material.
Top prosecutors from all 50 states are urging Congress to establish an expert commission to study how artificial intelligence can be used to exploit children through pornography and to expand existing restrictions on child sexual abuse materials to cover AI-generated images.
The infiltration of artificial intelligence into children's lives is causing anxiety and sparking fears about the perversion of children's culture, as AI tools create unsettling and twisted representations of childhood innocence. This trend continues a long history of cultural anxieties about dangerous interactions between children and technology, with films like M3GAN and Frankenstein depicting the dangers of AI. While there is a need to address children's use and understanding of AI, it is important not to succumb to moral panics and instead focus on promoting responsible AI use and protecting children's rights.
Attorneys general from all 50 states and four territories are urging Congress to establish an expert commission to study the potential exploitation of children through generative AI and to expand laws against child sexual abuse material (CSAM) to cover AI-generated materials.
Australia's eSafety Commissioner has introduced an industry code that requires tech giants like Google and Microsoft to eliminate child abuse material from their search results and prevent generative AI from producing deepfake versions of such material.
Australia's internet regulator has drafted a new code that requires search engines like Google and Bing to prevent the sharing of child sexual abuse material created by artificial intelligence, and also prohibits the AI functions of search engines from producing deepfake content.
AI on social media platforms, both as a tool for manipulation and for detection, is seen as a potential threat to voter sentiment in the upcoming US presidential elections, with China-affiliated actors leveraging AI-generated visual media to emphasize politically divisive topics, while companies like Accrete AI are employing AI to detect and predict disinformation threats in real-time.
Artificial intelligence (AI) poses a high risk to the integrity of the election process, as evidenced by the use of AI-generated content in politics today, and there is a need for stronger content moderation policies and proactive measures to combat the use of AI in coordinated disinformation campaigns.
State attorneys general, including Oklahoma's Attorney General Gentner Drummond, are urging Congress to address the consequences of artificial intelligence on child pornography, expressing concern that AI-powered tools are making prosecution more challenging and creating new opportunities for abuse.
Government agencies at the state and city levels in the United States are exploring the use of generative artificial intelligence (AI) to streamline bureaucratic processes, but they also face unique challenges related to transparency and accountability, such as ensuring accuracy, protecting sensitive information, and avoiding the spread of misinformation. Policies and guidelines are being developed to regulate the use of generative AI in government work, with a focus on disclosure, fact checking, and human review of AI-generated content.
Paedophiles are using open source AI models to create child sexual abuse material, according to the Internet Watch Foundation, raising concerns about the potential for realistic and widespread illegal content.
The generative AI boom has led to a "shadow war for data," as AI companies scrape information from the internet without permission, sparking a backlash among content creators and raising concerns about copyright and licensing in the AI world.
Generative AI is empowering fraudsters with sophisticated new tools, enabling them to produce convincing scam texts, clone voices, and manipulate videos, posing serious threats to individuals and businesses.
Deepfakes, which are fake videos or images created by AI, pose a real risk to markets, as they can manipulate financial markets and target businesses with scams; however, the most significant negative impact lies in the creation of deepfake pornography, particularly non-consensual explicit content, which causes emotional and physical harm to victims and raises concerns about privacy, consent, and exploitation.
Adversaries and criminal groups are exploiting artificial intelligence (AI) technology to carry out malicious activities, according to FBI Director Christopher Wray, who warned that while AI can automate tasks for law-abiding citizens, it also enables the creation of deepfakes and malicious code, posing a threat to US citizens. The FBI is working to identify and track those misusing AI, but is cautious about using it themselves. Other US security agencies, however, are already utilizing AI to combat various threats, while concerns about China's use of AI for misinformation and propaganda are growing.
AI-generated content is becoming increasingly prevalent in political campaigns and poses a significant threat to democratic processes as it can be used to spread misinformation and disinformation to manipulate voters.
While many experts are concerned about the existential risks posed by AI, Mustafa Suleyman, cofounder of DeepMind, believes that the focus should be on more practical issues like regulation, privacy, bias, and online moderation. He is confident that governments can effectively regulate AI by applying successful frameworks from past technologies, although critics argue that current internet regulations are flawed and insufficiently hold big tech companies accountable. Suleyman emphasizes the importance of limiting AI's ability to improve itself and establishing clear boundaries and oversight to ensure enforceable laws. Several governments, including the European Union and China, are already working on AI regulations.
AI-generated deepfakes pose serious challenges for policymakers, as they can be used for political propaganda, incite violence, create conflicts, and undermine democracy, highlighting the need for regulation and control over AI technology.
Criminals are increasingly using artificial intelligence, including deepfakes and voice cloning, to carry out scams and deceive people online, posing a significant threat to online security.
The European Union is warning about the risks posed by widely accessible generative AI tools in relation to disinformation and elections, calling on platforms to implement safeguards and urging ChatGPT maker OpenAI to take action to address these risks. The EU's voluntary Code of Practice on Disinformation is being used as a temporary measure until the upcoming AI Act is adopted, which will make user disclosures a legal requirement for AI technologies.
The UK Home Secretary and the US homeland security secretary have pledged to work together to combat the rise of child sexual abuse images created by artificial intelligence (AI), which are increasingly realistic and pose challenges for law enforcement and online safety.
Deepfake images and videos created by AI are becoming increasingly prevalent, posing significant threats to society, democracy, and scientific research as they can spread misinformation and be used for malicious purposes; researchers are developing tools to detect and tag synthetic content, but education, regulation, and responsible behavior by technology companies are also needed to address this growing issue.
Artificial intelligence (AI) has the potential to facilitate deceptive practices such as deepfake videos and misleading ads, posing a threat to American democracy, according to experts who testified before the U.S. Senate Rules Committee.
Artificial intelligence is now being used in extortion cases involving teens, making an already dangerous situation even worse. It is crucial for both teens and parents to remain vigilant and have open conversations about the dangers of online activities.
Internet freedom is declining globally due to the use of artificial intelligence (AI) by governments for online censorship and the manipulation of images, audio, and text for disinformation, according to a new report by Freedom House. The report calls for stronger regulation of AI, transparency, and oversight to protect human rights online.
A nonprofit called AIandYou is launching a public awareness campaign to educate voters about the potential impact of AI on the 2024 election, including using AI-generated deepfake content to familiarize voters with this technology.
Artificial Intelligence is being misused by cybercriminals to create scam emails, text messages, and malicious code, making cybercrime more scalable and profitable. However, the current level of AI technology is not yet advanced enough to be widely used for deepfake scams, although there is a potential future threat. In the meantime, individuals should remain skeptical of suspicious messages and avoid rushing to provide personal information or send money. AI can also be used by the "good guys" to develop software that detects and blocks potential fraud.
Deepfake videos featuring celebrities like Gayle King, Tom Hanks, and Elon Musk have prompted concerns about the misuse of AI technology, leading to calls for legislation and ethical considerations in their creation and dissemination. Celebrities have denounced these AI-generated videos as inauthentic and misleading, emphasizing the need for legal protection and labeling of such content.
Generative AI tools, including Facebook's AI sticker generator, are being used to create controversial and inappropriate content, such as violent or risqué scenes involving politicians and fictional characters, raising concerns about the misuse of such technology.
Generative artificial intelligence (AI) is expected to face a reality check in 2024, as fading hype, rising costs, and calls for regulation indicate a slowdown in the technology's growth, according to analyst firm CCS Insight. The firm also predicts obstacles in EU AI regulation and the introduction of content warnings for AI-generated material by a search engine. Additionally, CCS Insight anticipates the first arrests for AI-based identity fraud to occur next year.
The U.S. Space Force has temporarily banned the use of web-based generative AI due to security concerns, suspending the creation of text, images, and other media using government data until new guidelines are released, according to an internal memo.
U.K. startup Yepic AI, which claims to use "deepfakes for good," violated its own ethics policy by creating and sharing deepfaked videos of a TechCrunch reporter without their consent. They have now stated that they will update their ethics policy.
