Digital Insights - 21 July 2023

Is ChatGPT getting dumber? Evidence suggests yes

• A new study from Stanford and UC Berkeley suggests GPT-4's accuracy has declined from 97.6% in March to just 2.4% in June after model updates.

• The study demonstrates that GPT-4 now struggles with simple reasoning questions it previously answered correctly, showing no work to explain its (incorrect) answers.

• This matches widespread anecdotal observations from users that GPT-4's quality has degraded recently.

• The findings contradict OpenAI's claim that complaints are just a psychological effect amongst heavy users. For Open AI’s part, Logan Kilpatrick, who leads developer relations at the organisation, tweeted that they are looking into the report’s findings.

• Whilst methodological errors in the study cannot be ruled out, if the findings are true, there will be significant implications for businesses that have relied on GPT-4 to build out additional tools.

Anthropic launches Claude 2 chatbot to rival ChatGPT

• American AI company Anthropic has released Claude 2, a new chatbot aimed at competing with ChatGPT, Bard and other AI assistants.

• Claude 2 can summarise long passages up to 75,000 words, similar to a novel length. This exceeds ChatGPT's summarisation abilities. It is also able to accept document uploads, something ChatGPT doesn’t do natively.

• Claude 2 operates based on a set of principles from documents like the UN Declaration of Human Rights to guide its judgments. Anthropic claims this "Constitutional AI" approach makes it safer.

• However, Claude 2 still seems prone to factual errors in its responses, like incorrectly naming sports winners, limiting its capabilities versus rivals.

China introduces new rules for generative AI products

• China will introduce interim measures in August to manage its growing generative AI industry, attempting to balance its aim of maintaining security with support for technological development. China is one of the first countries to introduce such regulations.

• Firms such as Baidu and Alibaba Group have launched dozens of AI models but held back from rolling out chatbots until new rules were finalised.

• The Cyberspace Administration of China said providers who wanted to offer services to the public would need to submit security assessments, ensuring products did not infringe IP rights and used legitimate data sources. Providers must also register their algorithms with the government and ensure content is ‘in line with China's core socialist values’.

Zelenskyy’s Nato Summit tweet causes waves

• Ukraine’s government, and President Volodymyr Zelenskyy in particular, have previously been hailed for their effective use of social media in the midst of the Russo-Ukrainian war.

• At the NATO summit in Vilnius, Ukraine pushed for a concrete path to membership but draft text was vague.

• In what appeared to be a last-ditch attempt to influence outcomes, an outraged Zelenskyy took publicly to Twitter to slam the "unprecedented and absurd" lack of timeline as leaving an "opening" for Russia.

• Zelenskyy has frequently used social media to speak directly to allies and supporters during the war. Normally, he would take a friendlier and more conciliatory tone, so this tweet stood out strongly, as did its timing, when he was already en route to Vilnius.

• This tweet called out NATO leaders publicly and tried to shape negotiations through public pressure. While Ukraine ultimately didn’t receive commitments around membership, and there has been debate whether the tweet helped or harmed negotiations, it was widely covered in the media and shifted media focus to questioning Nato leaders as to the outcome, with France’s Emmanuel Macron, Czech Republic’s Petr Pavel and Nato’s Jens Stoltenberg acknowledging frustrations at the outcome.

Viral deepfake ad confronts bias towards women's football ahead of World Cup

•  A viral ad by Orange used deepfake technology to overlay male players' faces onto female players' highlights.

• It initially seems to show male stars like Griezmann and Mbappé, but reveals it's actually the French women's national team.

• The ad confronts and rejects the common bias that women's football is less exciting and skilled than men's in the lead up to the 2023 FIFA Women’s World Cup, which started this week.

• But studies show people rate men and women's performance similarly when gender is concealed. And, in reality, top female players like Marta have exceeded men's achievements in areas like World Cup goals scored.

• With projected viewership of 2 billion, the tournament is building the profile of women's football, and Orange aims to prove that women’s football is just as gripping as men’s football for fans.

Threads loses steam as active users plunge after launch

• Threads saw an enormous surge at launch, quickly amassing over 100 million signups, but according to analytics firm Similarweb, daily active users have already plunged from 49 million on July 7 to just 23.6 million by July 14.

• Engagement has crashed as well, with total daily time spent on Threads collapsing from 21 minutes to only 6 minutes over the same period.

• Threads initially capitalised on easy account creation via Instagram to gain users quickly. However, it lacks key features present on Twitter like emojis, hashtags, and chronological feeds.

• This is likely causing the massive user dropoff as the initial excitement wears off and flaws become apparent.

• Threads may struggle to retain users long-term and still has major work to build core features to compete with Twitter.

Apple expands retail presence in China with WeChat store

• Apple has opened a store on China's super app WeChat, marking an expansion of the US company's retail channels in the country. Apple already operates a shop on Alibaba Group's Tmall online marketplace.

• WeChat's 1.2billion users are now able to buy Apple products including iPhones, iPads and Macs from the store.

• While smartphone sales in China fell 5% year-on-year in the first quarter of 2023, iPhone sales grew 6%.

TikTok’s Chinese ownership raises concerns about its influence on Taiwan election

• Experts in Taiwan are worried China could influence TikTok content ahead of the Taiwanese presidential election in January 2024.

• Authorities in Taiwan have already barred TikTok from government-issued devices, in line with moves by countries including the US.

• With Taiwan often at the centre of the US-China diplomatic tensions, the election result would certainly influence how that relationship evolves in the years ahead.

• TikTok has already proven to be a source of political disinformation:  prior to the 2022 midterm elections in the United States, research showed that TikTok approved 90% of ads that researchers submitted which included false information.