As an Amazon Associate I earn from qualifying purchases.
Open-weight designs, consisting of Nvidia’s Nemotron and Alibaba’s Qwen, revealed strong outcomes similar to Anthropic’s finest designs. GPT-5.4– the best-performing design from OpenAI– likewise carried out reasonably well on the standard, offering “Exemplary” actions on 54 percent of concerns and attaining an 88.9 mean rating.
Unsurprisingly, current frontier designs revealed a much more powerful propensity to withstand Russian propaganda than designs from simply a couple of years back. Claude 3.5 Haiku– the highest-rated design launched in 2024– got a mean ranking of simply 73.1 on the criteria. That mark would put it in the bottom 3rd of designs launched in 2026 on this metric.
Comprehensive standards for Google’s Gemini 2.5 Pro design program especially level of sensitivity to harmful triggers and triggers in Russian.
Comprehensive criteria for Google’s Gemini 2.5 Pro design program especially level of sensitivity to destructive triggers and triggers in Russian.
Credit: Estonian Language Institute
That enhancement over time was not consistent throughout all LLM makers. Google’s a lot of propaganda-resistant LLM, Gemini 2.5 Pro, is almost a years of age now and has actually just reached a mean rating of 82 on the standard, mainly due to a specific vulnerability to maliciously worded triggers. The most current checked Google design, Gemini 3.5 Flash, just scored a 73 on the standard, equivalent to Anthropic designs launched almost 2 years earlier.
In a supporting post on the Propastop blog site, the company highlights the number of designs revealed much less resistance to Russian propaganda when questioned in Russian. Google’s Gemini 3.5 Flash got considerably lower benchmark ratings in Russian than in English, as did open-weight designs like Moonshot’s Kimi K2 and StepFun’s Step 3.5 Flash.
What one nation views as propaganda, obviously, another may view as a set of essential cultural realities that LLMs must support and show. A current research study from King’s College teacher Gregory Asmolov examines how the Russian federal government– through current technical alliances with other BRICS nations– is looking for to affect AI designs by predicting particular sociopolitical positions that are “culturally delicate” to Russia’s perspectives.
Learn more
As an Amazon Associate I earn from qualifying purchases.
