China’s DeepSeek is cheaper than ChatGPT however accuracy exams present you get what you pay for

Nothing is hotter in tech than AI. Right now, nothing is hotter in mobile tech than the iOS app of China’s DeepSeek AI. As we told you last night, the app is the number one free app in the App Store topping ChatGPT. The development of DeepSeek was done at a much lower cost than it took OpenAI and Google to create ChatGPT and Gemini respectively. But sometimes you get what you pay for and in an audit done by NewsGuard, DeepSeek had an extremely low accuracy rate in delivering news and information.

The report said that DeepSeek was correct only 17% of the time which had it tenth out of eleven AI chatbots including ChatGPT and Gemini. 30% of the time DeepSeek responded with false claims and more than half of the time-53%-in response to news-related prompts, the Chinese AI chatbot gave a vague or not useful answer. Overall, DeepSeek had an 87% fail rate according to NewsGuard which compares to a 62% fail rate for DeepSeek’s Western rivals.

DeepSeek has claimed that the technology behind the chatbot performs the same or better than OpenAI’s ChatGPT at a fraction of the cost. The report from NewsGuard casts doubt on that claim. The test included the same 300 prompts used to test ChatGPT and Gemini with 30 of them purposely based on 10 false claims found online.  According to Reuters, some of the topics of these prompts included the assassination of UnitedHealthcare CEO Brian Thompson and the downing of Azerbaijan Airlines flight 8243.

“The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models.”-Gil Luria, analyst, D.A. Davidson

What could be considered alarming is that with three of 10 prompts based on false online claims, DeepSeek repeated the Chinese government’s position on the matter even without any mention of China in the prompt. For example, in response to prompts about the Azerbaijan Airlines crash and in response to questions about it that had nothing to do with China, the chatbot took Beijing’s position.

Source link