Bilecik University study examines AI accuracy in social assistance information

A study conducted by Dr. Hacer Taşkıran Tepe at Bilecik Şeyh Edebali University has examined the accuracy and readability of large language models (LLMs) regarding social assistance information. The research, published in the Journal of the Society for Social Work and Research, evaluated how models like ChatGPT-3.5, ChatGPT-4o, Gemini, and Microsoft Copilot respond to frequently asked questions from the Ministry of Family and Social Services official website.

The findings indicate that no single AI model provided sufficient accuracy for all questions. Furthermore, the responses demonstrated only a moderate level of readability, suggesting that information provided via AI may not be easily understood by all users, particularly disadvantaged groups.

Dr. Taşkıran Tepe warned that incorrect or incomplete information regarding social assistance programs and eligibility criteria could prevent individuals in need from accessing their rights. The study suggests that AI tools should be used as supplementary resources that require verification through official sources rather than as sole information providers.