Evaluation of Translation Quality Between Chinese and Thai for Basic History Taking Using Google Translate, Baidu Translate, and ChatGPT

Abstract

Objective: To assess and compare the translation quality of basic history-taking content from Chinese to Thai and from Thai to Chinese using Google Translate, Baidu Translate, and ChatGPT. Methods: This evaluative study began with testing three prompt formulations for ChatGPT in order to identify the most appropriate prompt for translating health-related text. Google Translate, Baidu Translate, and ChatGPT were then used to translate 75 disease- and symptom-related terms and 86 phrases or sentences relevant to basic history taking, neurology, gastroenterology, and reproductive systems. The translation materials were selected from a patient history-taking handbook. The translations were performed in both Chinese-to-Thai and Thai-to-Chinese directions. The translation quality was assessed by two experts at two levels: 1) the lexical level, which was evaluated using acceptability and compared across the three tools using Bayesian generalized linear mixed models; and 2) the phrasal/sentential level, which was evaluated using a 5-point Likert-scale questionnaire across four domains: adequacy, fluency (grammaticality and comprehensibility), meaning, and severity (impact on health). Comparisons of translation quality among the three tools were conducted using an ordinal logistic mixed-effects model. Results: At the lexical level, no statistically significant interaction was found between translation tools and raters. ChatGPT demonstrated the highest probability of acceptable translations in both the Chinese-to-Thai and Thai-to-Chinese directions, with acceptability rates of 99.98% and 99.96%, respectively. ChatGPT showed significantly better lexical translation quality than Google Translate and Baidu Translate in both directions (P < 0.05), whereas no statistically significant difference was observed between Google Translate and Baidu Translate (P > 0.05). At the phrasal/sentential level, no statistically significant interaction was found between translation tools and raters. ChatGPT achieved significantly higher translation quality scores than the other two tools across all domains and in both translation directions (P < 0.01). In the Chinese-to-Thai direction, Google Translate scored significantly higher than Baidu Translate in all domains (P < 0.05). In the Thai-to-Chinese direction, Google Translate scored significantly higher than Baidu Translate in adequacy, meaning, and severity (P < 0.05), but no significant difference was found in fluency (P = 0.424). Conclusion: ChatGPT demonstrated the highest translation quality for words, phrases, or sentences used in basic history taking, in both Thai-to-Chinese and Chinese-to-Thai directions. However, it should be used only as a communication aid. Healthcare professionals should further verify critical information to prevent inaccuracies that may affect patient health.