Evaluation of Accuracy in Translating Pharmacy Related Terms and Sentences from Thai to English of the Google Translate in 2021 Compared to That in 2017
Article Sidebar
Main Article Content
Abstract
Objective: To evaluate the accuracy of Google Translate (GT) in translating terms and sentences related to pharmacy from Thai to English in 2017 and 2021. Methods: The researchers collected terms and sentences related to conversations encountered in community pharmacies from books and dictionaries in health and pharmacy. Subsequently, the study translated the terms and sentences through GT in 2017 and 2021. Appraisal of translation accuracy was done by three pharmacists with two levels of language structure i.e., word and sentence levels. Results: The average acceptances of translating pharmacy related terms in 2017 and 2021 were 76.32% and 83.42% respectively, with statistically significant differences between years (P < 0.001). At the sentence-level, the adequacy and fluency in 2017 were 3.28±1.46 and 2.97±1.57, respectively, while those in 2021 were 4.55±0.68 and 4.32±0.91, respectively, with significant differences from those in 2017 (P < 0.001). The accuracy evaluation based on the BLEU (BiLingual Evaluation Understudy) in 2017 and 2021 were 0.28±0.41 and 0.41±0.41, respectively. Assessment of a negative impact of wrong translation on patient health using a severity index revealed that the impacts in 2017 and 2021 was 3.85±1.25 and 4.79±0.48, respectively (P < 0.001) (higher values mean less impact). Conclusion: The GT has continuously improved its translation accuracy. However, application of this tool in pharmacy needs verification of translation output to ensure the accuracy and prevent negative impacts on patient health.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
ผลการวิจัยและความคิดเห็นที่ปรากฏในบทความถือเป็นความคิดเห็นและอยู่ในความรับผิดชอบของผู้นิพนธ์ มิใช่ความเห็นหรือความรับผิดชอบของกองบรรณาธิการ หรือคณะเภสัชศาสตร์ มหาวิทยาลัยสงขลานครินทร์ ทั้งนี้ไม่รวมความผิดพลาดอันเกิดจากการพิมพ์ บทความที่ได้รับการเผยแพร่โดยวารสารเภสัชกรรมไทยถือเป็นสิทธิ์ของวารสารฯ
References
Okpor M. Machine translation approaches: issues and challenges. Int J Comput Sci Appl 2014; 11: 159-65.
Al Shamsi H, Almutairi AG, Al Mashrafi S, Al Kalbani T. Implications of language barriers for healthcare: a systematic review. Oman Med J 2020; 35: e122. doi: 10.5001/omj.2020.40.
Dew KN, Turner AM, Choi YK, Bosold A, Kirchhoff K. Development of machine translation technology for assisting health communication: A systematic review. J Biomed Inform 2018; 85: 56-67.
Li C, Son N, Abdulkerim B, Jordan CA, Son CGE. Overcoming communication barriers to healthcare for culturally and linguistically diverse patients. N Am J Med Sci 2017; 10: 103-9.
Mohammad A, Saini B, Chaar BB. Pharmacists’ experiences serving culturally and linguistically diverse patients in the Australian community pharmacy setting. Int J Clin Pharm 2021; 43: 1563-73.
Google. Google Translate [online]. 2017 [cited Aug 10, 2017]; Available from: translate.google.com?hlth.
Rodriguez JA, Fossa A, Mishuris R, Herrick B. Bridging the language gap in patient portals: An evaluation of Google translate. J Gen Intern Med 2021; 36: 567-9.
Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, et al. Google's neural machine translation system: Bridging the gap between human and machine translation [online]. 2016 [cited May 2, 2017]. Available from: arxiv.org/abs/1609.08144.
Chen X, Acosta S, Barry AE. Evaluating the accuracy of Google Translate for diabetes education material. JMIR diabetes 2016; 1: e3. doi:10.2196/diabetes.58 48.
Cornelison BR, Al-Mohaish S, Sun Y, Edwards CJ. Accuracy of Google Translate in translating the directions and counseling points for top-selling drugs from English to Arabic, Chinese, and Spanish. Am J Health Syst Pharm 2021; 78: 2053-8.
Patil S, Davies P. Use of Google translate in medical communication: evaluation of accuracy. BMJ 2014; 349: g7392. doi: 10.1136/bmj.g7392
Phosri N, Lertnattee V. Google translate using for a communication tool between Thai pharmacists and Myanmar patients in drugstores. J Hlth Sci Res 2019.13: 104-14.
Phosri N, Lertnattee V. An evaluation of Google Translate in pharmaceutical language: Thai and English translations. The 6th international graduate study conference: Creative multi-disciplinary studies for sustainable development; 2016 July 11-12; Bangkok,Thailand. Bangkok: Silpakorn University; 2016. p. S1009 – 19.
Royal Institute. English-Thai, Thai-English pharma- ceutical vocabulary Royal institute dictionary. 2nd ed. Bangkok: Aksorncharoentat; 2008.
Phosri N, Lertnattee V. An evaluation of Google Translate in pharmacy domain between Thai and Myanmar. Adv Sci Lett 2018; 24: 2746-50.
Nahler G. Dictionary of pharmaceutical medicine. Vienna: Springer; 2009.
Bureau of Health Administration. Manual in four languages Thai-English-Myanmar-Cambodian for health personnel in providing services to foreigners. Nonthaburi: Bureau of Health Administration; 2014.
Chanyam W. Handbook of medical conversation; English conversation for nurses; Conversation etc. without fear ; Traveling conversation for army medical personnel. Bangkok: Army Medical Field Service School; 2013.
Royal Institute. English-Thai, the pharmaceutical dictionary. 2 nd ed. Bangkok: Royal Institute; 2008.
Eiamprapai P. The handbook of ASEAN language for medical personnel. Humanities and Social Sciences Mahasarakham University: Mahasarakham ; 2014.
Costa-jussà MR, Farrús M, Pons JS. Machine trans- lation in medicine. In: Mokryš M,Lieskovský A, editors. Proceedings of the 1st virtual international conference on advanced research in scientific fields; 2012 Dec 3-7; Žilina, Slovakia. Žilina: Advanced research in scientific areas; 2012. p.1995-8.
Seljan S, Brkić M, Vičić T. BLEU evaluation of machine-translated English-Croatian legislation. Proceedings of the eighth international conference on language resources and evaluation; 2012 May; Istanbul, Turkey. Istanbul: European language resources association; 2012.p. 2143-8.
Falotico R, Quatto P. Fleiss’ kappa statistic without paradoxes. Qual Quant 2015; 49: 463-70.
Aiken M. An updated evaluation of Google translate accuracy. Studies in Linguistics and Literature. 2019; 3: 253-60.
Internet World Stats. Internet world users by language [online]. 2021 [cited Dec 25, 2021]; Available from: www.Internetworldstats.com/stats7. htm
Taira BR, Kreger V, Orue A, Diamond LC. A pragmatic assessment of Google Translate for emergency department instructions. J Gen Intern Med. 2021; 36: 3361-5.
