Development of the Criteria for Credibility Evaluation of Websites with Advertisings of Dietary Supplement Version 2

Abstract

Objective: To revise the Credibility Evaluation of Websites with Advertisements of Dietary Supplements (CEWADS) criteria to the CEWADS-2 criteria and test their validity, reliability, and usability by consumers. Method: The researchers revised the 7 items CEWADS criteria developed by Wallapa Saisin in 2019 to the CEWADS-2 criteria by incorporating two additional criteria. The CEWADS-2 criteria underwent revisions based on feedback from eight experts and the results of tests involving ten consumers. Intra-rater reliability testing was performed by the researcher assessing the quality of 19 websites advertising dietary supplements over two separate occasions, one month apart, and calculating the intraclass correlation coefficient (ICC). Inter-rater reliability was assessed by having three experts evaluate these 19 websites independently, with ICC computed. A usability test was conducted with three groups of 77 consumers, who used the CEWADS-2 criteria to evaluate websites that experts had judged to be of high, medium, or low quality. Results: Intra-raters reliability was very good (ICC=0.94), while inter-rater reliability was at a good level (ICC=0.81). When considering the results on individual item in the criteria, seven items showed good or very good inter-rater reliability, whereas the other two items exhibited moderate and fair reliability. Differences of scores on the CEWADS-2 between consumer and expert were higher for the high-quality website (2.81 on a scale of 0-9), compared to those for the low or medium-quality websites (0.97 and 0.66, respectively). The participants evaluated high, medium, and low quality websites according to the CEWADS-2 criteria, yielding average scores of 5.86±1.80, 5.34±1.64, and 3.30±1.18, respectively (on a 9-point scale). The participants could effectively distinguish the low-quality website from those of high or medium quality, but they could not effectively differentiate between high-quality and medium-quality websites. The ability to discriminate website quality was found to be independent of health literacy. Furthermore, the scores on the CEWADS-2 criteria of the consumers exhibited significant correlations with various quality variables, such as website trustworthiness, affirming the validity of the criteria. Conclusion: The CEWADS-2 criteria exhibit an acceptable level of validity and reliability. Consumers are able to employ these criteria to differentiate low-quality websites from those of high and medium quality. Health literacy has no effect on this ability. Future research should investigate the acceptability of this criteria for use by consumers.