Abstract

Aim

Patients frequently use YouTube for information. This study aimed to evaluate the quality of Turkish YouTube videos on age-related macular degeneration (AMD).

Methods

Turkish translations of “AMD” and “yellow spot disease” were searched on August 16, 2023. The first 100 videos were grouped as useful, misleading, or irrelevant. Video properties and quality scores [Journal of American Medical Association (JAMA), Global Quality Score (GQS), DISCERN] were evaluated.

Results

We classified 74 videos as useful, 16 as misleading, and 10 as irrelevant. The most common uploaders were television shows and social media. The mean JAMA score was 1.94±1.09, the mean GQS score was 2.49±1.29, and the mean DISCERN score was 36.82±16.53. The mean JAMA, GQS, and DISCERN scores were higher in the useful group than in the misleading group (p=0.0001, p=0.001, p=0.0001). The number of likes and view ratio were higher in the misleading group than in the useful group (p=0.019, p=0.037). The main content of the misleading group was treatment, and the uploaders were related to industry and commercial interests.

Conclusion

The quality of the AMD videos was insufficient. Although misleading videos had lower quality, they were more popular. This may be due to patients’ hope for new treatments, as misleading videos have discussed this topic.

Introduction

Age-related macular degeneration, or AMD, was first described in 1985 as a disease in people over 50 that causes a loss of central vision and changes in the macula’s color and structure (1). In developed countries, it is the most important cause of visual impairment in people aged >60 years, accounting for 8.7% of legal blindness cases worldwide. The prevalence of AMD is believed to reach approximately 288 million by 2040 (2). With the prolonged life expectancy, the incidence of AMD has also increased.

Visual disorders, which have a negative impact on quality of life due to their physical and psychological effects, can affect patients on a daily basis, ranging from social relations to work concentration. With emotions such as tension, fear, and anxiety, patients with vision loss turn to online platforms where they can obtain information and visit a physician.

Due to high rates of advanced literacy, researchers looked into health-related internet use habits and found a strong relationship between health literacy and internet use (3).

Social networking sites, such as Facebook, YouTube, and Twitter, have opened new avenues for disseminating health-related information. Studies have been conducted to evaluate the content of health-related videos on different social networking sites (4, 5). YouTube is a free video-sharing platform with more than 1 billion users. Millions of users upload videos on YouTube, and uncontrolled sharing can cause information pollution that contradicts relevant standards, especially health-related standards. Studies evaluating the quality of health-related videos on YouTube have reported that nearly 16-30% present misleading or low-quality information (6, 7).

Video popularity is calculated using the video power index and view rate. The Journal of American Medical Association (JAMA), DISCERN, and Global Quality Score (GQS) scoring systems are used to evaluate the educational quality of videos. There are concerns regarding the quality and reliability of online medical information because uncontrolled individual video uploads can result in inaccurate and misleading information. It has become possible to access high-quality educational videos through scoring systems such as JAMA, DISCERN, and GQS.

In ophthalmology, numerous studies have evaluated the information quality and reliability of YouTube videos on different topics. Kayabaşı et al. (8) analyzed YouTube videos about myopia and concluded that they were of weak to moderate quality. The study of Tanyıldız and Oklar (9) concluded that YouTube videos about uveitis were poor in reliability and quality and were not sufficiently educational for patients. Ozturkmen and Berhuni (10) evaluated YouTube videos about pterygium surgery and reported that they were of low quality and inadequate in informing patients. Kaptı and Erdem (11) reported the quality of YouTube videos about congenital nasolacrimal duct obstruction as “average”. Unlike most other studies, İlhan et al. (12) examined YouTube videos on thyroid orbitopathy, and the quality was good in most videos.

This study aimed to evaluate the reliability, popularity, and quality of Turkish YouTube videos on AMD to examine the status of Turkey, compare low, medium, and high-quality videos in terms of general video characteristics, and determine the relationship between general video characteristics and video reliability and quality evaluation.

Materials and Methods

Compliance with Ethical Standards

The data collected for this study were acquired from publicly accessible YouTube videos. This study was not required to obtain Institutional Review Board approval or ethical approval as it involved only public access data.

Search Strategy and Data Collection

In this cross-sectional, record-based study, YouTube (www.youtube.com) was searched using the Turkish keywords “yaşa bağlı makula dejenerasyonu and sarı nokta hastalığı” on August 16, 2023. “Age related macular degeneration” and “Yellow spot disease” are translations for “Yaşa bağlı makula dejenerasyonu” and “sarı nokta hastalığı” in the Turkish language. Video search was performed after clearing the browser’s entire search history without the user logging in to prevent misdirection. The top 100 videos were selected for evaluation based on their “relevance”, determined by YouTube’s algorithm.

Video Categorization and Characteristics

Two ophthalmologist examiners (GDA, MO) watched and analyzed the videos.

All videos were grouped based on their content as useful, misleading, or irrelevant.

1. Useful: Video showing scientifically accurate and correct information regarding any aspect of the disease.

2. Misleading: The video presented inaccurate or unproven information based on available scientific evidence.

3. Irrelevant: This condition presents information that is not relevant or related to AMD.

Irrelevant videos were excluded from the study, and useful and misleading groups were included in the statistical analysis (Figure 1).

For each video, video metrics, including video length, time since upload, and number of likes, were recorded, and the view ratio was evaluated to assess popularity. The view ratio was determined by dividing the number of views by the time since upload.

The video and audio qualities of the videos were evaluated according to criteria previously described by Young et al. (13). Video quality was defined as good if it was professionally produced with excellent quality and effects, moderate if it was a home video, and poor if it was grainy, affecting the ability to see presentation details. Similarly, the audio quality was considered good if all words could be clearly heard without significant background noise or distracting audio effects, moderate if most words were understandable, with minimal background noise, and poor if it limited the understanding of the material.

Videos were also categorized into four groups based on their upload source: (1) ophthalmologist, (2) TV show/social media, (3) industry/commercial interest, and (4) medical school/academic center.  The main contents discussed in the video were noted, and groups were classified according to the main content under the following headings: (1) Only treatment, (2) General information about disease + diagnosis, and (3) General information about disease + diagnosis + treatment (all aspects).

In addition, we recorded whether the video included real procedures or animation. Video participants were also recorded.

Video Quality Scoring Systems

The information quality of each video was evaluated using the DISCERN, JAMA, and GQS scores.

The DISCERN score was developed to evaluate the educational quality and reliability of medical information, particularly treatment options available to the patient (14). The questionnaire contains three sections with 16 questions, and a higher score indicates superior quality. The initial eight questions related to reliability, whereas the latter seven assessed specific details of the treatments received. The final question relates to the overall quality of a publication (Appendix A). According to the DISCERN scoring system, the videos were grouped into excellent quality (63-80 points), good quality (51-62 points), fair quality (39-50 points), poor quality (27-38 points), and very poor quality (16-26 points).

The JAMA scoring system evaluates the quality of online health-related information according to four criteria: currency, authorship, disclosure, and attribution, each scored as 0 or 1 (Appendix B) (15). A high score on this scale indicates that the information was of good quality. A score of 4 indicates excellent quality, and a score of 0 indicates poor quality.

The GQS system is a 5-point scale used to evaluate the ease of use, overall flow, and accessibility of the information delivered in the video (Appendix C) (16). A score of 5 indicated excellent quality, and a score of 1 indicated poor quality.

Statistical Analysis

Statistical analyses were performed using the NCSS (Number Cruncher Statistical System) 2007 Statistical Software (Utah, USA). In evaluating the data, in addition to descriptive statistical methods (mean, standard deviation, median, interquartile range), the distribution of the variables was examined using the Shapiro-Wilk normality test. The Kruskal-Wallis test was used for intergroup comparisons of variables that did not show a normal distribution; Dunn’s multiple comparison test was used for comparisons of subgroups; and Dunn’s multiple comparison test was used for paired group comparisons. Mann-Whitney U and chi-square tests were used to compare qualitative data, and the Pearson correlation test was used to determine the relationships between variables. The results were evaluated at a significance level of p<0.05.

Results

Video Categorization and Characteristics

A total of 100 videos were initially included in the study. Of these, 74 (74%) were classified as useful, 16 (16%) as misleading, and 10 (10%) as irrelevant. Irrelevant videos were excluded, and 74 useful and 16 misleading videos related to AMD were included for further statistical analysis.

The video characteristics are summarized in Table 1. Both the audio and video qualities were generally good. Most videos (40%) were uploaded by television shows and social media platforms. More than half of the videos (57.8%) discussed AMD in terms of all aspects.

Table 2 summarizes the video metrics of the 90 videos analyzed. The mean JAMA score was 1.94±1.09 (moderate quality), the mean GQS score was 2.49±1.29 (moderate quality), and the mean DISCERN score was 36.82±16.53 (poor quality). When the videos were evaluated according to the uploader, no significant differences were observed in JAMA, GQS, DISCERN scores, and view ratio (Table 3).

However, the JAMA and DISCERN scores differed significantly according to video content (p=0.045 and p=0.021, respectively) (Table 4). Overall, the highest video quality scores were obtained for videos that discussed the disease in all aspects (general information about the disease, diagnosis, and treatment group). When Dunn’s multiple comparison test was performed, the JAMA averages of general information about the disease + diagnosis + treatment group were significantly higher than those of the treatment group (p=0.021). DISCERN averages of general information about the disease + diagnosis + treatment group were significantly higher than those of the treatment group (p=0.011). The view ratio was similar between the groups (p=0.061).

Correlation analysis showed a positive correlation between JAMA and GQS scores, between JAMA and DISCERN scores, and between GQS and DISCERN scores (r=0.862 p=0.0001; r=0.863 p=0.0001; r=0.874 p=0.0001), respectively. However, the scores were not correlated with the other parameters (Table 5).

After evaluating the data for all videos, the useful and misleading video groups were assessed separately and compared. Table 6 summarizes the video metrics of the two groups. Videos in the misleading group were newer than those in the useful group (p=0.006). The number of likes and view ratio were higher in the misleading group than in the useful group (p=0.019 and p=0.037, respectively). The JAMA, GQS, and DISCERN scores were statistically significantly higher in the useful group than in the misleading group (p=0.0001, p=0.001, and p=0.0001, respectively).

When the video and audio qualities and other characteristics of useful and misleading videos were compared, video content was the only significant difference between the two groups (p=0.0001). Videos giving all the details about the disease were most common in the useful group, whereas the most shared content in the misleading group was treatment (Table 7).

Discussion

The primary finding of this study is the poor to moderate quality and reliability of Turkish YouTube videos on AMD. When the factors that may affect the quality scores were examined, content was considered a main factor. The group that mentioned only treatment had lower quality than the other groups. When the useful and misleading groups were compared in terms of video quality, all video quality scores of the misleading group were lower than those of the useful group. Contrary to lower quality scores, the like and view rates of the misleading group were higher than the useful group. Interestingly, the most popular videos did not necessarily have the highest quality and reliability.

YouTube is a video-sharing website where patients can gain information about their diagnoses and treatments recently through open-access health videos. Although the number of people seeking health-related information has increased in recent years because of the ease of access to such information, it has been revealed that approximately one-third of patients do not trust such content (17). In addition to patients, healthcare professionals often watch YouTube videos. In particular, surgical videos are viewed by physicians on a learning curve or who want to discover and learn new techniques (18-20).

One of YouTube’s key features is that anyone can upload videos, regardless of background, medical qualifications, professionalism, or purpose. Therefore, health-related information on YouTube can be quite wide-ranging, including inaccurate or highly qualified information. In this process, obtaining online information from the appropriate sources is crucial because it can even change patient compliance with treatment (15). In our study, we found that 10% of videos were irrelevant to the subject, and 18% of the related videos were misleading. This result is consistent with studies conducted in different countries and with different disease titles in the literature (6, 7, 21, 22).

People often overlook the important factor of sound and image quality in YouTube videos. Good sound quality allows visitors to connect better with the video and engage more with the content. However, poor sound quality can reduce the video view rate even with the highest visual content. In our study, both the sound and video qualities were good. This may have occurred because most of the videos examined in our study were uploaded by TV shows and industries and, therefore, were prepared more professionally.

Video quality must be at the highest level, particularly for health-related videos. For this reason, several video quality questionnaires, such as HONcode, JAMA, DISCERN, and GQS, were used to evaluate the quality and reliability of the video content. In this study, the JAMA, DISCERN, and GQS scoring systems were used, and the results suggest that the YouTube videos on AMD are of  “poor to moderate” quality. When the scoring and uploading individuals were compared, it was observed that the highest JAMA score was obtained by ophthalmologists, and the highest DISCERN and GQS scores were obtained by medical faculties and academic centers, indicating that videos uploaded by users outside the health sector are of lower quality than those added by health professionals. In our study, the low rate of ophthalmologists (25.6%) and medical school/academic center unloaders (2.2%) compared with the total might be one reason for the low quality scores. Our findings are compatible with the results of other studies in the literature (23, 24). Non-profit organizations and academic-sourced videos are known to have the highest value for information. The high quality and reliability of healthcare professional videos can be attributed to several characteristics. First, healthcare experts base their recommendations on scientific evidence and clinical guidelines, which increases the accuracy and credibility of their information. Second, professional videos typically provide extensive coverage of the medical subject, encompassing various aspects, such as general information, diagnosis, and treatment alternatives, in a detailed manner. Third, accurate medical terminology enhances clarity, accuracy, and overall production quality. Hence, we advocate prioritizing videos produced by ophthalmologists or academic centers as the ideal educational resource for patients with AMD.

In this study, correlation analysis revealed a strong positive correlation between the DISCERN, JAMA, and GQS scores. This finding shows that the scales used in this study provide parallel results that reflect the reliability and educational quality of the videos.

The number of views indicates the popularity of videos, and the daily view rates reflect their currency (25). Our study found no correlation between JAMA, DISCERN, and GQS values and the number of views or view ratio; thus, high- and low-quality videos were watched at a similar rate. Our research shows that a video’s popularity level does not necessarily correlate with its quality. This situation may be caused by two different reasons. First, patients may not be equipped to evaluate video content quality and accuracy and may watch videos without making a choice. Second, the number of views on YouTube was considered an important parameter indicating video popularity. It can be considered that videos with a high view rate among people have high reliability (26). Today, video discoverability and reliability can be increased artificially by purchasing views. This situation may cause patients to think that these videos are better. Because patients cannot be consciously selective within this pool of videos, ophthalmologists and academic centers need to increase the number of videos that contain accurate information.

We did not find a significant difference in the view ratio between the videos uploaded by academic and non-academic ones, indicating that patients do not prioritize videos uploaded by healthcare professionals when searching. This result may be related to several factors. First, commercially purchasing views and likes, which are mostly used by non-academic uploaders, can increase the discoverability of videos. Although this created the impression that the video was artificially better, the situation may be the opposite. Second, despite their medical knowledge, physicians may struggle to explain complex medical information in a simple manner that patients can easily understand. The medical language and complexity of videos uploaded primarily by physicians and healthcare institutions may make them less accessible to the general public.

When health-related videos on YouTube are evaluated, not only videos that provide general information about the disease but also videos about diagnosis and treatment can attract more attention (27). When useful and misleading videos were compared in our study, we found that misleading videos mainly discussed treatment. Misleading videos might be more prominent than useful videos because they often share new, speculative, and promising content, whereas useful videos typically discuss proven and tried treatments. There is currently no treatment that can completely and permanently eliminate AMD, and there is no way to restore vision loss in advanced stages. Searching for new and alternative treatment methods may prompt patients to highlight new, misleading content. However, turning to unproven and unreliable treatment methods with the hope of a cure may harm patients both financially and healthfully.

Study Limitations

Our study has some limitations. Among this study’s weaknesses is the subjective nature of the analysis, which was attempted to minimize by having two experienced observers rate the videos and confirm the inter-rater reliability. Another limitation is that although terms in both everyday spoken language and academic language (yellow spot disease and age-related macular degeneration) were used in the search section, the search words used by the authors may differ from the search words of the patients and the videos they encounter. In addition, it is known that the keywords typed into the search engine may vary according to the geographic region of the searcher and the search history. As YouTube is an interactive and dynamic video-sharing platform, rearranging and uploading new videos may change the results. In this study, only Turkish videos related to AMD were evaluated. Evaluating Turkish videos and videos from other languages may change the results. However, because our primary target group was Turkish patients, we aimed to evaluate the quality of Turkish videos and the situation in Turkey by using videos in Turkish, which is the mother tongue.

Despite these limitations, examining the videos by two different ophthalmologists and the consistency of the scores across these evaluations were important in the present study. Furthermore, using multiple scales to assess various facets of the videos’ content and quality makes the study’s results comprehensive. Finally, our study reveals Turkey’s reality. To the best of our knowledge, this is the first study to evaluate the quality of Turkish YouTube videos on AMD published in Turkey.

Conclusion

Our findings demonstrate that the information quality of Turkish AMD-related videos on YouTube is poor to moderate. The number of irrelevant and misleading videos was high. Patients may not be able to distinguish between useful and misleading information. The potential effects of incomplete or unreliable YouTube videos on the patient-physician relationship, as well as patients’ perceptions and understanding of their disease, are important. In particular, patients who are in search of a new treatment for their disease may lose confidence in their current treatments with unproven options. Although the quality indices were lower than useful videos, it is noteworthy that patients watched misleading videos that mainly mentioned new and speculative treatments. Publicizing more health-related videos created professionally by health professionals may increase public health awareness, and the Internet may be a valuable tool for delivering high-quality and reliable information to the public.

Footnote