Significant health disparities exist between Hispanics and the general US population, complicated in part by communication, literacy, and linguistic factors. There are few available Spanish-language interactive, technology-driven health education programs that engage patients who have a range of health literacy levels. We describe the development of an interactive virtual patient educator for educating and counseling Hispanic women about cervical cancer and human papillomavirus. Specifically, we describe the iterative design methodology and rationale, usability evaluation, and pilot testing of the system with Hispanic women in a rural community in Florida. The pilot study findings provide preliminary evidence of the feasibility of the proposed patient education approach. The proposed application and the lessons learned will prove beneficial for future work targeted towards different cultural populations.
Keywords: Virtual Human, Health Disparities, Health Literacy, Patient Education, Cervical Cancer
1 INTRODUCTIONWhile quality of health care in the United States (US) is steadily improving, there remain large disparities in health care access and quality between non-Hispanic Whites and minority and low-income groups, particularly in the context of cancer prevention. Hypotheses concerning these disparities have centered on ethnic differences in risk factors, psychosocial and cultural factors, knowledge of cancer, and quality of care [8]. Research has identified culturally competent intervention methods, including deliberate modification of health care environments to serve patients of various cultural backgrounds, as critical in addressing these disparities [3]. Many studies have examined cultural competence within face-to-face interventions; however, few studies have done so in the context of health Information Technology (IT). Sources of media, such as the Internet, radio, television, and newspaper/magazines, have been established as an effective and desired way to deliver health information to Hispanic populations [10]. Furthermore, health education delivered through multimedia and interactive technology has been shown to assist Hispanics in improving health literacy [1].
In this study, we have developed a culturally-tailored, interactive tablet application to deliver information about cervical cancer (CC) prevention and screening to a low literacy, rural population of Hispanic women living in a migrant farmworker community in central Florida. In order to ensure the efficacy of the system, we designed a virtual patient educator (VPE) using an iterative, participatory design approach to embody the characteristics most desired by community members to deliver the health information. The VPE is deployed on a tablet computer with headphones in the waiting room at the clinic. For a patient to complete all educational modules and knowledge checks the VPE delivers, it takes approximately 25 minutes. Feedback regarding feasibility, acceptability, usability, and information retention was iteratively collected and synthesized to identify population-specific design principles and insights.
1.1 Disparities in Cervical Cancer Experienced by HispanicsHispanics represent the fastest growing minority group in the US, accounting for an estimated of 17.8% of the US population in 2016 [22]. However, Hispanics also exhibit the lowest level of health literacy among any demographic group [15] and continue to receive worse access and lower quality of care than non-Hispanic Whites [20]. U.S. Hispanic women in particular face marked disparities in CC, with incidence nearly 40% higher than that of White women [21]. Contributing factors to this population’s lack of participation in CC screening include poverty, lack of insurance, immigration status, difficulty accessing a provider who offers CC screening, lack of a regular health care provider, concerns regarding receiving CC screening for a provider of a different gender, low acculturation, religious beliefs that emphasize modesty, lack of knowledge, fear, and embarrassment [16]. In addition, provider behaviors and practices have been found to play a role in these disparities [23], leading to an increasing recognition of the importance of a culturally-informed health care workforce.
Although the VPE application is designed to address the particular needs of the Hispanic migrant farmworker community, we anticipate that this application and the lessons learned will prove beneficial for future work targeted towards different cultural populations. Studies of health care disparities in Hispanic populations in conjunction with other minority immigrant populations have found commonalities in health behaviors, practices, and utilization of health services between the different cultures [11].
1.2 Patient Navigation to Address Cervical Cancer DisparitiesThe system discussed in this paper serves as a supplemental technology for a patient navigator (PN) intervention to address CC disparities experienced by Hispanic women living in a rural farm-worker community in central Florida. PNs assist individuals in overcoming obstacles from screening to treatment and in coping with challenges during survivorship by addressing issues including lack of transportation, financial and insurance barriers, lack of childcare or language translation, low health literacy, or low literacy. The local Community Cancer Network Cervical Cancer Patient Navigation Program (CCN-CCPNP) was created in response to 3 gaps identified by our community partner, Catholic Mobile Medical Services (CMMS) [24]. The gaps include: (1) delays in reporting results of Pap tests to patients; (2) lack of coordination of follow-up care for patients with abnormal findings; and (3) lack of culturally and literacy relevant education about CC and human papillomavirus (HPV). Findings from formative research with community members and health care providers indicated members of the population had a general understanding of the purpose of Pap testing, but virtually no knowledge regarding HPV as the cause of CC [24]. Many women felt shame and embarrassment when undergoing a Pap test but also wished to learn more about their bodies. The women interviewed preferred getting Pap test information from another woman who has herself received the test. In addition, community members reported enjoying visual methods of learning and indicated that they were open to using technology to receive health information if guided appropriately. To address these learning needs, the CCN-CCPNP team initially developed a low literacy, Spanish language CC and HPV educational program was orally delivered by a PN. However, in a busy clinic environment, it is difficult for the PN to find adequate time to provide this educational program to all women who receive navigation.
1.3 Virtual Agents for Health EducationTailored healthcare information has been shown to promote better understanding of content and effectively improve health related behavior [17]. Tailored health information can also influence the extent to which underserved individuals retain knowledge, in turn leading them to follow up on recommendations by providers. One technology that has been shown to leverage the affordances of tailored healthcare information is virtual agents. They have shown particular promise in healthcare applications due to their adaptability to various situations and realistic representation of a human healthcare provider. Virtual agents can also deliver health care information consistently and accurately while maintaining a calm demeanor and avoiding trainer bias. Additionally, evidence has shown that virtual agents can improve understanding and acceptability of online health information by verbally explaining information normally viewed on a static page [5].
In recent years, the number of studies investigating the uses of virtual agents in healthcare has grown [9]. Virtual agents have been developed to address a variety of health topics, including hospital discharge [4], childhood obesity [19], and mental health [18]. While there has been in an increase in consumer health information technology targeted at the Latino population, the majority of technology-enabled health promotion programs are aimed primarily at educated, health-literate individuals and are not tailored to diverse Latino subpopulations [7, 12]. Novel health communication technologies such as virtual humans that do not rely on advance reading or computer skills have the potential to break down health communication barriers and culturally adapt programs to diverse populations.
To address CC disparities and augment a PN intervention, the proposed work leverages virtual agent technology to develop an efficient, consistent, and comprehensive interactive application that delivers Spanish-language education about CC and HPV. This study aims to supplement existing literature by elaborating population-specific design principles for design of health-information technology, as well as extend state-of-the-art technology mediated approaches to a medically underserved, low-literacy, and non-English speaking rural populations. Furthermore, this is the first design, development, and evaluation of a culturally tailored virtual human technology for Hispanic women in a farmworker community.
2 INITIAL FEASIBILITY STUDYBefore design and development, an initial feasibility study was conducted in order to assess the previous exposure of our female Hispanic farmworkers to computers and technology and the preliminary acceptability of a virtual agent application among this population [25]. Contradictory to our expectations that female Hispanic farmworkers have limited access to computers, after interviewing 26 Hispanic women who received health care at CMMS, 61.5% of them reported to have used a computer before. Additionally, 93.8% of those who had previous computer exposure felt confident in their ability to use one in the future. Regardless of whether or not they had prior experience using a computer, all women surveyed indicated an interest in receiving health information from a computer.
Regarding acceptability of the application, data from the feasibility study indicated a high degree of enthusiasm for technology-based Spanish language health education interventions to supplement existing healthcare resources. Participants also showed interest in adaptation of this technology to address various health topics, including disease prevention and early detection, breast health and breast cancer, and nutrition and diet. One-hundred percent of women interviewed emphasized that the application should be adapted from the English prototype into their native Spanish language. All women also reported that they would be interested in using this type of computer program to obtain health information and that they would trust health information provided through a similar program.
3 DESIGN AND DEVELOPMENT OF VPEWe used an iterative design methodology with distinct but dependent phases in the development of the VPE. Our process consisted of the following phases: 1) developing the design and functional requirements for the application; 2) prototype development of the user-interface and the health content script; and 3) usability testing. The iterative design process, from project start to completion of the user studies, lasted 1.5 years. We used a multi-faceted approach to designing a system that would effectively teach womenn about CC prevention methods using an interactive, technology-driven health education program.
The application was developed with Unity and was targeted for Android devices higher than Android 4.3. For our testing, we installed the application in Google Nexus 9 inch tablets which we handed out to the users. The tablets used during pilot testing came pre-installed with the application and were automatically configured to log how participants used the application. For each participant, a log file collected click behavior, the time spent in each module, and the flow from one module of the application to another.
3.1 Context of UseThe deployment site for the system is CMMS in Dover, Florida, a program that works with volunteer physicians and other health and social service professionals to provide free medical, health promotion and social services to a rural, low-income population. Services provided by this program include acute primary medical care and follow-up and referrals, routine well woman exams, health promotion and disease prevention, and health screenings for high blood pressure and diabetes. The program primarily provides services to persons with household incomes less than 200% of poverty level and without health care coverage. This study specifically focused on the population that attended one of the clinic locations for routine well woman exams. Currently, the clinics make use of a lay PN model of health care facilitation. PN seeks to overcome barriers that patients face in obtaining recommended health care. PN interventions are associated with an increase in timely receipt of cancer early detection services [14].
The VPE primarily interacts with users through a predefined script (see section 3.2 for a description of the content in the script). Users traverse the script by selecting buttons to respond to questions as inputs, with the application then routing the users to different topics. After several topics, a review session is presented to check the user’s knowledge comprehension level before proceeding to further topics and discussions. Each topic in the script has its own distinct and independent state. Users can choose to stay within the current topic by pressing the ’Repeat’ button or transition to the next state by pressing the ’Yes’ or ’No’ buttons which trigger different transition conditions (see Figure 1). Users can also move back to the previous topics for review (’Back’ button) or move to the next topic at the end of review session (’Next’ button). The VPE verbally states the functionality of buttons shown on the screen at the end of every module to reinforce user understanding of the interface.
Figure 1.Current VPE Design. The VPE is positioned on the left side of the interface. A pop-up, supplementary image in the top right conveys visual information corresponding to each module. Navigation buttons are displayed on the bottom of the screen.
3.2 VPE Content and Dialogue StructureThe content and dialogue delivered by the VPE was adapted from a low-tech PN-delivered CC education intervention developed by the research team. The organization of content and structure of delivery were motivated by the Elaboration Likelihood Model of Persuasion (ELMP) and Social Cognitive Theory (SCT). Studies have shown that ELMP and SCT are effective models for health promotion [2, 13]. Using ELMP, the VPE program can be directed by each individual patient based on her preferences for obtaining information about CC and HPV and designed in such a way to motivates patients to carefully consider information provided, as persuasion using this central route leads to more long lasting attitude and behavior change.
The final system contains 51 unique dialogue states including 3 review modules, each covering approximately 4 content modules. Figure 2 illustrates an example of VPE dialogue structure. Patients are given the option to skip information with which they are already familiar or in which they are not interested during the transitions between modules. Additionally, review modules are given to the user at several points in the dialogue to test patient understanding of key points with a few yes/no questions. If a patient incorrectly answers a question, the VPE directs the patient to the appropriate module to review of the material addressed. The results of each review were recorded for each user and were stored for later review. The final module of the application allows users to review the content of the application either by the specific module in which they are interested or through a comprehensive review module.
Figure 2.Dialogue Structure. Sample dialogue navigation from module to review. VPE quotes are listed on the left with appropriate screenshots and dialogue order displayed beside them
3.3 Virtual Agent Character DesignPrior research has shown that familiarity is an important factor in promoting the acceptability and trustworthiness of virtual agent applications [6]. Therefore, the design of the virtual agent was inspired by the current PN at the CMMS clinic. Using CMMS’ own PN as a model allowed us to leverage the established trustworthiness of the the human PN while also creating compellingly novel interactions. A screen shot of the current virtual agent/prototype design is depicted in Figure 1.
3.4 A Typical Session with the VPEOnce the patient had been guided to a private room in the clinic, the PN provided the patient with an orientation session in which they are told that the character will talk to them and they just need to listen and interact with the computer when appropriate. The VPE proceeds through the scripted content systematically, describing each section before moving on to the next. VPE modules provided information about the female reproductive system, CC, CC screening, HPV, or the development of cancer. In additional, some modules encouraged women to discuss these topics with their health care providers and prompted them to request CC screening.
We wanted to ascertain the patient’s ability to use the application and analyze common learning paths, as well as measure patients’ content understanding and retention. To meet this design goal, we developed a back-end data collection system that recorded user interaction data in order to help identify common user patterns and evaluate the general ease of use. The data collection in our prototype system is designed to record the timestamp and accuracy each time a click is made, which provides a measure of comprehension of instruction and suitability of language used. The duration of time that a user spends in each module of the application is also recorded. This quantified information allows a measurable evolution through the software design process, and ultimately, identification of both positive and problematic software features. Review modules were given to the user at several points in the dialogue, the answers to which are also recorded affording a measure of content understanding and retention. During a review, the user is verbally asked a yes/no question by the VPE and prompted to click one button respond ’Yes’ or another button to respond ’No’. Users are also able to click a button to have the VPE repeat the question. The ability to automate and quantitatively assess a patient’s use of the system provided meaningful usability measurements that ultimately influenced design changes.
4 EVALUATING THE VPEThe goal of this section is to describe the design, usability, and efficacy of the VPE. A total of 66 participants were recruited to test and provide feed-back about the VPE. 43 participants (Mage = 39, Meducation = 8.5, Income < 30, 000$) reported feed-back bout the design of VPE, 8 participants (Mage = 47, Meducation = 6.6, Income < 20, 000$) reported feed-back about the usability of the VPE, and 15 participants (Mage = 39, Meducation = 8.3, Income < 20, 000$) reported feed-back about what they learned from the VPE materials. All participants were required to be: 1) Hispanic; 2) female; 3) between 21 and 70 years; 4) able to speak Spanish; and 5) willing and able to provide informed consent. The following sections present the three iterative testing phases we conducted.
4.1 DesignWe conducted multiple iterations of design testing in order to determine the most acceptable design for the target population. A screen shot of the current virtual agent/prototype design is depicted in Figure 1. Various concepts were shown to participants during the design testing and iteratively modified according to the feedback from the intended audience.
Interviews were conducted by a PN in a private area located at the CMMS stationary clinic, and demographic data were collected in addition to the criteria for study eligibility. Each participant reviewed components of the VPE and then answered questions based on a standardized interview protocol. The interview protocol targeted specific sets of design characteristics as determined by the specific round of testing. In our analysis, quantitative summaries were generated for questions that had yes/no responses. For more subjective questions, the valence of the given response was prioritized over specific feedback in our analysis. The program design and overarching aesthetic appeal were refined based on the information generated from these interviews, and with the goal of creating an enjoyable and comfortable virtual setting for patients. Based on the extensive, iterative feedback collected, the majority of design and functionality modifications to the application were completed during this phase.
100% of participants provided positive feedback regarding the virtual agent’s initial appearance, authenticity, and trustworthiness. Prior research has shown that familiarity is an important factor in promoting the acceptability and trustworthiness of virtual agent applications [6]. Therefore, the design of the virtual agent was inspired by the current PN at the CMMS clinic. Using CMMS’ own PN as a model allowed us to leverage the established trustworthiness of the the human PN while also creating compellingly novel interactions. Individual feedback from early rounds of testing includes implementing more movement by the VPE to reinforce authenticity, adjusting the VPE’s hairstyle or clothing, and creating a bilingual version of the application. 80% of participants indicated that the VPE appeared like a real person, it appeared trustworthy, and they would trust the health information it provided them. Overall, participants expressed high levels of empowerment gained from the application, as indicated by their selection of ’Esperanza’, which means ’Hope’ in Spanish, as the name for VPE.
Feedback regarding the VPE’s explanation of concepts, without regard to module-specific dialogue, was overwhelmingly positive. 100% of participants surveyed preferred the version that used dialogue recorded by the local PN over unfamiliar individuals. The version based on the PN was the only version that 100% of participants agreed was familiar, authentic, real, and easy to follow and hear. The majority of participants also seemed to identify culturally with this version, as 83% of participants labeled the cultural identity of the VPE as Mexican, the same nationality as almost all of the participants. While 100% of participants understood the wording used by the VPE, feedback related to speed and visual presentation varied. 40% of participants requested slower animation of content, particularly with respect to the VPE’s audio explanation. Participants who provided feedback about content speed indicated that images disappeared or changed too quickly to allow for full comprehension of the material. 14% of participants also requested more images to supplement the audio explanation. Overall, participants responded positively to the content provided by the application. While participants said they use a diverse range of sources of health information, technological and otherwise, 100% of participants said that a computer program would be useful for giving people health information.
4.2 UsabilityFollowing design interviews, we conducted a user study to determine the usability and acceptance of the VPE using a high-fidelity prototype of the application, with the patients using the system on a tablet device and then taking part in an in-depth interview.
In contrast to the design testing phase, usability interviews focused on acceptability of design features as well as the technical usability. A prototype was presented to participants to evaluate usability of the technology, interface usability, and content comprehension through observation and an interview. These measures are meant to ensure that patients can perform intended system tasks efficiently, effectively, and satisfactorily.
The on-site PN conducted usability testing with participants in a private area located at the CMMS stationary clinic. The PN initially provided participants with instructions on how to use the VPE and then allowed them to interact with it as they wish, assisting participants with technological difficulties and answering questions as needed. Once the participant had used the VPE as long as she desired, the PN began the usability interview. Using a standardized form, the PN recorded observations about each patient’s VPE use and evaluated the usability of VPE application interface, acceptance, preferences, and the patient’s overall perceptions of the application, as well as demographic data.
All participants completed the interaction without any problems, with sessions lasting 2 to 20 minutes (mean 10 minutes). Patients repeated an average of 1.75 modules, with the most commonly repeated modules containing a description of HPV and how it is contracted. Most patients completed every review question correctly. The most challenging review question asked the patient to identify the virus that causes CC. Fewer than 50% of participants correctly answered this question. Every other review question had a correct completion rate of 100%. Although fewer than half of the patients answered this question correctly, the module containing the associated content for this question was one of the least frequently repeated modules. Additionally, the module for which the patients most frequently responded ’No’ (55%) is the transition which asks if the user has ever heard of the cervix.
4.3 Learner VerificationFollowing usability testing, the PN conducted learner verification (LV) interviews. The purpose of the LV testing was to assess the likelihood the VPE will influence the intended audience, Spanish-speaking women in a farmworker community, to obtain regular Pap tests. The LV process focused on the following elements of the VPE as guided by ELMP and SCT: acceptability, attraction, understanding, self-efficacy, and persuasion. The research team developed a semi-structured interview guide including 23 open-ended and neutral questions, as well as demographic questions. Probes were used to gather additional information.
Prior to interviews, the PN provided participants with instructions on the VPE, allowed them to interact with it as they wished, assisted participants with technological difficulties, and answered questions. Once the participant had used the VPE as long as she desired, the PN conducted a LV interview. Interviews lasted approximately 45 minutes. Interviews were recorded, transcribed verbatim, and translated into English. Data obtained from each participant were used to assess the participants’ ability to describe key concepts (e.g., why a Pap test is important).
100% of LV participants found the VPE to be both acceptable and attractive. 87% of LV participants understood the material provided in the VPE and provided specific information regarding what they had learned, but there was still come confusion regarding the content regarding HPV and HIV. In addition, LV participants noticed some new technical issues, such as diagrams depicting the process of cells changing to cancer that disappeared too fast, as well as issues with the low volume of the voice in places. In response to this feedback, changes were made in the application to fix transition bugs, correct confusing graphics, raise the voice pitch, and slow down the animations before final pilot testing occurred.
5 DISCUSSIONWe designed a Spanish-language VPE that could teach female Hispanic farmworkers about effective CC prevention practices. Design features for this population included the use of an acceptable virtual agent interface, redundant modalities for the medical information (on-screen images, text, and recorded speech), and integrated comprehension checks.
Collectively, our user studies involved 66 participants from the population of female Hispanics living in a migrant farmworker community in central Florida. Participants found the VPE very easy to use with minimal guidance from the PN. Overall, participants reported high levels of satisfaction with the system, with very few reporting reservations receiving medical information from an virtual human.
The development process of this culturally tailored technology has revealed many important insights into the design considerations and technological adaptations for health care technology that best serve the population of female Hispanic migrant farmworkers. Feedback collected during design interviews shows that the women strongly approved of the VPE when it resembled a community member and health care worker they already respected and trusted. Furthermore, the majority of participants identified the VPE as similar to themselves culturally, which distinctly contributed to the increased trustworthiness and anthropomorphism participants attributed to the VPE. Most learner verification participants understood the material provided by the VPE and provided specific information regarding what they had learned, demonstrating high information retention. The consistency of positive feedback and perceptions of the VPE application demonstrates the potential for the VPE to augment health education.
5.1 Future WorkThere are a number of different ways in which the findings from our study can be used in future work in health IT. The immediate plan for this line of research is to assess the efficacy of the VPE with or without PN on CC screening outcomes among members of the community who are not already connected to health care and in need of CC screening. In addition, there are a number of different ways in which the findings from our study can be used in future work in health IT. Spanish-speaking women interviewed for the feasibility study indicated that they would appreciate a virtual agent application that provided information for a wide range of health conditions. The VPE that we developed in the present study has the potential to be further developed, with content added to provide information for a range of different health conditions of interest to the intended audience, including breast health and breast cancer screening, family planning, and nutrition and diet. The intensive and iterative process of design of the application also provides a framework for developing other health applications to culturally diverse populations.
AcknowledgmentsThe authors wish to thank the women who assisted us with the development of the VPE application. Research reported in this publication was supported by the National Cancer Institute and the Office of Research on Women’s Health of the National Institutes of Health under Award Number R21CA167418.
FootnotesCCS CONCEPTS
• Human-centered computing → HCI design and evaluation methods; Interactive systems and tools
ReferencesRetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.3