The Challenge of Censored Participants in Community-Based Research
Janine E. Janosky, Qing Sun, Susan B. Laird, and Anna Kostura
Recruiting minorities is critical to the accuracy of clinical research. Sex, age, season, time of day of appointment, and lapsed time between registration and scheduled attendance are among factors influencing whether those who register to participate keep their appointment.
Overcoming barriers is essential to get more members of underserved populations to participate in clinical research. Adjusting recruitment procedures to fit the lifestyles and routines of the targeted participants is recommended to achieve the goals of Healthy People 2010 and 2020. There is a paucity of research regarding factors that contribute to whether participants follow through after registration. In two community-based prevention education programs for minority women and men, a research team identified some of the factors affecting participation. Individuals were more likely to attend a program after registering for it based on a mix of personal and program variables.
Recruiting and retaining participants from underserved populations for clinical research is a challenge. Yet, overcoming these barriers is essential to reducing health disparities in these communities, with the ultimate goal of achieving Healthy People 2010 (United States Department of Health and Human Services, n.d.) and 2020 (United States Department of Health and Human Services, n.d.) objectives. There is little research on the factors that influence whether individuals actually follow through as a participant after they have agreed to participate in a research study. In an extensive review of minority recruitment and retention, Yancey et al. (2006) reported factors that appear to influence the barriers. These included community involvement, incentives and logistical issues, type of study design, and passive versus active recruitment strategies. Passive strategies are the traditional means of recruiting. Examples are print and television ads, which require individuals to contact the research staff. Active strategies are those in which the staff goes directly to the community or contacts individuals through the mail or by telephone. These strategies depend more on community relationship building,
While the number of studies on recruitment and retention of minorities for research is increasing, the literature is still sparse regarding why participants do not appear for their scheduled research appointments. There is some evidence that missed primary-care appointments result from such things as forgetfulness (Martin, Perfect, and Mantle, 2005; Hussain-Gambles, Neal, Dempsey, Lawlor, and Hodgson, 2004; Neal, Hussain-Gambles, Allgar, Lawlor, and Dempsey, 2005), inconvenient appointment time (Neal, Hussain-Gambles, Allgar, Lawlor, and Dempsey, 2005), mistrust (Neal, Hussain-Gambles, Allgar, Lawlor, and Dempsey, 2005), or lack of satisfaction with office staff (Yancey, Ortega, and Kumanyila, 2006; Lacy, Paulman, Reuter, and Lovejoy, 2004). There are fewer studies that examine demographic variables correlated with missed primary-care appointments. The results of such studies include younger age (Weingarter, Meyer, and Schneid, 1997; Cashman, Savageau, and Lemay, 2004; Lasser, Mintzer, Lambert, Cabral, and Bor, 2005; Waller and Hodgkin, 2000; Neal, Lawlor, Allgar et al., 2001), being female (Neal, Lawlor, Allgar et al., 2001), and being African-American (Lasser, Mintzer, Lambert, Cabral, and Bor, 2005). Distance between the site of care and the patient’s home may be a significant factor in determining whether the patient keeps his or her appointment; however, results of these studies have been inconsistent (Cashman, Savageau, and Lemay, 2004). Moreover, these findings regarding missed primary-care appointments may not be generalizable to missed appointments for research studies.
Several strategies to increase recruitment and retention of minorities into clinical research have been proposed (Janosky, Kohley, Sciullo, et al., 2006; Janosky, Laird, and Sun, 2008; Davis, Bustamante, Brown, et al., 1994; Sadler, Peterson, Wasserman, et al, 2005). A number have been implemented by The Center for Primary Care Community-Based Research (CPCR) at the University of Pittsburgh’s School of Medicine. (Janosky, Kohley, Sciullo, et al., 2006). Specific strategies include partnerships with community and religious leaders, partnering with Commonwealth of Pennsylvania state health improvement project teams, recruiting directly in the community (health fairs, farmers markets, and community events), and more traditional or passive strategies such as television and print media. The costs associated with these various strategies are reported elsewhere (Janosky, Laird, Kohley, et al., 2008). Additional implementation strategies that CPCR has utilized include reminder phone calls, “sorry we missed you” letters sent subsequent to a missed research appointment, and providing a menu of available dates and locations for education sessions in an effort to make dates/times more convenient. The issue of participants’ missing or not following through on scheduled research appointments has the potential to directly impact the conduct and cost of the research.
The overall objective of this investigation was to describe the factors that contribute to participation versus censored participants in two community-based education programs. One program was designed to reduce cardiovascular disease in minority women, and the other addressed stroke and prostate cancer in minority men. Censored participants were defined as those who self-registered and consented to attend a specific self-selected community-based education program but failed to follow through. Participants who rescheduled their attendance in advance of the scheduled session were not considered as censored. Data were derived from intake forms that indicated date and time an individual registered for an educational program, referral source, where a session was being held (in the community or on a university campus), demographic information (address, sex, age), and project data indicating whether individuals who registered actually participated.
Our team collected data during the recruitment phase for the two studies: “Minority Women’s Heart Initiative,” funded by the Health Resources and Services Administration; and “Innovative Strategies in Reducing Stroke and Prostate Cancer in African-American Men,” funded by the Pennsylvania Department of Health. Both were community-based prevention and care education programs for men 35 and older and for women 25-75. The “Innovative Strategies” study utilized a community-based intervention that attempted to remove barriers to patient education with regard to stroke and prostate cancer prevention and to foster compliance with prevention and treatment activities.
The purpose of the minority women study was to identify and monitor a cohort of women at risk for cardiovascular disease, with a special emphasis on including women who identified as African-American, and to educate the community regarding prevention of cardiovascular disease. Both studies were under the direction of the primary author. The research was conducted through CCPR (Janosky, Laird, and Sun, 2008; Janosky, Laird, Kohley, et al., 2008).
Data from intake forms indicated date and time an individual initially registered for an educational session, referral source, where the session was being held (in the community or on campus), and demographic information (address, sex, age). Additional data available through CPCR records indicated the date and time the educational session was held, which participants attended the 90-minute session, lapse of time in days from registration to initial date of the session, and distance (calculated from zip codes) in miles from the registrant’s home to the site of the program.
Programs were scheduled in conference centers on the University of Pittsburgh campus and in the community at houses of worship, community centers, and the like. Separate sessions were scheduled for men and women according to each study’s protocol in each season of the year. Dates and times varied to allow the most flexibility for participants (midday, evening, weekends, etc.).
Distance from the participant’s home to the site where the research session was held, lapse of time between scheduling and participation, age of participant, season, starting time (before or after 5 p.m.), where the program was held (university or community), and referral source were all used as possible predictors of attendance. For comparisons between groups, chi-square analyses were used for categorical variables, and independent t-tests or two-way ANOVA was used for continuous variables. For the examination of concomitant effects, where suitable, either a linear regression or a logistic regression was used. Though statistical significance was defined as p < .05, actual significance levels are presented for other cutpoints.
We collected data from August 2005 until July 2006. A total of 872 individuals registered for both studies; 375 (43.0%) subsequently participated in the respective studies (27.5% men and 49.5% women).
Table 1 presents a summary of the results for those who attended and those who did not attend by the aforementioned predictors. Table 2 presents a summary of the results by sex and attendance status for the predictors of interest.
The mean age was 47 years (sd = 10.4). The mean age of those who attended (46.05) and those who did not attend (49.14) was statistically significant (p < .001). The men’s study of prostate cancer and stroke required that participants be 35 or older, while the women’s study of cardiovascular disease included those 25 to 75. Nonetheless, the differences reported here between the ages of attendees did not seem to be substantially influenced by this criterion. There were, however, significant differences between men and women. Registered men attended at the rate of 27.5%, and registered women attended at the rate of 49.5% (p < .001).
Table 1 also shows a statistically significant difference for attendance by season of the year (p = .005), where the session was held (p < .001), how the participant heard about the study (p < .001), time of session (p = .03), proximity to site (p < .001), lapse of time in days (p < .001), and age (p < .001). Our findings suggest participants are more likely to attend in the winter and spring, if the session is held in the community rather than on campus, and if the session is held before 5 p.m.
Table 2 shows the findings for each of the predictors by attendance status and sex. For those attending, season of session (p < .001), day of session (p = .001), where the session is held (p = .03), and how the participant heard about the study (p = .037) all differed for men and women. The attendance rate for women was higher when the session was held on a weekday; however, men were more likely to attend on the weekend. Women attended more in the winter and spring, while men were more likely to attend sessions in the summer or fall.
Women were more likely to attend a program if it was held within 14 days of their registration. Men, on the other hand, were more likely to participate if the program was held within 19 days of registration (p < .001).
Individuals were more likely to attend if the program was in the spring and winter than in the summer or fall (p = .021), and were more likely to attend a program held in the community (p < .001) than at a university site. Season made no difference (p = .361) for those who did not attend. Men were more likely to attend if the session was within 4.8 miles of their home zip code, and women within 5.2 miles (p < .001).
A binary logistic regression was performed with predictors that were statistically significant from the univariate comparison including where the program was held, how the participant heard about the study, season of the year, lapse of time from registering to participating, and age of the participant. The overall correct predicted percentage was 74.0%, and the cut value is .460. Significant predictors included the site of the session (p < .001) and how participants heard about the study (p = .001), driven specifically by television (p = .007), age (p < .001), season (p = .005) and lapse of time (p = .006) (Table 3).
These findings add to our understanding of what works in recruiting minorities into research studies. Scheduling programs in the community during the winter and spring and in the evening increases the probability that those who register for a study will subsequently participate. There were significant differences in attendance with respect to sex. Other predictors include how the participants heard about the study, age, and time lapse between registration and scheduled attendance. The reader is cautioned against generalizing these findings to other populations and settings for the following reasons:
1. Other minority groups might not respond in the same way as our participants, all self-identified as minority and/or African-American.
2. Recruiting for purposes other than a community-based prevention and care educational program result in difference rates.
3. A different conclusion might be reached if a broadened definition of censoring is used; the definition of censored used here was nonattendance at the scheduled and consented education session.
4. Generalizability to other climates might be limited since the study was conducted in Pittsburgh, which has four distinct seasons.
There has been a wealth of research on minority participation in research, including influences and barriers. However, unlike missed primary care appointments, there has been little research into missed research appointments by minority participants. Woolf et al. (2000) investigated the differences between office-based patients who consent to be surveyed at home and have their records reviewed and patients who do not consent. This study indicated that patients who consented to have their records reviewed were older, included fewer women and African-Americans, and reported poorer physical function than those who did not give consent. Through the use of multivariate analysis, older age, male sex, and lower functional status were significant predictors of giving consent.
Similarly, the current study highlights differences between research participants who register for a study and subsequently attend or do not attend, with the latter considered as censored here. Differences between our study and that of Woolf et al., include a differing rationale, different modes of invitation to prospective participants, profiles of possible participants, and other variables.
Our study adds to the understanding of some of the influences, limitations, and obstacles to minorities actually following through with research appointments once they have registered. Not only must researchers consider what brings minority participants to the door of interest and registration for research studies, but they must also consider what will facilitate their full participation and completion of a study. Ushering minority participants to and through the front door for participation in research is not enough. Important factors to consider and master to have minority participants register and successfully complete a research study include: (1) types of locations where they will most likely participate and follow through; (2) season of year and time of day that is most convenient; (3) proximity of the research site to the participant’s home; and (4) lapse of time between registration and study visit. Knowing these factors prior to initiation of a study will give researchers a better chance of meeting their goals.
Identifying and negotiating the factors found to be significant are part of the fabric and crux of community-based research. Research is not simply about the achievement of clinical or epidemiological goals, but for maximum success and participation, we must achieve these goals in harmony with participants’ lifestyles, environments, and desires.
Further research in this and related areas is needed to specify additional challenges to the successful recruitment and retention of minority participants in research. Navigating these challenges will be crucial to eliminating disparities and achieving the goals of Healthy People 2010 and 2020.
Cashman, S.B., Savageau, J.A., Lemay, C.A., & Ferguson, W. (2004). Patient health status and appointment keeping in an urban community health center. Journal of Health Care for the Poor and Underserved, 15, 474-488.
Davis, D.T., Bustamante, A., Brown, C.P., Wolde-Tsadik, G., Savage, E.W., Cheng, X, & Howland, L. (1994). The urban church and cancer control: A source of social influence in minority communities. Public Health Reports, 109(4), 500-506.
Hussain-Gambles, M., Neal, R.D., Dempsey, O., Lawlor, D.A., & Hodgson, J. (2004). Missed appointments in primary care: Questionnaire and focus group study of health professionals. British Journal of General Practice; 54, 108-113.
Janosky, J.E., Kohley, J., Sciullo, L., & Robinson, J. (2006). Addressing the challenges behind minority recruitment: Traditional and non-traditional recruitment strategies in community-based research. British Medical Journal, 333, 300-302.
Janosky, J.E., Laird, S.B., Kohley, J., Sun, Q. (2008). Cost of recruiting participants into a primary care community-based research registry. (Submitted Manuscript).
Janosky, J.E., Laird, S.B., & Sun, Q. (2008). Content and context of a research registry for community-based research. Journal of Community Health, 33, 270-278.
Lacy, N.L., Paulman, A., Reuter, M.D., & Lovejoy, B. (2004). Why we don’t come: Patient perceptions on no-shows. Annals of Family Medicine, 2(6), 541-545.
Lasser, K.E., Mintzer, I.L., Lambert, A., Cabral, H., & Bor D.H. (2005). Missed appointment rates in primary care: The importance of site of care. Journal of Health Care for the Poor and Underserved, 16, 475-486.
Martin, C., Perfect, T., & Mantle, G. (2005). Non-attendance in primary care: The views of patients and practices on its causes, impact and solutions. Family Practice, 22, 638-643.
Neal, R.D., Hussain-Gambles, M., Allgar, V.L., Lawlor, D.A., & Dempsey, O. (2005). Reasons for and consequences of missed appointments in general practice in the UK: Questionnaire survey and prospective review for medical records. BMC Family Practice, 6, 47.
Neal, R.D., Lawlor, D.A., Allgar, V., Colledge, M., Ali, S., Hassey, A., Portz, C., & Wilson, A. (2001). Missed appointments in general practice: Retrospective data analysis from four practices. British Journal of General Practice, 51, 830-832.
Sadler, G.R., Peterson, M., Wasserman, L., Mills, P., Malcarne, V.L., Rock, C., Ancoli-Israel, S., Moore, A., Weldon, R., Garcia, T., & Kolodner, R.D. (2005). Recruiting research participants at community education sites. Journal of Cancer Education, 20(4), 235-239.
United States Department of Health and Human Services. (n.d.). Healthy people 2010. Retrieved September 12, 2008, from http://www.healthypeople.gov/.
United States Department of Health and Human Services. (n.d.). Healthy people 2020. Retrieved September 12, 2008, from http://www.healthypeople.gov/.
Waller, J., & Hodgkin, P. (2000). Defaulters in general practice: Who are they and what can be done about them? Family Practice, 17, 252-253.
Weingarten, N., Meyer, D.L., & Schneid, J.A. (1997). Failed appointments in residency practices: Who misses them and what providers are most affected? The Journal of the American Board of Family Practice, 10(6), 407-411.
Yancey, A.K., Ortega, A.N., & Kumanyika, S.K. (2006). Effective recruitment and retention of minority research participants. Annual Reviews of Public Health, 27, 1-28.
Woolf S.H., Rothemich, S.F., Johnson, R.E., & Marsland, D.W. (2000). Selection bias from recruiting patients to give consent to examine data for health services research. Archives of Family Medicine, 9, 1111-1118.
About the Authors
Janine E. Janosky is vice provost for research and professor of mathematics at Central Michigan University. Qing Sun, Susan B. Laird, and Anna Kostura are all graduate assistants at the University of Pittsburgh. Janosky may be reached at Janine.E.Janosky@cmich.edu.
This study was completed while Janosky was executive director of the Center for Primary Care Community-Based Research at the University of Pittsburgh. The authors acknowledge the contributions of the center’s staff. The Pennsylvania Department of Health and Health Resources and Services Administration funded the research.
Research form the field
In planning JCES, the editors and editorial board wanted to include research from the field. We believed it would become one of the best read sections in the journal. We had all heard, or said ourselves: “I’ve got all this good stuff I’d like to share with others, but where can I publish it?” You can send it to JCES’ Research from the Field. We begin with two reports from University of Alabama scholars. We start locally only because our scholars knew about the section. Now that you know, we hope you will write up your field research and send it in. And spread the word.