Review by Jennifer W. Purcell
Kennesaw State University
This volume is the product of a five-year collaborative study on the impact of community leadership development programs spanning six states. The book is the third volume of the Rural Studies Series sponsored by the Rural Sociological Society, and the research presented was funded in part by the U.S. Department of Agriculture’s National Institute of Food and Agriculture. The purpose of the book is to engender thoughtful theory and evidence-driven design and redesign of community leadership development programs.
Organization and Content
The book’s nine chapters are organized into two parts. In Part I, the authors dedicate six chapters to the presentation of their analysis of data collected on community leadership development. In Chapter One, the authors provide a historical context of community leadership development (CLD). They define community-based programs as those that are (1) focused on community issues and needs; (2) organized and managed locally or by a community group; and (3) sponsored by local resources. Although CLD programs are diverse in their target audience, curriculum, delivery, and specific community-level outcomes, the authors contend the common element across programs is their goal to “increase local leaders’ capacity to improve the community’s well-being” (p. 21). The authors suggest investment in leadership development and capacity-building responds to observed deficits in the community. They contend current and future community leaders in smaller, rural communities have inadequate preparation to address the complexity of social factors impacting their communities. In closing Chapter One, the authors propose two significant reconsiderations for CLD. First, they suggest CLD program sponsors and practitioners examine evaluation strategies for how program outcomes for individual participants, organizations, and communities are achieved. Identifying how CLD achieves outcomes at various levels is critical to evaluating and improving programs. Second, the authors call for a conceptualization of leadership that moves beyond individual roles and talents to one that recognizes the collaboration and interaction through which leadership and leaders manifest. This conceptual view of leadership as a social process informs the learning intervention that is CLD and is the premise on which the study is built.
Chapter Two details the impact of CLD on individual participants. The key finding related to individual participants informed by data collected in 24 communities with CLD programs, or “treatment sites,” was respondents who participated in CLD programs had significant learning when compared to respondents who lived in counties without programs. CLD participation yields learning that is associated with measureable increased levels of civic engagement and enhanced social capital within communities. Chapter Three expands upon the impact of CLD on social capital by illustrating how involvement in community organizations is broadened and intensified among CLD participants. In Chapter Four, the authors utilize a modified application of Flora, Flora, and Fey’s (2003) community capitals framework to demonstrate the positive impacts of CLD on community development.
Chapter Five addresses participant diversity, curriculum design, and community effects. The authors present data related to the six individual-level outcomes of CLD identified in the study (Personal Growth and Efficacy, Community Commitment, Shared Future and Purpose, Community Knowledge, Civic Engagement, and Social Cohesion). In Chapter Six, the authors press the core idea of the book: community leadership is about community change (p. 119). Their intent is to reveal strategies for CLD program improvement through empirical evidence. Chapter Six concludes Part I with a series of questions for practitioners and sponsors to consider as they design or redesign CLD offerings.
Part II includes three chapters that address the implications of the research for practitioners and future research. In this section, the authors detail a conceptual framework for community leadership development that integrates existing literature and findings from their research. The authors first emphasize the individual motivations for developing community leadership capacities and the myriad expressions of civic engagement (Chapter Seven). The authors suggest civic engagement leverages individual capacities for leadership and builds upon networked, relational leadership within a community in order to influence positive change (Chapter Eight). Hence, developing individual leadership capacity is elemental to promoting civic engagement; however, such individual-level development is merely a first step in CLD.
As demonstrated through the study, effective CLD interventions include curriculum that addresses individual, organizational, and community-level outcomes. Absence of any one of the three curricular outcomes is a missed opportunity to influence change and impact the community. The authors conclude with a discussion of a model of CLD informed by their research through which participants first attain and/or demonstrate (1) improved human capital self-efficacy and governance skills, (2) increased community commitment, (3) shared future and purpose, and (4) increased political knowledge (political capital). Based on the model, CLD participants then demonstrate increased social cohesion and ability to bridge networks and increased civic engagement. These CLD participant behaviors then result in enhanced civic infrastructure and community capacity that yield community-level awareness of responsiveness
to social needs and the inevitable changing conditions experienced in communities.
The book is tailored for CLD practitioners, sponsors, and evaluators through its accessible theoretical literature, study design, and data analysis. The volume is valuable for researchers who will be impressed with the breadth of relevant citations and integration of multiple disciplinary literature bases. Additionally, the authors provide five sets of study questions throughout the text that would be helpful for structuring independent or group learning exercises for graduate students or as a means of professional development for community leadership development practitioners.
Community effects of leadership development education: Citizen empowerment for civic engagement is an essential read for anyone responsible for the design and evaluation of community leadership development programs. Although targeted program evaluations and assessment of participant learning exist, there is a dearth of empirical research on CLD effects on communities and the interplay between individual, organizational, and community-level outcomes. This much anticipated volume heralds a new era of scholarship and practice that engages interdisciplinary literature and empirical research to more critically and precisely explore the benefits of the vast investment in CLD. The authors are frank in their discussion of the research’s limitations; the factors influencing CLD outcomes are numerable and controls are challenging. Despite the acknowledged limitations of the research, this volume is a significant contribution to CLD. The research presented provides an unprecedented synthesis of literature and empirically based insights into the recruitment practices, curriculum design, and evaluation strategies of CLD. Furthermore, the volume is a welcomed summons for scholars and practitioners to build upon the recommendations contained therein and advance the design of community leadership development programs and the ways in which such programs’ outcomes and impacts are measured.
The volume will also appeal to community engagement practitioners and community-engaged scholars who support and participate in university-community partnerships. University-sponsored community engagement activities, including community-engaged scholarship, are championed for their positive impact on communities;
however, comprehensive monitoring and measuring the varied impacts of these activities remains a challenge. Pigg and his colleagues challenge us to reconsider the design of programs intended to support leadership and community development. While the emphasis of their research addresses community leadership development specifically, there are implications for the diverse array of university-community partnerships. For example, the authors address multiple program design elements: primary and secondary outcomes, sponsorship mobilization, collaborative learning, the political and value-laden nature of community-engaged efforts, and diversity among participants among others.
The text may also encourage university leaders to reflect on the ways in which academic, co-curricular, and outreach programs meet their communities’ leadership development education needs. As representatives of anchor institutions that contribute to their communities’ sustainability and advancement, university leaders must carefully consider how and to what extent myriad initiatives individually and collectively impact participants and communities. Through this rigorous investigation, universities may more holistically illustrate their significance and demonstrate appropriate stewardship of the vast public and private investment in higher education.
The authors provide substantial insight into the effects of community leadership development programs; however, even they acknowledge their contribution falls short in providing a general
theory of community leadership. Pigg and his colleagues provide a model of the effects of community leadership development programs on individuals and communities that, although limited, presents a more comprehensive conceptual framework for leadership development education. Their model synthesizes multiple disciplinary literatures related to civic leadership capacity and demonstrates the interconnectivity and complexity of community leadership development. Community effects of leadership development education: Citizen empowerment for civic engagement is crucial addition to the literature on community leadership development, civic engagement, and community capacity building that will advance the influence of citizen leaders in their communities. The volume is also a valuable resource for university leaders who advocate for campus-community partnerships and those whose work directly impacts such collaborations that are designed to have positive impacts on communities.
The editorial team of the Journal of Community Engagement and Scholarship would like to acknowledge and thank the West Virginia University Press for providing copies of the book for this review.
About the Reviewer
Jennifer W. Purcell is an assistant professor of Leadership Studies and faculty consultant for community engagement at Kennesaw State University.
Review by Sarah Stanlick
Higher education has become increasingly technology-centric with massive open online courses (MOOCs), hybrid courses, and classroom technologies pushing us to reconsider how knowledge is shared, how identities develop, and how learners are reached. With this shift, there are both challenges and opportunities for the field of service-learning and civic engagement to consider. At the recent Pathways to Achieving Civic Engagement Conference hosted by North Carolina Campus Compact, a clear theme emerged with regard to technology’s impact on the future of service-learning and civic engagement. Some spoke with trepidation, and others with bold hope for innovations that could make service-learning even more accessible in the online format.
As educators, if we seek to develop the civic knowledge, skills, and values/attitudes of our students, that practice can take place in many different forms and formats. Harnessing technology’s power to support that diverse sharing of ideas, connections to modes of communication across boundaries, and scaffolding of critical reflection and growth are all domains in which the experience of service-learning and civic engagement can be deepened. However, not all are hopeful for technology’s role in education or society. Psychologist and MIT Professor Sherry Turkle (2012) has done extensive work on the divisive nature of technology and its capacity to push people further apart and encourage less “human” interactions. Clearly, there is much to be learned and new horizons to explore if we are to meaningfully and effectively implement technology in the service-learning and civic engagement field.
Editors Jean Strait and Katherine Nordyke are ahead of the curve in this respect, having published a volume that delves into the many expressions, benefits, and potential pitfalls of engaging in the online world. In their work eService-Learning: Creating Experiential Learning and Civic Engagement Through Online and Hybrid Courses, the editors bring together a diverse collection of author experience to speak to the future of service-learning in an online capacity. Divided into three main sections, the book provides a scaffold for readers to think through the ways in which experiential learning and civic engagement can take place in online and hybrid courses.
Moving from essentials to models to future directions, the book clearly scaffolds the knowledge about e-service-learning for the reader to both learn new frameworks and imagine the possibilities for their own practice. In the opening chapter, the editors—joined by Jane Turk—provide a comprehensive review that sets up many of the issues and implications of service-learning, both digitally and analog. They identify civic knowledge, skills, and values/attitudes as the ultimate goal of the pedagogy that is service-learning, and thus posit the question of how technology helps us to realize those goals. Leora Waldner provides a critical understanding of the components of e-service-learning and, further, what the values of a high-quality experience should encompass. In chapter 3, Nordyke explores the ways in which service-learning at large can be realized in a course, and then asks us to think about the appropriate usage when technology is involved. Whether it be direct or indirect service, integrated or standalone, there are many options for what service-learning can look like in the digital realm, and it is incumbent upon the curriculum designer to make intentional choices about that delivery. Finally, Strait rounds out this section of chapters by exploring the role of technology as a support system for service-learning, and specifically e-service-learning models.
In the next set of chapters, models for e-service-learning across disciplines and geographic areas are explored. Each provides a specific context for e-service-learning, from university adoption at-large to an online business course to the assessment of experiences in an established service-learning office. One of the chapters that focuses on exemplars is a case study on service-learning after Hurricane Katrina. Striking a parallel between today’s political landscape and the intense focus on the Syrian refugee crisis, one can look at this chapter as especially salient. A spike in attention and awareness can lead to a situation that susceptible to largely technocratic or unsustainable engagement. In the chapter on post-Katrina efforts, the example of the Each One, Teach One program provides evidence of a successful model that integrated hybrid online, in-person experiences for sustainable engagement and community partner relationship development. This section reminds us that we must think critically and carefully about the way in which service-learning and civic engagement is being implemented in truly meaningful, high-quality ways regardless of the realm—online or in-person.
Moving into the final section of the book, the focus shifts to the future. The penultimate chapter is a call from John Hamerlinck to leverage technology and community engagement to make higher education more purposeful and impactful. Trends in neoliberalism and internationalization in higher education—coupled with an increased focus on providing asynchronous experiences—have driven the growth and development of MOOCs and hybrid courses. Hamerlinck cites a number of examples from domains of cost, student success, and debt that point toward the need to rethinking the purpose and usefulness of higher education. Within the space constraints of a chapter, it is impossible to exhaust all of the issues facing the wicked problem of higher education delivery and transformation. However, there is one notable absence: the issue of access. I can envision an opportunity for a critical examination of these issues of access and equity in future works, especially as the opportunities for e-service-learning continues to grow and concern about the digital divide remains very real. That being said, the exciting opportunity that he highlights is the ability re-imagine higher education, leverage technology to create community, and highlight one’s work in service-learning and civic engagement to shift the expectations and models of higher education. In his vision, social media can be a tool to promote civic ideals, the openness of online courses can disseminate knowledge across stakeholders, and the creative, collaborative capacity of joint projects can be ever-expanded across geographies. Community-engaged teaching with technology, connectivism, and the decentralization of knowledge bases can have implications for the field, while strengthening values of community and public purpose for all of higher education.
Strait and Nordyke have put together a timely and useful resource for imagining and reconsidering the role of technology as a support, facilitator, and space for service-learning and civic engagement to take place. As a scholar-practitioner with significant experience in using technology in the classroom, I found this book to be an immensely helpful tool for those at any level of technology adoption. The contributors give us a catalyst to spark creative thinking about the delivery of service-learning education while retaining a focus on the values and quality of community engagement. Whether you look toward this brave new world of technology to expand your practice or deepen its impact, this volume is an essential resource.
Turkle, S. (2012). Alone together: Why we expect more from technology and less from each other. New York: Basic Books.
The editorial team of the Journal of Community Engagement and Scholarship would like to acknowledge and thank Stylus Publishing for providing copies of the book for this review.
About the Reviewer
Sarah Stanlick is director of the Center
for Community Engagement and professor
of practice in sociology and anthropology at
Review by Paul H. Matthews
University of Georgia
Is social entrepreneurship education just a kind of service-learning found in business schools? Or is it grounded in such different history, assumptions, and practice that its commonalities with community-based pedagogy are only superficial at best? The field of social entrepreneurship—an “innovative, social value-creating activity” (Wei-Skillern, Austin, Leonard, & Stevenson, 2007, p. 4)—is sometimes construed as analogous to service-learning, but on many campuses the two fields seem to occupy disparate and non-overlapping spaces. Sandra Enos’s brief volume (84 pages in four chapters, plus prefix, one appendix, and references), published in 2015 as part of the Community Engagement in Higher Education series (Palgrave Macmillan), seeks to change that, and to illustrate some of the ways in which these pedagogies of service-learning and social entrepreneurship can, do, and do not intersect on college campuses.
In the first chapter (“The Landscape of Social Change Education”), Enos—a sociology professor with experience in both service-learning and social entrepreneurship—sets the context with a sketch of some recent discussions of university engagement’s place in higher education, and contends that both fields focused on in this volume should be considered a subset of “social change education.” She provides evidence for the growing institutionalization of each, for instance through journals and professional associations, and draws parallels specifically between the existence of overarching organizations that encourage campus involvement in both movements (e.g., between Campus Compact and Ashoka U). Next, she summarizes prior work presenting these two fields’ commonalities (e.g., these are social-change oriented movements seeking greater campus reach; entail community-based partnerships; and provide opportunities for real-world applications of knowledge) as well as differences (e.g., a stronger focus on student learning in service-learning, versus on community/societal change in social entrepreneurship; a stronger orientation toward “disruption” of existing structures and practices in the latter; and differing language as well as desired learning outcomes). While Enos also contends that “[S]ocial entrepreneurship prizes the founding of new organizations” (p. 18), other researchers have noted the growth and value of social “intrepreneurship”—i.e., working to implement change within existing structures and organizations (Bornstein, 2007; Bornstein & Davis, 2010)—a construct not included in Enos’s overview.
Her second chapter (“Organizing for Engagement”) comprises the bulk of the book’s research contribution, an attempt to determine how these two pedagogies are organized and interrelate on campuses that offer both. Enos reports on website analyses and interviews she conducted with staff at the 10 campuses currently recognized with both Carnegie’s community engagement classification and the “Changemaker Campus” designation by Ashoka U. She contends that these “exemplars of service-learning and of social entrepreneurship education” (p. 23) should offer a view into the current status of institutionalization and potential relationships between these subfields of community engagement on U.S. campuses. As such, she investigates whether there are “patterns in the organization of these programs, whether there are typical disciplinary homes for service-learning and social entrepreneurship courses, …and how programs that offer [them] are related to each other” (p. 27). However, Enos reports that little consistency was found; instead, “what we see are individual profiles, tied to institutional size, history, culture and leadership” (pp. 32–33). While she summarizes information about each campus in a table, its content reads more like field notes than an analysis. As might be expected from a range of institutional sizes, types, and locations, she found that the colleges and universities studied had varying amounts of coursework, centralization, and support structures, and used different terminology in talking about their initiatives.
In chapter three, “Challenges for Service-Learning and Social Entrepreneurship,” Enos iterates several of the criticisms that have been extended toward these two sets of practice, including (for service-learning) a lack of “deliberate and intentional incorporation of civic skills” (p. 45), as well as issues of privilege, social justice, and the problematization of “service” (pp. 47–48). For social entrepreneurship, Enos notes that this field sometimes uses different terms (e.g. “social enterprise”) implying different foci; that innovations may struggle in being implemented at scale; and that the “celebratory nature of innovations in the social entrepreneurship space makes it difficult to mount a critical review of these practices” (p. 55). She also summarizes concerns that social entrepreneurship may tend to “paint the state, unions, public employees and other existing nonprofits as the enemy” (pp. 56–57), undermining rather than enhancing existing efforts, which may contrast with service-learning’s focus. In terms of social entrepreneurship education, Enos notes that existing courses have been criticized for a lack of emphasis on complex problem-solving, and for mismatches between desired competencies identified in course syllabi and those recommended by practitioners in the field. While interesting, and a good overview of some of the critiques offered for each field, this chapter does not contribute much new to the discussion beyond summarizing.
In the book’s final chapter, “Educating for Engagement: A Turning Point,” Enos renews her recommendations that, as two subsets of campus/community engagement, service-learning and social entrepreneurship should become better integrated institutionally. She provides examples from her own and other universities as demonstrations of how these two fields may fit into a broader conceptualization of civic engagement, as well as how particular disciplines might implement both. After repeating her calls for more cross-fertilization, Enos suggests the need “to challenge each other on what works—in teaching and research—and certainly on what strategies get us closer to working with community partners in problem-solving reciprocal ways” (p. 83). Finally, the book’s appendix provides a list of campus representatives for Chapter 3’s interviews.
Several aspects of this book raise issues worthy of contemplation for those in the community engagement field. The explicit comparison and contrasting between the two sets of philosophies and practices illuminates several interesting points. For instance, Enos contends:
Among the most important differences between service-learning and social entrepreneurship include the former’s focus on groups and collaboration vs. the latter’s on individualism, and vocabulary borrowed from the nonprofit sector vs. vocabulary taken from the business world. Other points that differentiate these fields are skepticism about market-based approaches to social problems on part of the service-learning community compared to an embrace of market-based solutions in the social entrepreneurship field. The service-learning field aims to educate students for civic engagement and democracy while some versions of social entrepreneurship work at solving problems outside of or independent of politics (pp. 16–17).
Likewise, a careful consideration of what each field might offer the other is a valuable exercise. Could some of the tools and concepts used in social entrepreneurship—e.g., the double/triple bottom line for determining return on investment—be helpful for assessing service-learning programs’ impacts? Could service-learning’s established expertise in reflection and in highlighting the importance of reciprocity with community partner organizations bolster the success of social entrepreneurship education and application?
In several areas, the book could be stronger. For instance, while couched at times in the language of research, this volume’s findings from campus interviews do not seem to follow many standards of qualitative methodology (e.g., thick description, coding, etc.), and as such fall somewhat short. Likewise, while the author includes a solid listing of prior articles that relate to this work’s guiding questions, Enos’s style of reviewing literature often seems more focused on enumerating a list of each author’s points, rather than synthesizing them as a whole. While in two separate chapters Enos offers other authors’ definitions for “social entrepreneurship,” she does not ever clarify how, specifically, she is using the term; and, while she notes that “the practice of social entrepreneurship is distinct from social entrepreneurship education” (p. 11), that distinction is not consistently apparent in the volume.
Several chapters enumerate details about certain journals, organizations and initiatives, yet omit others of similar potential interest and merit without explanation. For instance, for service-learning and community engagement, she lists several journals (JCES; the Michigan Journal of Community Service Learning) and conferences and organizations (IARSLCE; Campus Compact), while other equally relevant resources (e.g., the Journal of Higher Education Outreach and Engagement; the Engagement Scholarship Consortium) are omitted. On the social entrepreneurship side, Enos provides a strong focus on both the Skoll Foundation and Ashoka, but does not acknowledge other important resources and players such as the Acumen Fund, the Schwab Foundation for Social Entrepreneurship, or Echoing Green (Bornstein & Davis, 2010). Likewise, her “short history of social entrepreneurship” seems to lay the founding of this field at the feet of Jeffrey Skoll and Bill Drayton, with no mention of key international “pioneers” (Bornstein & Davis, 2010, p. 13) such as Muhammad Yunus (Grameen Bank) and Fazle Abed (Bangladesh Rural Advancement Committee). Finally, frequent typos and minor errors (e.g., in names of national organizations, journals, and funders, or in referring to the “2012 [sic] Carnegie round of applications” [p. 29]) also distract somewhat from the book’s content.
In the end, Enos defends and extends her thesis statement: “Some may believe that service-learning and social entrepreneurship models of educating students for community engagement are incompatible but I am going to argue that they are not” (p. 70); however, as her own research shows, even on campuses considered exemplary in each area, at present these two fields tend not to converge in practice. For readers interested in how these two branches of engaged pedagogy might interweave more productively, Service-Learning and Social Entrepreneurship in Higher Education may be a good starting point, from either side of the equation.
The JCES editorial team would like to acknowledge and thank Palgrave Macmillan for providing an electronic copy of the book for this review.
Bornstein, D. (2007). How to change the world: Social entrepreneurs and the power of new ideas (updated edition). New York, NY: Oxford University Press.
Bornstein, D., & Davis, S. (2010). Social entrepreneurship: What everyone needs to know. New York, NY: Oxford University Press.
Wei-Skillern, J., Austin, J.E., Leonard, H., & Stevenson, H. (2007). Entrepreneurship in the social sector. Thousand Oaks, CA: Sage Publishing.
About the Reviewer
Paul H. Matthews is associate director of the University of Georgia Office of Service-Learning.
Dr. Andrew J. Pearl
I am honored to have been asked to serve as the new Book Review Editor for the Journal of Community Engagement and Scholarship. Not only do I view this as a great opportunity to serve in the advancement of higher education community-engaged scholarship, but also as an opportunity for me to learn from authors, scholars, and community partners from across the country and around the globe.
I am currently director of Academic Engagement at the University of North Georgia, where my overarching responsibility is to institutionalize community-engaged scholarship across our five campuses. Among the ways that we accomplish this is to engage with and develop faculty members, work with students and let them know about the educational opportunities available to them, and develop mutually beneficial community partnerships.
I am a relatively recent graduate of the University of Georgia’s Institute of Higher Education, and my scholarly agenda includes faculty member motivations related to the scholarship of engagement, student motivations and outcomes related to service-learning experiences, and institutional and public policies that affect community engagement in higher education. In my research, I utilize the complementary strengths of both quantitative and qualitative analytic methods to fully explore scholarly questions, often through interdisciplinary collaborations. I am also interested in exploring how graduate students are socialized for and participate in community engagement, as well as strategies for integrating engagement into the STEM fields.
My goal with the book reviews in JCES will be to introduce readers to literature that advances knowledge, provides practical advice, disseminates best practices, and encourages conversation and dialogue. Book reviewers will not only provide summaries of recently published titles relevant to our field, but will also critically analyze and examine the works through the lens of community engagement.
I invite faculty members, administrators, staff members, students, and community partners to offer their interpretations of the literature. From its beginning, JCES has always intentionally been a “different kind of journal,” which helps to establish a niche among scholarly publication outlets. In addition to providing established scholars and researchers with a platform to disseminate their work, we also seek to hear from all of the people who work to make these multi-faceted community-campus partnerships work. In the true spirit of community engagement, we encourage authorship from first-time authors, students at all levels, and community partners.
To paraphrase Ernest Boyer, higher education has the potential, and I might even say the responsibility, to advance both intellectual and civic life, and I believe that JCES plays an important role in this process. I gratefully look forward to the opportunity to work with the rest of the JCES editorial and publishing team as we continue to build on the wonderful foundation already in place.
Eric Conrad, Meghan Shewmake, Courtney Shows, and Jen Nickelson
Sustainable community partnerships require more than good intentions, necessitating the facilitation of true collaboration to be successful (Blouin & Perry, 2009). McLean and Behringer (2008) assert that for a truly equitable partnership to exist, each party involved must both contribute to, and receive benefits from, the relationship. These contributions from students, campus partners, and communities provide meaningful experiences that result in reciprocal benefits among all partners. Practices that enable equitable collaborations include the establishment of personal relationships, reciprocity, flexibility in adapting to unexpected outcomes, and collaboration to overcome barriers (King, Williams, Howard, Proffitt, Belcher, & McLean, 2004). Fostering these practices result in increased sustainability with the potential to expand collaborative efforts.
The Holt Health Fair is a community event with the goal of providing health screenings and information dissemination to the community of Holt, Alabama. The inception of this annual event was developed through the formation of a collaborative between The University of Alabama (UA) students within the Delta Xi Chapter of the Eta Sigma Gamma (ESG) National Health Education Honorary, and members of the Holt community. Through the maturation of this partnership, stakeholder capacity has increased through mutual support and empowerment. This evolution is the direct result of a partnership that strongly values community voice and input. The following is a student perspective illustrating pertinent community engagement principles, as adapted from McLean and Behringer (2008), and their application in the development of the Holt Health Fair and community partnership.
Partnerships Thrive on Personal Connections
Two commonly asserted theoretical approaches to community change are that of “cultural invasion” and “cultural synthesis.” Cultural invasion occurs when external institutions or academics impose values and agendas with brazen disregard for community voice. Cultural synthesis necessitates those same external forces support and supplement community efforts to collaboratively solve community need (Green & Kreuter, 2005). The primary distinction between these theoretical approaches is in the formation of personal connections within the community. Development and fostering of these relationships demonstrate an investment in the community’s well-being that extends beyond the cessation of singular research projects and funding.
In 2008, a collective of concerned citizens established the Holt Community Partnership (HCP) in an effort to develop initiatives within Holt that “transform lives through opportunity, education, unity, and safety” (HCP, 2010). The HCP was comprised of Holt community members, local law enforcement, religious leaders, board of education affiliates, as well as faculty and staff from UA. This partnership established a foundation in which personal connections between UA and the Holt community could be fostered through cultural synthesis.
The establishment and cultivation of this relationship led to the inclusion of UA student associations, such as the ESG health organization. Since the integration of ESG and Holt, students have been afforded the opportunity to foster relationships within the community through interaction and participation at HCP meetings as well as through additional community collaboratives and initiatives. This community immersion is a transformative process in which programs and initiatives are no longer simple extracurricular activities, but vested experiences in which tangible change can result within the community where these relationships are formed.
Participation and Communication Requires Reciprocity
A true collaborative is one that includes an empowering and power-sharing dynamic that promotes an equitable partnership (Israel, Schulz, Parker, Becker, Allen, & Guzman, 2008). This dynamic may be challenging for academics due to the relinquishment of power or the perceived threat to validity, but is necessary to ensure cultural synthesis and capitalize on community expertise to enhance initiatives and improve outcomes. Support for this synergistic relationship can be achieved through the mobilization and utilization of university resources to supplement community efforts. The reciprocal nature of an equitable partnership ensures community collaborators have input in the development and implementation of programs that can be tailored to meet the diverse needs of the community (Ahmed & Palermo, 2010).
Prior to the inception of the health fair, the HCP and UA collaborators established a community festival within Holt. Over time, attendance and participation in the festival waned, necessitating a reevaluation of the initiative which ultimately resulted in the conclusion that the festival was no longer feasible and meeting the needs of the community. Collaborators began an iterative process to develop a new initiative within Holt to maximize the collective capacity of UA stakeholders and the community. Consistent with the theoretical approach of cultural synthesis, this cyclical method emphasized concerns created by community members and the capacity of stakeholders to address this need. The outcome of this reciprocal dynamic was a community proposal for the establishment of a Holt Health Fair and the creation of processes to support the initiative. Critical to this process was the explicit attention to the knowledge and expertise of community members and emphasis on empowerment inclusive of information sharing, decision-making power, resources, and support within the partnership.
The importance of reciprocity does not diminish following the approval of the initiative, but must be maintained throughout its entirety. As a health education organization, ESG was enthusiastic regarding the community initiated proposal for a health fair within Holt. While well intentioned, unbridled enthusiasm may result in an overzealousness that neglects or excludes community input and precipitates cultural invasion. Eta Sigma Gamma student and faculty representatives perpetuated an equitable dynamic through attendance at HCP sessions to ensure continued communication that emphasized community-initiated development, implementation, and evaluation. Respect for the unique perspective and expertise of Holt collaborators produced a reciprocal transfer of knowledge, skills, and resources that enhanced the capacity of all stakeholders.
Communication and Cooperation Overcome
Partnerships sustained through personal relationships that prioritize reciprocal communication and power facilitate leveraging of collective capacity to overcome project barriers (Green & Kreuter, 2005). A significant challenge following the decision to develop a health fair was creating a suitable event on an abbreviated timeline. Previously established relationships with the HCP facilitated efficient role delineation, allowing for the attainment of resources and services necessary for event implementation. Following the initial implementation of the Holt Health Fair, a stakeholder evaluation identified further barriers that adversely affected project outcomes. Promotion efforts in year one were primarily managed by ESG students which utilized radio and television broadcasts, as well as flyer distribution, yet intended reach was lower than expected. Year two of the initiative improved these processes through increased community involvement in the dissemination, utilizing the cultural influence of local church networks and businesses. Collaborations with Holt High School teachers and coaches resulted in presentations from high school students as well as performances by the dance team and band. The expansion of entertainment activities and the addition of childcare were incorporated to increase event appeal and facilitate easier access to services for community participants with children. Auxiliary barriers that were also addressed included the improvement of issues in regards to venue, transportation, as well as the expansion of vendors and services. Addressing and reducing barriers throughout the evolution of the health fair was expedited through cooperative mechanisms that prioritized continual communication and assessment between ESG, the HCP, as well as UA and community organizations.
Progress Necessitates Increased Capacity
The readiness of the partnership to be adaptable and amenable to increase contributions to the program strengthens the collaborative and improves program success (Sandy & Holland, 2006). Compared to the health fair, the Holt Festival required less resources and investment from each collaborator in the partnership. During the initial conception of the Holt Health Fair, the partnership determined current stakeholders needed to expand their capacity to accomplish the successful implementation of the initiative. The UA stakeholders increased capacity through incorporation of ESG student members and the expansion of university collaborators able to provide resources and health services. Barriers identified served as the impetus for increased community contributions resulting in systematized processes to increase promotion and engagement. Maintaining the equitable relationship through this expansion necessitated that each collaborator augment their contribution to improve the initiative and preserve reciprocity.
Expanding capacity to increase contributions of the partnership should also result in increased benefits to all stakeholders (King et al., 2004). Through the development and improvement of the Holt Health Fair, offerings were expanded from each collaborator which in turn increased the benefits each party received. University students within ESG and other collaborating organizations were provided the opportunity for experiential learning in their respective disciplines. Through application of classroom content, students benefit from deeper understanding of course material and are afforded the opportunity to strengthen cultural competency, leadership attributes, and self-efficacy (Powell & Conrad, 2015). Community members gained the expertise of UA students and faculty, access to health services through the university and external organizations, as well as media coverage and publicity. The benefits resulting from increased capacity were cultivated through relationships that foster trust and reliance in the collaborative process. The increased contribution and capacity can then be extended and utilized in other community partnerships.
Successful Partnerships Breed Successful Partnerships
The principles and benefits of equitable partnerships are transferable regardless of initiative, event, or program (Rhodes, Malow, & Jolly, 2010). The same practices of fostering and sustaining partnerships of reciprocity through cultural synthesis serve to reinforce relationships facilitating initiative success. Partnerships capable of overcoming project barriers promote capacity building through multidirectional learning on an individual or community level. Cultivating improved competence and proficiency through existing partnerships result in increased knowledge, skills, and expertise that enhance diverse community initiatives.
In addition to being active partners of the Holt Health Fair, ESG is also involved in additional health related initiatives within the Holt community. One such coalition is the Holt afterschool program in which ESG members educate elementary students regarding health topics in collaboration with the community organization Tuscaloosa’s One Place. An additional initiative is the Holt Health Lab in which UA and Holt collaborators aim to empower the community to meet health needs through screening, education, and policy. Similar to the Holt Health Fair, these initiatives incorporate processes to ensure community voice and reciprocity to enhance program success. Lessons learned in each initiative may serve to build on successful aspects and avoid barriers that other projects may have encountered. In this way, the expertise and capacity of each partnership is extended beyond itself to ultimately benefit the community through pervasive processes.
Partnership Length Increases Success
Successful partnerships that provide meaningful contributions and benefits to the community and university is a long-term process that requires commitment from both parties (Israel et al., 2008). Cultural synthesis asserts that the establishment and maintenance of trust and rapport among communities necessitate that commitments extend beyond single projects or funding. This prioritization on relationships over outcomes mediate an equitable power-sharing dynamic in which the partnership can more efficiently overcome barriers and increase capacity to achieve goals. These approaches allow for the development of an infrastructure that promotes longevity and sustainability which increases partnership and project success.
The successes of the Holt Health Fair and other community initiatives are direct results of an equitable partnership facilitated through cultural synthesis. Utilizing this approach, UA collaborators prioritized people over processes to develop and strengthen relationships among the Holt community. Through this genuine engagement and interaction within Holt, ESG student perspectives were altered from a mentality that prioritized a tangible product to a mindset that strongly desired an increase in community wellness, well-being, and empowerment. This vested interest fostered reciprocity among ESG students, UA faculty, and Holt community partners through the establishment of processes that respected community voice and perspective. Utilizing the collective strengths of the partnership, event deficiencies were identified and barriers were surmounted as each stakeholder evaluated and subsequently expanded contributions to the initiative, increasing student and community capacity. Lessons learned from the development, implementation, and evaluation of the Holt Health Fair can be applied in diverse community initiatives to further transform lives through opportunity, education, unity, and safety. Establishing an equitable partnership within Holt did not deny differences among multiple perspectives, but embraced diversity by affirming undeniable support through cultural synthesis to create improved outcomes in community engagement.
Ahmed, S.M., & Palermo, A.S. (2010). Community engagement in research: Frameworks for education and peer review. American Journal of Public Health, 100(8), 1,380–1,387.
Blouin, D.D., & Perry, E.M. (2009). Whom does service learning really serve? Community-based organizations’ perspectives on service learning. Teaching Sociology, 37(2), 120–135.
Green, L.W., & Kreuter, M.W. (2005). Health program planning: An educational and ecological approach. New York: McGraw-Hill.
Holt Community Partnership. (2010). How the HCP developed. Retrieved from https://holtcommunitypartnership.wordpress.com/about/.
Israel, B.A., Schulz, A.J., Parker, E.A., Becker, A.B., Allen, A.J., & Guzman, J.R. (2008). Critical issues in developing and following community based participatory research principles. In M. Minkler & N. Wallerstein (Eds.), Community-based participatory research for health: From process to outcomes (pp. 53–76). San Francisco: John Wiley & Sons.
Jacob, W.J., Sutin, S.E., Weidman, J.C., & Yeager, J.I. (2015). Community engagement in higher education. In Community Engagement in Higher Education (pp. 1–28). SensePublishers.
King, B., Williams, W., Howard, S., Proffitt, F., Belcher, K., & McLean, J.E. (2004). Creating the bridge: The community’s view of the expanding community partnerships. In B.A. Behringer, B.C. Bach, H. Daudistel, J.W. Fraser, J. Kriesky, & G.E. Lang, (Eds.), Pursuing opportunities through partnerships: Higher education and communities (pp. 75–85). Morgantown, WV: West Virginia University Press.
Minkler, M., & Wallerstein, N. (Eds.). (2011). Community-based participatory research for health: From process to outcomes (2nd ed.). San Francisco, CA: Jossey-Bass.
McLean, J.E., & Behringer, B.A. (2008). Establishing and evaluating equitable partnerships. Journal of Community Engagement and Scholarship,1(1): 66-71.
Powell, B., & Conrad, E. (2015). Utilizing the CIPP model as a means to develop an integrated service-learning component in a university health course. Journal of Health Education Teaching, 6(1), 21-32.
Rhodes, S. D., Malow, R. M., & Jolly, C. (2010). Community-based participatory research (CBPR): A new and not-so-new approach to HIV/AIDS prevention, care, and treatment. AIDS Education and Prevention, 22(3), 173-183.
Sandy, M., & Holland, B. A. (2006). Different worlds and common ground: Community partner perspectives on campus-community partnerships. Michigan Journal of Community Service Learning, 13(1), 30-43.
About the Authors
Eric Conrad is a doctoral student, Meghan Shewmake is working on a master’s degree, Courtney Shows is an undergraduate student, and Dr. Jen Nickelson is an associate professor, all in the Department of Health Science at the University of Alabama.
Yawah Awolowo, Debra Clark, and Darlene Robinson
Project UNITED is a federally funded grant that addresses obesity issues in the Alabama Black Belt. The Black Belt represents some of the poorest counties in the United States and is plagued with chronic health conditions—obesity, diabetes, high blood pressure, cancer—which are predominantly found in African Americans. Community leaders from the Black Belt Community Foundation, a 501(c)3 nonprofit corporation whose purpose is to help improve the quality of life of residents in the Alabama Black Belt, participated in the project as community health scholars. Project UNITED partnered community health scholars and academic scholars to develop proposals that addressed obesity issues in rural communities. A team of three community health scholars and three academic scholars collaborated and formed, Home Sweet Home, a two-year, $50,000, multigenerational obesity intervention in Greene and Sumter counties. This paper provides reflections from the three community health scholars regarding their experience in a community based participatory research (CBPR) project.
The paper is presented as first person testimonials describing the experiences of each participant.
Darlene Robinson, Greene County
Being one who is always looking for ways to improve myself and my community, I took a step of faith in response to the invitation I received to become a community health scholar with Project UNITED. I also believed I would acquire knowledge that would help my community. In the community my mission is to improve everyday life of parents and their children. Even though my plate was already full, I felt it would be an opportunity to find ways to improve myself and my community.
The Project UNITED journey introduced us to individuals from around the state who also wanted to improve themselves and their communities. I could tell the training was well thought out, as we visited a small garden at a rural school where the youth were being introduced to growing their own foods. This was the Project UNITED pilot project. Educating our community about healthy lifestyle changes is very important and we learned ways to take it home. Project UNITED brought professors from the University of Alabama and community people together as a learning tool that showed that we all share the same passion of making Alabama a healthier place to live.
The circle was complete when our team formed. We had three members from the community and three members from academia, with a mission of curtailing obesity by educating ourselves. As a group we merged our thoughts, hashed out our ideas, made a plan, found an audience—the preschoolers and their parents—and formed a roadmap to get us where we wanted to go. We named our program “Home Sweet Home.” Our mission was to introduce and teach a new way of thinking about eating. We sought to change current statistics that says we are an obese society.
If you’re busy, think you have that all you can handle, feel you know all you need to know, then CBPR is probably for you, because it takes all you’re doing and all you can handle and turns it into a mission that you’ve been on all the time but gives you directions to get to a better place. Then it will not just be your idea of what the community needs, but the community will be able to tell you what they want.
Debra Clark, Sumter County
In 2013, I was chosen by the Black Belt Community Foundation to be a community health scholar. This was because of my health-related awareness work at the Health and Wellness Education Center (HWEC) Disease Management Program. At the first meeting my first thought was “Why am I here?” realizing that I did’t have time to take on an additional project. But my second thought was “how can I not participate?” considering the impact this project could have on my community. Certainty, it is through strong community partnerships with local agencies that has made HWEC successful. The agency has made great strides to implement initiatives to address obesity, diabetes, cardiovascular disease, and other chronic illnesses to improve the health of local residents.
As a result of this project, I
1. Was invited to present a paper entitled “Fighting Obesity for the Health of It” at the Rural Health Conference in Tuscaloosa in 2013.
2. Received the Rural Heroes Award from the UA College of Community Health Sciences.
3. Was able to expand the Jump To It (what is it?) program for additional communities.
4. Produced a simple handout on the traffic light diet to promote healthy eating for distribution at health fairs and community events.
5. Have witnessed the greatest impact within my own family. My 5-year old grandson was never introduced to yogurt, but now eats yogurt and encourages his mom to “add stuff” to make it good.
Although this fight has been tedious and time consuming, I must admit that rewards have been greater than the struggle. The program has helped build capacity and strengthen my agency by connecting me with a team of experts with genuine concern and interest, which correlates with the agency’s mission of promoting healthier lifestyles through education and prevention.
Yawah Awolowo, Sumter County
As a community health scholar participating in Project UNITED, I have enjoyed adding to what I already knew about health-related issues and unhealthy eating. I nurtured my family in an organic, self-sufficient, natural homebirth/home schooled permaculture environment and have sustained five seriously committed years of organic farming on a 66-acre, family-owned farm. My journey with CBPR and the development of a partnership with academic researchers not only expands my baseline knowledge, but the collaborative process has also allowed me to continue to put into practice my knowledge in a way to benefit my community. I have learned that while academic researchers are very knowledgeable about health and treatment, I also bring a great deal of knowledge about nutrition and health. Additionally, I have expertise about the rural community where I live, which I know is valuable for eliminating obesity in this community.
My participation in Project UNITED enlightened my interest in heart disease and obesity, major health concerns in my family. My mother, who was 90 years young developed heart disease. She had no hypertension, diabetes, no medicines for her entire life. The doctors informed the family that there was a blockage in one of her heart valves; surgery was necessary to remove the blockage. One of the major reasons I chose to be involved as a community health scholar in Project UNITED was because I watched my 90-year-old mother go through heart surgery and witnessed her spectacular recovery as she completely changed her eating habits. The other reason for my interest is because there are so many people in my family who have serious health conditions. My sister had surgery two weeks prior to my mother and my daughter has also had medical issues. For these reasons, my desire has been fueled to establish alternative eating habits as a means to prevent disease, especially with kids at an early age. I believe this can be done by letting them be involved with the purchase, preparation, cooking, and serving of good healthy foods.
Project United was the right program for my personal and community needs to learn more about prevention of this disease in our community. I realized that the cultural normality of food intake always consists of some type of animal products on a daily basis that cause blockages and slow down the process of the heart pumping blood and oxygen.
Involvement in Home Sweet Home Project
As a community health scholar I have been able to work with a team of people who share my interest of eliminating diseases through education and awareness. We met as a group utilizing a tool called speed dating to develop a professional relationship/marriage that has grown into a big happy family. Consider the three-legged stool; one cannot stand without the other. Each of us brings so much to table that I wonder how we could have been so close without ever crossing paths. Nevertheless, we have bonded and now look for ways to improve our efforts by enhancing our writing skills and seeking additional resources to promote sustainability to enhance our community for a better future. Community health scholars: What better name can you give to such individuals who are a community’s first choice when it comes to health care. We are the movers and shakers in our community.
Home Sweet Home is our project. We named it, we designed the program, we listened to the community, we respected the community, we educated ourselves, and we educated the community. The program was not handed to us; it was made by us for us with the mission in mind of changing lifestyles by teaching our families how to spend time together, talk to each other, share with others, cook with love, train our children how to prepare a dish and making home truly Home Sweet Home. The mission of introducing new ways of cooking to our parents was well received. The preschoolers enjoyed the experience and the parents appreciated the information because our eating habits have led to many health improvements. The traffic light playing cards (red light = foods to avoid; eat; yellow = sometimes foods; green = go foods) were designed as a way for families to learn what foods to stop eating, for example cautiously eating the yellow foods and eating lots of all the green foods. Each family received a deck of Home Sweet Home cards designed by the team.
The program was introduced in the communities represented by the community health scholars. The team was glued together by a passion to make a difference, and we believe it is the start of great things to come. Our coming together was not by accident; it was time for academia and the community to unite on a mission of bringing the idea of healthy eating to the community.
Project United has been a perfect learning catalyst. One of the major goals of the project was to partner community leaders with academic researchers to develop and pilot test an obesity intervention program. We spent the year getting to know each other’s interests and personalities with an emphasis on building trust and working toward sustaining a lasting relationship. This was important to me because I have had several business encounters where I have shared ideas and people have either ignored or taken my ideas. These experiences have made me wary to trust experts sometimes. Therefore, any emphasis on building trust and sustainability are going to be important for me and my community.
I eventually ended up working with a team of six. We talked about potential projects based on our interests where we would be competitive for funding internally. We also evaluated the best approaches to write the grant using all our strengths. As a member of the team, I also earned my IRB certificate, which builds on the training I received in medical ethics and research. We titled our submission: Home Sweet Home, with a focus on home food environment. This project addresses obesity in the age group 2 to 5 years (pre-k). Our primary goal was to get 2-5-year-old children, along with their parents, to learn how to choose good, healthy produce, prepare food in a clean area, practice safety measures, make it a child friendly environment, create their own recipes using the things they like to eat, and spark their interest in healthy eating habits from an early age.
Thankfully, our collaborative proposal received funding for one year through a competitive internal grant process.
As I reflect on the past year of my role as a Community health scholar and the value that CBPR has added to my life, I believe that the time invested in the program will save lives in my community. And that is time well spent. As the program began its journey into the community, the best began to happen with children and the parents. We prepared familiar food with new twist, collards greens with no meat; to make home-cooked foods with fresh herbs and olive oil; the children preparing salads, constructing parfaits and serving their parents and grandparents; providing chef hats and aprons and gifts for families to continue their healthy journeys.
About the Authors
Yawah Awolowo is an organic farmer and chef at Mahalah Farms in Cuba (Sumter County), Alabama.
Debra Clark is founder and executive director of the Health and Wellness Education Center in Livingston (Sumter County), Alabama.
Darlene Robinson is founder of Imagine Me Youth Development Program in Greene County, Alabama.
Lola Baydala, Fay Fletcher, Melissa Tremblay, Natasha Rabbit, Jennilee Louis, Kisikaw Ksay-yin, and Caitlin Sinclair
In response to high rates of substance abuse in their communities, members of the Maskwacis four Nations invited university researchers to partner in culturally adapting, implementing, and evaluating an evidence-based substance abuse and violence prevention program, the Life Skills Training program (Botvin & Griffin, 2014). This project used a community-based participatory research (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003) approach, and was carried out by university and First Nation community partners. To evaluate the impact of the adapted program, students completed pre and post questionnaires, and community members participated in focus groups. The adapted Maskwacis Life Skills Training program was delivered in schools for three years. Students’ knowledge increased significantly during program delivery, and strong support was documented from community members. This project demonstrates the impact that can be facilitated by culturally adapting and delivering a prevention program, and by forming a community-university partnership.
The Maskwacis First Nations, located in central Alberta, Canada, include four communities: LouisBull, Montana, Samson, and Ermineskin. These neighboring Plains Cree Nations have a combined population of approximately 15,000 with roughly 53% of the population aged 17 or under (Grekul & Sanderson, 2011). The Nations are governed by independent chief and councils, and have separate education directors and schools.
National media attention has focused on high rates of crime and gang violence in Maskwacis, largely ignoring the rich and vibrant Cree culture that threads the four Nations together. In Maskwacis, Cree history, culture, and language are featured prominently in traditional community ceremonies and cultural events that regularly take place. Community Elders view their culture and language as a means to combat the social and public health problems that face Maskwacis community members as a result of the destructive impact of colonization, residential schools, and forced assimilation. Research evidence supports this perspective, demonstrating that a positive cultural identity can be a protective factor against substance abuse and violence for Indigenous
children and youth (French, Kim, & Pillado, 2006; Gazis, Connor, & Ho, 2009; Kulis, Napoli, & Marsiglia, 2002).
The current project stemmed from a previous pilot project led by a partnership between university researchers and Alexis Nakota Sioux Nation members (Baydala, Fletcher, Worrell, Kajner, Letendre, & Rasmussen, 2014). Alexis community members sought to address the root causes of fetal alcohol spectrum disorder, and invited university researchers to partner with them in working toward substance abuse prevention. This pilot project involved the cultural adaptation, delivery, and evaluation of a substance abuse and violence prevention program in the Alexis community. The Botvin Life Skills Training (LST) program (Botvin, Baker, Renick, Filazzola, & Botvin, 1984; Botvin & Griffin, 2014) was chosen for the project, based on the results of a literature review that identified the program as having extensive, high-quality evidence to support its effectiveness, including multiple randomized control trials (National Registry of Evidence-based Programs and Practices, 2008). The three-year LST program has three levels, each including between 8 and 14 one-hour modules, and begins in either elementary or junior high school (see Table 1).
In 2010, a Maskwacis community member attended a presentation delivered by university researchers and Alexis community members. Following this presentation, Maskwacis community members invited university researchers to partner in introducing and adapting the LST program in their own community.
The partners identified three broad objectives for the current project: (1) culturally adapt the LST program to reflect the language, culture, and visual images of the Maskwacis community; (2) deliver the adapted program in Maskwacis schools; and (3) evaluate the impact of the adapted program. Given the established effectiveness of the LST program in reducing substance abuse and violence and increasing skills related to prevention, the aim of the current project was to deliver the culturally adapted program with fidelity to the original curriculum, and to document the impact of the adapted program.
Research has demonstrated the importance of programming for Indigenous children to reflect Indigenous worldviews and culture (Hare, 2011). Further, the cultural content of programming has been correlated with enhanced learning for Indigenous children (Tsethlikai & Rogoff, 2013). A recent policy report also declared that the representation of Indigenous cultures, languages, and traditions in school classrooms is essential for promoting the academic success of Canadian Indigenous children and youth (Toulouse, 2008). For minority populations in general, culturally adapted evidence-based programs have been shown to be more effective than standard programs (Kumpfer, Magalhaes, & Xie, 2012).
The LST program is a generic program proven highly effective with students from different geographic regions, socioeconomic circumstances, and racial-ethnic backgrounds (Botvin, Griffin, Diaz, & Ifill-Williams, 2001). However, because the LST program does not incorporate Cree values, language, culture, and identity, Maskwacis community members and Elders decided to culturally adapt the program. In keeping with the principles of knowledge to action research (Graham, Logan, Harrison, Straus, Tetroe, Caswell, & Robinson, 2006), adaptations to the LST program, and the corresponding creation of the Maskwacis Life Skills Training (MLST) program, began in the first year of the project.
During the first year, a six-member adaptation committee was formed consisting of Elders from each of the Maskwacis four Nations. This committee met weekly to complete adaptations to the first level of manuals. Adaptations to the second and third levels were completed by a rotating group of Elders. During adaptation meetings, over 30 different Elders from all four communities reviewed the original LST curriculum and provided recommendations for adaptations, consisting of Cree language and syllabics, Elders’ teachings, and personal life stories. Following committee meetings, community and university partners worked together to adapt the manuals. Additionally, a community member created visual images for the manuals that reflected the Maskwacis culture and community. After adaptations were completed, Elders reviewed and provided approval for the adapted manuals.
Throughout the adaptation process, Elders were instrumental in ensuring that the adapted program accurately reflected the Cree culture. For example, one of the modules in the original LST curriculum focused on the harmful effects of tobacco use. However, Elders shared the importance of distinguishing between “poison tobacco” and “sacred tobacco” (i.e., kistemaw). In Maskwacis, kistemaw has important spiritual, cultural, and ceremonial purposes, such as being offered to communicate gratitude in advance of a request. Accordingly, the original LST module on smoking was adapted to focus on both the detrimental effects of poison tobacco as well as the healthy traditional use of kistemaw. Additionally, in order to reinforce program adaptations, MLST staff created digital stories to accompany each program module; these were guided and narrated by community Elders.
As Figure 1 shows, the adaptation process resulted in a program that incorporated both Western and Indigenous foundations of substance abuse and violence prevention. Rather than infusing Indigenous pillars of effective substance abuse prevention into an existing Western framework, the integrity of the lessons from each worldview was maintained.
Community-Based Participatory Research
This project used a community-based participatory research (CBPR) approach (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003). With an emphasis on tangible benefits for communities, co-learning, equitable involvement, and respect for multiple forms of knowledge, CBPR can contribute to leveling power imbalances between community and university partners.
Because Indigenous peoples are calling for research to be done with and for Indigenous communities rather than on Indigenous communities, CBPR has been recognized as particularly suitable for research involving Indigenous peoples (Castleden, Morgan, & Lamb, 2012; Koster, Baccar, & Lemelin, 2012; Smith,1999). The partners in this project applied the principles of CBPR in multiple ways. In particular, academic team members traveled to the community each week to engage in team meetings; meetings began with smudging and a prayer conducted in the local Cree language; a consensus-based decision-making model was followed with contributions from all team members; program evaluation was participatory, with input and guidance from all partners; and all reports and program materials were created collaboratively and approved by all partners. The project was approved by the research ethics board at the partner university and by the Maskwacis First Nations through a Band Council Resolution (the authority mechanism by which elected representatives in First Nation communities support and authorize an action).
A total of 25 focus groups were held over the course of the three-year project with 42 school personnel, 102 students, 18 Elders, four parents, and 12 MLST facilitators. With the exception of student focus groups, which were held with classes of approximately 20 students, focus groups included between six and eight participants to enable effective information sharing (Kreuger & Casey, 2015). Focus groups took place at the end of each of the three program years, and were held separately with school personnel from each school, students from each school, Elders, parents, and facilitators. The length of focus groups was between one and two hours.
Using purposive sampling, university partners recruited MLST facilitators, school personnel, and students to participate in focus groups, while community partners recruited Elders and parents. Community and university partners worked together to gather consent for participation. Although university partners facilitated all focus groups, community partners were present during Elder focus groups to translate from Cree to English where necessary. University research team members who conducted focus groups had graduate-level training in research methods and focus group facilitation as well as specific experience working with Indigenous communities.
Different focus group guides were developed for school personnel, students, Elders, parents, and MLST facilitators. Questions focused on participants’ experiences with the program adaptation and delivery, program impact on students, schools, and the community, unanticipated successes and challenges, and suggestions for improvement. Focus groups were conducted using a conversational style whereby participants were invited to speak to one another rather than directly to the facilitator, disagreements were encouraged, and participants were guided to express their thoughts in their own words and on their own terms (Kitzinger, 2005).
With participants’ permission, focus groups were audio recorded and transcribed verbatim. Data were managed with ATLAS.ti software, and analyses were completed using content analysis. Data were grouped according to common content, and a preliminary coding scheme was developed. The coding scheme was refined during discussions among partners, and themes were subsequently identified. Transcripts and preliminary data analyses were presented to participants as a means of member checking.
To evaluate learning of program content, questionnaires were distributed to students before and after each year of program delivery. The LST questionnaire (National Health Promotion Associates, 2011a; 2011b) and Piers-Harris Self-Concept Scale, Second Edition (Piers & Herzberg, 2002) were distributed to students. The wording of LST questionnaires was adjusted to accommodate students’ reading levels. The elementary school version of the LST questionnaire included items regarding knowledge and attitudes toward smoking and drinking, as well as social and personal self-management skills. The junior high version of the LST questionnaire included similar content with an added focus on drug refusal skills. Additionally, LST questionnaires were adapted to include a cultural knowledge scale consisting of items created by community members and Elders. From the initial stages of the project, community partners emphasized the importance of using a strengths-based approach. Accordingly, questionnaire items that directly questioned frequency of substance use were eliminated. Instead, questionnaire items
focused on constructs that are highly related to decreased substance use (i.e., knowledge, attitudes, self-management, refusal, and social skills).
Children who provided verbal assent and whose parents provided written consent for participation completed questionnaires before and after each year of program delivery (see Table 2). Where assent and/ or consent was not provided, children still received the MLST program but did not complete questionnaires.
A sequential longitudinal cohort design was used, and a linear mixed model was used to analyze the questionnaire data. The linear mixed model has advantages over repeated measures ANOVA for working with repeated measures data (Kreuger & Tian, 2004). Primarily, a linear mixed model gives structure to the error term by adding additional random-effect terms. This is important to properly account for error that can arise due to correlations between data points within the same subject (intra-subject correlation). Also, the linear mixed effects model can accommodate missing data points, and is less likely to lead to spurious results (both Type I and Type II error) with categorical data.
Focus group findings were grouped into three overarching themes, consisting of (i) program impact; (ii) factors that contributed to the program’s success; and (iii) suggestions for improvement.
Program impact. Focus group participants described the impact of the MLST program in a number of areas. School impact was a particular
area of focus. Participants described how the program allowed for traditional Elders’ teachings to be brought in to the schools; that the program facilitated both student and teacher learning about Cree culture and the Maskwacis community; and how the program had resulted in positive classroom changes. As one teacher commented, “there’s a more positive environment in the classroom…we’re all communicating and we’re talking and there’s no yelling.”
Student impact was also described. Participants not only discussed how students had demonstrated learning of program content; they also described how the program had contributed to
elevating students’ self-esteem, respectful attitudes, and listening skills. According to one student, “I learned to like myself for who I am.” Similarly, as a school staff member stated, “I noticed their self-esteem was brought up…I noticed with their attitudes, they’ve been more respectful…to the class teacher and they’ve learned to respect themselves and their fellow peers…” Additionally, focus group participants reported that the MLST program had contributed to students’ enhanced sense of pride in their identity. One teacher described how the program had “opened up a lot of doors for them to try and realize that being who they are is okay.” Similarly, Elders reported an increase in youth attendance at cultural events in the community.
Focus group participants also described Elder impact. Elders recounted how the program had facilitated their own learning by reminding them about ancestral teachings shared by other Elders. Elders also felt that their participation in the program allowed them to contribute to their community in meaningful ways, resulting in positive personal impact. As one Elder described, “it warmed my heart to make me feel wanted.”
Similarly, facilitator impact was described. Facilitators felt that they had benefited from being immersed in cultural teachings, and described bringing traditional teachings into their own homes for their children and grandchildren to learn: “I had pride in what I was teaching. And for my personal life…I’m raising my grandson, and he gets to have a bulk of what I teach.” Additionally,
facilitators were impacted by the opportunity to build strong relationships with Elders, and also described feeling proud to make a difference in their community.
Factors contributing to program success. Among the multiple contributors to the MLST program’s success, participants described the focus on cultural teachings as paramount. According to participants, it was critical for facilitators and Elders to teach students about the importance of respect, honoring the Creator, speaking Cree, learning Maskwacis history, and developing cultural pride. These teachings were fundamental to student engagement and overall program acceptability. One teacher described how,
This content would not have worked had it not been presented culturally…we prayed before every class, we burned sweet grass…and the Elder led them in prayer and that just locked them in, they knew they were home once they did that.
In order to enrich the program’s cultural content, Elder involvement was also critical. Facilitators felt that Elders’ guidance and knowledge made program adaptations successful, while students, facilitators, and school personnel appreciated Elders’ presence in the classroom. As one facilitator described, “it seems like the more we bring the Elders in, the more the children benefit.”
Additionally, focus group participants commended the program’s community relevance.
Because the program was adapted specifically for the Maskwacis community by Maskwacis Elders and community members, the program incorporated community relevant language and visual images as well as local knowledge. Additionally, because MLST facilitators were community members, they were familiar to many students; even where facilitators and students did not have pre-existing relationships, focus group participants reported that students could easily identify with facilitators from their own community.
Facilitator skills were also important for the program’s success. Equipping facilitators with the necessary skills was a challenge in the first year of the program, with school personnel noting steady improvements in the second year of implementation, and providing exclusively positive feedback in the third year. The most important facilitator skills described by focus group participants were sincerity, confidence, the ability to engage students, the flexibility to accommodate students’ learning and reading levels, and classroom management competence. One teacher noted that, “it was really great that [the facilitator] was so enthusiastic about the program…it got the kids engaged, and they could sense that she really cared.”
Although it was important for facilitators to possess skills in leading students through program teachings and activities, teacher involvement was also described as essential. Facilitators noted that when teachers supported and contributed to MLST classes, students demonstrated increased engagement. It was helpful for teachers to assist with classroom management, to provide feedback to facilitators, and to supplement facilitators’ teachings with their own knowledge and experience. By the third year of implementation, all facilitators and school personnel reported strong relationships.
Finally, it was important for the MLST program to demonstrate compatibility with the school’s core curriculum. According to school personnel, the MLST program complemented students’ regular social studies curriculum, and was a suitable replacement for regularly scheduled health classes. This simplified the task of fitting MLST classes into busy classroom schedules. Even when classroom schedules were hectic, however, teachers were willing to prioritize the program: “They’re life skills, they’re things that are literally going to get them into their adulthood, so maybe math can wait a little while.”
Suggestions for improvement. Focus group participants made a number of suggestions for program improvement related to program delivery. In the first year of implementation, it was strongly suggested that facilitators take part in additional training to enhance their teaching and classroom management skills. This was addressed during the second and third years of implementation. In addition, focus group participants communicated that the program should be delivered for the full school year rather than for only four months of the year. One teacher felt that, “In order to really get the full benefit of the program, it needs to be reinforced all year long…so they are always reminded and supported.”
Suggestions for improvement were also made related to program content. During the first year of implementation, a recommendation was made to increase the involvement of Elders in program delivery, to incorporate more cultural teachings, and to improve the readability of manuals. Each of these suggestions were addressed by setting a goal to bring Elders into each MLST class, and by adapting manuals to include stronger cultural elements and lower reading levels. Suggestions were also made to add modules related to grieving and gender roles, as well as to implement a parent component. Finally, focus group participants strongly suggested that the program include more hands-on activities: “When we would do activities, it was amazing, it was the most interaction I would get out of my kids all year.”
Questionnaire results are summarized below, with scores at baseline and at the final data collection point (i.e., year 3 post) depicted. Tables 3 and 4 show LST questionnaire scores for elementary and junior high students respectively. Analyses demonstrated statistically significant increases in scores for elementary students on all LST questionnaire scales. For elementary students, the Overall Knowledge scale included items relevant to anti-smoking knowledge (i.e., the harmful effects of smoking) and life skills knowledge (i.e., communication, decision-making, advertising, self-esteem, dealing with stress, and assertiveness). Items relevant to cultural content (e.g., Cree words and cultural protocol) were included in the Cultural Knowledge scale. The Attitude scale included items relevant to anti-smoking and anti-drinking attitudes. Finally, the Life Skills scale included the same content as the life skills knowledge items, but inquired about behaviors related to these content areas rather than inquiring about knowledge.
Statistically significant differences between baseline and the end of year 3 were also demonstrated for junior high students on all LST questionnaire scales with the exception of relaxation skills. For junior high students, Overall Knowledge included anti-drinking knowledge and life skills knowledge. The Cultural Knowledge scale included items with the same content as that of the elementary questionnaire. Also similar to elementary students, the Attitude scale included anti-smoking and anti-drinking attitude content. Finally, the Life Skills scale included items relevant to drug refusal skills, assertiveness skills, relaxation skills, and self-control skills.
Table 5 depicts elementary and junior high students’ scores for the Piers-Harris Children’s Self-Concept Scale-2. There were no statistically
significant differences between baseline and the end of year 3 for elementary or junior high students on the Piers-Harris-2.
The current project demonstrates the meaningful impact that can be achieved by culturally adapting and delivering an evidence-based prevention program in a First Nations community. Quantitative results obtained from pre and post LST questionnaires showed significant positive increases in knowledge, skills, and attitudes between baseline and the end of year three. Findings reflect that students not only retained the knowledge communicated to them through the MLST program, but that students learned progressively more from the program each year. Further, because students’ scores increased related to both original program content and additional cultural content, these results demonstrate that students improved their knowledge, skills, and attitudes related to substance abuse from both Euro-Western and Indigenous perspectives.
With regard to the Piers-Harris questionnaire, there were no statistically significant differences between scores at baseline and the end of year three. This may be due in part to students’ scores being average at baseline (i.e., t scores approximated 50). It may not be realistic to expect mean self-concept scores to increase beyond an average range. Further, like other social-emotional measurement tools, the Piers-Harris-2 was validated using a U.S. sample of primarily Caucasian children (Piers & Herzberg, 2002), and its applicability to Indigenous children has not been established. Moving forward, it will be important to establish and utilize pre and post measurement tools that reflect the unique realities of Indigenous children.
In addition to pre and post questionnaires, focus groups were implemented to provide insight regarding program impact, factors contributing to program success, and suggestions for improvement. Conducting focus groups after each year of program delivery allowed for suggestions to be addressed continually, contributing to growing community support. Overall, focus group participants communicated strong support for and investment in the program as a promising means to address substance abuse and violence and to enhance traditional culture in Maskwacis. Focus groups provided information regarding the widespread impact of the program on schools, Elders, facilitators, and students.
Key to program impact was the utilization of a community-based participatory approach that incorporated Western and Indigenous pillars of knowledge. In this way, the MLST program honored the concept of “two-eyed seeing” (Figure 2), developed by Mi’kmaw Elders Albert and Mudena Marshall. “Two-eyed seeing” recognizes multiple diverse perspectives as valid without privileging one viewpoint over another. Enacting this principle also means acknowledging that multiple perspectives can lead to a richer understanding of health issues than one perspective alone. By acknowledging the value of Western and Indigenous substance abuse prevention models, our community-university partnership provided an example of how Indigenous and Western research paradigms can co-exist in a space that honors both worldviews. Focus groups revealed that this was critical to the success of the project. In particular, schools and community members identified the central importance of cultural adaptations to the program’s acceptance by students, schools, and the wider community. Focus group participants also reported that adaptations elevated the program’s potential to reach and engage students by presenting content in a way that was relevant to their community and cultural context. Similarly, adaptation of MLST questionnaires was important to honor community partners’ perspectives regarding the importance of capturing the program’s impact on students’ cultural knowledge.
The current project also suggests that CBPR can act as a catalyst for community change. However, although pre and post questionnaires and focus groups provided the opportunity to obtain important community feedback on the MLST program, our research team identified a number of unanticipated and peripheral community impacts that were not accounted for by the use of our
conventional evaluation methods. For example, community partners took on leadership roles in the program and formed a non-profit society to sustain the MLST program beyond the terms of our original research grant. In order to more fully understand these additional impacts, we supplemented the current evaluation methods with Outcome Mapping (Earl, Carden, & Smutylo, 2001; Tremblay, Baydala, Rabbit, Louis, & Ksay-yin, submitted for publication). Outcome Mapping is a tool that is sensitive to community change and development, and that emphasizes the significant process of culturally adapting and implementing programs in the context of a CBPR partnership.
Finally, a potential limitation of this study should be acknowledged; namely, the current study does not include a control group. In order to engage all four Nations in this project, it was necessary to deliver the MLST program in all Maskwacis schools. An intervention group and control group were initially defined; however, because these groups came from the same schools, it was not possible to maintain the integrity of the control group. In particular, students often switched back and forth between control group and intervention group classrooms, and teachers indicated that students were sharing MLST teachings with students in the control group. Consistent with our experience, recent literature indicates that treating complex community change initiatives as controlled experiments may be inappropriate, particularly in the context of CBPR (Kelly, 2010). In addition, we intentionally selected an evidence-based program with extensive research to support its effectiveness. As a result, it was not necessary to once again prove the effectiveness of the program; rather the aim of this project was to deliver the culturally adapted program with fidelity to the original curriculum, and to document the impact of the adapted program.
Maskwacis community members and Elders established a partnership with University of Alberta researchers to culturally adapt, implement, and evaluate an evidence-based substance abuse and violence prevention program. Students’ overall knowledge increased significantly during the three years of program implementation, and strong support was documented from schools, Elders, students, and other community members. This project demonstrates the considerable impact that can be facilitated by culturally adapting and delivering a prevention program, and by forming and maintaining a strong community-university partnership.
Baydala, L., Fletcher, F., Worrell, S., Kajner, T., Letendre, S., Letendre, L., & Rasmussen, C. (2014). Partnership, knowledge translation, and substance abuse prevention with a First Nations community. Progress in Community Health Partnerships, 8(2), 145–155.
Botvin, G.J., Baker, E., Renick, N.L., Filazzola, A. D., & Botvin, E.M. (1984). A cognitive-behavioral approach to substance abuse prevention. Addictive Behaviors, 9(2), 137–147.
Botvin, G.J., Griffin, K.W., Diaz, T., & Ifill-Williams, M. (2001). Drug abuse prevention among minority adolescents: Posttest and one-year follow-up of a school-based prevention intervention. Prevention Science, 2(1), 1–13.
Botvin, G.J., & Griffin, K.W. (2014). Life Skills Training: Preventing substance misuse by enhancing individual and social competence. New Directions for Youth Development, 2014(141), 57–65.
Castleden, H., Morgan, V. S., & Lamb, C. (2012). “I spent the first year drinking tea”: Exploring Canadian university researchers’ perspectives on community-based participatory research involving Indigenous peoples. The Canadian Geographer, 56(2), 160–179.
French, S.E., Kim, T.E., & Pillado, O. (2006). Ethnic identity, social group membership, and youth violence. In N. G. Guerra & E. P. Smith (Eds.), Preventing youth violence in a multicultural society (pp. 47–73). Washington, DC: APA Books.
Gazis, N., Connor, J.P., & Ho, R. (2009). Cultural identity and peer influence as predictors of substance use among culturally diverse Australian adolescents. Journal of Early Adolescence, 30(3), 345–368.
Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in translation: Time for a map? Journal of Continuing Education in the Health Professions, 26(1), 13–24.
Grekul, J., & Sanderson, K. (2011). “I thought people would be mean and shout.” Introducing the Hobbema Community Cadet Corps: A response to youth gang involvement? Journal of Youth Studies, 14(1), 41–57.
Hare, J. (2011). “They tell a story and there’s meaning behind that story”: Indigenous knowledge and young Indigenous children’s literacy learning. Journal of Early Childhood Literacy, 12(4), 389–414.
Israel, B.A., Schulz, A.J., Parker, E.A., & Becker, A.B. (1998). Review of community-based research: Assessing partnership approaches to improve public health. Annual Review of Public Health, 19, 173-202.
Kelly, T. (2010). Five simple rules for evaluating complex community initiatives. Community Investments, 22(1), 19-23.
Koster, R., Baccar, K., & Lemelin, R.H. (2012). Moving from research ON to research WITH and FOR Indigenous communities: A critical reflection on community-based participatory research. The Canadian Geographer, 56(2), 195-210.
Kitzinger, J. (2005). Focus group research: Using group dynamics to explore perceptions, experiences, and understandings. In I. Hollway (ed.), Qualitative research in health care (pp. 56–70). Maidenhead: Open University Press.
Kreuger, R.A., & Casey, M. (2015). Focus groups: A practical guide for applied research (5th ed.). Thousand Oaks, CA: Sage Publications.
Kreuger, C., & Tian, L. (2004). A comparison of the general linear mixed model and repeated measures ANOVA using a dataset with multiple missing data points. Biological Research for Nursing, 6(2), 151-157.
Kulis, S., Napoli, M., & Marsiglia, F.F. (2002). Ethnic pride, biculturalism, and drug use norms of urban American Indian adolescents. Social Work Research, 26(2), 101–112.
Kumpfer, K.L., Magalhaes, C., & Xie, J. (2012). Cultural adaptations of evidence-based family interventions to strengthen families and improve children’s developmental outcomes. European Journal of Developmental Psychology, 9(1), 104-116.
Minkler, M., & Wallerstein, N. (Eds.). 2003. Community-based participatory research for health. San Francisco, CA: Jossey-Bass.
National Health Promotion Associates, Inc. (2011a). Life skills training questionnaire: Elementary school version. Retrieved from www.lifeskillstraining.com.
National Health Promotion Associates, Inc. (2011b). Life skills training questionnaire: Middle school version. Retrieved from www.lifeskillstraining.com.
National Registry of Evidence-based Programs and Practices. (2008). Retrieved from http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=109.
Piers, E.V., & Herzberg, D.S. (2002). Piers-Harris Children’s Self-Concept Scale™, Second Edition. Los Angeles, California: Western Psychological Services.
Smith, L., (1999). Decolonizing methodologies: Research and Indigenous peoples. London, UK: Zed Books.
Toulouse, P.R. (2008). Integrating Aboriginal teaching and values into the classroom. Sudbury, ON: Laurentian University.
Tsethlikai, M., & Rogoff, B. (2013). Involvement in traditional cultural practices and American Indian children’s incidental recall of a folktale. Developmental Psychology, 49(3), 568–578.
About the Authors
Lola Baydala is an associate professor in the Department of Pediatrics at the University of Alberta.
Fay Fletcher is professor and associate dean academic, Faculty of Extension, the University of Alberta.
Melissa Tremblay is a program evaluator in the Department of Pediatrics at the University of Alberta.
Natasha Rabbit is the executive director of the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta.
Jennilee Louis is a research assistant for the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta.
Kisikaw Ksay-yin is an Elder in the Nehiyaw Kakeskewina Learning Society, Maskwacis, Alberta.
Caitlin Sinclair is research coordinator, Department of Pediatrics, University of Alberta.
Melissa Tremblay, Lola Baydala, Natasha Rabbit, Jennilee Louis, and Kisikaw Ksay-yin
The aim of the current paper is to discuss the use of Outcome Mapping as a tool for evaluating community and stakeholder changes that occurred when a prevention program was culturally adapted and delivered through a community-university partnership. To the authors’ knowledge, this paper represents the first account of using Outcome Mapping as an evaluation tool in a Canadian Indigenous context. Changes in the behavior, actions, activities, and relationships of five boundary partners were retrospectively documented using the tool. Data demonstrated positive impact on Elders and students; growing community investment in and support for the Maskwacis Life Skills Training program’s cultural components; progressive increases in community ownership of the program; and growth in the community-university partnership. Overall, Outcome Mapping provided a systematic method for understanding peripheral changes that are often overlooked in conventional research and evaluation, but that nonetheless indicate progress toward community changes and long-term impact.
Growing evidence for the value of community-based participatory research (CBPR) (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003) has resulted in a proliferation of studies that utilize this approach (Jagosh, Macaulay, Pluye, Salsberg, Bush, Henderson, Sirett, Wong, Cargo, Herbert, Seifer, Green, Greenhalgh, 2012). With an emphasis on translating research findings for use in communities, bidirectional learning and capacity building, equitable involvement, and honoring multiple forms of knowledge, CBPR does away with the conventional hierarchy between researchers and those being researched. In this way, CBPR is particularly suitable for work with Indigenous communities (Castleden, Morgan, & Lamb, 2012; Cross, Friesen, Jivanjee, Gowen, Bandurraga, Matthew, & Maher, 2011; Gauld, Smith, & Kendall, 2011). Indeed, the option of community participation in research affecting Indigenous peoples is recognized as an ethical imperative (Canadian Institutes of Health Research, 2007). When community control and agency are emphasized and Indigenous community members contribute in meaningful ways to the research process, local capacity is strengthened and community-level changes can occur (Kelly, Saggers, Taylor, Pearce, Massey, Bull, Odo, Thomas, Billycan, Judd, Reilly, & Ahboo, 2012; Salimi, Shahandeh, Malekafzali, Loori, Kheiltash, Jamshidi, Frouzan, & Majdzadeh, 2012). The task of evaluating and measuring community-level change is not simple or straightforward. Although the randomized controlled trial has long been recognized as the gold standard for academic research, experimental approaches are often inappropriate for evaluating programs that take place in community settings (Judge & Bauld, 2001; Victora, Habicht, & Bryce, 2004). Kelly (2010) notes that communities are complex and constantly evolving; as such, evaluations of community-based initiatives cannot adhere to a single theory or model, and cannot test direct cause-and-effect relationships as in a closed system. In response to concerns with utilizing experimental designs to evaluate complex community initiatives, innovative evaluation approaches are beginning to emerge (Dart & Davies, 2003). Outcome Mapping (Earl, Carden, & Smutylo, 2001) represents one such approach. The aim of the current paper is to discuss the use of the tool for evaluating community and stakeholder changes that occurred when a substance abuse and violence prevention program was culturally adapted and delivered through a community-university partnership. To the authors’ knowledge, this paper represents the first study that uses Outcome Mapping as an evaluation tool in a Canadian Indigenous context.
The Community and the Project
The Maskwacis community, located in central Alberta, Canada, is made up of four Cree Nations: Louis Bull, Montana, Samson, and Ermineskin. The four nations have a combined population of approximately 15,000. These neighboring nations function independently, with separate chief and councils, education directors, and schools. Approximately 53% of the Maskwacis population is under the age of 18 (Grekul & Sanderson, 2011). As a result of colonization and its ongoing impact on Indigenous peoples, high rates of substance abuse and acts of violence pose a challenge for many First Nations communities in Canada. In response to these challenges, Maskwacis community members invited University of Alberta researchers to partner with them to culturally, adapt, deliver, and evaluate a substance abuse and violence prevention program to children and youth in Maskwacis schools. In order to realize this goal, a community-university partnership was formed in the spirit of CBPR. The first stage of the project involved culturally adapting the evidence-based Life Skills Training (Botvin, Baker, Renick, Filazzola, & Botvin, 1984; Botvin & Griffin, 2014) program to incorporate Cree culture, language, and values. Elders and community staff led the cultural adaptation process (Baydala, Fletcher, Tremblay, Rabbit, Louis, Ksay-Yin, & Sinclair, 2016). The adapted Maskwacis Life Skills Training (MLST) program was subsequently delivered for three years in Maskwacis schools by community program facilitators.
Evaluation of the MLST program was carried out over three years of program implementation. The evaluation consisted of (1) focus groups conducted at the end of each program year with school personnel, Elders, program facilitators, parents, and students to obtain feedback on the program, discuss suggestions for improvement, and understand the successes and challenges of the program; and (2) pre- and post- program questionnaires distributed to student participants (National Health Promotion Associates, 2011a, 2011b). By the third year of implementation, it was clear that MLST program effects were extending into the wider community. Partners were aware of a web of individual and community-level outcomes that could not be comprehensively captured with questionnaires or focus groups alone. Further, program facilitators often shared meaningful stories about program impact, but did not feel that evaluation processes allowed for documenting this informal data. Accordingly, Outcome Mapping (Earl et al., 2001) was utilized to supplement the evaluation of the MLST program. This paper will share project findings documented through Outcome Mapping, describe the use of Outcome Mapping as a retrospective evaluation tool, and discuss the suitability of this method for use in a CBPR partnership as well as with Indigenous communities.
Outcome Mapping was developed in Canada by the International Development Research Centre, and was released to researchers and practitioners in 2001. Since that time, Outcome Mapping has mainly been used in Africa, Latin America/ Caribbean, and Asia, with only 2% of reported Outcome Mapping use in North America (Smith, Mauremootoo, & Rassmann, 2012). Although the tool was created for use in an international development context, research has documented its potential applicability to more diverse settings, including economically developed countries (Smith et al., 2012).
Outcome Mapping is a participatory approach to planning, monitoring, and evaluation. It represents a shift from defining results in terms of long-term impact to defining results in terms of observable changes in partners’ behaviors, actions, activities, and relationships. Although long-term impact is the ultimate goal of community-based projects, Outcome Mapping recognizes the importance of also tracking smaller-scale, incremental changes that indicate meaningful progress. An exclusive focus on impact can preclude a focus on these incremental changes, “severely limiting…potential for understanding how and why impact occurs” (Earl et al., 2001, p. 6). Further, the complex nature of community initiatives causes difficulty in linking large-scale impacts to discrete projects or initiatives. For this reason, Outcome Mapping acknowledges that the contributions of multiple projects, programs, organizations, and individuals are necessary to achieve large-scale impact. The work of a single initiative in isolation is not sufficient to achieve social change. Outcome Mapping is therefore useful in focusing on a project’s contributions to outcomes and impacts rather than attempting to attribute outcomes and impacts to one particular project.
The Outcome Mapping manual (Earl et al., 2001) describes 12 steps divided into three stages. The first stage is Intentional Design. This is a planning stage during which a project defines the changes it will aim to bring about as well as the strategies that will be used to contribute to the change process. The second stage is Outcome and Performance Monitoring. During this stage, monitoring priorities are identified, and methods are provided for monitoring progress toward desired outcomes. The third and final stage is Evaluation Planning. Appendix A (Page 67) lists the three stages and the steps required. Moving through the stages requires a series of workshops, with workshop implementation instructions provided in the manual.
Our Approach to Outcome Mapping
Recent research indicates that the procedure is especially relevant when adapted to meet the needs of an individual project (Smith et al., 2012). In practice, many researchers have employed the tool as part of a retrospective evaluation (Nyangaga, Smutylo, Romney, & Kristjanson, 2010). In this case, only the first stage, Intentional Design, is applicable. Stages 2 (Outcome and Performance Monitoring) and 3 (Evaluation Planning) are applicable when Outcome Mapping is used to prospectively plan, monitor, and evaluate a project.
For the current project, Outcome Mapping was used as a retrospective evaluation tool. From January to December 2013, approximately 150 hours were dedicated to workshops designed to move through the required steps described in stage one. These workshops were facilitated by the team’s program evaluator and always took place in the community setting. Workshop participants consisted of four academic team members, seven community team members, and two community Elders; this team is referred to as the working group committee (WGC).
When the WGC began the process, the project’s mission and vision statements had already been established (see Appendix B). Subsequent steps were followed in order, with boundary partners, outcome challenges, progress markers, and strategies developed by the WGC. After generating retrospective progress markers, the evaluator examined a number of data sources to collect evidence for progress. In particular, the evaluator examined minutes from four years of weekly WGC meetings; bi-annual reports to the project’s funder; transcripts from post-program focus groups conducted over three years; facilitator reports completed after each MLST class session over three years; and pre and post program questionnaires completed by MLST students over three years. These data were systematically extracted, coded, and classified according to the identified progress markers. After data were extracted and compiled, the WGC engaged in a series of meetings to discuss findings. These discussions allowed the WGC to reflect on areas where progress had advanced considerably as well as areas where less progress had occurred.
After progress markers were identified, workshops focused on identifying the strategies used to achieve the outcome challenges. In consultation with a practitioner, the WGC decided not to apply the strategy categorization process described in the Outcome Mapping manual. Instead, the evaluator separated identified strategies into those that the WGC had already successfully employed and those that would be useful in moving forward. In this way, the technique was not only useful as a retrospective evaluation tool, but was informative for ongoing program planning and improvement. The WGC decided not to employ Step Seven: Organizational Practices, as it was determined that the first six steps provided sufficient information and detail for the retrospective evaluation.
Changes in the behavior, actions, activities, and relationships of five boundary partners including Elders, leadership/education directors, schools, community and university partners were documented using the first stage. Changes in the student boundary partner were also documented, but were included in a separate paper (Baydala et al., 2016). These changes indicated meaningful progress toward desired outcomes. Consistent with the program’s vision and mission statements, outcome challenges and examples of progress markers are presented below for each of the five boundary partners. Examples of progress markers are presented in table form along with a summary of changes to support each progress marker. In keeping with guidelines outlined in the manual, progress markers were divided according to changes that the WGC expected to see (indicating an early response to project activities), liked to see (indicating more active engagement of boundary partners), and loved to see (indicating profound change in boundary partners). Finally, strategies for achieving progress markers are summarized for each boundary partner.
As in other Cree communities, Maskwacis Elders are held in the highest regard and act as community educators, historians, and storytellers. Because Cree culture and language were at the heart of the MLST program, Elders were critically important to the program’s planning and implementation. Following from the program’s vision toward healthy First Nation communities, the outcome challenge identified for Elders was to strengthen relationships between Elders and youth. Table 1 lists progress markers and a summary of changes for this outcome challenge.
Outcome Mapping allowed the WGC to evaluate a bi-directional learning relationship between the MLST program and community Elders. In particular, not only was the MLST program founded on the knowledge and direction of Elders; the WGC was also promoting behavioral changes in Elders by encouraging the formation of Elder-youth relationships in the community.
The primary strategy toward building relationships between Elders and youth was to invite Elders into Maskwacis classrooms as part of the MLST program. When Elders could not be physically present during MLST classes, digital stories, narrated in Cree by community Elders, were used to reinforce cultural teachings. Also, facilitators deliberately emphasized the importance of respecting community Elders and created opportunities for Elders and youth to interact in positive ways both in the classroom and through extracurricular events including a hide tanning cultural camp. In addition to allowing the WGC to identify existing strategies for promoting Elder-youth relationships, Outcome Mapping prompted the WGC to generate ideas for additional relationship-building activities. For example, the WGC planned to implement a tea and bannock day where students could invite their grandparents to engage in storytelling at their schools.
The Maskwacis four Nations are governed by separate leaders (i.e., chiefs and council members), and have separate education directors. In Maskwacis as in other First Nation communities, formal approval from community leaders during the project’s initiation was required; this approval was granted in the form of band council resolutions from each Nation. The outcome challenge here was to reawaken the spirit of our leaders’ and education directors’ Nehiyaw mamitonecikan (i.e., Cree spirit). Progress markers are listed in Table 2.
Toward the outcome challenge of reawakening their Cree spirit, leaders and education directors were strongly supportive of the MLST program’s goal to promote culture in Maskwacis schools.
Although communicating with leaders was difficult due to their busy schedules and demanding jobs, the WGC did establish and maintain ongoing relationships with a number of prominent community members, resulting in one chief taking on an advocacy role in support of the program, and in all leaders and education directors signing letters in support of the program.
In order to facilitate progressive changes in leaders and education directors, community staff employed a number of relationship building strategies. In particular, leaders and education directors were invited to all MLST events, were given MLST newsletters, and were provided with a comprehensive business plan. Community staff also regularly attempted to schedule meetings to provide updates. Regarding prospective strategies, the WGC committed to provide leaders and education directors with monthly updates, and to invite leaders from all four Nations to meet as a collective.
Evaluating leaders and education directors as boundary partners allowed the WGC to recognize how important they were to program sustainability and to facilitating positive community change. As part of the process, the WGC discussed that it was important to maintain relationships with leaders and education directors beyond obtaining band council resolutions and formal written approval. In order for the program to thrive and have a community-wide impact, it was necessary for the WGC to provide leaders and education directors with multiple opportunities to become familiar with and involved in the MLST program.
While the support of community leaders and education directors was necessary to allow the MLST project in the Maskwacis community, the support of school personnel was necessary to allow for the project to experience success in Maskwacis schools. The outcome challenge for schools was to support and promote the program in the school environment, particularly the cultural aspects of the program. Progress markers and a summary of changes are listed in Table 3.
The above summary highlights that school personnel became increasingly welcoming, supportive, and interested in the MLST program and the program’s cultural teachings. During the first year of the program, teachers were hesitant to accept the program, rarely supported facilitators with classroom management, and reported a number of issues with facilitator reliability. By the third year, teachers provided expressly positive feedback about facilitators and the program, allowed extra MLST class time, began to advocate for the program, and incorporated cultural teachings into core subjects. Outcome Mapping allowed for the WGC to identify these progressive changes in school personnel.
To promote these changes, a number of strategies were utilized. An important strategy at the outset of the project was to invite schools to participate in program training delivered to facilitators. This allowed for schools to gain an understanding of the program before it began. Additionally, schools were given the opportunity to participate in focus groups at the end of each project year in order to provide feedback. Each year, this feedback led to additional strategies being employed for schools. In particular, the second and third years of the program saw more emphasis placed on facilitators being punctual and reliable, facilitators communicating with teachers, and facilitators being present in schools outside of MLST class time. Evaluation updates and promotional material were also provided to schools as a means of keeping school personnel informed about the program. Through Outcome Mapping, the WGC was able to identify and reflect on each of these strategies and the extent to which they were successful with schools.
Using the technique, a number of prospective strategies were also identified for schools. Primarily, because all three years of the program saw low attendance from school personnel at MLST events, the WGC determined that they needed to employ more targeted efforts to involve schools in MLST events. Additionally, it was determined that facilitators should spend time in their classrooms before the MLST program began in order to develop rapport with students and teachers and to establish mutual expectations. The WGC also committed to sharing a facilitator code of conduct with teachers, providing more regular updates to school principals, and annually revisiting the program’s memorandum of understanding with schools. Finally, unlike schoolteachers, facilitators were not University educated; rather, they were well-informed and knowledgeable regarding their culture and language. It was important for teachers to understand and respect the cultural qualifications of facilitators from the outset of the project. Accordingly, it would be important for the MSLT program to more clearly communicate with school personnel regarding the qualifications of facilitators at the beginning of the school year. Overall, Outcome Mapping provided a means for the WGC to generate prospective strategies in a systematic way that allowed for targeting areas where additional progress could be made.
Community partners (Table 4) included both MLST facilitators and program administrative staff. The outcome challenge for community staff was to practice, promote, and support a Nehiyaw (i.e., Cree) worldview through the program.
In identifying and evaluating progress markers for community partners, the WGC began to recognize the complexity of community partners’ roles, which extended beyond delivering the MLST program to students. These included developing their leadership capacity, as well as building relationships with Elders, students, schools, other MLST staff, the wider community, and university partners.
It was important to clarify strategies that facilitated the success of community partners in their complex roles. Strategies to this end were focused on promoting culture within the workplace. In particular, the job descriptions of community staff emphasized a significant cultural component, and it was clearly communicated to community staff that they were expected to spend time each day engaging with and learning from Elders. Additionally, community partners were permitted to participate in ceremonies and other cultural events during work hours, and took part in multiple culturally relevant professional development opportunities. To the extent that community partners were able to strengthen their cultural knowledge and connection with Elders, relationship building and accountability were similarly strengthened.
Moving forward, the WGC determined that it would be important to regularly revisit the staff code of conduct and handbook to ensure consistent staff expectations and standards. Another prospective strategy was to focus more on enhancing Cree language skills, and relatedly, to institute a Cree naming ceremony for all community staff.
University partners (Table 5) included the project’s principal investigator, research coordinators, research assistants, and a program evaluator. For university partners, the outcome challenge was to practice authentic CBPR.
The mapping technique highlighted that University partners experienced significant growth in their capacity to practice authentic CBPR. This was made possible by community partners’ willingness to educate university partners, which in turn facilitated a strong and trusting partnership.
A number of strategies were relevant to university partners. Perhaps most importantly, university partners were physically present in the community as often as possible. The WGC held weekly meetings in the community; accordingly, university partners were present in the community at least once per week. Having regular meetings between partners was essential for relationship building. University partners also traveled to the community to work with community partners on program adaptations, conference abstracts, presentations, and to complete day-to-day project tasks together. Community partners shared that it was also imperative for university partners to attend community events and ceremonies, and to create opportunities for partners to informally socialize. While these strategies were employed throughout the project, the WGC had not evaluated why and how these strategies were effective until they began to use the technique. In this way, the process allowed for partners to discuss and clarify their assumptions regarding CBPR and the strategies necessary for establishing a strong and equitable community-university partnership.
Through these discussions, the WGC identified a number of additional strategies for community-university partnerships. Primarily, at the outset of a project, partners should establish a mutual understanding of expectations in order to avoid later confusion and misunderstanding. In this vein, the WGC recommended that community partners be involved in budget discussions from the beginning of the project in order to understand how and why budget decisions are made; this would help community partners to develop a more solid understanding of budget management when projects later transition into sustainable, community-led programs. Additionally, both partners should be aware of the power imbalance that is inherent to grant funds being held at the university rather than in the community. Most, if not all, community-university partnerships must deal with the reality that large funding agencies award research grants solely to university partners, who are then accountable for managing funds. Although in the current project, a portion of the overall grant was allotted to community partners to manage themselves, funds were still administratively filtered through the university, which was reportedly frustrating for community partners. In order to deal with these frustrations and the power imbalance that accompanies this situation, partners must be open and honest with one another, and willing to discuss the uncomfortable circumstances that such a funding arrangement can result in. Again, the overarching strategy for working through these challenges was to develop strong and trusting relationships between partners throughout the project.
To supplement the MLST program evaluation, Outcome Mapping was employed as a retrospective evaluation tool by community and academic research partners. Through the process, a number of boundary partners were identified, including Elders, community leaders and education directors, schools, as well as community and university partners. Outcome challenges, progress markers, and strategies were identified for each boundary partner through a series of team workshops, and evidence to support progress was systematically collected from meeting minutes, focus groups, reports to funders, and program facilitators’ daily reports.
Overall, partners were in strong agreement that Outcome Mapping provided a meaningful method for evaluating and telling the MLST program’s story of progress. In this way, the process was congruent with Indigenous worldviews. In particular, Indigenous research methods often emphasize narrative and relationality. Through workshops, team members told stories of their experiences with the program, informing the creation of outcome challenges and progress markers. This also allowed for the project’s milestones to be structured as more of a narrative than conventional evaluation methods, resulting in a coherent story of progress. Similarly, because the WGC created progress markers, the project was able to capture outcomes that were significant to community partners.
Importantly, the process was extremely beneficial for the current project. Of particular importance was the opportunity that Outcome Mapping offered for relationship building, both among community staff members and between community and academic partners. Bringing team members together to collaboratively define evaluation outcomes served as a reminder that partners were working toward a common goal. Participating in interactive workshops also allowed for partners to more fully understand multiple perspectives, and to consider ideas that were often innovative and novel. In this way, rich opportunities were provided for partners to learn from one another, and provided a catalyst for community partners to articulate and communicate expectations of university partners. Further, it provided a means for all WGC members to feel a sense of ownership over and investment in the evaluation process. The MLST evaluation was perceived as less of an academic endeavor, and more of a learning opportunity for both community and academic partners. Disentangling the complex web of progress also served to enhance staff morale. The WGC described feeling a sense of immense accomplishment at the evidence for progress elucidated by the process.
Similarly, Outcome Mapping prompted the WGC to recognize outcomes that might otherwise be overlooked by more conventional evaluation tools. Although there is obvious importance in remaining focused on long-term impacts such as reduced substance abuse and healthy First Nation communities, the process allowed for the WGC to take pride in smaller accomplishments indicating progress toward these long-term goals. A systematic examination of smaller-scale changes and strategies also allowed for the WGC to examine how changes occurred, and why less progress occurred in certain areas. Although the WGC had used tools such as phase diagrams in the past in order to illustrate progress, Outcome Mapping demonstrated far more complexity as well as peripheral and unanticipated changes. Likewise, it allowed for the WGC to examine outcomes and strategies that reached beyond the end of the evaluation, and to consider strategies and markers of progress moving forward into a phase of program sustainability.
Although the program was highly beneficial for the current project, a number of challenges must be highlighted. Primarily, it required a large time commitment from both partners. Because Outcome Mapping is a participatory process, it is essential for all team members to be present at workshops. Additionally, one person must be assigned to facilitate workshops and to organize and track data, which in itself represents a significant time commitment. Further, because Outcome Mapping is a relatively new method, it can be difficult to secure a facilitator who has an in-depth understanding and experience with the process. Investing time and energy into it also requires a flexible project funder. For the current project, funders did not impose particular evaluation methods, and trusted team members to employ appropriate evaluation tools. This flexibility is essential for the success of Outcome Mapping. Indeed, framing knowledge with its use could be seen as a risky undertaking in the world of academia, where randomized controlled trials and experimental methods are the standard. However, in complex community settings, the technique is highly valuable and arguably necessary for evaluating and understanding community progress and change.
This project represents the first account of Outcome Mapping in a Canadian Indigenous community. Given the need for evaluation methods that align with the non-linear, multi-faceted processes of community change and development, this project makes an important contribution to the evaluation literature. Our project not only demonstrates how Outcome Mapping can accurately and comprehensively capture the change process in community projects; it also highlights how the method can be used to bring together and engage community and university partners in the evaluation process.
Finally, this project details how Outcome Mapping can be effectively adapted by a community-university partnership.
While conventional research and evaluation methods can provide valuable information regarding the effectiveness of a program, CBPR projects require the additional use of innovative methods that can account for the complex nature of community change. For the current project, Outcome Mapping provided a systematic method for understanding peripheral changes that are often overlooked in conventional research and evaluation, but that nonetheless indicate progress toward community changes and long-term impact. Among other findings, data demonstrated positive impact on Elders and students; growing community investment in and support for the MLST program’s cultural components; progressive increases in community ownership of the program; and growth in the community-university partnership throughout the project. The method is a highly valuable tool for CBPR projects. It supports growth in community and academic capacity and relationship building between community and academic partners. Outcome Mapping can enhance research and evaluation quality and contribute to project sustainability by offering a framework for capturing outcomes that are meaningful for community partners.
Baydala, L., Fletcher, F., Tremblay, M., Rabbit, N., Louis, J., Ksay-yin, K., & Sinclair, C. (2016). A community-university approach to substance abuse prevention. Journal of Community Engagement and Scholarship, 9(1), 67–75.
Botvin, G.J., Baker, E., Renick, N.L., Filazzola, A.D., & Botvin, E.M. (1984). A cognitive-behavioral approach to substance abuse prevention. Addictive Behaviors, 9(2), 137–147.
Botvin, G.J., & Griffin, K.W. (2014). Life skills training: Preventing substance misuse by enhancing individual and social competence. New Directions for Youth Development, 2014(141), 57–65.
Canadian Institutes of Health Research (CIHR). 2007. CIHR guidelines for health research involving Aboriginal people. Ottawa, ON: CIHR.
Castleden, H., Morgan, V.S., & Lamb, C. (2012). “I spent the first year drinking tea”: Exploring Canadian university researchers’ perspectives on community-based participatory research involving Indigenous peoples. The Canadian Geographer, 56(2), 160–179.
Cross, T.L., Friesen, B.J., Jivanjee, P., Gowen, L., Bandurraga, A., Matthew, C., & Maher, C. (2011). Defining youth success using culturally appropriate community-based participatory research methods. Best Practices in Mental Health, 7(1), 94–114.
Dart, J., & Davies, R. (2003). A dialogical, story-based evaluation tool: The most significant change technique. American Journal of Evaluation, 24(2), 137-155.
Earl, S., Carden, F., & Smutylo, T. (2001). Outcome Mapping: Building learning and reflection into development programs. Ottawa, ON: International Development Research Centre.
Gauld, S., Smith, S., & Kendall, M.B. (2011). Using participatory action research in community-based rehabilitation for people with acquired brain injury: From service provision to partnership with Aboriginal communities. Disability and Rehabilitation, 33(19-20), 1,901–1,911.
Grekul, J., & Sanderson, K. (2011). ‘I thought people would be mean and shout.’ Introducing the Hobbema Community Cadet Corps: A response to youth gang involvement? Journal of Youth Studies, 14(1), 41–57.
Israel, B.A., Schulz, A.J., Parker, E.A., & Becker, A.B. (1998). Community-based research: A partnership approach to improve public health. Annual Review of Public Health, 19(), 173–202.
Jagosh, J., Macaulay, A.C., Pluye, P., Salsberg, J., Bush, P.L., Henderson, J., Sirett, E., Wong, G., Cargo, M., Herbert, C.P., Seifer, S.D., Green, L.W., & Greenhalgh, T. (2012). Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. Milbank Quarterly, 90(2), 311–346.
Judge, K., & Bauld, L. (2001). Strong theory, flexible methods: Evaluating complex community-based initiatives. Critical Public Health, 11(1), 19–38.
Kelly, T. (2010). Five simple rules for evaluating complex community initiatives. Community Investment 22(1), 19–23.
Kelly, J., Saggers, S., Taylor, K., Pearce, G., Massey, P., Bull, J., Odo, T., Thomas, J. Billycan, R. Judd, J., Reilly, S., & Ahboo, S. 2012). “Makes you proud to be black eh?”: Reflections on meaningful Indigenous research participation. International Journal for Equity in Health, 11(40), 1–8.
Minkler, M., & Wallerstein, N. (Eds.). 2003. Community-based participatory research for health. San Francisco, CA: Jossey-Bass.
National Health Promotion Associates, Inc. (2011a). Life skills training questionnaire: Elementary school version. Retrieved from www.lifeskillstaining.com.
National Health Promotion Associates, Inc. (2011b). Life skills training questionnaire: Middle school version. Retrieved from www.lifeskillstaining.com
Nyangaga, J., Smutylo, T., Romney, D., & Kristjanson, P. (2010). Research that matters: Outcome Mapping for linking knowledge to poverty-reduction actions. Development in Practice, 20(8), 972–984.
Salimi, Y., Shahandeh, K., Malekafzali, H., Loori, N., Kheiltash, A., Jamshidi, E., Frouzan, A. S., & Majdzadeh, R. (2012). Is community-based participatory research (CBPR) useful? A systematic review on papers in a decade. International Journal of Preventive Medicine, 3(6), 386–393.
Smith, R. Mauremootoo, J., & Rassmann, K. (2012). Ten years of Outcome Mapping adaptations and support. Ottawa, ON: Outcome Mapping Learning Community.
Victora, C.G., Habicht, J., & Bryce, J. (2004). Evidence-based public health: Moving beyond randomized trials. American Journal of Public Health, 94(3), 400–405.
About the Authors
Melissa Tremblay is a program evaluator in the Department of Pediatrics at the University of Alberta. Lola Baydala is an associate professor in the Department of Pediatrics at the University of Alberta. Natasha Rabbit is the executive director of the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta. Jennilee Louis is a research assistant for the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta. Kisikaw Ksay-yin is an Elder in the Nehiyaw Kakeskewina Learning Society, Maskwacis, Alberta.
Deborah Thomson, Rebecca Dumlao, and John Howard
College students face a complex world filled with pervasive social problems that require strong knowledge bases, critical thinking abilities, and sustained engagement in civic life. This article details key pedagogical practices for our innovative health puppetry program, in which undergraduate honors students use puppets to share information about healthy eating, diabetes prevention, and active lifestyles with children and their families in community settings. We articulate a notion of “flexible thinking” as the ability to take on and perform new roles within the public/civic arena by seeing complex social problems from multiple perspectives and responding with creative solutions and engaged action. We look to the written reflections of our student puppeteers to share, in their own words, multiple ways their thinking and communication changed as they grew as puppeteers, community partners, and citizen-leaders. We also offer insights about promoting flexible thinking through in-depth service-learning.
In the spring of 2011, 13 undergraduate honors students from a variety of majors were part of the inaugural “Puppet Shows that Make a Difference!” class, a service-learning honors seminar team-taught by two of us with another of us serving as guest lecturer and project advisor. Our goal was to give our students the experience of using health puppetry to speak with children in our community about ongoing childhood overweight and diabetes issues. We spent the first half of the semester training our students as puppeteers using large colorful puppets and scripts purchased from the longstanding educational puppetry organization The Kids on the Block (n.d.). We also taught our students an interdisciplinary course curriculum focused on interpersonal, intercultural, and small group communication, with guest speakers on topics like healthy eating, childhood obesity and diabetes, child development and family relations, and educational principles for children. In the last month of our class, we visited nine different low-cost (or no-cost) after-school programs that partner with our university’s service-learning center. Our students performed the puppet shows to nearly 300 children. Each show was approximately 30 minutes long, with 20 minutes of scripted performance and 10 minutes of interactive dialogue.
But the puppet shows were more than just a set of community-based experiences. We recognized that students engaged in rich, well-designed, service-learning projects learn not only through hands-on experiences in the community; they also learn through sustained self-reflection. In-depth written reflections can help teach students to consider their experiences thoughtfully to “generate, deepen and document” their learning (Ash & Clayton, 2009; Rama & Battistoni, 2001). Throughout our course, we asked our students to write weekly reflections, called articulated learnings (ALs) in response to prompts about key course topics. We also required a final essay that took the form of a longer AL that integrated and highlighted learning from across the semester. The ALs were based on a model for critical reflection (Ash & Clayton, 2009). This model—known as the DEAL model—provided a clear framework for students to organize and understand their own experiences via (D)escription, (E)xamination, and (A)rticulation of (L)earning in the journals so that they would glean new meaning from their community interactions, rather than just to have an experience outside the classroom. This model not only encourages students to think in-depth and critically about topics being explored, but also easily lends itself to scholarly analysis.
Below we will argue that our health puppetry project led students to develop and engage in flexible thinking and communication with members of their community. In our thematic analysis of student journals kept over the course of the semester, we find flexible thinking in a) perspective-taking shifts experienced by students as they enact the roles of puppeteer, teacher, leader, and citizen and b) increasing awareness of students’ sense of civic responsibility and agency in “making a difference” as citizen-leaders. We end our analysis with suggestions for scholars and practitioners to promote flexible thinking through in-depth service-learning courses. But first, we provide a brief literature review and a detailed explanation of our class and how it worked.
Literature Review and Background Information
Today’s college students face an increasingly complex world filled with pervasive social problems that require the knowledge, skills, and informed agency to put learning into action in their communities. The National Task Force on Civic Learning and Democratic Engagement (2012) points out that “full civic literacies cannot be garnered only by studying books; democratic knowledge and capabilities also are honed through hands-on, face-to-face, active engagement in the midst of differing perspectives about how to address common problems that affect the wellbeing of the nation and the world” (p. 3). For students to truly care about problems, they need to experience them outside of what they may read as an example or statistic in a textbook. Or, as Battistoni (2013) states, a broad understanding of civic knowledge (and subsequent action) “includes a deeper knowledge of public issues, including their underlying causes as well as of how different community stakeholders understand issues” (p. 115–116). Students studying a health issue might, for instance, learn about the multiple facets of community and family life that could contribute to health issues as well as to become more sensitive to the cultural traditions surrounding food and health.
To be well-prepared to work in communities, then, students must be able to think about different perspectives on a problem, consider varied courses of action, and determine ways to collaborate effectively to create solutions that serve a larger purpose (Saltmarsh & Hartley, 2012). Indeed, students must be empowered to embrace an ethical civic identity (Knefelkamp, 2008). For example, students could learn that there are diverse professionals who could address complex health issues from varied perspectives and areas of expertise. Students could also be given opportunities to actively experience new ethical civic identities by playing unique roles in the community.
Further, as Minnich (2012) points out, “education and democracy both thrive on inquiry, on experimentation that may enable discovery” (p. 25). Effective civic action requires what we are calling “flexible thinking.” Flexible thinking is necessary to take on and perform new roles within the public/civic arena. It requires the ability to see the “big picture” of a complex social problem, along with the ability to look at that problem from multiple perspectives, responding with creative solutions and engaged action. It requires the type of “fluid intelligence” Cattell (1963) described as necessary for “adaptation to new situations” (p. 3). Fluid intelligence is “the ability to be creative, make leaps of insight, and perceive things in a fresh and novel manner” (Potter, 2013, p. 78). Community-based performance helps students to develop this kind of flexible thinking, as the performer must venture into unfamiliar territories of self and other, both as they take on the role of a character in a puppet show and as they take on the role of citizen-leader within their community. Structured reflection about community experiences can also help students develop flexible thinking as they must look back at what worked (or didn’t) in one situation and then be creative in considering new alternatives to use in the future.
Using Service-learning and Performance
To Address Complex Social Problems
Recognizing the importance of students’ civic learning as well as building their capacity for flexible thinking and action, we created a service-learning course to take on the problem of childhood overweight and diabetes, both of which are severe problems in our community. In North Carolina two thirds of all adults (67.5%) are overweight or obese. The state also ranks 50th in the United States for rates of childhood obesity (Pitt County, 2008). In our county, it is estimated that 40% of elementary-age children and adolescents are overweight or at risk of becoming overweight Pitt County, 2008). Given the pressing social issues facing us, these interactive puppet shows were not just “fun” (although they were that); they were a way to share accurate information on healthy
eating and active lifestyles with children at-risk of developing obesity and eating-related health problems such as diabetes.
These puppet shows were also a way for our students to learn from the children in their community about what kids face when it comes to eating well, using fluid intelligence to gain new perspectives on these health problems. All of our puppet shows contained both a scripted scene and a time to interact, so children in the audience could speak directly with the puppets. Our puppets, for instance, asked the children questions like “What could you eat for a healthy snack?” And our puppeteers were frequently surprised by the children’s answers. In this sense, our project created a unique learning partnership between college student learners and elementary school learners, with puppets in the middle. As Bringle and Hatcher (2002) point out, campus-community partnerships are developed through person-to-person interactions that are dynamic. Moreover, partnerships between campus members, such as our university students, and community members, such as the children, are recognized to be both complex and challenging (Jacoby, 2003), since the partners come to the interactions “from different worlds” (Sandy & Holland, 2006, p. 30). So, as a primary learning goal, we hoped our students would “learn to put themselves back into the mix of humanity…working ‘with’ people (in the community) rather than ‘on’ them” (Boyte, 2009, p. 15), being able to shift their thinking accordingly. We also hoped that our students would see themselves increasingly as citizen-leaders “making a difference” in the very communities where they were performing.
Block-Schulman and Jovanovic (2010) state that “service-learning programs work because they engage students wholly—involving the intellect, the body, and the emotions in a social arena to assert an ethical posture (as active citizens)” (p. 93). Performance lends itself to this holistic level of engagement, because the student-as-performer engages the flesh, the memory, the senses, the emotions, the voice, and even the human spirit in the act of performing. Pineau (2002) describes performance as “a medium for learning,” one that “requires the rigorous, systematic, exploration-through-enactment of real and imagined experience in which learning occurs through sensory awareness and kinesthetic engagement” (p. 50). This type of performative learning through embodiment is closely aligned with Minnick’s (2003) notion of deeply democratic thinking as “play” in which students get “caught up in imaginative moments, not tied down to or locked within what he or she already knew or what logically followed” (p. 24). We found that this form of “play” as a learning strategy was quite effective, as we discuss below.
Our Course: Puppet Shows That Make A Difference!
Our “troupe” included 13 undergraduate honors students who were majoring in disciplines as diverse as chemistry, criminal justice, communication, biology, accounting and nursing, among others (see Figure 1). They were primarily sophomores and juniors, and they were fairly evenly split by gender. Almost half of the class planned to go into a health profession, and most of these intended to attend medical school after college. Only 3 of our 13 claimed to have any performance experience; only one had ever worked with puppets. We were also lucky to have three teaching assistants, graduate students in our department’s master’s program in health communication. We had trained these students as puppeteers during the previous semester, both for our own process of learning to work with our new puppets and so that they would, in turn, be able to help our undergraduate students with the difficult skills of puppeteering such as eye focus, puppet gesture, and lip sync.
We had two faculty members team teaching this course. One of us has extensive performance experience, and the other has extensive service-learning experience. Both of us are also graduates of our university’s Engagement and Outreach Scholar’s Academy, an intensive program to learn about community-engaged scholarship with institutional and financial support to develop new projects. Because both of us study and teach communication, we invited a number of colleagues from areas such as nutrition, pediatrics, and childhood development to present on these topics to give our students interdisciplinary knowledge. These areas of understanding were key to our project, not just because we wanted our students to see the “big picture” of childhood overweight and diabetes, but also because they would need to be able to answer children’s questions on these topics in their puppet shows.
To provide a strong sense of the challenge our students faced in learning to be puppeteers, we must first describe these puppets. These were not hand puppets. They were huge, kid-sized puppets with arm rods, requiring both physical strength and substantial performance skills by puppeteers (see Figure 2). This style of puppetry is based on the ancient Japanese Bunraku style in which the performer stands behind the puppet dressed in black, as a kind of shadow to the puppet. The performer also wears a black mesh hood in order to see the audience, allowing for better interaction. Although the shows were fully scripted (and copyright demanded no deviation from these scripts), each performed puppet script was followed by an interactive question and answer session between the kids in the audience and the puppets. Thus the need for interdisciplinary learning by the students and another need for them to quickly flex their thinking from being a puppet using a script to being a puppet that could accurately and responsively answer each child’s questions.
We were able to purchase the puppets and scripts with grant funding from our university. As part of this, we were also able to bring in two trainers from the head office of The Kids on the Block, who conducted a one-day puppeteering workshop with our students. Consequently, the students were very well prepared as puppeteers and had many chances to practice and develop their puppet skills both in and out of class before actually doing the community puppet shows.
Throughout our course, we asked our students to write weekly reflections in response to prompts about key course topics. Both the prompts and the students’ writing followed the DEAL model for critical reflection (Ash & Clayton, 2009; Ash, Clayton, & Moses, 2009; Ash, Clayton, & Atkinson, 2005). The DEAL model encourages students to describe and then explain their learning using the following sentence stems: “I learned that…; I learned this when…; This learning matters because…; In light of this learning….” Our discussion prompts covered topics within three categories: personal growth, civic learning, and academic enhancement. Topics included community partnerships, leadership, performing with children, teamwork, exploring cultures, and making a difference, among others. The ALs also traced students’ experiences as they learned about the academic foundations of their practices and as they learned from their interactions with community members as engaged citizen-performers. The final AL essay called for an integration of the multiple dimensions of their experiences.
Research Questions and Method
Our investigation of service-learning and flexible thinking considers the intersections among role playing/perspective taking, civic engagement, and flexible thinking in students’ writings. That is, we looked carefully at the essays to see whether our primary learning goals for students were being met. To this end, we posed the following research questions:
RQ1: How did playing new roles (puppeteer, teacher, etc.) contribute to students’ abilities to take the perspective(s) of others (and enact flexible thinking)?
RQ2: What did the service-learning experiences mean to students, relative to their citizenry and perceived ability to facilitate change?
Students were asked to participate in the study, but had the option not to include their responses in the research. They were told that their reflections would be used to analyze the content and critical thinking. So, while we recognized that the prompts would influence the content of their responses, we also expected that the structure would give our students some grounding in what to write about (as opposed to open-ended journaling where students may choose to simply recall experiences without delving deeper). The DEAL model encourages students to think deeply about the content of their learning, as well as their related thoughts, feelings, and developing skills. As researchers, our hope was to gain some understanding of the meanings and significance our students would make of their service-learning and related classroom experiences.
Consequently, our primary method of analysis is focused on generating understanding as it emerges from students’ own voices (i.e., grounded theory).
We chose the “constant comparative method” (Lindlof, 1995, p. 222) so students’ voices could be heard without imposing a rigid theoretical frame. This approach enables researchers to develop common and overarching themes, frames, and principles through a systematic but flexible form of inquiry. Two important aspects of this method are that “it specifies the means by which theory grounded in the relationships among data emerges through the management of coding (hence, grounded theory), and it shows explicitly how to code and conceptualize as field data keep flowing in” (Lindlof, 1995, pp. 222–223). In this study, thematic elements were identified in the student reflections and compared across authors. Below we share the themes that emerged from our analysis and the connections among them.
Learning New Roles: Puppeteer, Community
Minnick (1985) lists “play” as a characteristic of the kind of thinking that we should be teaching in the college classroom, noting that “Such play is not always fun: It can take us to scary places. But it also unclenches, releases” (p. 24). Many of our students began the semester feeling “clenched” about what lay ahead. As honors students, they were accustomed to performing well on tests and receiving high marks on essays. But, for most, performing was an entirely new venture, one squarely outside of the proverbial “comfort zone.”
Interestingly, in their first week of journaling about the class, these 13 students collectively used the word “comfort” 21 times, all basically describing how uncomfortable they were about performing. Representative of this was Mike, a pre-med student, who described the discomfort he experienced when students were asked to sing their names as part of a warm-up activity. Mike wrote that the activity “forced students like me out of their zones of comfort.”
I was slightly taken aback on the first day of this class when asked to sing in front of my classmates for several short periods as a part of warm-up exercises. I do not often sing in front of people, and much less often in front of those I barely know.
Ron, too, described the anxiety he felt upon reading scripts aloud for the first time, writing, “I sink into my chair, excessively, unnecessarily fearful of being selected to assume the vocal identity of a character about whom I know slightly more than nothing at all.” This process of taking on a character would eventually prove liberating for our students, freeing them up for other possibilities of self. But this “liberation” was only after hours of practice and increasing self-confidence as performers and educators.
Moving Between Self and Other
Richard Schechner (1985) theorizes the restored behavior of taking on a theatrical role as “me behaving as if I am someone else” (p. 37). Schechner describes the performer’s stance as she inhabits the character as a dance between the “not me” and the “not not me,” explaining:
While performing, a performer experiences his own self not directly but through the medium of experiencing the others. While performing, he no longer has a “me” but has a “not not me,” and this double negative relationship also shows how restored behavior is simultaneously private and social. A person performing recovers his own self only by going out of himself and meeting the others—by entering a social field (p. 112).
For our students, this interplay between self and other was not an easy process. The challenges of performance went beyond learning the physical mechanics of puppetry such as lip sync, eye focus, and gesture. It was entering Schechner’s “social field” in which the self would be doubly displaced, first as our students took on a puppet character and then a second time as they came into dialogue with their young audiences, who offered them a child’s eye view of the world.
Interestingly, it was the mental image of this future performance partner, the imagined children, which brought Jada some comfort as she navigated the anxieties of performing before her peers in these early days of our class:
During my reading I focused so much on the part that everyone else in the room seemed to disappear. While I continued to talk as Christine I began to imagine myself in a room full of young children. … I believe that I was able to get into the reading as I began to imagine children because I love working with children. I know that they are not judgmental and that they love it when adults act crazy.
Ron also navigated performance anxieties with an eagerness for learning this new role when he wrote in his first reflection: “I am eager to start practicing with the puppets, and molding my own personality to theirs.” Ron likely intended to communicate that he would start the process of shaping his character’s personality (rather than his own). But the mistake is revealing of what would happen for many of these students, as the process of taking on a character freed them up for other roles and for seeing the world from other points of view, to think more flexibly.
Teaching and Leading
In Hersey and Blanchard’s (1977) situational leadership model, leaders can delegate to their followers (in other words, let their followers lead), when the follower’s readiness level is high. In our case, this happened when the students were ready to go off script in the interactive portion of their shows. Having embodied the knowledge about healthy eating and diabetes through rehearsal and performance, they were ready to teach, and they were ready to lead others. After eight weeks, our students stepped out into their community to perform. As they did this, they took on a new role, that of teacher. Arnold, another pre-med student, spoke to this challenge and the learning opportunities it presented when he wrote:
Questions like “Can diabetes kill you?” were hard to answer when dealing with children, but showed that they were relating to the topic and trying to understand as much about it as possible. Dr. Collier came and instructed us on the medical aspects of diabetes in order to improve our knowledge so we could answer questions with factual information. This was knowledge that we could apply immediately to the performances that were in our immediate future, something that is a rarity in college courses.
Arnold imagined this learning continuing into his future role as physician, stating “I will also continue to learn as much as possible about diabetes, and continue to educate my community about the disease in some way.”
As our students became more comfortable with their roles as puppeteers, they shifted into new roles as teachers and leaders. This was clearly demonstrated during the one show when students had to think on their feet because they received no questions from the audience. We had become accustomed to a room full of little hands popping up when the puppets took questions. But in this location the children had been highly disciplined to sit still and say nothing, and this discipline continued into the interactive portion of our show. After a prolonged silence ended the show, our students came out and took their bows. They put their puppets down, took their black hoods and gloves off, and began to interact with the children as themselves. Maggie started off by saying to the group, “You really don’t have any questions about healthy eating or diabetes?” Then a question came, and then another, turning into our longest audience dialogue and the only one that took place between performer and audience with no puppet in between.
These sorts of dialogic experiences with children in our community challenged our students to see the beyond their own frames of reference. Melanie writes about this learning when she first had to answer the question “What is diabetes?” posed by a young person in her audience. Melanie stated that her “first instinct […] was to begin by explaining the function of glucose in the cells and its conversion to energy in order to be used for carrying out daily activities.” She quickly realized that a highly technical explanation would likely not be met with wide understanding from the K-2 crowd. Melanie continues:
As I gained performance competence, I started to think more about who my audience was, and what kind of answers they would understand. This learning matters because it made me realize who I was answering, and take into account what their level of understanding about biology would be as an elementary school student. In light of this learning, I began to think like a kid when the question and answer portion of the show began so that I would be able to answer questions in a way the child asking the question would comprehend.
Melanie not only had to think like her character to answer the question, she had to think like her audience, displaying the kind of entry into Schechner’s (1985) “social field” that enables a performer to escape the perspective of the self, however briefly.
Maggie also spoke about this when she was surprised by the willingness of the children in her audience to share their own stories, writing:
Once we began talking about diabetes, I realized through the children’s numerous comments that many of them had firsthand experience with diabetes through family members, but also that they did not know much about the condition.
Our students were now not just learning about the problem of diabetes in our area from the perspective of a physician or a professor, they were learning from the personal stories of children with a diabetic grandmother who had died from the disease or a diabetic uncle who took shots of insulin every day.
Making a Difference
We had titled our course “Puppet Shows that Make a Difference!” hoping that our planned puppet shows would do just that in our community as our students shared information about healthy eating and diabetes with youth at risk of diet-related health problems. Although we did not assess the impact that these puppet shows had on our audiences, what we found was that the perception of making an impact was deeply meaningful to our students. Caitlin’s final essay exemplified this:
I felt at the beginning of this class that it would not be possible for me to change the world, the nation, or even a city on my own. I seemed to block out the “…That Make a Difference” part of the class. I did not see how it was possible for a freshman in college to do anything that would have enough impact to be considered “making a difference”. I realized that if I could help one child, this class, and ultimately I, would be a success. Yet, the complete breakthrough did not occur during [the lectures]. […]Finally, on the day in which we performed for the children, a complete breakthrough occurred. I saw the children and how receptive they were. I saw their excitement, and the excitement of the adults around me. It finally seemed real. I had made a difference.
Of course, we should be skeptical that these puppet shows could affect the kind of dramatic societal changes that Caitlin articulates. But we can see in her optimism and investment in community that spark of what it feels like to be an agent of change.
Astin and Astin’s (1996) Social Change Model of Leadership Development posits that there are three levels of change for engaged citizenship: individual, group, and community. At the first level, individuals gain a consciousness of self and a commitment to the value of civic engagement. Next, at the group level, individuals come together for collaboration and common purpose. This sets the stage for the final level: community engagement. Christine Cress (2011) also writes about how service-learning can engage students in these three stages, noting that “students are able to extend their intellectual capacities to include empathy and problem solving that will have a real community impact” (p. 78). She continues: “in this spirit, service-learning offers students the opportunity to become critically conscious citizens with the knowledge and skills for creating more equitable democratic communities” (p. 78). Our student Caitlin echoes this notion as she ties the work of performance to the work of democracy and citizenship, writing:
A belief in the ability to make a difference is even more important to the citizens of this and any other democratic nation. Democracy thrives on the belief of its citizens that they can make a difference. This is what drives people to the voting booths, to rallies, and to signing or forming petitions. If the belief in a single person’s ability to change the world was lost, I believe that democracy itself would be lost as well.
Caitlin’s conception of perceived civic agency as fundamental to a thriving democracy offers an insight into the kind of flexible thinking that our service-learning puppeteers experienced as they went out to perform “making a difference.”
Part of perceived civic agency, for our students, was embracing the idea that they were part of multiple communities, including the sometimes overlooked ones surrounding their university, and that they needed to be an active participant, a citizen, in those communities. Justin, in his final paper, thought about this as he discussed the importance of “teamwork,” writing:
Last, but definitely not least, the most important teamwork was with our class and our partners in the community, both the adults and the children. This was the most delicate and important version of teamwork we learned about this semester. I learned that we, as East Carolina University students, as current Pitt County residents, as citizens of the United States of America, have a civic responsibility to improve and take part in each and every community we are involved in. Every community from my immediate family to the enormous student body we have at East Carolina flourishes only when we work together to try to improve it.
Justin here identifies several “communities” that he is a part of—from his family, to his university, to his current county of residence. Justin’s sentiments speak to the power of service-learning to help invest students with what Justin aptly terms the “civic responsibility to improve and take part in” the communities of which we are a part. Another way to think about this is that Justin was moving from the “not me” to the “not not me” on his return from his journey through the “social field.” The larger community of Pitt County was now not-not him. His statements also demonstrate that he was able to think about community from multiple perspectives.
By the end of the semester, many of our students were reconceptualizing themselves as community leaders. Arnold wrote about this new citizen-leader role when he stated:
We are becoming leaders by consciously choosing to symbolically communicate the importance of eating healthily to children through puppetry. … Leaders are not perfect people. They can be anyone. A leader is simply someone with the right skills who, when the opportunity arises, has the courage to step forward and say, “I can make a difference, starting today.”
Melanie had this same realization, noting the importance of each individual’s contributions as part of the larger collective effort that is community leadership. She wrote:
I had a few epiphanies about making differences in the world around us. I realized that you already have what it takes to make the world a better place. Making a difference to the world may seem like an enormous task, but it is in fact the collective effort of everyone to make small contributions with a lot of heart. The size of the contribution is not what matters most. The key here is to have the heart to do it.
In both Arnold’s and Melanie’s sentiments, we see students who are conceiving of themselves as leaders as they rethink what citizen-leadership is, something that does not necessarily come only from “perfect people,” in Arnold’s conception, or from “enormous tasks,” in Melanie’s. It can come from those who choose to step forward, even in small acts.
Building Capacity for Civic Engagement
Our inaugural puppetry course demonstrates what can happen when students are given the opportunity to focus in-depth on a community-based issue/problem. That is, our students were asked to consider multiple human perspectives, to draw on insights from multiple disciplines as well as across specialties within a single discipline (i.e. small group communication, leadership communication, and interpersonal communication) and to do this in a single semester. This type of interdisciplinary and cross-disciplinary learning is, unfortunately, a rarity in most college classes that are instead structured to focus on subject matter rather than giving students a “breadth of understanding” about an issue. Admittedly, there are many structural and functional obstacles in the way. As Fitzgerald, Burack, and Seifer (2010) note:
Pressures to build strong university-community collaborations pose difficult problems for the academy because they demand interdisciplinary cooperation, rejection of provincial disciplinary turfism, changes in the faculty reward system, a re-focusing of unit and institution missions and the breaking down of firmly established and isolated silos (p. x).
Still, we must overcome such challenges, because providing students with a greater breadth of understanding is essential to enabling them to think deeply about the complex issues we face in today’s world. Being a contemporary problem-solver frequently requires more than just disciplinary knowledge; civic action necessitates higher ordered flexible thinking that applies, evaluates, and integrates knowledge. Indeed, focusing on a single community-based issue rather than on a subject specialty area is one way to help students capture the complexity of a contemporary problem (in our case, eating-related health problems for children). This offers a new possibility for what others have identified as the important process of “doing democracy” and “fostering civic action” (The National Task Force, 2012).
Our course, heard through the student voices in their reflections, reiterated to us how important “big” (pressing, social) issues are in our classes and how students can be empowered to address some aspect of those “big” issues. Just as there may be many partners necessary to solve complex problems, there could be possibilities for students representing many disciplines to work together. Working with different partners and being able to think about an issue from different perspectives is important far beyond any single issue. In a diverse global society, everyone must interact regularly with people representing different experiences and assumptions. So, the abilities to look at a problem from multiple perspectives will be essential to college graduates functioning as active citizens in very dynamic and ever-changing democracies.
Our project had several challenges and limitations. One was simply the labor-intensive nature of this type of performance work. Many hours were spent outside of the classroom conducting
rehearsals and scheduling performances. There was a very steep learning curve here! For example, one of the lessons we learned was that our child audience members were more attentive audience members when they had their teachers sitting nearby. This was a lesson we learned the hard way at an after-school program where the teachers left and we had to monitor behavior while trying to do our jobs as performers.
Another limitation had to do with the depth of partnering that took place in our project. While individual college courses can be subject to the limitations of a semester-long commitment to a community program (See Stoecker & Tryon, 2009), classes that are issue-focused like ours can be part of a longer-term programmatic effort or what Heath and Frey (2004) call “community collaboration.” These authors rightly note that “in most communities today, it is a necessity for groups, organizations, and institutions to work together collaboratively to confront complex issues” (p. 189). In subsequent puppetry projects, we have worked to partner more deeply with one agency, and to write our own scripts based on interviews and focus groups with members of our community on the topic of healthy eating. Similarly, scholars from other disciplines could partner with community members to develop interactive puppet shows to address other community issues. We see nutrition students developing interactive puppet shows to teach children about healthy eating and to learn about children’s eating habits. We see dentistry students doing the same with dental health or sport science students doing the same with exercise (two topics our continuing puppetry students have been working on).
Although we chose not to conduct pre- and post-tests to attempt to quantify our student’s growth, their reflections showed us what they learned while interacting with the children. These opportunities to learn-while-doing were invaluable and went far beyond the cognitive content emphasized in many, even most, college courses. They offered positive emotion, excitement, and “in the moment” kinds of thoughtful responding. We saw that allowing our students the opportunities to “play” both in and out of the classroom helped them to get more excited about not just the “thinking” part of their work, but also about the community-based “doing.” We hope that future scholars and practitioners will develop new ways to capture the element of this rich “play” that can invigorate and motivate both learning and doing. Staying motivated and having fun is not just nice, it is fundamental to opening up new possibilities and to keeping us enthusiastic enough to overcome problems and dilemmas we face along the way.
This service-learning course has also shown us the value of in-depth written reflections. We agree with Ash and Clayton (2009) that critical thinking through reflection requires careful consideration and planning to specify detailed learning outcomes and to design reflections that help students achieve those specific outcomes. We also recognize the importance of aligning our student outcomes with assessing the overall community impact—something that we admittedly did not address in the single semester allotted for this course. We did not gather formal feedback from the children we performed for, but we did learn from the places where children laughed or were especially engaged and from questions and comments they gave. In the future, seeking more formal feedback can help us to improve the quality of our performances and to understand what our audiences are getting out of them. Still, we realize that teaching toward community impact and designing related assessments will also require broad, interdisciplinary efforts over time. Researchers need to work with practitioners to determine what will work to demonstrate
impact and to show the long-term sustainable commitment to making positive changes. Students can have a part to play in identifying what really matters. Working as co-educators (Zlotkoswki, Long, & Williams, 2006) and solo educators for others (like the children in our audiences), service-learning students offer real potential for looking at long-standing situations with fresh eyes… and that may be just where innovative responses to “what’s never been done” can begin to take root.
While our study demonstrated a single semester’s course projects, we are well aware that the community partnerships started through these classes require our long-term commitment and efforts, even as we train new puppeteers and work with other community-based programs focused on our chosen issue. As Enos and Morton (2003) aptly note, ideal service-learning is not just about the single transactions between partners, but rather about “the continuing possibility (that partners) will be transformed in large and small ways” (p. 20). We wholeheartedly agree. Teaching for flexible thinking has real possibilities to change students, faculty, universities, and communities as we, together, address the issues that demand our attention. Our students, working collaboratively with others on and off campus, began to see how their own efforts are part of the larger whole, a part of a complex problem’s long-term solution. They saw how they “can make a difference” as players in community collaboration. This is the exciting civic challenge we face; it is also what is most needed for a bright future for our communities, for us all.
Ash, S., & Clayton, P. (2009). Generating, deepening, and documenting learning: The power of critical reflection in applied learning. Journal of Applied Learning in Higher Education, 1(1), 25–28.
Ash, S., Clayton, P., & Atkinson, M. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service-learning, 11(2), 49–59.
Ash, S., Clayton, P., & Moses, M. (2009). Learning through critical reflection: A tutorial for service-learning students. Self published.
Astin, A.W., & Astin, H.S. (1996). A social change model of leadership development: Guidebook, Version III. Los Angeles, CA: Higher Education Research Institute.
Battistoni, R. (2013). Civic learning through service-learning: Conceptual frameworks and research. In P. Clayton, R. Bringle, and J. Hatcher (Eds.), Research on Service-learning Conceptual Frameworks and Assessments, Vol. 2A: Students and Faculty, pp. 111–132. Sterling, VA: Stylus.
Bloch-Schulman, S., & Javonovic, S. (2010). Who’s afraid of politics: On the need to teach political engagement. Journal of Higher Education, Outreach, and Engagement, 14(1), 83–100.
Boyte, H. (2009). Repairing the breech: Cultural organizing and the politics of knowledge. Partnerships: A Journal of Service-learning and Civic Engagement, 1(1), 1–21.
Brammer, L., Dumlao, R., Falk, A., Hollander, E., Knutson, E., Poehnert, J., Politano, A., & Werner, V. (2012). Core competencies in civic engagement. Journal of Service-Learning in Higher Education. University of Louisiana System. Retrieved from https://www.merrimack.edu/live/files/160-core-competencies-in-civic-engagement.
Bringle, R., & Hatcher, J. (2002). Campus-community partnerships: The terms of engagement. Journal of Social Issues, 58(3), 503–516. doi:10.1111/1540-4560.00273.
Cattell, R.B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54(1), 1–22.
Pitt County Pediatric Dietitians and Nutrition Educators, (2008). Achieving a healthy weight in children. North Carolina Healthy Weight Initiative.
Cress, C. (2011). Consensus, collaboration and community: Mutually exclusive ideas? In C. M. Cress, D.M. Donahue and Associates (Eds.) Democratic dilemmas of teaching service-learning: Curricular strategies for success, pp. 73–81. Sterling, VA: Stylus.
Enos, S., & Morton, K. (2003). Developing a theory and practice of campus-community partnerships. In B. Jacoby and Associates (Ed.). Building partnerships for service-learning (pp. 20–41). San Francisco, CA: Jossey-Bass.
Fitzgerald, H., Burack, C., & Seifer, S. (2010). Handbook of engaged scholarship: Contemporary landscapes, future directions, Vol. 2: Community-campus artnerships. East Lansing, MI: Michigan State University Press.
Heath, R., & Frey, L. (2004). Ideal collaboration: A conceptual framework of community collaboration. In P. Kalbfleisch (Ed.), Communication yearbook 28 (pp. 189–231). Mahwah, NJ: Erlbaum.
Hersey, P., & Blanchard, K.H. (1977). Management of organizational behavior, (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.
Jacoby, B., and Associates (Ed.). (2003). Building partnerships for service-learning. San Francisco, CA: Jossey-Bass.
The kids on the Block Puppets. (n.d.). Available at: http://kotb.com/.
Knefelkamp, L.L. (2008). Civic identity: Locating self in community. Diversity and Democracy, 11(2), 1–2.
Lindlof, T.R. (1995). Qualitative communication research methods. Thousand Oaks, CA: Sage.
Minnich, E. (2012). Educating democratically: An interview. Higher Education Exchange, (Kettering Foundation), pp. 24–34. Available at https://www.kettering.org/wp-content/uploads/4_HEX2012_Minnich.pdf.
Minnich, E. (2003). Teaching thinking: Moral and political considerations. Change, 35(5), 18–24.
Pineau, E.L. (2002). Critical performative pedagogy: Fleshing out the politics of liberatory education. In N. Stucky and C. Wimmer (Eds), Teaching performance studies, pp. 41-54. Carbondale, IL: Southern Illinois University Press.
Potter, W.J. (2013). Media literacy, 6th edition. Los Angeles: Sage.
Rama, D., & Battisoni, R. (2001). Service-learning: Using structured reflection to enhance learning from service. Available at http://www.compact.org/disciplines/reflection/index.html.
Saltmarsh, J., & Hartley, M. (2012). “To serve a larger purpose”: Engagement for democracy and the transformation of higher education. Philadelphia, PA: Temple University Press.
Sandy, M., & Holland, B.A. (2006). Different worlds and common ground: Community partner perspectives on campus-community partnerships. Michigan Journal of Community Service-learning, 13(1), 30–43.
Schechner, R. (1985). Between theatre and anthropology. Philadelphia, PA: University of Pennsylvania Press.
Stoecker, R., & Tryon, E. (Eds.). (2009). The unheard voices: Community organizations and service-learning. Philadelphia, PA: Temple University Press.
The National Task Force on Civic Learning and Democratic Engagement. (2012). A crucible moment: College learning and democracy’s future. Washington, DC: Association of American Colleges and Universities.
Zlotkowski, E., Longo, N.V., & Williams, J.R. (2006). Students as colleagues: Expanding the circle of service-learning leadership. Providence, RI: Campus Compact.
This research project was approved by the
University and Medical Center’s Institutional Review Board as research project #11-0147. Following our IRB protocol, we sought written informed consent to access the student journals at the start of the semester. A graduate student collected the consent forms, and they were kept by one of our administrative staff members until after grades were posted at the end of the semester, so as not to create pressure on
students to consent. We have changed the names of our
students to preserve anonymity.
About the Authors
Deborah Thomson is an associate professor and John Howard and Rebecca Dumlao are professors, all in the School of Communication at East Carolina University.
Diana M. Ruggiero
This essay presents a brief ethnography of a small Latino community in Tennessee and their interaction with local volunteers following a disastrous flood that occurred in July 2014. The ethnography, in this case in the form of a screenplay, depicts the overall intercultural sensitivity of the volunteers, the affected, and the interpreters. In the process, this essay also considers such creative analytic practice (CAP) ethnographies may help students involved in Spanish and community service-learning courses as well as communities bridge the “self”/“other” gap that so often distances Latino immigrants and locals.
It is well known that migrant populations
contend with numerous challenges in their adopted society, from secondary citizenship and discrimination to cultural differences. This exclusion is exacerbated, if not caused, by language barriers. Indeed, as scholars in the social sciences have long noted (e.g., Hill, 1995, 2008), it is often language that demarcates the difference between “us” and “other.” Unfortunately, this polarizing form of thinking can be extremely detrimental to the progress of both marginalized and dominant populations within a city. When we fail to realize that we do indeed have new residents who are monolingual in Spanish and then ignore the needs that this population has, we are not only doing a disservice to them but to the population in general in that we reify and maintain the very structural differences that keep Latinos in an undesirable and marginal position in relation to the dominant society (see, for example, Gomberg-Muñoz, 2011).
As a language professor with community
service-learning courses, I cannot help but attend to the needs of Latino immigrants and connect students with this reality. While grammar instruction and cultural activities such as dancing salsa and eating at a Mexican restaurant are valid forms of teaching and learning Spanish, learning a foreign language also entails applying the language in the context of daily life. For me, this means visiting a local community and using the language as much as possible in a positive and beneficial manner. Though situating language use within a lived context can be achieved through study abroad, an immersive language experience can likewise often be accomplished within one’s own local community. The benefits in this case, however, extend well beyond the individual student and the development of language abilities.
This essay chronicles an actual event that took place during the week of July 1, 2014 in Memphis, Tennessee, a city of more than 650,000 residents, following flooding in June. Demographically, the city is majority African American (63.3%) and Caucasian (29.4%), with a growing Latino population (6.5%) for which there are few resources in this heavily monolingual city (http://quickfacts.census.gov/qfd/states/47/4748000.html).
The events deal with the aftermath of a period of unseasonably heavy rainfall during the month of June. There was major flooding and aid was needed throughout the city. Many agencies, such as the Red Cross, were there to aid those who needed it, but as the Red Cross is staffed locally, they reflected the demographic majority of the city; there were very few, if any, Spanish speakers available. This created a huge need for third party interpreters who could speak both English and Spanish and help the affected families receive aid. This third party was vital to the success of both parties by being a bridge across the language barrier.
Language differences provided enough of a barrier for the disaster relief team that the process to receive aid was stalled to the point of barely functioning. The need for bilingual volunteers was vital, but the availability and turnout was abysmally low. Agencies that exist to bridge the gap between the two communities were working overtime to get the word out and to find those willing to volunteer their services. Emails were sent and social media was alerted, but despite this there were very few people willing to help. Some of the responses were that the holiday was approaching and there was no time; others were that they had no confidence in their Spanish speaking ability. Both responses, while seemingly reasonable, were in fact forms of further exclusion of the communities. Such responses are understandable given that being a formal interpreter takes several years of training; however, for this cause, all that was required of the volunteers was assistance with a form that was already bilingual (Spanish-English). Issues that arise commonly in this situation are the ability to read and write on the part of the Spanish speaking residents.
During the week of July 1, 2014, I was able to help with interpreting and also to pass out cleaning kits for homes damaged by the flood. From observations and personal experience during the event, the jobs were very simple and straightforward; they were about helping to ease the fear for those affected and to provide help through the aid process. This required at minimum an intermediate command of the language. For those who passed out the cleaning kits, a beginning Spanish level or less was required. The questions were the same, and with the correct teacher anyone could successfully parrot the sentences. That is to say, anyone with the desire to help would have not only been welcomed, but much needed and of great service. Those who showed up found a lively community that, despite the devastation, continued to remain optimistic. The community was an open place, and they continued to do the best they could while supporting one another. The children were running around and playing and neighbors were sharing what little they had. There was some worry that permeated the air, some unease and fear…worries about where to go, what would come next for them, and how they would be able to pick up and carry on. There was a tangible tension in the air, mostly in the form of questions such as: How are the damages going to be paid for? What aid will the management of the housing community affected by the flood give? But mostly the community was grateful for the help and the assurance that things would get better.
Though the language skills required to assist the Red Cross in this particular case were indeed minimal, it was perhaps the relative cultural awareness and sensitivity of the interpreters that made more of a difference in serving the affected families as shown below. It is perhaps here, at the intersection of community service-learning and language instruction, that language instructors such as myself can make a lasting contribution toward the betterment of society.
Indeed, the service-learning approach to the acquisition of a foreign language is an approach that I would have previously benefited from before the incident of the week of July 1, 2014. Prior experience with such a disaster relief event within the community would have quelled the butterflies in the pit of my stomach. As it so happened, I had not taken any service-learning courses as a student. It becomes very difficult to ease another person’s fears as your fears and misgivings run rampant. There was not enough preparation in the world for the emotional impact of the job that I was to perform. As previously noted, the translation and transcription of information to the form was not in and of itself difficult. The hardest part was occupying a neutral territory, reminiscent of Homi Bhabha’s (1994) “third space,” as a Latina and naturalized U.S. citizen with first-hand knowledge of the trepidation and uncertainty that comes with being an immigrant and an immigrant in need of assistance amid crisis. It was unsettling knowing that, at the end of the day, I could return to my cozy home when the homes of those I was helping, those I identified with in many ways more than with my fellow neighbors, were destroyed by the flood.
Following this experience, I realized that I needed a productive way to help me process what I had encountered and experienced. I also realized that, if this were the case for me as a Latina, than certainly my own students could and would also benefit from similarly creatively processing their service-learning experiences. Rather than journal, I opted to create a creative analytic practice (CAP) ethnography to represent what I had observed and encountered during a total of 12 hours worth of volunteer work and observation. My experiences were additionally informed by in-depth interviews with those affected by the flood and with the director of Latino Memphis. The brief screenplay presented below exemplifies the dialogs and interactions that occurred during volunteer hours.
CAP Ethnography: The Screenplay
The term “creative analytic practice ethnographies” was coined by Laurel Richardson to refer to a host of interpretive processes informing research and representation in the social sciences and humanities following the postmodern turn (Richardson & St. Pierre, 2005). Though hardly new, the various forms of creative ethnographic representations now practiced across different disciplines and fields share a fundamental concern with the nature of social reality, the subjective positioning of the researcher, and the relationship between the researcher and those being researched. What CAP ethnography offers is a way of approaching and representing social reality that is perhaps most reflective of the way reality is actually experienced in everyday life; as heterogeneous, contested, and contradictory.
I was drawn to CAP ethnography, and specifically screenplay writing (as a form of CAP ethnography), in part as a result of my experience with the service-learning course, but also because of its
potential to draw audiences into the interpretive process. As Nathaniel Kohn (2000) notes in his observations about the collaborative and dialogic nature of screenplay writing, the format of the screenplay encourages academics to treat their writings as an open dialogue. The screenplay requires openness to critique and frankness that traditional academic writing does not often employ (aside from the peer review process). It also invites the audience to share the sights, feelings, and sounds of the characters that it portrays, as well as in the final interpretation. By helping the reader contextualize everyday activities and the emotion behind them, the use of the screenplay as a creative analytic practice allows for the evocation of the human element within research. It also allows for a potential transformative experience for both the writer and the audience in that it necessitates active engagement on both parts in elucidating meaning from the text. This is especially important when considering the politics surrounding the issue of immigration and Latinos in the United States.
The screenplay excerpt (see Appendix A), then, is written with the intention of inviting audiences to reflect on their encounters with Latino immigrants as well as with their experience of natural disasters, loss, and need. They are called to think past their stereotypes, preconceptions, and beliefs concerning Latinos and the issue of immigration and to empathize with the struggle and humanity of the people affected by the flood (both victims and aid workers).
Regarding the choice of presenting the screenplay in both Spanish and English, Richardson (2005) explains that “language is a constitutive force, creating a particular view of reality” (p. 960) and that it is tied to the idea that the ethnographer cannot be separated from who he/she is; thus the play was originally written in Spanish (my first language).
The screenplay excerpt, though brief, depicts a telling exchange between a Red Cross volunteer, interpretr, and Latino family, and captures the greater function of intepreter as cultural interpreter. The Red Cross volunteer in this case, though humane in his/her treatment of the flood victims, necessarily acts within strict boundaries and guidelines dictated by the task assigned—in this case, processing claim forms and handing out cleaning kits. Mr. Lopez, in dire need of assistance extending far beyond what the Red Cross could offer, is understandably nervous and wary of approaching and divulging personal information to the Red Cross. Indeed, though the status of Mr. Lopez is left unspoken, it is presumed on the part of the Red Cross volunteer that he is undocumented. The reasons for Mr. Lopez’ reticence, however, may be a result of numerous factors, an element of the screenplay left open to interpretation and debate. The end result, however, is that there is a deep mistrust of authority, despite the fact that an organization like the Red Cross transcends national boundaries (as the volunteer worker alludes to in his/her comment regarding his/her unconcern with the place of origin of Mr. Lopez’s document). Into this scene where much is left unspoken the interperter enters.
Though speaking little in the script itself, the interpreter plays a crucial role in mediating and bridging the divide between the two distinct social realities represented here by the Red Cross volunteer and Mr. Lopez. Though there to help, the Red Cross volunteer inadvertently alienates Mr. Lopez in two interrelated ways: by failing to acknowledge and respond to his particular story and concern for his daughter as well as by denying him agency in assigning him to a preconceived and generic category (or categories) distinct and opposite from that occupied by the Red Cross volunteer (i.e., victim, Latino, foreigner/immigrant, undocumented/illegal, and “other”). Mr. Lopez’s alienation is further exacerbated by yet another major factor; the language barrier. The interpreter in this case becomes the voice and therefore means of agency for Latinos such as Mr. Lopez. Thus even though the task of the interpreter in this scenario is rather banal (communicating the content and meaning of a form already written in Spanish), the significance of the interpreter’s role as a mediator navigating the power dynamics inherent in the space between the Red Cross volunteer and Mr. Lopez is greatly heightened and cannot be understated.
The impact of the gulf between the Red Cross and the Latinos affected by the flood and of the relative weight of the responsibility we (meaning myself and my students) had assumed during this service-learning project was overwhelming. Horror, sorrow, compassion, guilt, shame, and anger coupled with an intense sense of urgency in my desire and determination to help were but a few of the emotions that overtook me. I vividly recall a woman with five children pleading with me to take her youngest, an infant, home with me for the evening, so damaged was her home. With tears in my eyes, I returned to her the child she had placed in my arms as the Red Cross volunteers adamantly shook their heads and sternly warned us: “Unless you have room for them all, do not take anyone home with you.” We were pained by the limitations of our role. Yet, we also knew that we were providing something far greater than interpreting skills and cleaning kits: We provided a human element to an otherwise cold and alienating process. We listened when the Red Cross could not. We navigated cultural differences that could have potentially led to misunderstanding, anger, and further mistrust. And in so doing, we advocated and provided a voice for an otherwise voiceless “other” in a potentially dehumanizing moment of crisis and need.
Perhaps it was the overwhelming sense of voicelessness and the desire to give voice to the families affected by the flood that led me to process my experience through the form of a screenplay. I now realize that the reality or truth of this experience lies in neither the voice of the Red Cross volunteers, the Latino families, nor the interpreter alone, but somewhere between them as well as between what transpired during the event (my memory creatively captured in the screenplay) and my reflection and reading of the event now several months later. The process has been at once illuminating, therapeutic, and cathartic on an intellectual and personal level.
Though journals are indeed valuable tools for processing such experiences, I contend that CAP ethnographies like the screenplay offer teachers and students undertaking similar service-learning projects a unique opportunity to unpack, reflect on, and critically assess their experiences in a personally meaningful way. Within my own class, the screenplay and its actual performance by the students (using different readings or interpretations of the roles) provided a rich space for exploring the role, responsibilities, and significance of language interpreters. It also served as a launching point for deep and meaningful conversations about the social, cultural, and political conditions and dynamics surrounding Latino immigrant communities, as well as about disaster relief and the nature of the relationship between the Latino community and local, national, and transnational service providers. It also reminded us of our individual responsibilities within our own communities to look after and care for one another in times of distress and need, regardless of language, culture, or any other social barriers that may exist. As my class concluded by the end of the semester, we are all human beings, not categories and labels to be checked off on a form; and just as we are all liable to be in a position of need, we are all also capable of lending assistance. Following this service-learning experience, I am confident that my students will not hesitate to respond to the call: “Volunteers needed.”
Bhabha, H.K. (1994). The location of culture. New York, NY: Routledge.
Bolaños, A.F., & Verdesio, G. (Eds). (2002). Colonialism past and present: Reading and writing about colonial Latin America today. Albany, NY: State University of New York Press.
Hill, J. (1995). Mock Spanish: A site for the indexical reproduction of racism in American English. Language and Culture, 2. Retrieved from http://language-culture.binghamton.edu/symposia/2/part1/.
Hill, J. (2008). The everyday language of white racism. Chichester, U.K.: Wiley-Blackwell.
Kohn, N. (2000). The screenplay as postmodern literary exemplar: Authorial distraction, disappearance, dissolution. Qualitative Inquiry, 6(4), 489–510.
Gomberg-Muñoz, R. (2011). Labor and legality: An ethnography of a Mexican immigrant network. New York, NY: Oxford University Press.
Richardson, L., & St. Pierre, E.A. (2005). Writing: a method of inquiry. In N.K. Denzin & Y.S. Lincoln (Eds.), The Sage handbook of qualitative research, 3rd edition (pp. 959–978). Thousand Oaks, Ca: Sage Publications.
About the Author
Diana M. Ruggiero is an assistant professor in the Department of Foreign Languages and Literature at the University of Memphis.
David Phipps, Joanne Cummings, Debra Pepler, Wendy Craig, and Shelley Cardinal
Knowledge mobilization supports research collaborations between university and community partners which can maximize the impacts of research beyond the academy; however, models of knowledge mobilization are complex and create challenges for monitoring research impacts. This inability to sufficiently evaluate is particularly problematic for large collaborative research networks involving multiple partners and research institutions. The Co-produced Pathway to Impact simplifies many of the complex models of knowledge mobilization. It is a logic model based framework for mapping the progress of research dissemination uptake implementation impact. This framework is illustrated using collaborative research projects from Promoting Relationships and Eliminating Violence Network (PREVNet), a pan-Canadian community-university network engaging in knowledge mobilization to promote healthy relationships among children and youth and prevent bullying. The Co-produced Pathway to Impact illustrates that research impact occurs when university researchers collaborate with non-academic partners who produce the products, policies, and services that have impacts on the lives of end beneficiaries. Research impact is therefore measured at the level of non-academic partners and identified by surveying research partners to create narrative case studies of research impact.
Knowledge mobilization helps make academic research accessible to non-academic audiences and supports collaborations between academic researchers and non-academic partners such as community-based organizations. Knowledge mobilization is a process that supports action oriented research and finds novel approaches to persistent social, economic and environmental challenges. Knowledge mobilization has elements of: 1) university “push” of research beyond the academy; (2) community “pull” of research from the academy; 3 “knowledge exchange” between community and the academy; but extends those to include 4) the co-production of research that has academic merit and also has relevance for community action (Phipps & Shapson, 2009). Knowledge mobilization can thus support community engaged scholarship and community-based research as well as service-learning when the learning opportunity is meeting the needs of a community derived research question. There has been increasing attention paid to knowledge mobilization and related activities as the academic research community seeks to articulate and maximize the various impacts of university research beyond the academy (Donovan, 2011; Grant, 2015).
Despite this increasing attention to articulating the impacts of research there is little evidence that research is creating extra academic impacts (Bhattacharyya & Zwarenstein, 2009). Sandra Nutley and colleagues point out that “a central irony is the only limited extent to which evidence advocates can themselves draw on a robust evidence base to support their convictions that greater evidence use will ultimately be beneficial to public services” (Nutley, Walter, Davies, 2007, p. 271). Although it is feasible to measure the
impact of a single knowledge mobilization intervention by testing indicators pre- and post-intervention, it is challenging to evaluate a complex system of knowledge mobilization where there may be multiple research collaborators practicing a diversity of knowledge mobilization methods with diverse end users.
In a recent review of leading models for knowledge mobilization such as the circular Knowledge to Action Cycle (Graham, Logan, Harrison, Straus, Tetroe, Caswell, & Robinson, 2006) and the models of Bennet and Bennet (2008), Phipps, Jensen and Myers (2012) concluded that many models of knowledge mobilization are highly complex. This conclusion is not surprising because knowledge mobilization is a complex process described by Bennet and Bennet (2008) as collaborative entanglement: “Collaborative entanglement means to purposely and consistently develop and support approaches and processes that combine the sources of knowledge and the beneficiaries of that knowledge to interactively move toward a common direction such as meeting an identified community need” (p. 48).
Knowledge Mobilization Pathway to Impact
In an effort to simplify a system of knowledge mobilization that reflects movement toward a common direction of impacts we turned to a logic model (Frechtling, 2007) where activities produce outputs that in turn produce outcomes that then produce impacts (Figure 1a). By mapping such a logic model onto knowledge mobilization processes, it is possible to draw a sequence of stages that lead from research to impacts (Figure 1b). In addition, it allows for the development of metrics at each stage of the logic model.
Dissemination. Knowledge mobilization supports dissemination beyond traditional academic publishing and conference presentations. This dissemination can include publishing activities such as press releases, clear language research summaries, as well as more iterative tools such as social media. It also involves active, in person methods such as research events where researchers engage actively with organizations seeking to engage with research and research expertise (Phipps, 2011). The goal of dissemination is to move research out of the academic setting and into practice and policy settings where it can progress towards impact.
Uptake. Once an organization has received research information from a dissemination activity it takes that research into the organization with a goal of determining whether the research is useful for informing decisions about policy, professional practice, and/or social services. Uptake can include presentations at staff meetings (that may or may not include the original researcher), internal evaluation, as well as comparisons to the literature and existing practice.
Implementation. Once the research has been taken up and passed through internal assessment, the organization may choose to use the research when developing new or improved products, policies, and services. Implementation in the knowledge mobilization context is an activity internal to the non-academic partner that uses research evidence to inform organizational decisions.
Impact. Impact is the effect the research-informed products, policies, and services have on end users as measured by the non-academic organization. It is measured not only in metrics of utilization but also by changes in the lives of citizens, the health of the environment, or animal welfare, depending on the ultimate end user the organization is seeking to address.
This model as illustrated in Figure 1b creates a pathway to impact that enables the monitoring of progress. By understanding the goals of each stage of the pathway to impact, it is possible to assess the benefits accruing along the pathway; however, the linearity of this model may be a limitation. Linear models of research use have long been abandoned in favour of more iterative models of research use that show sustained engagement between researchers and non-academic partner organizations (Nutley, Walter, & Davies, 2007; Greenhalgh & Wieringa, 2011). Linear models create risks that research evidence merely “transferred” to end user organizations may be misinterpreted or misused. Linear models create challenges of attribution which is the extent to which impacts can be attributed to the use of specific research outputs (Boaz, Fitzpatrick, & Shaw, 2008). By requiring a moment of transfer from the academic to the non-academic setting, linear models also reinforce academic and non-academic silos.
Knowledge Mobilization Co-produced Pathway to Impact
Academic research networks are expected to collaborate with non-academic partner organizations to make new discoveries and transform those discoveries into impacts. This requires a more iterative version of the pathway to impact than is shown in Figure 1b because this process requires collaboration and co-production at each stage of the pathway. A circular or iterative logic model has previously been recommended for evaluating knowledge translation (Davison, 2009) such as the Knowledge to Action Cycle (Graham et al., 2006) and a cyclical model proposed for education research (Amo, 2007). The iterative aspects of circular and cyclical models can be embedded into the knowledge mobilization logic model of Figure 1b to produce a Co-produced Pathway to Impact, as illustrated in Figure 2. The Co-produced Pathway to Impact maintains collaboration throughout the process and creates an iterative relationship between the non-academic partners and academic researchers, while maintaining an overall progression from research development to ultimate impact. As illustrated in Figure 2 there are domains where academic research and policy/practice activities remain distinct; however, the central overlapping space is a shared space of collaboration where co-production occurs at each stage of the pathway.
Co-production occurs at each stage of the pathway and accelerates the impact of research. For example, co-production at the research stage ensures partners’ readiness to take up findings because of their input on the nature of the research questions, methods, and interpretations. Co-production in the research stage enhances partners’ motivation and engagement with research content—the new knowledge will be relevant to them. At the dissemination stage, the research findings are tailored to meet the partners’ needs from knowledge mobilization products. These products are produced in an accessible format for the partners. Different partners can then tailor the same research findings into their own relevant and actionable knowledge mobilization products that further heighten network engagement and increases dissemination. Partners enhance dissemination through their organizational channels with a breadth and depth that researchers cannot achieve. The ongoing mutual and reciprocal support and collaboration between the researchers and partners in the uptake and implementation stages enables organizational transformation in response to the new research findings. Traditionally, as research moves to impact, there is a decrease in engagement across the four stages of the pathway and engagement of the academic partner is lowest in the ultimate impact stage. Unlike the traditional process of research dissemination with research “handed” to partners, our framework supports an ongoing relationship through the knowledge mobilization processes. As illustrated in Figure 2 each stage of the pathway confers benefits for both researchers and partners, leading to new research questions, knowledge, and potential knowledge mobilization products.
This Co-produced Pathway to Impact is illustrated with examples from PREVNet (www.prevnet.ca). PREVNet is a multi-disciplinary and multi-sectorial network founded in 2006 on the premise that to prevent bullying strategies are required in every setting where Canadian children and youth live, learn, work, and play. PREVNet includes 121 researchers from 21 disciplines (e.g., psychology, education, social work, law, business, criminology, policy, psychiatry, nursing) collaborating with 63 national public and
community sector organizations.
PREVNet addresses the increasingly recognized and urgent need to provide all adults responsible for socializing children and youth with knowledge and tools to choose, implement, evaluate, and sustain effective bullying and violence prevention initiatives. Although many bullying prevention programs are available, they often lack empirical evaluation, and have the potential to be ineffective or, in some cases, harmful (Dodge, Dishion, & Langsford, 2006; Farrington & Ttofi, 2011). Programs based on science with evidence of effectiveness are not well disseminated, particularly to isolated and vulnerable communities; moreover educators are most likely to choose programs and resources on the basis of word-of-mouth, rather than evidence (Cunningham, Vaillancourt, Rimas, Deal, Cunningham, Short, & Chen 2009). PREVNet promotes engaged scholarship by collaborating with its member organizations to develop evidence-based initiatives that rest on four pillars: education/training, assessment/evaluation, prevention/intervention, and policy/advocacy (Pepler & Craig, 2011). PREVNet has a focus on participation by non-academic partners and a target of action oriented impacts which are hallmarks of authentic community engagement (Stoecker, 2009).
PREVNet’s research, training and knowledge mobilization projects are at various stages of development from research to impact. The projects below are presented as a snapshot in time to illustrate the different stages of the Co-produced Pathway to Impact. Each of the projects is a collaboration between academic and non-academic partners. The ongoing, sustained collaboration of each project described below creates the critical feedback loops illustrated at each stage of the pathway (Figure 2). In this manner the academic and non-academic partners not only contribute their complementary expertise to the project, but the collaboration enables critical reflection on the creation of new knowledge and its application to the prevention of bullying.
An example of partner-led co-produced research. The Quazar Positive Behaviour Recognition Program: Wynford Motivation Works is collaborating with PREVNet and the Toronto District School Board’s Build Character Build Success initiative to produce animated videos and lesson plans to build elementary students’ motivation to behave in ways that exemplify each of this initiative’s 10 positive character traits shown to be important for healthy relationships: respect, responsibility,
empathy, kindness and caring, teamwork, fairness, honesty, co-operation, integrity, and perseverance.
PREVNet academic researchers and Wynford entered into an intense collaborative co-production process for program development and evaluation research. The first draft of program content was collectively reviewed and subsequently revised and enhanced to reflect current scientific evidence about character development and violence prevention. A manual for school implementation was produced to ensure program fidelity. Now named the Quazar Positive Behaviour Recognition Program, it is ready for dissemination to end users, with an ongoing evaluation component. There is a website that introduces and enables schools to register for the program. It is currently launched and being evaluated in four Toronto and five Kingston Ontario elementary schools.
Research benefits. New knowledge about the positive characteristics important for healthy relationships; new collaborative activities between researchers and partners, such as the Toronto District School Board; engaged graduate student experiences.
An example of dissemination—Public Health Agency of Canada (PHAC) Canadian Best Practices Portal (CBPP). CBPP is an authoritative repository for annotated and evidence-based health promotion practices. PREVNet researchers and graduate students have collaborated with PHAC since 2009 to create and populate the Violence Prevention Stream for the CBPP as a tool for disseminating evidence-based violence prevention practices, tools, and interventions. Each year, violence prevention programs developed in Canada and internationally are reviewed by PREVNet academic researchers and PHAC and those meeting the stringent inclusion criteria are included on the portal. The Violence Prevention Stream currently hosts 80 programs on the site, and there are 3,000 unique visitors annually.
After conducting six focus groups with educators and community organizations to explore the usability of the portal, feedback has resulted in improvements to the site. PREVNet and PHAC have developed a Needs Assessment Toolkit, to enable stakeholders to select programs that will be effective, relevant and appropriate for their specific populations and local needs, further enhancing the utility of the CBPP as a dissemination tool. PREVNet researchers have actively promoted the CBPP violence prevention portal through public presentations and professional conferences.
Dissemination benefits. These provide improved functionality using the Needs Assessment Toolkit; web based and social media promotion; improved accessibility of the 80 evidence-based programs, as well as four academic presentations at conferences; and increased decision maker awareness regarding the importance of evidence-based violence and bullying prevention programs.
An example of Uptake—Family Channel StandUP! Campaign. Family Channel is a commercial-free network offering family television entertainment in 5.8 million homes across Canada. Its target audience is children aged 9–12. Family Channel has been involved in Bullying Awareness Week every November for the past for 11 years, and approached PREVNet to be its official research partner in 2006. In 2012, a comprehensive Bullying Awareness Teacher’s Guide for Grades 4–6 was written by a team of graduate students from across Canada under the leadership of a PREVNet researcher. The 80-page guide brought together current evidence-based information about bullying, cyberbullying, bullying and LGBTQ students, bullying and students with exceptionalities, building a respectful classroom climate, and a plan for lead-up activities before and daily activities during Bullying Awareness Week. After reviewing and evaluating this resource and the accompanying tip sheets and associated activities, Family Channel contributed professional graphics and design. The 2012 Teacher’s Guide was downloaded 2,250 times. In February 2012, Dr. Wendy Josephson, professor, Department of Psychology, University of Winnipeg, and three students held a series of focus groups with 41 elementary and high school teachers from Winnipeg and the surrounding area to review the 2012 Teacher’s Guide. Based on this input, the 2013 guide was revised and refined.
Uptake benefits. Family Channel validated the academic research in a real world setting; graduate students gained skills working with non-academic audiences (Family Channel and teachers); user audience input was used to refine the resource; resource made available to end users.
An example of implementation—Girls United Training, Girl Guides of Canada (GGC). Beginning in 2006, consultations with GGC leadership revealed that the training provided to Girl Guide Leaders, known as “Guiders,” did not specifically address bullying and relational aggression, nor was bullying addressed in the GGC Code of Conduct even though Girl Guide leadership identified bullying and relational aggression as needing to be addressed. A working group with leading researchers on girls’ aggression was convened, and then a PREVNet researcher and graduate student worked with senior GGC training developers to co-create the Girls United Training Module for adult leaders and Girls United Badge for Girl Guides. The initial iteration of the training was presented to the PREVNet Social Aggression Working Group, attended by leading Canadian researchers working in the field of social aggression and by staff from several youth-serving community organizations. A training module was developed based on the feedback from the working group. GGC training developers simplified the language, sharpened the messaging, and supplied graphic design. Between 2006 and 2008, PREVNet delivered the training to over 75 Senior Trainers (who in turn trained other local trainers, who then trained Guiders) in British Colombia, Nova Scotia, and Ontario. PREVNet collected participant evaluations of these training sessions (N = 129) and found high levels of satisfaction, with a mean rating of 4.7 on a 5-point scale assessing perceptions of value and relevance, understanding of topic, and increased confidence about addressing social aggression among girls. Similar ratings were found by PREVNet from 27 participants who took the training from a Senior Girl Guide Trainer (mean = 4.6), providing evidence that the “Train the Trainer” model was effective (Daniels & Quigley, 2009).
From 2007 until August 2013, 1,445 Guiders completed on line training. Between October 2007 and August 2013, 18,873 Girl Guides achieved the Girls United Badge, indicating they had fulfilled the required activities designed to develop their understanding of healthy relationships with their peers. This example illustrates how co-produced evidence informed training (the Girls United Training Module for adult leaders, and Girls United Badge for Girl Guides) was disseminated to the PREVNet Social Aggression Working Group, was taken up by GGC training developers and implemented in a national train-the-trainer campaign.
Uptake benefits. These included a research informed training program; graduate student experience working in a practice setting; expansion of program to an online version; and Girl Guides developing an enhanced understanding of healthy relationships
An example of impact—the Healthy Relationship Training Module (HRTM). The HRTM was developed through a Community of Practice that included PREVNet academic researchers, students and three youth-serving non-profit organizations: Big Brothers Big Sisters Canada, Canadian Red Cross, and Scouts Canada. By sharing resources and exchanging knowledge, the goal was to enhance each organization’s capacity to
foster respectful, safe, caring, and inclusive environments for children and youth.
Adult leaders play a critical role socializing children and youth: they serve as role models, mentors, guides, supports, and teachers. To be effective in their work with children and youth, they need explicit training about how healthy development depends on healthy relationships, and how to identify and address bullying and other unhealthy relationship dynamics. There was an assumption that professionals and volunteers who work with children and youth have the knowledge, confidence, and skills they need to create healthy social climates and prevent bullying, yet explicit, comprehensive, and evidence-based training was missing. Working collaboratively, the Community of Practice co-created the HRTM to address the gap in relationship training.
The module consists of a comprehensive Facilitator’s Guide, a slide presentation deck, and a Participant’s Handbook. Following a multi-step process in which PREVNet and Community of Practice members move from visioning to design to evaluation and training, the HRTM was co-created in stages, with a graduate student preparing a first draft that was extensively presented and critiqued through multiple Community of Practice meetings. Based on participant feedback and questionnaire results, an extensive revision of the HRTM Facilitator Guide was completed.
Within the three partner organizations, the HRTM was integrated into existing training resources and procedures. Pre- and post-training pilot data were collected using the “Knowledge Confidence Skills: Healthy Relationship Questionnaire,” with a pilot data set of 505 participants from the partners. Analysis of these data revealed significant increases in participants’ confidence and commitment to fostering healthy relationships.
The following comments from partner organization leaders speak to the rapid uptake and implementation of HRTM that occurred by the end of 2012:
…if you look at knowledge mobilization, that knowledge that was presented, all the research and best practices made its way down to the field, which I think was a huge benefit. Across Canada, we incorporated portions of the Healthy Relationships training into all our prevention education materials. For example, in our training for teachers in bullying prevention we have integrated a module on healthy relationships. These teachers train Youth Facilitators and share information on healthy relationships. Our Youth facilitators deliver workshops to younger students and talk about healthy schools and healthy relationships. We have 3,500 Youth facilitators across Canada and reached over 260,000 youth with information on bullying prevention and healthy relationships last year. We also recently updated our Be Safe! Program for children ages 5 to 9 (formerly known as c.a.r.e.). Our 8th edition contains a section on healthy relationships. We hope to reach over 30,000 children, parents
and teachers with the new kit over the next year. — Lisa Evanoff, National Training Manager Canadian Red Cross
This year we have the potential of reaching more than twenty-four thousand youth, right from Vancouver Island to Newfoundland. So that’s a goodly number of youth and as far as adults, potentially more than twenty-seven thousand volunteer leaders. If you include all of our paid staff as well as our volunteers, we’re looking at over one hundred and two thousand individuals. — DeEtte Bryce, past Training Representative for Fraser Valley Council, B.C. Scouts Canada
The impact has been very exciting given our magnitude across the country – we work in every province and now have some relationships and programs in each of the territories as well. We’re able to bring these new resources to children and youth, parents/guardians, volunteers/mentors, service delivery staff, and executive staff and boards across the country. In 2012 Big Brothers Big Sisters agencies across Canada served over 40,000 children and youth—every child and youth, along with their volunteer mentors and parents/guardians, benefits from the Healthy Relationship training. Susan Climie, director of training, Big Brothers Big Sisters of Canada,
Impact benefits. Gaps in training identified and addressed; training developed and provided to make safer spaces for children and youth across Canada. Training contributes to economic and social benefits.
These examples illustrate how sustained engagement between academic researchers, students, and non-academic partners enables: the co-production of research (Wynford); the dissemination of research (PHAC); the uptake of research
evidence into non-academic programs (Family Channel); the implementation of research evidence into products and services (Girl Guides); and, the eventual impact of evidence informed training on the lives of end beneficiaries (HRTM). The use
of the Co-produced Pathway to Impact has a number of implications for the practice of knowledge mobilization as described below.
Reflecting on the PREVNet experiences of collaborations between academic researchers and students with non-academic partners including (but not limited to) Wynford, Toronto District School Board, PHAC, Family Channel, Girl Guides, Red Cross, Scouts Canada and Big Brothers Big Sisters we have not only developed and implemented the Co-produced Pathway to Impact but can draw conclusions on its utility as a framework describing knowledge mobilization processes.
It is clear from the HRTM collaboration that impact is measured at the level of the non-academic partner. Academic impacts arise from research and dissemination, but impacts on the lives of end beneficiaries are mediated through the products, policies and services of non-academic partner organizations. The HRTM was a training program co-produced with academic researchers but delivered nationally though Community of Practice partners such as Red Cross, Scouts Canada, and Big Brothers Big Sisters. That impact is a function of non-academic partners has also been demonstrated by Sarah Morton, who has shown the critical role of research users in mediating impacts of research beyond the academy (Morton, 2014).
As illustrated by the HRTM example, in a co-production process, research can skip dissemination and uptake and move directly to implementation, which then has an impact. There was no need for dissemination and uptake because the end users of the HRTM were involved in its creation. This outcome is unique to co-production where the process of undertaking the research can have an impact (i.e., influence decision making) even before the research has been disseminated. Co-production can therefore help to address issues of attribution (Boaz, Fitzpatrick & Shaw, 2008).
When these impacts are measured by partners evaluating the effects of their efforts on their stakeholders and end beneficiaries, the stories of impacts can be told through narratives and case studies. Structured impact case studies were the unit of assessment for the UK Research Excellence Framework (REF; www.ref.ac.uk) and research on the REF confirmed this method as the best method for articulating impacts of research beyond the academy (Grant, 2015).
Because the pathway from research to impact may be measured over years, researchers and academic institutions need to remain in contact with non-academic partners to be able to capture the narrative case studies of impact. Without this active follow up or continual engagement with partner organizations, academic researchers may have little appreciation of the impacts of their research. This has been confirmed by an evaluation of knowledge mobilization programs by the Social Sciences and Humanities Research Council of Canada (SSHRC, 2013).
If funders, such as SSHRC, want to generate impacts from their investments in research, then they need to fund uptake and implementation activities within partner organizations. These activities can be supported by funding graduate student internships and post-doctoral fellowships in partner organizations to support uptake and implementation. This strategy will provide the non-academic organization with ready access to academic research expertise and will provide the student/fellow with experience in non-academic professional environments.
Finally, knowledge mobilization is often described using the metaphor of “bridging the gap” between the silos of research and policy/practice; however, this metaphor maintains the academic and non-academic silos. In co-production there is no gap to bridge. Academic researchers and non-academic partners come together in a shared space of collaboration (see Figure 2). They maintain their own independent spaces but research, dissemination, uptake, and implementation occur in a collaborative environment. In contrast impact beyond the academy is expressed in the non-academic environment only.
Future Work/Issues Arising
This theoretical framework is a snapshot in time of a number of research collaborations at various stages along the Co-produced Pathway to Impact. Some of the observations are retrospective and are not intended to make predictions of future benefits arising from the research. To establish how the Co-produced Pathway to Impact works for a single collaborative research project, one would follow a co-produced research project such as the Quazar Positive Behaviour Recognition Program as it progresses through dissemination to uptake to implementation and eventually to impact. However, a number of potential challenges arise. It may take years for impact to be realized. Many research projects will not proceed all the way to impact, as other factors such as availability of resources and competing products may prevent good research from proceeding to impact. Therefore, the question arises about the unit of measurement and evaluation: should the Co-produced Pathway to Impact be applied at the project, program/unit,
institutional or network level?
Additionally, the linearity of the logic model underpinning the Co-produced Pathway to Impact may not be an issue. A number of linear logic model based frameworks describing the flow of research to impact have been described by Alberta Innovates Health Solutions (Graham, Chorzempa, Valentine, & Magnan, 2012), and by the Commonwealth Scientific and Research Organisation in Australia (Morgan, 2014) and is linearly referred to as research uptake, use and impact at the Centre for Research on Families and Relationships, University of Edinburgh (Morton, 2014). What is interesting about this convergent thinking is that knowledge mobilization professionals seem to be getting comfortable with the linearity of these pathways. Linear models for a single knowledge mobilization project have been abandoned in favour of iterative models such as the Knowledge to Action Cycle (Graham et al., 2006). When working in a system of knowledge mobilization, however, a portfolio of projects, such as described for PREVNet, does move towards impact. And this movement is linear from research to impact. Different projects at different stages in the in the linear Co-Produced Pathway to Impact create the opportunity to further examine projects and develop indicators describing each to the stage of the pathway.
The Co-produced Pathway to Impact requires that researchers and research partners engage in ongoing collaboration throughout the process from research to impact. PREVNet’s deep and sustained collaborations may not be feasible or desirable for some community organizations or university researchers; however, in collaborative networks that have a mandate to not only create new knowledge but also to translate that knowledge into improved economic, social, health, cultural or environmental impacts the Co-produced Pathway to Impact creates a framework that describes the progress of collaborative research as it develops from research into new products, policies and services. It also illustrates that getting to impact is a shared enterprise and activities in both academic and non-academic partner sites need to be eligible expenses in research funding programs. A number of recommendations arise for those wishing to use the Co-produced Pathway to Impact to describe knowledge mobilization processes.
For academic researchers: Since impacts of research beyond the academy are mediated by non-academic partners it is important to stay in touch with non-academic partners who may be using academic or co-produced research to inform new products, policies, and services. Only by working with partners to tell those stories will academic researchers be able to articulate the impacts of research.
For non-academic partners: The role of the non-academic partner in community-campus collaborations is more than a co-creator of research knowledge or passive recipient of academic research. It is the community partner, not the academic researcher, who will implement research evidence into products, policies and services to benefit stakeholders. The Co-produced Pathway to Impact highlights the critical role of non-academic partners in mediating research impact.
For research institutions: Public policy drivers such as the UK’s Research Excellence Framework (REF) are driving UK academic institutions to articulate the impacts of university research beyond the academy (Grant, 2015). It can take many years for research to be taken up by partners and implemented into the products, policies and services that will then have an impact on end beneficiaries. Without an institutional office like a Knowledge Mobilization Unit (Phipps & Shapson, 2009), research institutions have little ability to identify these impacts. Sustainable research networks such as PREVNet maintain relationships with non-academic partners and over time collaborate with them to articulate the benefits of research projects at each stage of the Co-produced Pathway to Impact.
Amo, C. (2007). Conceptualizing research impact: the case of education research. Canadian Journal of Program Evaluation, 22(1), 75–98.
Boaz, A., Fitzpatrick, S., and Shaw, B. (2008) Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation. Retrieved from http://www.psi.org.uk/pdf/2008/bridgingproject_report_with_appendices.pdf
Bennet, A. & Bennet, D. (2008). Knowledge mobilization in the social sciences and humanities: moving from research to action. Frost, West Virginia, USA: MQI Press.
Bhattacharyya, O., & Zwarenstein, M. 2009. Methodologies to evaluate effectiveness of knowledge translation interventions. In S. Strauss, J. Tetroe, and I. Graham (Eds.), Knowledge translation in health care: Moving from evidence to practice (pp. 249–260). Oxford, UK: Blackwell Publishing 2009.
Cunningham, C.E., Vaillancourt, T., Rimas, H., Deal, K., Cunningham, L., Short, K., & Chen, Y. (2009). Modeling the bullying prevention program preferences of educators: A discrete choice conjoint experiment. Journal of Abnormal Child Psychology, 37(7), 929–943.
Daniels, T., & Quigley, D. (2009). Girls united: Creating a nationwide intervention program to address the use of social aggression within girls’ groups. In W. Craig, D. Pepler, & J. Cummings (Eds.) Rise up for respectful relationships: Prevent bullying (pp. 145–168.) PREVNet Series, Volume 2. Kingston, Canada: PREVNet Inc.
Davison, C.M. (2009). Knowledge translation: implications for evaluation. New Direction for Evaluation, 2009(124), 75–87.
Dodge, K.A., Dishion, T.J., & Langsford, J.E. (2006). Deviant peer influences in intervention and public policy for youth. Social Policy Report, 20(1), 3–19.
Donovan, C. (2011) State of the art in assessing research impact: introduction to a special issue. Research Evaluation, 20(3), 175-179.
Farrington, D., & Ttofi, M. (2011). Bullying as a predictor of offending, violence and later life outcomes. Criminal Behavior and Mental Health, 31, 90–98.
Frechtling, J.A. (2007). Logic modeling methods in program evaluation. San Francisco, CA: Jossey-Bass.
Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: time for a map?. Journal of Continuing Education in the Health Professions, 26(1), 13–24.
Graham, K.E.R., Chorzempa, H.L., Valentine, P.A., & Magnan, J. (2012). Evaluating health research impact: Development and implementation of the Alberta Innovates – Health Solutions impact framework. Research Evaluation, 1–14. doi: 10.1093/reseval/rvs027.
Grant, J. (2015) The nature, scale and beneficiaries of research impact: An initial analysis of Research Excellence Framework (REF) 2014 impact case studies. Higher Education Funding Council of England. http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independentresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf Accessed June 20, 2015.
Greenhalgh, T., and Wieringa, S. (2011). Is it time to drop the ‘knowledge translation’ metaphor? A critical literature review. Journal of the Royal Society of Medicine, 104(12), 501–509.
Morgan, B. (2014) Income for outcome: Australia and New Zealand are experimenting with ways of assessing the impact of publicly funded research. Nature, 511, S72–S75. Retrieved from: http://www.nature.com/nature/journal/v511/n7510_supp/pdf/511S72a.pdf.
Morton, S. (2014) Creating research impact: The roles of research users in interactive research mobilisation. Evidence & Policy: A Journal of Research, Debate and Practice, 11(1), 35–55. Retrieved from http://dx.doi.org/10.1332/174426514X13976529631798.
Nutley, S., Walter, I., & Davies, H. (2007). Using evidence: How research can inform public services. Bristol, UK: The Policy Press.
Pepler, D., & Craig, W. (2011). Promoting relationships and eliminating violence in Canada. International Journal of Behavioral Development. 35(5), 389–397.
Phipps, D., & Shapson, S. (2009). Knowledge mobilisation builds local research collaborations for social innovation. Evidence & Policy, 5(3), 211–27.
Phipps, D. (2011). A report detailing the development of a university-based knowledge mobilization unit that enhances research outreach and engagement. Scholarly and Research Communication, 2(2), 1–13.
Phipps, D.J., Jensen, K.E., & Myers, J.G. (2012). Applying social sciences research for public benefit using knowledge mobilization and social media. In Asunción López-Varela (Ed.), Theoretical and methodological approaches to social sciences and knowledge management. Rijeka, Croatia: InTech.
SSHRC. (2013). Evaluation of SSHRC’s knowledge mobilization funding opportunities. Retrieved from www.sshrc-crsh.gc.ca/about-au_sujet/publications/KMb_evaluation_2013_e.pdf.
Stoecker, R. (2009). Are we talking the walk of community-based research? Action Research, 7(4), 385–404.
About the Authors
David Phipps is executive director, Research and Innovation Services in the Office of Research Services at York University in Toronto, Canada. Joanne Cummings is knowledge mobilization director, PREVNet. Debra Pepler is a professor in the Department of Psychology at York University. Wendy Craig is a professor in the Department of Psychology, Queen’s University, Kingston, Ontario, Canada. Shelley Cardinal is National Aboriginal consultant to the Canadian Red Cross RespectED: Violence and Abuse Prevention program (Vancouver).