PerriKennedy.com

Introduction: As part of my graduate assistant duties for Dr. Yonnie Chyung, I researched the topic of evidence-based practice (EBP). I looked at its origins in medical practice, its adoption by other fields, and how it might apply to the field of Human Performance Technology.  I was particularly interested in the barriers to implementation and how they might affect HPT professionals who wanted to introduce EBP into their practice. The following is my brief literature review on the subject.

Evidence-Based Practice

 

1. What is evidence-based practice?

Definition

Evidence-based practice (EBP) originated in the field of medicine. Sackett et al. (1996) defined it as the pairing of a practitioner’s clinical expertise with empirical evidence from clinical research in order to make the best decisions about a patient’s care. Management science has adapted EBP into evidence-based management (EBM) which incorporates research findings from organizational and behavioral sciences into organizational decision-making processes (Rousseau, 2006).

Evidence

Although empirical evidence from randomized control trials (RCTs) is considered the standard due to its quantifiable and objective nature, Rycroft-Malone, Gill, Seers, and Kitson (2004) maintained that contextual evidence collected through other methods was both credible and necessary to form a complete picture. The American Psychology Association (APA) (2005) defined three categories of credible evidence: research studies, practitioner experience, and situational context. Research is not limited to RCTs and other experimental studies; ethnographic research, case studies, and studies linking interventions to outcomes are also valid sources of data which retain contextual elements (APA 2005, Edwards 2004).

Principles

McKibbon & Wilczynski (2009) laid out the five steps to conducting evidence-based practice as defining a question, collecting the evidence, evaluating the evidence, integrating evidence into context to create a resolution, and evaluation of the resolution. Sackett et al. (1996) stressed the importance of integrating the evidence, as EBP is an adaptation of evidence to suit both the practitioner’s level of expertise and the uniqueness of each EBP research question.

 

2. Evidence-Based Practice in Use

Fields

The concept of evidence-based practice arose in clinical medicine and spread to closely-related fields such as nursing, dentistry, mental health, and public health (Trinder & Reynolds, 2001). Studies on the use of EBP have been conducted in fields such as physiotherapy (Turner & Whitfield, 1997), dental hygiene (Asadoorian, Hearson, Satyanarayana, & Ursel, 2010), occupational therapy (Bennett, et al., 2003), education (Pirrie, 2001), school libraries (Barron, 2003), public safety (Lum, Koper, and Telep, 2011), and social work (Webb, 2001).

Processes and Guidelines

EBP is still an emerging concept, with even the early-adopting fields of clinical medicine and nursing questioning its reliance on clinical data (Clarke, 1999; Seidel, 2011) and arguing its limitation (Nevo, I. & Slonim-Nevo, V. (2011).

The five steps to conducting evidence-based practice are defining a research question, collecting the evidence, evaluating the evidence, integrating evidence into context to create a resolution, and evaluation of the resolution (McKibbon & Wilczynski, 2009). Sackett et al. (1996) stressed the importance of integrating the evidence, as EBP is an adaptation of evidence to suit both the practitioner’s level of expertise and the uniqueness of each EBP research question. 

Relative Value of Evidence

Rice (2008) ranked sources of evidence by the consistency and universality of results, judging systematic reviews of RCTs to be the most valuable for use in EBP, followed by individual RCT studies and structured case studies. APA (2005) did not rank its evidence sources, but did recognize the limitations of each category in terms of the generalizability of results and the transportability of results into usable data for practitioners.

 

3. Limiting Factors of EBP Usage

Challenges

Each field presents its own unique challenges to implementing EBP. Business-oriented disciplines such as management science (Pfeffer & Sutton, 2006), human resource development (Gray, Iles, & Watson, 2011; Rynes, Giluk, & Brown, 2007), and industrial/organizational psychology (Briner & Rousseau, 2011; Thayer, Wildman, & Salas, 2011) all share the common challenge of a gap between the interests of academia and practitioners. In each case, the subjects of academia-based research may vary significantly from the topics written about in practitioner-oriented publications. Articles written for publication in peer-reviewed academic journals are not written with the practitioner in mind.

Other challenges to EBP typical in business-related fields include the quality of available evidence, (Rice, 2008; Thomas, 2006), the applicability of results from one context into another (Clark, 2006) and the limited number of systematically-conducted studies from which to draw relevant evidence (Pfeffer & Sutton, 2006).

Barriers

Barriers to implementing evidence-based practice can be categorized as environmental and individual.

Environmental barriers include authoritative organizational hierarchies which limit practitioners’ ability to base their decisions on research rather than traditional practice (Asadoorian, Hearson, Satyanarayana, & Ursel, 2010; Gambrill, 1999) and the lack of incentives to take the extra effort needed to conduct EBP (Thayer, Wildman, & Salas, 2011). External factors such as legal requirements for practice may also complicate using EBP (Hasson, Andersson, & Bejerholm, 2011).

The greatest environmental barrier to implementing EBP may be a lack of time and resources available for practitioners (Hannes, Pieters, Goedhuys, & Aertgeerts, 2010; McCluskey, 2003; Metcalfe et al., 2001) to search the available research and formulate interpretations.

Individual barriers will vary depending on the individual, but studies indicate that the lack of the skills required to conduct adequate research and synthesize the findings is a common problem. Such skills are usually found in individuals with higher levels of education, and those individuals are more likely to accept EBP because they are comfortable with the cognitive skills required (Aarons, 2004; McCluskey, 2003). However, when the higher education consists of a practitioner-oriented curriculum with less emphasis on research, holding a graduate degree does not necessarily correlate to familiarity with searching and comprehending academic research (McCluskey, 2003; Rynes, Giluk, & Brown, 2007).

4. Evidence-Based Practice and Human Performance Technology

Needs Assessment

The analytical nature of the needs assessment aspect of HPT practice call out for theoretical frameworks tested in practice in order to address the complexity of diagnosing organizational problems (Cho, Jo, Park, Kang, & Chen, 2011). However, the models in use by HPT practitioners were developed with little documentation from their creators about the theories or evidence upon which the models were structured; the creators were consultants in practice rather than pure academics, and drew primarily from their own client experiences (Rummler, 2007).

For now, HPT practitioners can draw upon extant research in management science, human resource development, organizational behavior, and industrial/organizational psychology in order to develop intervention strategies based on researched evidence.

Instructional Design

There is currently a broad array of research evidence available to instructional designers, as evidence-based practice has already been introduced into the education field (Pirrie, 2001) and additional research has been done on workplace-directed topics such as transfer of training (Hutchins, 2009) and learning environments (Hardré, 2008).

Change Management and Motivational Issues

The concept of change management, which provides structure to moving an organization from one state to another, draws from research and concepts from industrial/organizational psychology as well as management science and human resource development, particularly in the realm of acceptance and motivation (Dormant, 1999). Motivation in organizations has been among the most published topics in human resource development publications over the last three decades, so the evidence base is substantial (Deadrick & Gibson, 2009).

 

5. Conclusion

Among the principles of the International Society for Performance Improvement (ISPI)’s Code of Ethics is the statement that human performance technology (HPT) practitioners use validated practice concepts consistent with existing research and practice knowledge (ISPI, 2002). Despite the very real challenges to be faced in adopting evidence-based practice in human performance technology, it is an essential step in establishing the credibility of the discipline (Clark, 2006). Academics and practitioners must work together to develop both lines of research and adaptations of research findings which are accessible to practitioners. Systematically-documented case studies drawn from practitioner experience would both provide evidence, adding to the discipline’s knowledge base, and point towards new lines of research for academics to explore.

 

References

Aarons, G. A. (2004). Mental health provider attitudes toward adoption of evidence-based practice: The Evidence-Based Practice Attitude Scale (EBPAS). Mental Health Services Research, 6(2), 61-74.

American Psychological Association Presidential Task Force on Evidence-Based Practice. (2005). Draft policy statement on evidence-based practice in psychology. Washington, DC: American Psychological Association.

Asadoorian, J., Hearson, B., Satyanarayana, S., & Ursel, J. (2010). Evidence based practice in dental hygiene: Exploring the enhancers and barriers across disciplines. Canadian Journal of Dental Hygiene, 44(6), 271-271-276.

Barron, D.D. (2003). Evidence based practice and the school library media specialist. School Library Monthly, 20(4), 49-51.

Bennett, S., Tooth, L., McKenna, K., Rodger, S., Strong, J., Ziviani, J., Mickan, S., & Gibson, L. (2003). Perceptions of evidence-based practice: A survey of Australian occupational therapists. Australian Occupational Therapy Journal, 50(1), 13-22.

Briner, R. B., & Rousseau, D. M. (2011). Evidence-based I-O psychology: Not there yet. Industrial and Organizational Psychology:Perspectives on Science and Practice, 4(1), 3-22.

Cho, Y., Jo, S.J., Park, S., Kang, I., & Chen, Z. (2011). The current state of human performance technology: A citation network analysis of Performance Improvement Quarterly, 1988-2010. Performance Improvement Quarterly, 24(1), 69-95.

Clark, R.C. (2006) Evidence-based practice and the professionalization of human performance technology. In Pershing, J.A. (Ed.), Handbook of human performance technology: Principles, practices, and potential (3rd ed.)(pp. 873-898). San Francisco, CA: Pfeiffer.

Clarke, J. B. (1999). Evidence-based practice: A retrograde step? the importance of pluralism in evidence generation for the practice of health care. Journal of Clinical Nursing, 8(1), 89-89.

Deadrick, D.L. & Gibson, P.A. (2009). Revisiting the research-practice gap in HR: A longitudinal analysis. Human Resource Management Review, 19(2), 144-153.

Dormant, D. (1999). Implementing human performance technology in organizations. In H. Stolovitch & E. Keeps (Eds.), Handbook of human performance technology (1st ed., pp. 237-259). San Francisco, CA: Jossey-Bass/Pfeiffer.

Edwards, D. Dattilio, F. & Bromley, D. (2004). Developing evidence-based practice: The role of case based research. Professional Psychology: Research and Practice, 35 (6), 589-597.

Gambrill, E. E. (1999). Evidence-based practice: an alternative to authority-based practice. Families in Society: The Journal of Contemporary Social Services, 80(4), 341-350.

Gray, D. E., Iles, P., & Watson, S. (2011). Spanning the HRD academic-practitioner divide: Bridging the gap through mode 2 research. Journal of European Industrial Training, 35(3), 247-263.

Hannes, K., Pieters, G., Goedhuys, J., & Aertgeerts, B. (2010). Exploring barriers to the implementation of evidence-based practice in psychiatry to inform health policy: A focus group based study. Community Mental Health Journal, 46(5), 423-432.

Hardré, P. (2008). Designing effective learning environments for continuing education. Performance Improvement Quarterly, 14(3), 43-74.

Hasson, H., Andersson, M., & Bejerholm, U. (2011). Barriers in implementation of evidence-based practice. Journal of Health Organization and Management, 25(3), 332-345.

Hutchins, H.M. (2009). In the trainer’s voice: A study of training transfer practices. Performance Improvement Quarterly, 22(1), 69-93.

International Society for Performance Improvement. (2002). ISPI code of ethics. Retrieved from http://www.ispi.org/uploadedFiles/ISPI_Site/About_ISPI/About/Code-of-Ethics.pdf

Lum, C., Koper, C. S., & Telep, C. W. (2011). The evidence-based policing matrix. Journal of Experimental Criminology, 7(1), 4-26.

McKibbon, A. & Wilczynski, N. (2009) PDQ evidence-based principles and practice. Shelton, CT: People’s Medical Publishing House.

McCluskey, A. (2003). Occupational therapists report a low level of knowledge, skill and involvement in evidence-based practice. Australian Occupational Therapy Journal, 50(1), 3-12.

Metcalfe, C., Lewin, R., Wisher, S., Perry, S., Bannigan, K., & Klaber Moffett, J. Barriers to implementing the evidence base in four NHS therapies: Dietitians, occupational therapists, physiotherapists, speech and language therapists. Physiotherapy, 87(8), 433-441.

Nevo, I. & Slonim-Nevo, V. (2011). The myth of evidence-based practice: Towards evidence-informed practice. British Journal of Social Work, 41(6), 1176-1197.

Pfeffer, J., & Sutton, R. I. (2006). Evidence-based management. Harvard Business Review, 84(1), 62-74.

Pirrie, A. (2001). Evidence-based practice in education: The best medicine? British Journal of Educational Studies, 49(2), 124-136.

Rice, M.J. (2008). Evidence-based practice in psychiatric care: Defining levels of evidence. Journal of the American Psychiatric Nurses Association, 14(3), 181-187.

Rousseau, D. M. (2006). Is there such a thing as ‘evidence-based management’?. Academy of Management Review, 31(2), 256-269.

Rummler, G.A. (2007). The past is prologue: An eyewitness account of HPT. Performance Improvement, 46(10), 5-9.

Rycroft-Malone, J., Gill, H., Seers, K., & Kitson, A. (2004). An exploration of the factors that influence the implementation of evidence into practice. Journal of Clinical Nursing, 13(8), 913-924.

Rynes, S.L., Giluk, T.L., & Brown, K.G. (2007) The very separate worlds of academic and practitioner periodicals in human resource management: Implications for evidence-based management.  Academy of Management Journal, 50(5), 987–1008.

Sackett, D. L., Rosenberg, W. M. C., Muir Gray, ,J.A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312(7023), 71-71.

Seidl, K. L. (2011). To EBP or not to EBP … why is it a question? Bariatric Nursing and Surgical Patient Care, 6(2), 53-53-54.

Thayer, A. L., Wildman, J. L., & Salas, E. (2011). I–O psychology: We have the evidence; we just don’t use it (or care to). Industrial and Organizational Psychology, 4(1). 32–35.

Thomas, M. N. (2006). Evidence-practice partnership. Performance Improvement, 45 (6), 8-12.

Trinder, L. & Reynolds, S. (2001) Evidence-based practice: A critical appraisal (2nd ed.). Malden, MA: Blackwell Science, Inc..

Turner, P. & Whitfield, T.W.A. (1997). Physiotherapists’ use of evidence based practice: A cross-national study. Physiotherapy Research International, 2(1), 17-29.

Webb, S.A. (2001). Some considerations on the validity of evidence-based practice in social work. British Journal of Social Work, 31(1), 57-79