Measuring Professional Identity

Complexity can make us uncomfortable. But when talking about a complex idea such as professional identity, a construct that encompasses multiple dimensions of how physicians and medical students make sense of their role, there is wisdom in adopting an approach to teaching and assessment that is also complex and nuanced. The Carnegie Foundation's influential report (Cooke et al. 2010) recommended that medical schools adopt a constructive developmental approach to teaching and assessing professionalism or professional identity. This approach acknowledges that physicians and medical students are meaning-making beings, continually making sense of a barrage of stimuli, requirements, demands, and feelings. Carnegie pointed to the importance of reflection in the process of the development of a professional identity. Carnegie steered educators away from traditional didactic pedagogies, to ones featuring ongoing reflection and feedback. For educators or administrators whose goal is to measure student progress, the complexity of the construct has generated more than a decade of debate on approaches that generate the least amount of discomfort. 

How do we then make sense of the need to measure outcomes of curriculum aimed at professional identity? Research in healthcare ethics education in the professions provides us with an established, empirically validated map building on constructive developmental theory, as recommended by Carnegie and others (Bebeau, 2002; Cruess et al. 2014). Bebeau and Lewis (2004) developed an in-depth reflective short essay assessment of professional identity based on Kegan's theory of adult identity formation (Kegan, 1994). This assessment subsequently became known as the Professional Identity Essay (PIE) and has since been adapted for dentistry, medicine, law, and counseling (Bebeau & Monson, 2012). Interpreting the PIE requires training in how to assign an overall estimate of level of development, from less to more complex, or from external to internal motivation (Kalet et al. 2016). PIE can be used as a measure of group outcomes, or as an individual formative assessment in the context of the curriculum or co-curricular programming related to resilience, lapses in professionalism, or wellness. 

PIE coding produces an overall score for research or evaluation. This score is not reported to students; instead, a narrative summary and developmental feedback are provided. Instructors may revisit the topic at several points in the pre-clinical curriculum. The rationale for this approach is to help students become more aware of how competition for grades fuels the ways they make sense of assessments. As a holistic measure, PIE attempts to normalize the idea that we are all on a lifelong path of growth, and there is no "bad" score, per se. The PIE also provides instructors with a focal point for learning reflective writing skills. Using rubrics to guide students to write PIE essay responses in sufficient depth and honesty is helpful in ensuring PIE essays are meaningful (see, e.g., Wald et al. 2012). 

The PIE measures (1) developmental levels of the meaning of professionalism and the expectations of the profession as a moral enterprise (Bebeau, 2002); and (2) the structural or the developmental levels of mental complexity of Kegan (1994). The methodology for interpreting and scoring the PIE was further developed to improve construct validity and provide rubrics to rate specific themes emergent from the PIE related to professional role and resilience in constructing meaning when prompted by a hypothetical failure or setback (Monson, 2016, 2019; Monson & Hamilton, 2012; Kalet et al. 2016). 

Kalet et al. (2016) explored construct validation of the PIE methodology in medical education finding good rater agreement (inter-rater ICC .83, 95% CI [.57 - .96], intra-rater ICC .85, 95% CI [.50 - .93]) and positive correlation with the overlapping construct of moral judgment (Kalet et al. 2018; Monson et al. 2008; Monson & Bebeau, 2009). A construct validation study using the Subject-Object Interview (SOI) and PIE similarly showed strong rater agreement (Monson & Hamilton, 2012). 

In studies of the predictive validity of PIE, Kalet et al. (2017) found that from the matriculation in medical school through eighteen months of preclinical, PIE scores increased in the aggregate (0.460, X²(8) = 16.4, p = 0.04). In a predictive validity study of students over a three year period (n=351), Kalet et al. (2018) found thatPIF stage predicted small but significant increases in clinical communication skills measured three weeks into the curriculum, including (1) patient counseling (6.5%, p < 0.01), information gathering (4.3%, p = 0.01), organization and management (4.1%, p = 0.02), patient assessment  (3.6%, p = 0.04), and relationship development (3.5%, p = 0.03) skills. 

PIE is effective in meeting the criteria for good assessment in medical education (Norcini et al. 2011), e.g., 

Criteria must recognize the growing awareness of the intimate relationship between assessment, feedback, and continued learning… relevant and useful feedback must be provided in a way that encourages and supports the examinees’ progress. Ideally, this feedback would be adaptive to the individual, his/her place in the developmental continuum, and the broader system of assessment.

When used as a formative assessment in courses, instructors provide students with developmental feedback, and follow-up with additional reflective writing assignments or in-class discussions on the meaning of professional identity. The development of rubrics on specific themes produced by the PIE is underway, and allows instructors to involve students in self-assessment or peer discussions and feedback (Monson, n.d.).

Essential Readings for Learning to Code PIE Essays 

Bebeau, M.J. & Monson, V.E., (2012). Professional identity formation and transformation across the life span. In McKee, A. and Eraut, M. (Eds.) Professional Learning Over the Life Span:  Innovation and Change. (Springer) 

Berger, J.G. (2012). Key concepts for understanding the work of Robert Kegan. Retrieved on November 2, 2016, from http://www.shiftingthinking.org/?page_id=449.

Berger, J.G. (2002). A summary of the Constructive-Developmental Theory Of Robert Kegan. Retrieved on March 12, 2019, from https://wiki.canterbury.ac.nz/download/attachments/6358104/berger+on+kegan+narrative.doc

Cruess, R.L., Cruess, S.R., Boudreau, J.D., Snell, L., & Steinert, Y. (2014). Reframing medical education to support professional identity formation. Academic Medicine, 89, 11. 

Selected Research Articles

Kalet, A, Buckvar-Keltz, L., Harnik, V., Monson, V., Hubbard, S., Crowe, R., Song, H.S., & Yingling, S. (2017). Measuring and promoting professional identity formation early in medical school. Medical Teacher, 39(3), 255-261. Available from: https://www.researchgate.net/publication/311982407_Measuring_professional_identity_formation_early_in_medical_school

Kalet, A., Buckvar-Keltz, L., Monson, V., Harnik, V.. Hubbard, S., Crowe, R., Ark, T., Song, H., Tewksbury, L., Yingling, S.. (2018). Professional Identity Formation in medical school: One measure reflects changes during pre-clerkship training. Available from: https://www.researchgate.net/publication/323314347

Monson, V.E., & Hamilton, N.W. (2012). Ethical professional (trans)formation: Early career lawyers make sense of professionalism, University of St. Thomas Law Journal. Available at http://ssrn.com/abstract=1733282

Recommended Readings

Bebeau, M.J. (2002). The Defining Issues Test and the Four Component Model: Contributions to professional education. Journal of Moral Education 31(3), 271-295.
Bebeau, M.J. (2008). Promoting Ethical Development and Professionalism: Insights from Educational Research in the Professions, 5 U. St. Thomas L.J. 366, 386.

Berger, J.G. (2012). Changing on the job: Developing leaders for a complex world. Stanford, CA: Stanford Business Books. 

Cruess, R.L., Cruess, S.R., & Steinert, Y. (2016). Amending Miller's Pyramid to Include Professional Identity Formation. Academic Medicine: 91(2):180-5.

Kegan, R. (1994). In over our heads: The mental demands of modern life.Cambridge, MA:Harvard University Press.

Kegan, R. & Lahey, L. (2009). Immunity to change: How to overcome it and unlock the potential in yourself and your organization.Cambridge, MA: Harvard Business Press.

King, P., Baxter Magolda, M.B., Barber, J.P., Brown, M.K., & Lindsay, N.K. (2009). Developmentally effective experiences for promoting self authorship. Mind, Brain, and Education, v3 n2 p108-118. 

Lahey, L., Souvaine, E., Kegan, R., Goodman, R., & Felix, S. (1988). A guide to the Subject-Object Interview: Its Administration and Interpretation.
Mezirow J., & Associates. (2000). Learning as transformation: Critical perspectives on theory in progress.  
Norcini, J. et al. (2011). Criteria for good assessment: Consensus statement and recommendations from the 2010 conference. Medical Teacher,33: 206–214 
Rest, J., & Narvaez, D. (1994). Moral development in the professions, Hillsdale, NJ: Erlbaum.
Rest, J., Narvaez, D., Bebeau, M.J., & Thoma, S. (1999a). Postconventional moral thinking: a neo-Kohlbergian approach.Hillsdale, NJ: Erlbaum.

Rest, J., Narvaez, D., Bebeau, M.J., & Thoma, S. (1999b). A neo-Kohlbergian approach: The DIT and schema theory. Educational Psychology Review, 11(4), 291-324. 
Wald, H.S., Borkan, J.M., Taylor, J.S., Anthony, D., & Reis, S.P. (2012). Fostering and evaluating reflective capacity in medical education: developing the REFLECT rubric for assessing reflective writing. Academic Medicine, 87(1):41-50. doi: 10.1097/ACM.0b013e31823b55fa.

Wynia, M.K., Papadakis, M.A., Sullivan, W.M., & Hafferty F.W. (2014). More than a list of values and desired behaviors: A foundational understanding of medical professionalism. Academic Medicine, 89:712–714.