A Look at Well-Being Measures to Evaluate
As a corporate communications trainer I work with many people on a short term basis. Over a two day session I can see an enormous impact on their personal and professional growth. Invariably I am always asked, “Does this training really work?” Aside from the pile of positive evaluations I have received I do not have a scientific answer for them. Implementing well-being measures can bring scientific rigor to my field and can fine tune the work we do to serve the client in the most effective way possible.
I. Corporate Training
The company we work with has a policy requiring their high performing women, after a selection process, to take part in our training. It is this training program that I will be evaluating. We work with high performing women in cohorts of 20-25 at a time. They are involved in programs that meet approximately three times over the course of the year. In the interim my company provides individual executive coaching, 360 feedback and on the job assignments. During our training sessions we help them focus on how to be strategic in their careers, how to become known internally, how to develop business, how to promote their accomplishments, give them interaction with successful role models and how to incorporate the ‘do’s and don’ts’ of executive presence. We specifically focus on the specific technical skills theater professionals use to project presence. We look at voice, physicality, word choice, non-verbal communication and energy. We also focus on how to combat performance anxiety when giving a presentation.
II. Goals of Well-Being Measures in Training
Because of the number of trainers and the different content of our training, we divide the training sessions that occur in multiple offices all over the country. But since we only see them one or two times in a year, how do we know the training has had a lasting impact? The primary goal with incorporating well-being measures would be to determine the impact of these training programs on the participants and what specific elements were the most effective. Our secondary goal is to discover if adding these skills to their ‘professional tool belt’ added to their level of subjective well-being at work by testing job satisfaction, productivity, resilience and positivity.
Diener, E., Lucas, R. E., Schimmack, U., & Helliwell, J. (in press) mention a meta analysis of sixteen studies (as cited in Petty et al., 1984) that examined the association between job satisfaction and job productivity. In this study they found that the correlation between these two variables was .31. Other studies mentioned had a similar numerical correlation. Diener again clarifies that a simple correlation does not tell us about the “direction of causality” (in press). The majority of the studies reviewed by Diener, et al. reveal that job performance affects satisfaction. If this is the case, then there is a stronger need for the training programs we do. The better these women perform (i.e. the more efficient they are with managing teams, the more clients they are able to serve, and the more work they sell) the more satisfied they will be which can lead to higher profits for the company (as cited in Harter, Schmidt, & Keyes, 2003).
Diener, et al. (in press) clarify that if there is a causal effect of job satisfaction on performance, it is likely to be confounded with other variables such as the relation between pay and performance and the self-esteem of the employee. From anecdotal comments I believe that the pay and performance correlation is at a high enough level to be satisfactory for the women, but I would need to evaluate this. We would also need to include self-esteem measures before and after our training programs to verify the satisfaction to performance connection. Here I would draw on Dianne Tice’s estimable work in this area (Tice, D., Gailliot, M., 2006).
In addition to retention of skill building and increased job satisfaction, I would like to evaluate resilience after our programs. These women work incredibly long hours and I believe resilience is the one factor that keeps women at the firm versus stepping off of the work treadmill for jobs that have fewer hours. This is something that I would like to evaluate. While we can add facilitating disputation and the ABCDE method to our programs in the future (Reivich, 2003), I would like to evaluate the base level of resilience these women have and if it increases after our programs that are not addressing resilience directly. In union with this, I would also like to evaluate positivity. Based on Barbara Fredrickson’s work, I see that positivity also leads to higher functioning teams and to more profitable teams (2009). If we can test for baseline positivity and compare to the results we find in successful teams, perhaps we can also add a training session on positivity to our programs.
III. Measurements
Currently we have subjective written evaluations after every two day session. The questions included are: What worked well in Section A, B, C? What did not work so well in Sections A, B, C? What would you improve for next time? And our one quantitative measure: Rank the effectiveness of this program on a scale of 1 to 10. Our goals are not only to analyze the impact of the programs on a subjective level but on an objective level as well. In addition to the questions we currently ask about the qualitative impact of the programs, we wish to add their level of positivity, their resilience, their level of job satisfaction after these programs (controlling for self-esteem) and to analyze the objective results of well-being measures.
Their resulting job performance can be objectively analyzed in a number of ways. We currently do measures across the firm to discover the retention and promotion percentage for the women who have gone through our programs. Job retention, in this instance, is measured by the number of women who stay at the firm from the previous year. These numbers have grown with each year of the program, which is impressive, but we still need more rigor in order to determine the reliability of our programs. I would like to add a quantitative measure evaluating profitability for our high level women that records how much business they sell after our programs.
In order to obtain the subjective and objective measures, I propose that we create a scale that covers the subjective measure and is specific to the training and the firm. I will draw on Diener, et al’s The Satisfaction With Life Scale (SWLS) as a model and adjusting to use with more narrow questions (1985). We will have to get a baseline before the beginning of the year-long program and follow up with one at the end when we look at the objective measures of success. In addition, we will design a scale tailored to each specific program to determine the effectiveness of each element of the training session. And finally, we can obtain self report and 360 feedback at the end of our year long program to assess which training elements were retained over the course of the year. After obtaining the information about job satisfaction, productivity, resilience and positivity we could then use that information to decide which programs to retain the following year. If there was one element that particularly resonated with the women we might be able to add on the job assignments or personal coaching around the issue. In addition, we would have further follow up on the most salient elements to see how we could tailor those programs to the women’s needs.
IV. Objections
The number of measures proposed above is ambitious. It will take patience on the part of the participants and a certain amount of our precious training time to have the participants fill out these evaluations. Email after the session is an option, though with busy schedules it is more effective to capture their information at the training session. It is also difficult to decipher which exercises or sessions were the most effective for which people. It would be impractical to give a survey before and after each two hour session. While resilience and positivity can add to the effectiveness of these professionals alone and in teams, our programming does not specifically address these issues. An evaluation of these elements might be the basis for designing future programs for the women. And finally, the objective measures might leave out productivity that cannot be measured in goods and services sold. The work these women do in mentoring each other and leading teams may or may not lead to an increase in the company’s receivables at the end of the year, but it does add to employee retention which is a major goal of our programs.
V. Conclusion
Well-being measures can be an effective tool for highlighting what is effective in corporate communications training programs. To determine what specific sessions impacted our participants in the realms of job satisfaction, job productivity, positivity, resilience, retention and promotion would be a boon not only to our company but to the field of training and development. Too frequently we have been accused of lacking rigor in our research and our outcomes have been questioned. In order to combat these accusations and remain effective for our clients we will have to employ both qualitative and quantitative measurements of well-being assessments.
References
Diener, E., Emmons, R., Larson, R. & Griffin, S. (1985). The satisfaction with life scale, Journal of Personality Assessment, 49(1), 71-75.
Diener, E., Lucas, R. E., Schimmack, U., & Helliwell, J. (in press). Well-Being and Policy, Chapter 4: Contributions of Well-Being Measures; Chapter 10: Work, the Economy, and Well-Being: Policy Examples.
Fredrickson, B. (2009). Positivity: Groundbreaking research reveals how to embrace the hidden strength of positive emotions, overcome negativity, and thrive. New York: Random House.
Reivich, K., Shatte, A. (2003). The Resilience Factor: 7 keys to finding your inner strength and overcoming life’s hurdles. New York: Broadway Press.
Tice, DM. (2006). How Self-Esteem Relates to the Ills and Triumphs of Society. In Self-esteem issues and answers: A sourcebook of current perspectives. (pp. 412-).New York, NY, US: Psychology Press.