New Research Introduces New Way to Gauge Efficacy of Well-Being Improvement Programs

Madison Agee

Organizations are searching for new ways to determine the efficacy of well-being improvement programs, a search that new studies from the Healthways Center for Health Research help to support. Published in Population Health Management, “Regional Economic Activity and Absenteeism: A New Approach to Estimating the Indirect Costs of Employee Productivity Loss”, introduces a new method for measuring the economic impact of work-related absences. Another recently published study makes an additional important contribution to the science of measuring the effectiveness of well-being improvement programs.

The study, “Extending Coarsened Exact Matching to Multiple Cohorts: An Application to Longitudinal Well-Being Program Evaluation within an Employer Population”, appeared in Health Services and Outcomes Research Methodology (HSORM). As with the regional productivity loss (RPL) study, this study expands upon an existing methodology and introduces a more robust and comprehensive measure useful for gauging the efficacy of well-being improvement programs.

The study focuses on a methodology known as Coarsened Exact Matching (CEM), which is used to remove imbalance (i.e., biases) between two groups and therefore create an “apples to apples” comparison of impact between them over time. To measure the effectiveness of a well-being improvement program, CEM compares two groups – those who participate in the program and those who do not but are of similar prospective risk – and systematically removes imbalance between the groups based on shared characteristics such as age, gender and disease status. This process enables an unbiased estimation of the program’s causal effect.

This “binary group” application of CEM for measuring the impact of well-being improvement programs is useful, but does not reflect the reality of these programs in which there is usually a wide spectrum of participation levels within a population. The recent HSORM study expanded the scope of CEM beyond the traditional binary group application to determine whether CEM could be an effective method for quantifying causality among multiple groups with different levels of program participation.

The study was able to show that the new multi-category specification of CEM accounted for and removed more bias than the traditional binary group application. Thus, multi-category CEM was found to produce a more comprehensive and robust measure of the impact of well-being improvement than the traditional application.

In addition to showing the validity of a multi-category CEM application, the study also demonstrated that higher levels of participation within a well-being improvement program led to a larger improvement in overall well-being. Well-being among non-participants improved 0.57 points, while well-being improved 1.32 and 1.48 points among the low- and high-intensity participants, respectively.

Together with the RPL study, these two studies make important contributions to the scientific literature and provide employers with new methodologies for measuring the impact of their well-being improvement programs.

Topics: Measure Wellness Methodology Science and Research ROI