Pacific Gas and Electric Company, a subsidiary of PG&E Corporation, is one of the largest combined natural gas and electric utilities in the United States. Based in San Francisco, PG&E has approximately 20,000 employees. The company provides natural gas and electric service to 16 million people throughout a 70,000-square-mile service area in northern and central California.
PG&E Academy has spent years investing in and improving its training programs to ensure employee competence across all critical roles in the company. Executive leadership increasingly requires measurable data that demonstrates training’s value and contribution to attaining business goals. To position themselves as critical business partners, PG&E Academy has instituted a portfolio approach to measuring training effectiveness.
This portfolio uses a phased approach that includes three categories of measurement:
All Academy training programs include Expectation measurement. High-risk, high-consequence programs are subsequently evaluated for performance outcomes based on the following criteria:
The Academy uses traditional L3 assessments administered through focus groups, surveys to employees and supervisors (30/60/90 days), and field observations. This L3 data, combined with L4 data from the QA/QM team is consolidated into the Academy’s performance outcome initiatives and used to demonstrate how training programs are contributing to meeting the business goals and objectives of the company.
Currently, 22% of the Academy’s portfolio is evaluated for performance outcomes (L3 and L4). The longer-term goal is to focus exclusively on training initiatives that can be aligned to business metrics, with the idea that those that cannot may not be necessary and/or needed.
Because L3 assessments can be difficult and expensive, the Academy uses predictive L3’s as another method of measurement. Four questions are added to L1 surveys that help predict knowledge transfer from training to the field. While not perfect, student responses to these questions are a proven indicator of whether the training received will be applied back on the job. The following questions are rated on a scale of one to five:
The Academy’s target for predictive L3 average ratings across courses is 4.52. According to Metrics that Matter, with the addition of a 60-day post training survey, the predictive L3 ratings are within 5% of observed L3’s across all industries.
The Service Mechanic scorecard shown below highlights how training contributed to increased capacity and quality of work. This level of transparency and detail has enabled the Academy to demonstrate the value of training and build enhanced credibility as a strategic business partner. The result is an increase in executive support of training programs and ongoing collaboration between the lines of business and the Academy.
This demonstrates value to the impacted audience, and positions training to become a valuable resource for other groups.