OE21 C.1 DATA ANALYSIS > ACTION PLANS > EXCELLENCE

Strategic Objective:

Strategic Objective:

Create Solutions to Important Organization Problems

Quality Objective:

Quality Objective:

Ensure that Solutions are Reliable, Accurate and Timely

Responsibility:

Approved: DD-MMM-YY

Responsibility:

Approved by: (Name) Chair, Leadership Focus Team (LFT)

All Focus Teams (LFT, CFT, OFT, WFT)

VALUE ADDED       

VALUE ADDED: - Data Analysis to Action Plans

 

  1. Improve process of data collection, analysis, problem and solution implementation.

  2. Improves synthesizing, analyzing, and interpreting quantitative and qualitative data, turning data into useful information, and acting operationally and strategically.

  3. Improves impact and benefits analysis, requirements definition and solution creation.

  4. Improving analysis of performance factors listed in Baldrige Excellence Commentary 4.1.

  5. Makes excellent use of OE21 Innovator and OE21 Project Manager (simple project plans).

  6. Ensures that problem definition and solution creation processes are logical and effective.

  7. Ensures that all OE21 Focus Teams have a common and shareable approach. 

Applicability of C.1 Data Analysis to Action Plans Standard

This C.1 Common Standard applies to the majority of the OE21 Surveys and their associated Data Analysis spreadsheet models, excluding the following:

  • LFT P.1 Organizational Profile Part 1 and 2 surveys

  • LFT 2.2 CPA CFO Performance Assessment (survey)

  • OFT 6.2a SM Team Charter (survey)

  • OFT 6.2a SM Data Log (survey)

  • OFT 6.2a SM Story (survey)

C.1 Process Chart

Figure C.1-1 Process Chart for Data Analysis to Action Plan

C.1 Implementation Instructions

Task C.1.1 Conduct Surveys and Export Results – Each Focus Team implements its unique set of OE21 Standards and decision support tool, including setting up and conducting surveys. All focus teams use the common OE21 Survey Process instructions to execute the surveys and to apply the Survey Methods application properly. Once the surveys are complete, the OE21 Facilitator (FAC) or another Focus Team member exports the Survey Methods results into a flat excel file suitable for insertion into the appropriate OE21 spreadsheet model. Most spreadsheet models have a Matrix tab in an Excel Table format and suitable for analysis. 

Task C.1.2 Examine OE21 Model and Address Key Questions - The OE21 FAC inserts the Survey export file into the appropriate OE21 spreadsheet model. Once this is done the FAC contacts the Focus Team to setup an online analysis meeting to examine results and to address three key questions. The results are displayed on a tab of the OE21 spreadsheet model. See example Figures C.1-2 (left side) and C.2-2 (right side) of the matrix model used in OE21 3.1 Customer Satisfaction and Value Matrix.

Figure C.1-2 Matrix Table for Data Analysis to Action Plan (left side)

Figure C.1-3 Matrix Table for Data Analysis to Action Plan (right side)

Figure C.1-2 Observations: (We suggest you walk through each of these carefully)

  • The averages of each attribute appears at the top of columns C to L; 

  • The weights (10) appear in row 2 of columns C to L); note weights can be unique for each attribute

  • The service being rated is Senior Hockey Training (note that other services/products could appear)

  • The Customers (names) who responded to the rating are listed in Column B 

  • The location (zip) age group (senior/adult/youth-adult/ or youth) and sex (M/F) appear in C, D, E

  • The customer Ratings (1-5) for ten attributes appear in columns F to L 

  • The Scores for each customer (all rated attributes) appear in column M 

  • The Scores are calculated as (Weight x Rating) and summed for all attributes per customer

  • Columns U, P, Q and R contain date, measure, target and max score pulled from the Table

  • The 3-month history are hand-entry based on earlier survey data collected

  • It is significantly good that the 3-month history shows the measure (231.3) is moving steadily toward the target (300). Since the measure in column P is the average of all scores, it provides a somewhat global indicator of how the customers that responded to the survey are feeling. 

  • Once the measure gets within 10% of the target (at 270), the focus team should raise the target to a higher level, which is like "raising the bar" on a high jumper training for perfection.

  • Since the highest possible score is 350, the customer ratings are about 66% (210/350) of the way toward achieving what we call Customer Excellence - and Customer Excellence IS the GOAL.

  • The list of CUSTOMER SUGGESTIONS is pulled from the Survey tab. These are highly valuable inputs to understanding the problem(s) and in setting the stage for possible solution actions.

  • In Figure C.1-3 the focus team has added categories alongside the customer suggestions

  • The category counts are: schedules (11), hockey sticks (4), menu (4) and uniforms (4) 

  • Likely conclusion: Schedules are not the only issue that customers are unhappy about 

Key Questions for the Focus Team - At this point in the Focus Team Meeting, the participants should address the first of three key questions:

 

  • Q1 - Do we have sufficient evidence that the data collected are valid?

  • Q2 - Do we have sufficient data and information to understand the problem?

  • Q3 - Do we have sufficient data and information to decide on the solution?

 

 

Q1 Considerations: Suppose that the organization has a customer population of about 1875 and only 30 of the customers have responded to the survey. Is 30 of 1875 (1.6%) good enough? To learn more the Focus Team might use an online Sample Size Calculator to get a better idea of how many responses are really needed. Figure C1-4 shows that to get a reasonably high confidence (95%) that you might need to survey at least 680 (36%) of the customers.   

 

This free sample size calculator includes very good explanations for determining sample size, confidence level, and confidence intervals for a given population. 

Suppose you want to survey the 270 million people in the USA. Would you believe that only 1067 people could be surveyed and you would have high confidence that their responses are valid. (Try it yourself).

Figure C.1-4 Sample Size Calculator Example

Conclusions for Q1: The organization needs to offer benefits to its customers to get more responses. The objective is to get a lot more than 30 responses and hopefully a lot closer to at least 600. If the customers still respond in low numbers we have to live with it and to be especially careful with assumptions about how well we understand the "global" population of all customers. 

Q2 Do we have sufficient data and information to understand the problem? Suppose we repeated the survey, added benefits to those who respond, and received about 400 responses.

 

In this case the considerations might including the following:

  • We can count the number of similar suggestions that the customers offered

  • We can estimate the percentage of similar suggestions (let's suppose it comes to 55%)

  • We can use the Sample Size Calculator to get an idea of how the total population is likely to offer the same suggestions

  • We can observe the AVERAGE scores at top of each of the ten Attribute Columns (in Figure 3.1-2 Column G the average for AVAILABLITY attribute is 2.8 of 5 (which is fairly low). This might correlate with the suggestions that the Senior Customers are telling us that the senior hockey training is UNAVAILABLE at the times that many of them want.

  • We must not overlook the other three issues in customer suggestions (menu, hockey sticks, and uniforms)

Based on these observations we might postulate that the PROBLEM is fairly well understood:

  • The seniors want the hockey lessons to be available Wed at 7PM.

  • The seniors want better menus, hockey sticks and uniforms

At this point the focus team has an idea of the impact of making schedule changes (e.g. need more instructors, need to shuffle other customer schedules, etc.) as well as the cost and feasibility of adding new menu items, uniforms and hockey sticks, however, it may be useful to dig a little deeper before we jump ahead to solutions.

One way to do this is speak with ALL senior hockey lesson customers. We might talk to them as they come in and out of the Sports Center for lessons. That will take time but may well be worth it - especially if the impact of the desired schedule and other changes is significant in cost and possible non-senior hockey customer shakeups.

Another way is to use the one question Voice of Customer survey process. This short version usually brings more responses, including customer summary ratings and suggestions.

Q3 Do we have sufficient data and information to decide on the solution?

An ideal answer is YES if our personal interview or short voice of customer survey of seniors gave us high assurance that we understand the problems. If NO, then we need more information to understand the real issues. Let's assume that we think we are ready now to create solutions that meet senior customer needs, are affordable and that can be implemented without negative impact on other customers needs and desires.

Q4 Do we have sufficient data/information to understand the Solution Requirements? If we think we know the solution well enough to define it clearly and then go implement it, then we should do exactly that. But what if there are alternative solutions? In that case, we might use the ACI-Innovator process. See Task 6.1d-3 in OE21 Standard 6.1d Innovation and Risk Management.  If you follow the instructions in 6.1d-3, you will perform several steps:

Step 1 - Create a Target Question. In our solution scenario for senior hockey schedules, our question might be stated like this:

  • What is the best way to make sure seniors receive their hockey lessons at times they want?

Step 2 - Edit the survey instructions inside the survey so that responders understand what to do.

Step 3 - Launch the ACI-Innovator survey and send it to all focus team members and workers who participate in the senior hockey training process. 

Steps 4, 5 and 6 - Responders provide 1 to 10 ideas and categories for each idea, in response to the Target Question, then they submit the survey.

Steps 7 and 8 - After sufficient responses are collected in the survey, the OE21 Facilitator (FAC) closes the survey and exports the results (an excel file containing ideas and categories). Next the FAC inserts the survey export file into the ACI Innovator spreadsheet model. 

Next, the focus team meets online and reviews the ACI Innovator results. Now we have additional ideas about the solutions for hockey schedule changes. This is where the good ideas usually surface and people begin to agree on how solutions will be implemented. In some cases, it may be necessary to create additional Target Questions and repeat the ACI Innovator process to drill down to learn more. When

Q5 Do we have sufficient data/information to understand the cost and schedule impacts for the solutions? Ok, let's assume that at this point in our analysis, we believe that the solutions to losing our senior customers boil down to four tasks:

  • Task 1 - Shuffle existing hockey schedules to make senior hockey lessons available on Wed 7PM

  • Task 2 - Add new menu items to Snack Bar offerings (food, beverages, snacks)

  • Task 3 - Purchase new hockey sticks

  • Task 4 - Purchase new hockey uniforms

The next step is to create an estimate of the cost of implementing these tasks. A good way to do this is to use the OE21 Project Manager (simple plan) tool, as shown in Figure C.1-5. 

Figure C.1-5 Simple Plan Example showing Cost and Schedule Estimates for small project

Figure C.1-5 shows an example of how our rough estimate for this data analysis scenario might look.

 

Observations:

  • The project is less than one month in duration

  • The project rough estimate cost is $15,900

  • $13,850 of the cost is for purchases

  • About 54 hours of labor are estimated

After the focus team reviews and agrees that the Simple Plan cost and schedule looks reasonable, the final question comes up.

Q6 Does the organization have sufficient resources to implement the solution?
 

This is a question that involves the entire workforce. The reason is that all additional tasks added to ongoing workloads often has a "ripple effect" that may cause conflicts with what to work on at any particular timeframe. For example if the hockey team is fully occupied with daily tasks of training customers, administering hockey team competitions, getting equipment ready for customers, cleaning up and restoring everything after training or competition events, then asking them to stop and do a few new tasks may have what we call "unintended consequences." 

Some data analysis and improvement projects may have a significant impact on ongoing workloads, while others may have little or no impact. 

In the OE21 system, there are four (4) excellence projects, each administered by one focus team:

The OE21 approach is that while each of the four focus teams (LFT, CFT, OFT, WFT) has their own project, it is also a good idea to elect the most qualified and experienced person in the organization to be the "Super Project Manager" the SPM. The SPM is normally a PMI-certified professional project manager who knows exactly how to integrate the four excellence projects into one master project that can be used to plan how ALL resources should perform their assigned tasks, including allowance for the times they must spend on the most common daily work tasks (hockey, figure skating, admin, etc.). The SPM knows how to apply "critical path" project techniques to help determine who does what and when - organization wide. 

The OE21 approach at this point is to follow a few additional guidelines that help control the planning, estimating and controlling of the four excellence projects. As new projects like the one in Figure C.1-5 are created, they flow through the following process:

  1. The Lead Focus Team submits new Solution Action Plans (like C.1-5) to the SPM for further analysis.

  2. The SPM works with the task performers to finalize labor hour and non-labor purchases, including both schedule and cost estimates. 

  3. The SPM may setup temporary changes to the master excellence project to see how the new Solution Action Plans impact the current master plans

  4. The SPM submits the updated version of the Solution Action Plan (C.1-5) to the Chief Financial Officer (CFO) who has the responsibility of approving the budgets needed for the new Solution Action Plan(s).

  5. If the CFO approves the budget, then the CFO submits the final Solution Action Plan to the CEO or President of the organization who will either approve, disapprove and ask for changes.

  6. Once the CEO approval is completed, the Solution Action Plan becomes part of the appropriate excellence project and everyone involved is notified to proceed ahead with implementation, including labor hours and purchasing of materials or other non-labor items.

  7. If the CEO disapproves, then the Solution Action Plan is terminated.

FINAL NOTES - Now that you have reviewed this OE21 Data Analysis > Action Plans > Excellence standard, you will find that all this applies to the majority of the OE21 surveys and tools. The message is this: STUDY, LEARN AND APPLY THIS OE21 STANDARD!