OE21 Guide to Assessment/Survey Process
OE21 Guide to Assessment/Survey Process
The OE21™ Guidelines require many assessments (surveys), with questions and rating scales. This practice provides instructions for using the Survey Methods programs to design, launch, and analyze assessments. An example assessment is included to communicate the process.
GENERAL GUIDELINES FOR OE21 SURVEYS
a. Decide who the responders will be, and collect their names, work unit or department name and email addresses. If possible get their phone numbers.
b. Decide on how long you will give them to respond to the survey. Example: The P.1 Organizational Description survey responders should be given about two days to respond.
c. When you launch the survey, email the survey link and a short message about the survey to the responders.
d. SurveyMethods can be used to determine when the responders have finished their surveys by logging in, going to the survey, looking at the value (count) under the Analyze column and next to the small bar chart icon. The number of responses is the count.
e. If the survey response time is passed and you have fewer inputs to the survey then the number of responders, contact (phone or email) the late responders and ask them to finish by the end of the next workday.
f. After you close the survey use SurveyMethods to export the survey to an Excel file and save or upload this file to the organization's cloud or another storage site. (IT may help with this).
DESIGN - LAUNCH - ANALYZE INSTRUCTION
Step 1: Estimate the Sample Size for the Assessment
Sample Size Guideline: In cases where a large number of people will respond to a survey, the use of a Sample Size Calculator is advisable to reduce the number of responses without loss of significant information or data. As of the release of this OE21™ Guideline, a sample size calculator is available on the web at this URL:
Use the sample size calculator to determine the size of your sample group of Stakeholders in cases where you have large Stakeholder sizes. In most cases, use a (+/-3%) confidence interval and a 95% confidence level.
Example 1: Suppose you wanted to survey the number of TV viewers in the USA, and the population of TV viewers is estimated to be 290,000,000. The sample size calculator shows that if only 1067 TV viewers respond to your survey, your Confidence Level would be 95% as accurate as if all 290 million responded.
Example 2: Suppose you wanted to survey the company workforce of 500 workers. The sample size calculator shows the need to survey at least 341 workers to maintain your Confidence Level at 95%.
Less than 100 workers in a company would require close to a 100% Sample size to maintain the Confidence Level at 95%.
Step 2: The OE21 surveys use Survey Methods (SM) to collect survey and quiz/test data and export the results to an Excel file format. The Excel format is designed to integrate with corresponding OE21 spreadsheet models that require SM surveys for source data.
NOTICE: Your organization may receive copies of all SM surveys in the OE21 system, depending upon the licenses you purchased. If you have questions on this please email them to
Login to www.surveymethods.com, and follow onscreen instructions to design a new survey, or use an existing survey. During the survey design process, you name the survey. You may be invited to edit an "Introductory Statement," which is normally used to introduce the purpose of the survey to the respondents. The following is an example.
Example of an Introductory Statement: To improve and maintain excellent services and products for our customers and our surrounding communities, we request that you respond to this survey. Your responses will help us determine your specific issues or recommendations concerning our products, services, and operations. We promise to respond to your concerns in a timely and appropriate fashion. We greatly appreciate your taking the time to complete this survey and ask that you complete no later than 24 hours from the time this email is received.
Step 3: Design the Survey. The OE21™ guideline for assessments (surveys) normally uses a combination of a measured rating question, followed by a narrative response question. The example in Figure A-1 is a captured screen from Survey Methods:
Figure A-1 Example of Rating Question Paired with Narrative Question
Figure A-1 shows a paired set of questions. One question captures a rating measure on a scale from 1 to 5. The other question captures narrative suggestions or ideas that the responder feels that deserve the highest rating of 5. If the rating is already 5, question 2 may be left empty or used to bestow compliments. When you have designed your assessment or survey questions, you should proceed to Step 4.
Step 4 - Target the Assessment. Determine the list of people or organizations and the EMAIL ADDRESSES, of those that should respond to the survey. If the list is lengthy (500 or more), than use the Sample Size Calculator to reduce the number of respondents who receive the survey.
Step 5 - Test the Assessment. Before emailing the survey to your respondents, test it by sending it internally to other members of the OE21™ Focus Teams (LFT, CFT, OFT, WFT). Ask them to respond to the survey as if they were "wearing the hats" of the target response group. If you have any improvement suggestions from the focus teams, then make the necessary changes or improvements.
Step 6 - Launch the Assessment. Use the Survey Methods program instructions to "launch" the survey. The preferred method is to create a web link that takes a respondent to the online survey. The link is normally copied into an Email. The email may include an Introductory Statement suggested in Step 2. Use the Email system to send the survey to the selected respondents.
Step 7 - Monitor the Survey. After the planned time allocated for respondents to take the survey, use Survey Methods to determine if the desired number of respondents have submitted their responses. If the time-limit has expired and too few people have responded, you may extend the time-frame with a reminder email.
Step 8 - Close the Assessment. When you have sufficient responses, then use Survey Methods to close the survey. This action blocks stragglers or unwanted responders.
Step 9 - Quick Look the Assessment Results. Use Survey Methods internal instructions to view the survey results. Survey Methods has "magnifying icons" nearby survey results. Click these icons and see more results details. After your quick look, proceed to Step 10.
Step 10 - Export the Survey Results for Further Analysis. Use the Survey Methods internal instructions to export the survey results to a spreadsheet file for further analysis, as presented in the following example.
Rating Score Analysis - OE21 spreadsheets have formulas in the SurveyData tabs. These formulas are used to average all rating points to get a "total score" for the survey. For example, the average survey score and date of the survey are tracked as a "measure" in the OE21™ Performance Measurement System. The average survey score can be compared to a target such as "5" or "4.5" - FYI, 4.5 / 5 = .9 or 90% target.
The survey Rating Total Score and Date may be used as a Key Performance Indicator (KPI) in the Performance Measurement System. The survey Rating score versus a target of "5" provides a good trend chart indicator for Performance Reviews.
Narrative Analysis - Create a new tab in the spreadsheet for narrative analysis. Copy and paste narrative suggestions or ideas into "a single column list" then add "categories alongside each idea as shown in Table A-1.
When all ideas are categorized, then you can use a spreadsheet to "sort" ideas by category. This group's similar ideas together and makes the overall survey results more useful.
Example Narrative Responses from Surveys:
The XYZ organization products are excellent and appreciated in our areas
The XYZ customer service has a poor telephone complaint process
The XYZ salespersons do not talk to the product support teams
We cannot directly communicate with XYZ support reps
The XYZ traffic creates too many accidents at 2nd & Lee streets
Step 11 - Create Survey Results Report and Action Plans. Based on results of ratings (average score) and narrative ideas sorted by categories, you should have enough data and information to decide whether or not you need to create a survey Results Report or an Action Plan, with these contents:
Summary Rating Score (total or averaged) Score (e.g., 3.5 of 5 converted to percent - 3.5.5 = 70%)
Major Findings Summary - with references to the number of similar idea categories or other significant narrative ideas reported.
Candidate Strategic Objective - If the survey was one of the strategic planning assessments suggested in Policy 2.0 Strategy Excellence, then the results of the survey may yield candidate Strategic Objectives. These findings may be a significant part of the Assessment Results Report and could become a new Strategic Action Plan.
Action Plan - Basic list of activities that the survey results suggest would be beneficial, including task title, brief description, suggested completion date and suggested focus team to be responsible for the task.
Step 12 - Upload Assessment Report. The OE21™ focus team responsible for the assessment (survey) should upload the Assessment Results Report, and Action Plans to the organization's Intranet, and alert the other focus teams of its availability.
- End of this Guide -