Chapter Four: Organizational Health Surveys

Research Methodology

The first step in this phase of the project was to conduct an organization-wide survey to assess employees' attitudes and perceptions about organizational issues. Two surveys were developed; one for managers (see Appendix B) and the second was for all other employees (see Appendix C). The management and employee surveys contained the same types of questions, although they were worded slightly different, with some questions in the employee survey being focused specifically on the individual's job.

Both surveys included two main sections. The first section contained questions assessing general organizational issues, including structure, climate, supervision, innovation, human resource development (HRD), performance management, accountability, technology, communication, and motivation and retention. Some of these issues or dimensions were included to gain more information on topics that seemed to stand out in the interviews we conducted in Phase 1; others were included to assess general organizational health, in accordance with standardized instruments, such as the Survey of Organizations. ⁽⁵⁾ The second section of the survey contained questions that focused more specifically on issues related to employee motivation and retention, including job characteristics, organizational commitment, career development, and compensation.

5. Taylor, J. & Bowers, D. (1972). Survey of organizations: a machine scored standardized questionnaire instrument. Ann Arbor: Institute for Social Research, University of Michigan.

The surveys were comprised of several different items. Various statements requiring ratings, open-ended questions, and demographic items all were included in the surveys. Ratings of statements were based on a one to five scale, with a rating of one representing strongly disagree and a rating of five representing strongly agree. All statements were written in the positive frame of reference to avoid confusion about the statements' meaning. Open-ended questions asked for respondents' opinions about how to deal with motivation and retention issues. Responses to these questions were categorized and summarized to protect the confidentiality of the respondents. The demographic items included in the survey assessed whether the employee was located in the Field or Central Office, the tenure as a NDDOT employee, gender, age, level of education, and functional job category.

Once an initial draft of the survey was completed, it was sent to every member of the technical advisory committee for their review and input. Some members of the committee gave constructive feedback for changing the survey in various ways. Suggested changes were made before the final draft of the survey was sent to the management team at the NDDOT for approval.

Surveys, which were accompanied by a letter from the NDDOT Director encouraging participation, were distributed to every employee at the NDDOT in January 2002. Employees initially were given one week to return the surveys in the business reply envelopes to the Upper Great Plains Transportation Institute. However, the response rate in the first week was less than originally predicted. To increase the final return rate, employees were given two extra weeks to return their surveys. Fifty-two management and 968 employee surveys were originally distributed. Of those surveys, 39 manager and 672 employee surveys were completed and returned, giving an overall return rate of 70 percent.

Survey Results: Demographics

Although much care was taken to limit the amount of demographic information that was requested on the survey, many people left several items, or the entire section, blank. The goal was to get enough information that important comparisons could be made, but to avoid specific requests for information that could identify a certain individual or small groups of individuals. Despite the effort to protect anonymity of respondents, the results indicated that between 4 percent and 7 percent of the demographic information was missing or blank. The demographic information available from the completed management and employee surveys is displayed in Appendices B and C, respectively. A brief examination of the demographics of the survey respondents showed that they matched those of the population of NDDOT employees as a whole. As a result, it is reasonable to generalize the survey results to the entire population of the NDDOT.

Survey Results: Section 1

Responses on the first section of the survey were averaged across all respondents. Responses on the questions for each dimension also were averaged to obtain overall ratings. As a result, the mean or average response was obtained for each of the dimensions measured in Section 1. Analyses were conducted to ensure reliability of each of the dimensions. These analyses were calculated to show that all questions assessing a dimension were strongly related to one another. Results indicated that each of the dimensions had an acceptable level of consistency or reliability among the questions making up the dimension. The following table displays the mean responses (from ratings on the 1 to 5 scale) of employees and managers on each of these dimensions.

Survey Results: Section 1

By looking at the means in the table, one can see that there were areas in which the organization was doing well. These organizational strengths all had means over 3.2 on the 1-5 scale. For employees, the areas of strength in the organization were climate, supervision, and technology. In this project, organization climate referred to work environment and co-workers. The supervision dimension was assessed by whether supervisors encouraged and provided help and training to employees for the purpose of improving their performance. The technology questions focused on the level and timeliness of individual support. Managers also thought that organizational climate and supervision were strengths, but instead of technology, saw communication as an additional strength. Communication was assessed by questions about the access to other employees, voicing opinions, dealing with conflicts, and generally being informed. The actual number of employees who gave each response for every question in the communication and other dimensions are displayed in Appendix C. The management responses are documented in Appendix B.

There also were some dimensions that were rated more negatively, having means under 3.0 on the 1-5 scale. The dimensions having the lowest ratings for employees were human resource development, performance management, and accountability. Human resource development questions in the survey assessed the recruiting, hiring, developing, and promoting of employees. The performance management dimension included items assessing whether there was an appropriate difference between pay to high performing and average employees, whether the performance assessment system was perceived as fair and met the NDDOT's needs, whether rewards and recognition were prevalent, and whether performance standards were clear. Accountability questions focused on formal authority and decision making, rewards for good performance, and reprimands for poor performance. The lowest rated dimensions by managers were performance management and accountability, as well. However, managers rated HRD relatively positively, showing a large discrepancy between manager and employee ratings.

Section 1: Different Results Based upon Demographic Information

Analyses were conducted to look at potential differences among the job categories included in the survey: administrative, engineering, information technology, maintenance, driver and vehicle services, and construction. Results showed that there were differences among the groups in the organizational climate, supervision, HRD, performance management, accountability, communication, and motivation and retention. In most cases, responses by the maintenance group were more negative than those in the other job categories. However, in the HRD, motivation and retention, and accountability dimensions, the responses by those in construction also were more negative than those from other job groups. In the survey, motivation and retention was assessed by questions focused on the morale and motivation of employees, in addition to whether the NDDOT is able to retain qualified workers.

Analyses also were conducted to examine differences between the job locations (Central Office or the Field) and the gender of respondents. Those in the Field gave responses that were more negative than in Central Office on the organizational climate, supervision, innovation, HRD, communication, and motivation and retention dimensions. In addition, males were more negative about HRD than were females, even when the length of tenure was taken into account. However, tenure of employees was found to be related to the responses given on many dimensions. Results showed that the longer the employees had been with the DOT, the more negative they were about structure, climate, supervision, HRD, performance management, accountability, and communication. Organizational structure was assessed by questions concerning employee and group roles, structure of jobs, and the hierarchy.

Survey Results: Section 2

As in the first section of the survey, analyses were conducted to determine whether the questions associated with each dimension were consistent or reliable. Again, for each of the dimensions listed above, the analyses showed acceptable reliability among the questions assessing a particular category.

The survey results of Section 2 were positive overall. The following table displays the mean responses (from ratings on the 1 to 5 scale) of employees and managers on each of the dimensions measured in Section 2 of the survey.

Survey Results: Section 2

Both employee and manager results showed that responses concerning characteristics of the job (variety of tasks, autonomy, feedback, etc.), commitment to the organization, and career development (learning, training, and advancement opportunities) were favorable. All of these dimensions had means greater than 3.1 on the 1-5 scale. In addition, employees reported high levels of job satisfaction at the NDDOT, with a mean of 3.5. Alternatively, the mean ratings associated with the compensation dimension were below 2.6 for managers and employees.

Survey respondents also were asked to prioritize 18 issues in terms of how well they motivate employees. Employees and managers ranked compensation as the highest priority. Other issues rated highly by employees were benefits and job security. This result was surprising since surveys conducted in other organizations typically show that employees are most motivated by career growth and opportunities. Managers were more consistent with the typical findings, in that they ranked opportunities for growth and advancement and rewards for performance as the next highest priorities following compensation. Employees and managers rated formalized rules and procedures as the issue of least priority in terms of motivation. All of the employee ratings are displayed in Appendix C; management ratings are shown in Appendix B.

Managers also were asked to mark which of the 18 potentially motivating aspects of the job and organization would require additional funding. Only four of the issues were reported as needing additional funds by a substantial number of managers. The largest percentage of managers indicated that compensation needed additional funding (69 percent), followed by benefits (59 percent), new equipment and technology (54 percent), and rewards for good performance (46 percent). Unfortunately, two aspects of the job and organization that employees rated as having the highest priority also are the two aspects that the majority of managers reported as needing extra funding. This finding suggests that funding will continue to need consideration in the development of any plans focused on enhancing employee motivation.

Section 2: Different Results Based on Demographic Information

As was done for the Section 1 results, analyses were conducted to examine differences in the ratings of dimensions among job categories. Results showed that for the job characteristics, career development, and compensation dimensions, responses given by those in maintenance were more negative than other job categories. Construction also was lower than other job groups on the job characteristics dimension. Even though there were differences on many of these dimensions, there were no differences in job satisfaction among respondents in various job categories. Employees generally were satisfied with their jobs at the NDDOT.

There also were differences in responses based on the respondents' job location, gender, and tenure. Ratings given by those in the Field were more negative than in Central Office on the job characteristics, organizational commitment, career development, and compensation dimensions. Also, males were more negative than females on questions assessing job characteristics and career development. Tenure of employees also was found to be related to the responses given on one dimension. Results showed that the longer the employees had been with the NDDOT, the more negative they were about career development opportunities.

Survey Results: Open-Ended Questions

In addition to having respondents rate various statements throughout the survey, open-ended questions were used to obtain opinions about how to deal with retention and motivation issues at the NDDOT. Employees were asked to report aspects of the job and organization that kept them working for the NDDOT. The most common responses were the benefits (272 people) and the actual work duties and tasks (234 people). However, 136 people also cited coworkers as another reason they have continued to work for the NDDOT. Employees also were asked what could be done to retain qualified employees at the NDDOT. The most common responses to this question were:

• to increase pay (381 people)
• to treat employees well and with respect (33 people)
• to improve benefits (49 people)

Using a pay-for-performance plan also was suggested by 29 respondents.

Managers were asked for their opinions regarding motivation and retention, as well. In response to a question asking for a realistic approach for dealing with NDDOT employee motivation and retention, the most common response was to improve pay and/or benefits. The managers made this suggestion. Other common answers to the question were:

• to challenge employees (five people)
• to offer rewards or incentives (four people)
• to deal with negative or poor performing employees (four people)
• to encourage less micro-management (four people)

The numbers of people who gave each response were small because suggestions from managers varied widely on this question. These findings indicate that there is not a consensus about the best way of dealing with employee retention and motivation issues.

Current Survey Results Compared to Previous

To determine whether there have been any changes in perceptions of employees over the last few years, the current survey results were compared with those that were obtained from an employee survey conducted in 1999. Because the surveys measured some different dimensions and included different questions, only general comparisons could be made.

Results from the earlier survey showed that employees rated the categories of equal opportunity, working conditions, and supervision most positively. These results are similar to findings obtained in the current survey, which indicated that employees were most positive about the work environment and supervision, as well. However, equal opportunity was not specifically measured in the current survey, instead, technological resources and support received high ratings in this most recent survey.

Similar findings were evident in the areas where employees had more negative perceptions. In both surveys, compensation was rated most negatively. In the earlier survey, professional ethics and training also were rated more negatively. Although the current survey did not have those exact same categories, the HRD and performance management dimensions that were rated more negatively did include some of the same issues. For example, the HRD dimension in the current survey did have one question specifically addressing training opportunities at the NDDOT. However, it may be that training is an area that has improved over the three years since the last survey was conducted. By examining responses given to the question assessing training (displayed in Appendix C), it seems that training was rated more positively than other issues included in the HRD dimension, thus supporting the idea that perceptions of training have improved.