Alumni Assessment: Providing Insight to University Leadership

Janine M. Kraus, Ph.D.

Director of Student and Young Alumni Programs

Texas Christian University

TCU Box 297440

Fort Worth, TX 76129

(817) 257-6563

FAX (817) 257-7564

Email: j.kraus@tcu.edu

Alumni Assessment: Providing Insight to University Leadership

When determining the effectiveness of an academic program it is important to assess the program internally through the institution and academic leadership. But, it is also important to assess the graduates that have benefited from the program. This study assessed a doctoral program at a regional university in a metropolitan area. The doctoral program was eliminated due to financial constraints. However, an alumni evaluation was not conducted on this program prior to it be eliminated. The purpose of this study was to conduct an alumni evaluation on this doctoral program to determine its effects with alumni.

Alumni Assessment and Evaluation of Academic Experiences

Alumni perceptions are important to the assessment of academic programs in higher education (Davidson-Shivers, Inpornjivit, and Sellers, 2004; Heywood, 2000; Rice, Stewart, and Hujber, 2000; McGuire and Casey, 1999; Delaney, 1997; Pike, 1993; Moden and Williford, 1988; Steveson, Walleri and Japely, 1985). An alumni assessment can provide invaluable information to university administration about their academic programs. Park (1994) identifies alumni as a relevant population for assessment, because they can share insight that has not been realized by current students, due to their objective distance from the program. Alumni can contribute to the assessment of student outcomes, program goals, policy, management, and instruction within graduate programs (Knapper and Cropley, 2000; Delaney, 1997; Moden and Williford, 1988; Kinnick, 1985).

Alumni are the clients of higher education. According to Melchiori (1988), “the purpose of such research is to identify broad educational, psychological, and sociological changes or to assess the impact of higher education on its consumers” (p. 9). When surveying alumni they are providing “graduates’ self-assessment of what they have learned, how well they have been able to apply their knowledge in practice, and the extent to which their master’s program enabled them to achieve professional competence” (Delaney, 1997, p. 243).”

Surveying alumni assesses the knowledge and skills developed in their college education and how those developments relate to their skills and leadership in the workplace (Rice, Stewart and Hujber, 2000; Hoey and Gardner, 1999; Amiran, M., Schilling, K.M., Schilling, K., 1993; Jennings, 1989). According to Haworth and Conrad (1997), “…alumni, employers, policymakers, state legislators, and members of the media – render quality judgments as they evaluate alumni performances in the workplace, scrutinize faculty productivity on campus, and critique the content and character of undergraduate and graduate curricula in our nation’s colleges and universities” (p. 164).

Baird (1996) identifies the need for information on long-term outcomes. It is important to obtain information from graduates to determine their success in academics, research, or their professional careers. The accomplishments of former students are most easily obtained through surveying graduates (Baird, 1996). This option is relatively inexpensive, versus time and money spent on face to face interviews with alumni.

Ohio University provides an example of effective use of alumni assessment. The institution leaders use alumni to determine if the general education is serving its purpose (Moden and Williford, 1988). At this institution the Dean’s Council is given annual reports on alumni research. Several colleges within the university were able to make adjustments based on the alumni research gathered. “Each department in the college develops profiles of its graduates to document their success, the quality of their preparation, and their ratings of the department’s academic program” (Moden and Williford, 1988, p. 73).

Stevenson, Walleri and Japely (1985) discuss the importance of following up with former students. They talk about the positive impacts of assessing alumni, which can help in program reviews and improvements, improve the program’s community relations, and develop marketing for the program. The authors note the importance of using the follow up surveys in institutional planning, rather than simply reading them and filing them away.

Research Procedures

The study population consisted of alumni from an eliminated doctoral program in the field of college and university teaching at a regional metropolitan university. The population consisted of all graduates from the program. The alumni graduating years ranged from 1951 to 2000.

The instrument used for this study was a survey. The survey instrument was a mixed method approach by using both quantitative and qualitative research (Creswell, 2003). The alumni provided quantitative research with a past picture of alumni experiences and satisfaction. However, the addition of qualitative open-ended questions allowed alumni to generate responses not considered by the researcher (Rice, Stewart, and Hubjer, 2000).

A stepwise procedure was used for the survey research and it consisted of three phases (Creswell, 2003). An initial mailing of the survey was sent with a second mailing being sent two weeks later. A third mailing occurred three weeks after the second mailing.

Research conducted by Fisher (1988), states that a response rate of 35 percent on two mailings to alumni can be expected. A response rate of 39.4 percent was achieved in this study.

Findings

Participants were asked if they thought the doctoral program prepared them for their first post-graduation position. Participants in the study either strongly agreed (35%; n = 48) or agreed (35%; n = 48) that their doctoral program did prepare them. A small percentage (5.8%; n = 8) of the participants disagreed that the program prepared them for their first post-graduation position.

When asked if their doctoral program helped with their current position, over one-third of the participants (38.5%; n = 52) indicated they strongly agreed that the doctoral training did help. Another 34.1% (n = 46) participants agreed the doctoral training helped them in their present position.

Participants were also asked about their preparation for their profession as a whole. The majority of participants (76.2%; n = 106) were either very satisfied with the preparation for their profession from their doctoral program, or were satisfied with the preparation they received.

Participants were questioned about the quality of student and faculty interaction. The majority of participants (82.4%) either strongly agreed (43%; n = 61) or agreed (39.4%; n = 56) there was quality student and faculty interaction while in the doctoral program. The majority of participants (65.4%; n = 87) were very satisfied with their work with faculty. And 26.3% (n = 35) of participants indicated they were satisfied with their work with faculty.

Participants were asked about the instruction provided in the program. The majority of participants (54.9%; n = 73) were very satisfied or satisfied (38.3%; n = 51) with their quality of instruction.

Courses offered in a doctoral program are another area that was evaluated. The majority of participants (51.1%; n = 68) were satisfied or very satisfied (38.3%; n = 51) with the variety of courses offered.

Participants provided their thoughts on the overall doctoral program. The majority of participants (50.4%; n = 71) thought the overall quality of the program was very satisfactory. Another 29.8% (n = 42) of the participants stated they were satisfied with the overall quality of the doctoral program.

Participants were asked about academic advising. The majority of participants (51.4%; n = 72) indicated they strongly agreed with the amount of supervision from their academic advisor. And 29.3% (n = 41) of participants agreed with the amount of supervision from their academic advisor.

The doctoral program coursework was another area evaluated. A large percentage of the participants (92.4%; n = 122) were either very satisfied (46.2%; n = 61) or satisfied (46.2%; n = 61) with the doctoral program coursework.

Participants were asked if they would enroll in the program again if it was still in existence. The majority of participants (80.2%; n = 105) said they would enroll in the program if it still existed.

Participants gave their thoughts as to whether they would recommend the program if it was still in existence. Almost half of the participants (49.6%; n = 66) surveyed said they would recommend the doctoral program with enthusiasm if it were still in existence. Over 33% of participants (33.1%; n = 44) said they would recommend the academic program. The majority of participants (89.2%; n = 116) said the program should definitely be reinstated and continued.

Discussion

The alumni involved in this study provided valuable information regarding their doctoral program. The alumni gave insight into the effects the doctoral program had on them. They provided a description of how the program helped them in various positions throughout their career. This information helped provide a long-range perspective of the alumni from this program.

Many of the alumni identified positive aspects of the program. The majority of the participants were either “very satisfied” or “satisfied” with the following areas: doctoral program coursework; quality of instruction; work with faculty; student and faculty interaction; academic advising; and variety of courses offered. All these specific areas would conclude that alumni had a positive experience in the doctoral program.

The alumni provided an overwhelming positive response towards the doctoral program as a whole. The majority of participants thought the overall quality of the program was very satisfactory. The majority of participants said they would enroll in the program if it still existed. Not only would alumni enroll in the program again, most of the participants stated they would recommend the program to others if it were still in existence. In addition, participants said they would recommend the doctoral program with enthusiasm if it were still in existence.

The majority of participants thought the program should definitely be reinstated and continued. This again relates to the many positive responses regarding the program. Alumni demonstrated a positive experience in the program since they would not only enroll in the program again, but also thought it should be reinstated.

References

Amiran, M., Schilling, K.M., and Schilling, K. 1993. Assessing outcomes

of general education. Banta, T and Associates (ed.) In Making a

difference. Outcomes of a decade of assessment in higher education.

San Francisco, CA: Jossey-Bass Publishers.

Baird, L. 1996. Documenting student outcomes in graduate and professional

programs. New Directions for Institutional Research 92: 77-87.

Creswell, J. 2003. Research Design: Qualitative, quantative, and mixed

methods approaches. Second Edition. Thousand Oaks, CA: Sage

Publications.

Davidson-Shiver, G., Inpornjivit, K., and Sellers, K. 2004. Using alumni and

student databases for program evaluation and planning.

College Student Journal 38 (4): 510.

Delaney, A. 1997. Quality assessment of professional degree programs.

Research in Higher Education 38 (2): 241-264.

Fisher, M. 1988. Surveying your alumni. New Directions for Institutional

Research 60: 25-38.

Haworth, J., Conrad, C. 1997. Emblems of quality in higher education.

Developing and sustaining high-quality programs. Boston, MA:

Allyn and Bacon.

Heywood, J. 2000. Assessment in higher education. Student learning,

Teaching, programmes, and institutions. London: Jessica Kingsley

Publishers.

Hoey, J., Gardner, D. 1999. Using surveys of alumni and their employers

to improve an institution. New Directions for Institutional Research

101: 43-59.

Jennings, E. 1989. Accountability, program quality, outcome assessment, and

graduate education for public affairs and administration. Public

Administration Review 49 (5): 438-446.

Kinnick, M. 1985. Increasing the use of student outcomes information.

New Directions for Institutional Research 47: 93-109.

Knapper, C., and Cropley, A. 2000. Lifelong learning in higher education. London:

Koganage Limited.

McGuire, M., and Casey, J. 1999. Using comparative alumni data for policy

analysis and institutional assessment. New Directions for Institutional

Research 101: 81-99.

Melchiori, G. 1988. Alumni research: An introduction. New Directions for

Institutional Research 60: 5-11.

Moden, G., and Williford, M. 1988. Applying alumni research to decision making.

New Directions for Institutional Research 60: 67-76.

Park, L. 1994. Good accounting skills: What more does a successful

accountant need? Journal of Education for Business 69 (4): 231-234.

Pike, G. 1993. The relationship between perceived learning and satisfaction

with college: An alternative view. Research in Higher Education 34 (1):

23-40.

Rice, R., Stewart, L., and Hujber, M. 2000. Extending the domain of instructional

effectiveness assessment in student evaluations of communication

courses. Communication Education 49 (3): 253-266.

Steveson, M., Walleri, R., and Japely, S. 1985. Designing follow-up studies

of graduates and former students. New Directions for Institutional

Research 47: 81-91.