We must not assume everything that employs technology is going to be successful. That is why evaluation is so important. And then we must use that evaluation to create positive change. (Former U.S. Secretary of Education Richard W. Riley, cited in McNabb, Hawkes, & Rouk, 1999, p. 1)

The underlying assumption in a technological perspective is that everyone shares a common interest in advancing the innovation. The only issue is how best to implement it. (Hargreaves, Earl, & Schmidt, 2002, p. 73).

Purpose, Scope, and Rationale

Professionals are being expected to function in a progressively complex environment in all fields (Twale & Kochan, 2000). Technology plays a significant role in this challenge (Diem, 2002; Karlen, 2001), particularly for educational leaders (Mendis, 2002). Teaching and leading have become increasingly multifaceted art forms with the advent of learning technologies. University and school faculty are being expected to design, deliver, and assess successful online courses (Fuks, Gerosa, & de Lucena, 2002; Mendis, 2002), often without the necessary training and support (Walker, 2002). As face-to-face contact becomes reduced through online environments, effective communication becomes essential and barriers more pronounced (e.g., Creanor, 2002). It is important to note that distance education studies have associated high-quality interaction with satisfaction for remote learners (e.g., Sorensen & Baylen, 2000). The results that this study shares provide support for this important finding in the context of doctoral learning.

Distance education is commonly viewed as “thinking outside of the box” that can empower learners to engage in better ways of learning in cyberspace and throughout their lives (e.g., Serwatka, 2002). This nontraditional learning opportunity transpires during the physical separation of teacher and student. A shift of emphasis from teaching to learning, or from the teacher to the learner, occurs through the mediation of print or technology (Wheeler, 2000). Web-enhanced learning, the electronic course delivery model discussed herein, is a modification or hybrid that incorporates some face-to-face contact. Importantly, web-enhanced courses emphasize human communication through a variety of teacher-student and student-student formats. This distinction between distance learning and hybrid learning is critical for learners, but it does not imply that interaction is somehow more important in the modified online environment. Regardless of the instructional medium selected, high-quality interaction in the educational relationship continues to be regarded as indispensable (Sorensen & Baylen, 2000).

This article explores some of the contradictions and possibilities of web-enhanced learning through the qualitative analysis of a pedagogical experiment. The parameters for this study are, broadly, doctoral education and, specifically, one doctoral course. However, extrapolations beyond these lenses are ventured.

An overriding lesson from my study suggests that web-enhanced instruction is, either inherently or at this early stage of development in the computer era, “a mixed bag.” Distance educators (e.g., Cifuentes & Murphy, 2000) generally seem to believe that while distance [ME1]technologies may exacerbate the frustration of individuals, they have the proven capacity to meaningfully connect learners across time/space/language barriers. This stance probably needs further study beyond the scope of this article—ongoing and continued assessment of pedagogical experiments that use technology need monitoring. One related issue concerns the need for substance over form, and peer learning over networking (Hargreaves, Earl, & Schmidt, 2002).

Open-mindedness toward the e-learning medium itself is necessary. Benefits and drawbacks need to be carefully weighed and contextual issues carefully considered. The issue of pedagogical effectiveness via distance learning becomes more confused than it should if an inherent value is attributed to instructional technologies. It is from this critical perspective that I have analyzed the mixed results of my online experiment. As a curious newcomer, I am striving for “a more balanced perspective” on distance learning that pursues a “critical understanding of the gains and losses connected with the educational use of computers” (Bowers, 1998, p. 77).

Written from the standpoint of distance learning experts, much of the web-based educational research seems colored by a positive, even strongly advocating, tone. This seems to underlie even assessment models for technological effectiveness. An overridingly favorable bias poses challenges for the novice distance educator, such as myself, who is trying to objectively examine the pedagogical experience of remote learning. Those researchers (e.g., McNabb, Hawkes, Rouk, 1999) who offer critical perspectives on distance education represent the minority voice. Bowers (1998), for example, argues that the discourse in this field is “dominated by advocates who now control the direction of educational reform” for schools and universities (p. 76). Technology has, notably, been ascribed status apart from its pedagogical function, reified as “a transformer” (Mendis, 2002). Such attribution potentially detracts from the teaching/learning focus that is the agenda for transformation in education today.

By using this opportunity to examine the results of my first experience teaching via electronic delivery, I offer a twist on the proliferating studies in distance learning. On the one hand, the technology community has given the rest of us a new way to educate that is potentially very exciting and practical. On the other hand, experienced educators who are new to the technological infusion of teaching can offer fresh insights. In my experience, complex, unresolved pedagogical issues characterize this medium of learning and its effect on the success of students and instructors. Importantly, the call for systematic review of pedagogical experiments using technology requires critical, balanced assessment; former U.S. Secretary of Education Richard W. Riley, among others, has voiced this need (cited in McNabb, et al., 1999).

Building on one doctoral cohort’s responses, I conduct a systematic analysis at the local level that incorporates multiple sources. I hope to contribute to other faculty’s pedagogic needs while generating topics for leaders who make decisions about distance learning as a new facet of teaching and evaluation. This analysis covers contextual issues (institutional, cultural, technological, evaluative, and pedagogical) related to the electronically delivered instruction. Methods of assessment and themes from the data, including contradictions that surfaced, are described. Finally, ideas for better managing this revolution in education are ventured.

The Study: Multiple Contexts and Issues

Institutional Issues

The technology-infused course called Teacher Evaluation was developed by me for three reasons: (1) to address the mission of my university to offer distance learning courses to serve the Florida community and to extend them to the doctoral level; (2) to respond to the requests of the administration and full-time professionals to engage in an opportunity that, by minimizing on-campus attendance, would maximize writing opportunities; and (3) to provide a vehicle for novices to develop skills in communications technology, and to share the knowledge gained.

This experiment had a double agenda: To determine what online capacities are expected of learners/leaders today and to discover how well distance learning can be applied to doctoral education. The population identified for this online experiment, an “open” doctoral cohort studying at one of the university’s remote campus sites, is unusual. The major target of distance learning, as reflected within my institutional context and the literature itself, is teacher education. Distance courses are designed mostly for undergraduate students (e.g., Serwatka, 2002), with some emphasis on the masters’ level (e.g., Matthews, 2002). A related target focuses on technology infusion and evaluation for inservice teachers and the nation’s schools. Contexts for such study include high poverty with high minority ratios (Lanahan, 2002), student learning and achievement aimed at school improvement (McNabb, et al., 1999), and university innovations that promote forays into institutional and global partnership (Cifuentes & Murphy, 2000).

Closer to home, faculty who are teaching using distance [ME2]technologies will need to critically assess the learning outcomes (e.g., Karlen, 2001). At this time, too few in-depth, systematic assessments of doctoral education have been made available for beginning virtual pedagogues like myself (among the exceptions are Boyer, 2001; Sorensen & Baylen, 2000). This scope needs widening. Comprehensive study of doctoral education in relation to emerging trends in electronically delivered courses is needed. Toward this goal, this article takes a modest step.

Two contrasting if not conflicting pictures arise from reading the distance education literature. On the one hand, an image is conveyed of distance learning as a rapidly growing innovation in education, with substantive support already developed for technologically-enriched global communities (Matthews, 1999); on the other hand, the picture that emerges is one of “hype” around the “rampant progress” that is claimed, with very limited actual use of “web-based virtual classrooms.” The problem, some researchers claim, is that gains from technologically supported courses have been limited due to “insufficient technical and educational knowledge, reluctance from educators and lack of institutional support” (Hsu, 1999, p. 9).

At the center of this intense debate lies the recognition that “the role of faculty is changed when viewed in the context of distance learning” (Karlen, 2001, p. 3). Advantages and disadvantages need to be carefully examined at the cultural level, beyond the testimonies of individuals and course assessments. According to the instructional technology experts with whom I have corresponded, a gap exists between the ideal (vision) and the reality (practicality) of the online learning phenomenon. Forces of reluctance and resistance, combined with institutional rhetoric and insufficient support, are viewed as obstacles.

Ongoing political concerns probably hinder experiences and assessments of online learning. Some of the reluctance of online delivery in higher education comes from faculty who are not technology experts. “Significant training” in technology and pedagogy is required beyond one’s areas of academic specialization (Karlen, 2001). Also, distance education courses tend to be problematic for different reasons from traditional classroom delivery, as reflected in weaker student course evaluations combined with inappropriate surveys for online classes (personal communication, associate dean, November 2001). The rewards seem slim not only because of the negative evaluations that can result but also because of the time-intensive demands on instructors.

The return on investment, then, may be low for distance educators. For example, although a stipend is paid to faculty in my university for preparing masters courses for online delivery ($6,000 US per course above one’s 9-month salary), no such financial incentive is available for doctoral courses. Without the necessary support for planning and delivery (e.g., course release), faculty are being expected to carry out the mission for technology advancement, but at a personal and professional cost. Senior faculty are, generally, less technologically invested and knowledgeable, which means that the responsibility for electronic course design and delivery mostly falls on the shoulders of tenure-earning faculty and technology professors.

Another problem is that online materials can be freely used without the instructor’s consent. Copyright control and the weight of ownership favors the institution (personal communication, professor of law, March 2002). The policy is that course materials produced with university resources and posted on a university’s website are, by extension, institutional property. This is not to overlook that more empowering situations may exist for faculty in some contexts.

A further challenge of the online learning environment is that the traditional format of office hours and scheduled classes becomes irrelevant: New mindsets are forced to emerge. Not all students are ready for the change in perspective that “postmodern” forms of learning demand on issues of time, space, and interaction, despite the premium placed on minimizing campus visits. Not all faculty or institutions are ready either: New modes of course delivery require a rethinking of such commonplaces as instructional preparation time, teaching load, class size, contact hours, feedback and timeliness, and incentives and rewards (Karlen, 2001).

Importantly, the issue of knowing/not knowing the students that one will be teaching via distance must be brought to the fore. Doctoral students can fare much better if they are operating within an established peer culture. Although the assessment results of my course were mixed, the positive feedback received can be partly attributed to the fact that most students already knew each other and myself. As a cohort they had been working intensively together for three years to build a resilient, scholarly culture. Because of my opportunity to teach this same group the previous semester (fall 2001), I had a gauge for knowing the students. I was also able to compare, using the same group, differences between the traditional and the electronic delivery of courses.

Although this discussion focuses more on the quality of the electronic experience than comparison with the traditional context per se, the students’ course evaluation was stronger the first semester. For this face-to-face, traditional delivery of a graduate seminar, my teaching scores were consistently 5.0 out of 5.0 on every item that was measured, using both the college and university’s standardized forms. However, the web-enhanced course resulted in mixed feedback.

Cultural Issues

Online learning has the potential to significantly reshape the doctoral culture nationally and globally (Boyer, 2001). Doctoral education in particular depends on high-quality interaction for student success and well-being (Mullen, 2001, 2005; Nyquist & Woodford, 2000), and deserves protection from poorly planned institutional expectations and media hype. Distance learning must be reexamined in this light so that informed decisions about this medium can be made, with consideration for the assessment needs and learning goals of particular populations.

A new national expectation has “set the bar” for reform and delivery of educational leadership doctoral programs. School leaders are required not only to learn new technologies for their jobs but also to model state-of-the-art practices (Mullen, Gordon, Greenlee, & Anderson, 2002). Reenvisioning doctoral programs within this context challenges the profession to dramatically change. Most universities have yet to be turned into contemporary institutions that prepare students for academic and professional goals (Nyquist & Woodford, 2000). Technology has been identified as a major foothold for achieving this vision. My discussion is framed by the question, “‘How can we re-envision the Ph.D. to meet the needs of the society of the 21^st century?’” (Nyquist & Woodford, 2000, p. 2). The specific question guiding my study is, “What was the quality of educational experience of a doctoral cohort’s first online learning experience?”

Technology-based learning has become integral to the advancement of graduate schools of education and public schools. Regarding public schooling, national organizations advocate for the redesign of university preparation programs that will enable aspiring leaders to better deal with the escalating challenges currently facing schools. Learning technologies play a significant role in forwarding educational change: NCATE program standards (National Council for Accreditation of Teacher Education, http://www.ncate.org), and national educational technology standards, specifically the Technology Standards for School Administrators (National Policy Board for Educational Administration for the Educational Leadership Constituent Council, http://cnets.iste.org/tssa) all support schoolwide technology infusion as a standard for excellence.

Additionally, technology has been identified as a contemporary leadership skill. Competency with technology is required for licensure (Standards for School Leaders of the Interstate School Leaders, 1996, Council of Chief State School Officers, http://www.ccsso.org) and the high performance ratings of schools (Mullen with Stover & Corley, 2001). Public policy on school improvement and accreditation is inextricably tied to technology standards.

As faculty in educational leadership programs prepare courses for today’s climate, we need to consider anew such critical issues for changing schools. In many ways, these issues parallel the transformations occurring within our higher education environments and professional lives. Leadership and vision, learning and teaching, productivity and professional practice, and social, legal, and ethical issues have all been rethought in light of technology-oriented values. Educational leaders are being expected to foster a culture conducive to the realization of this multifaceted vision. Appropriate technologies must be integrated to maximize learning and teaching. They should be used to enhance professional practice and increase productivity.

Comprehensive systems of effective assessment and evaluation are expected to emerge from these contexts. Faculty are by default responsible for designing our own assessments suitable to this medium. Further, the social, legal, and ethical implications of the decisions we make relative to technology and learning must be examined (Bowers, 1998; Mullen, et al., 2002).

One such major area for assessment involves the impact of technology on the development and viability of school-communities (Bowers, 1998). Applying this contemporary goal to doctoral education is yet another road to be traveled. Aligned with the leadership program redesign efforts at my university, we have proactively responded to the pressure from the school districts and national standards. In their workplaces, our students are expected to collaboratively develop a learning organization by functioning successfully with technologies. In their graduate studies, critical reflection is being encouraged for assessing such directions for change.

The promise of distance education is that it can assist with the very process of learning how to learn. Educators must constantly adapt to the changes within the workforce, as reflected in the explosive popularity of e-learning. Also, “knowledge workers” must “learn to work within a group,” which is required by businesses and schools, and they must “learn how to creatively change an old knowledge set into new knowledge,” hence becoming web-based educators themselves (Fuks, et al., 2002, p. 23). Faculty that prepare educational leaders to engage in purposeful inquiry (Wallace, Acker-Hocevar, & Sweatt, 2001) must take all this into account.

Technological Issues

Distance learning courses use a variety of technologies to offer students new educational experiences and convenient access to selected courses. In the role of instructor, I considered what options would be most appropriate to the learning of a doctoral group, many preparing to write their dissertations. For example, unlike distance courses that use interactive videoconferencing I avoided using a one-way lecture format, given the learner-centered goals of intensive scholarly development and practical inquiry. My aim was to simplify form and highlight substance so that learning could be maximized. This approach fits with the premium placed on high intellectual learning levels and critical thinking skills in universities. Diem (2002) believes this is the purpose of hybrid instructional technologies, which implies that online learning contexts should be aligned with the goal of developing the capacity for scholastic and applied learning.

Currently, other formats such as WebCT are being used to conduct online learning, often without video and audio interfaces. WebCT, a software program for supporting the administration and delivery of web-based instruction, is a provider of integrated e-Learning systems for universities. My own institution has implemented WebCT’s course management system (Project Team of Bruce Landon, http://www.webct.com, 2002). I found this format for course design and delivery appealing for its simplicity, although it is not suitable for complex forms of technical manipulation or group interaction. For my course, interaction with multiple stakeholders—the student groups and whole class, the instructor and other professors, and school leaders—required that interactivity would have to occur beyond the online environment.

To assist me in learning and using the WebCT model, I relied heavily upon a computer expert’s guidance. Once assigned the course, I had six weeks to prepare for online delivery. Without any computer training in distance learning or the WebCT model, I was completely dependent upon another’s technical know-how and goodwill. This individual’s firsthand experience with distance learning made this project much more doable. She carried out all functions related to network setup and maintenance as well as the display of course material, including my graphic designs. She also used electronics to convey my belief that the doctoral student is responsible for self-directed inquiry within a supportive context. The syllabus and all other material were then converted for web access. We collaborated on and reviewed all of these dimensions of the work. Throughout, I assumed the role of eager “test case.”

Evaluative Issues

Educational systems show signs of moving away from the uncritical building of technology infrastructure toward evaluating the effectiveness of application in classrooms (McNabb, Hawkes, & Rouk, 1999). Nonetheless, the assessment of outcomes is still relatively new in all areas, including cognitive and technical skills (Ainley, Banks, & Fleming, 2002). Assessment of technology-delivered instruction and student learning must be carefully undertaken with an eye toward “choosing critical indicators for evaluation” and overall improvement (Clark, 2000, p. 5). This goal is a major challenge for faculty experiencing computer-mediated instruction for the first time, and without the necessary expertise, training, and support. It is within these emerging evaluative and instructional contexts that my study has been produced and that learning outcomes have been tentatively determined.

As previously indicated, I approached the new course experience with a belief that electronic learning is not inherently good or bad but should be judged on a number of factors, including the experience of students and myself. It makes sense to me that technology use is important for the supportive role it can play in teaching and learning, rather than as an end in itself (Eva Baker, part of the McNabb et al. report, 1999). Also, assessment—as an ongoing part of any course—can be approached as a form of reflective inquiry: “Evaluation is part of a reflective process. The more reflective we are, the more likely we are to improve our practice” (Charol Shakeshaft, see the McNabb et al. report, 1999, p. 8).

Pedagogical Issues

In the spirit of innovation that Simonson (2000) describes as the “willingness-to-change” by readily adopting new ideas, my institution, my students, and I embarked on this trial. With this online endeavor, I offered the first web-enhanced course taught at the doctoral level in my program area, and one of the first in the university. Twenty-five students began and successfully completed this course, renamed “Teacher Evaluation: Big Top Event” by one learner.

The purpose of the course extended beyond the scope of teacher evaluation. The idea was to probe learning and assessment with different educational groups, including students and administrators. Not all students were interested in teacher development and evaluation as a topic of inquiry. However, a critical direction needed for the growth of the entire group concerned issues of evaluation in education. This course also encouraged a positive change in school cultures through alternative assessment processes and methods, with an emphasis on peer models (e.g., peer coaching) and constructs of empowerment (e.g., collaborative mentoring).

As the instructor, I approached distance delivery as another opportunity for engaging doctoral students in intensive feedback within an identified area of dissertation research. As a doctoral supervisor at the university, I was prepared to provide in-depth responses and guidance on all writing assignments—with rewriting as the norm—and problem-solving expertise.

However, although previously “sold” on the use of technology for enhancing my own teaching and electronic communications with students, I practiced a healthy reservation with this online experience. I did not assume that doctoral courses, including my own, should automatically go online and students become distant users. The masters program in which I teach is being completely converted to online delivery. Separate consideration for the doctoral level and its unique emphasis on the socialization and culturing of scholar-practitioners is needed, which should not be overlooked in the enthusiasm of the online learning era.

Communications technology experiences leading up to this course were significant in the lives of this doctoral group. In this sense, although collectively we were novices in the world of distance learning, we were not new to technology. In fact, most reported an average to high comfort level with technology. Like the Nobel Laureates found to have been mostly active technology users, we identified with its benefits: Technology has aided the scholarly community with a significant increase in writing, productivity, communications with colleagues and constituents all over the world, and access to databases (Cisco Systems, 2001). Writing-based mentoring networks for educational leaders are also part of this growing phenomenon.

Extra time was needed for the preparation and maintenance of my course website, as Hsu (1999) warns. Faculty cannot assume that once the course site is available to users that the work ahead resides only with assignments and interaction. To the contrary, an ongoing challenge exists in the need to update the site in order to encourage student engagement. For instance, because website locations can change within short periods of time, I was vigilant about checking the URLs and updating critical resources. As another example, I published the students’ article reviews at the course website and then submitted them for formal review to a journal editor. Additionally, I posted all exemplary assignments and projects (e.g., learning contact) as sources of stimulation and peer mentoring.

Finally, I submitted daily entries to the calendar page and encouraged students to do the same, a feature that attracted daily interest. Such entries congratulated students on their academic successes, new positions, and births. Also on the calendar page, reminders were posted of upcoming assignments with blurbs that helped sustain motivation, clarify timelines for the submission of work, and summarize longer entries contained within the syllabus-based modules.

With the availability of this online course, students were spared extensive travel—as much as five hours (return trip) to the university campus. (One school superintendent in the group who lives in a state other than Florida has always been accommodated at a distance.) This online solution was also for the “in-between,” as Bothel and Enfinger (1999) describe those “who may be close enough to attend some courses but have other restrictions on their time and availability” (p. 5). My goal, then, was to combine distance learning with individual and group meetings.

Importantly, I believed that the online experience could help this student population develop an informed perspective about technology use for adult learners. We would have the opportunity, in keeping with the new policies, to improve our skills as leaders responsible for integrating and modeling technology in various contexts. Early on I had reassured the group that this learning curve would entail a period of disequilibrium for everyone (Mendis, 2001). The trust we had established was necessary for accepting this particular challenge of learning via distance.

Doctoral Course Structure and Process

In preparing for this Teacher Education course, I read descriptions of courses adapted for online delivery (e.g., Bothel & Enfinger, 1999; Mendis, 2001). My own approach within the web-enhanced world included some unique elements: The days of instruction (one-time, all-day retreat and final session), as well as the entire course, were structured using a circus motif. Upon entry to the website, learners were greeted with the message: “Welcome to the fairgrounds!”

An outstanding feature of the circus theme was the use of “tent act” as modules. Each contained critical components of the course, ranging from weekly assignments to resources and exemplars of work. The seven modules, each identified with the flap of a tent designed for opening at the user’s hand, were titled “Module 1-Tent Act 1” (and so forth), as well as “Examples,” “Articles,” and “Resources”—all “acts of wonder” aimed at engaging practitioners.

After I launched the course online, I encountered a recently published article about how a company used the circus motif. Related themes were integrated into training modules for corporate executives to help uncover new perspectives about work and life (Hammonds, 2001). The trainer also played the role of “ringleader” (lead educator) and her associates were assigned those of “tightrope walkers,” “jugglers,” and so forth. Our parallel idea is that if life and work were approached as a circus, then knowledge workers would become passionately committed to bringing our whole selves to what we do—treating learning and community as “the greatest show on Earth!” One “leadership ensemble”—groundbreaking collaborators in the world of chamber music—believes “Ideas, initiative, creativity, energy, passion—these are among the most vital qualities. … A respectful environment engages workers on a personal level, [which] can have an enormous impact …” (Seifter, Orpheus Chamber Orchestra, & Economy, 2001, p. 144).

Because of the trust, risk, and safety already established within the doctoral cohort, I felt empowered to take risks by introducing an overtly imaginative overlay. This is an example of how familiarity and continuity with the same group of learners can make a significant difference in the benefits accrued from web-enhanced instruction. As one dimension of the circus metaphor, I invited a leader of the cohort to help design the theme to reflect her classmates’ and professors’ personalities. During the January retreat, the group playfully selected roles, tagging each person’s back with a circus character (e.g., fortune teller). Everyone guessed at their preassigned identities as hints were dropped about their unusual gifts (e.g., propensity for knowing future events).

During this first encounter, the group freely associated upon hearing the words “teacher evaluation.” The responses, including “waste of time,” “humiliation,” “red tape,” and of course “circus,” revealed problematic experiences. The education literature validates these criticisms. For example, Gottesman’s (2000) argument is that evaluation amounts to a “dance of the lemons” for most teachers—a meaningless ritual without integrity or effect. Alternative (e.g., peer-based coaching) systems can turn around this “circus.”

Students were then launched into a semester-long inquiry into the need to analyze commonplace forms of evaluation and to offer solutions. They were asked to share all assignments with their working groups, and to integrate any desirable recommended changes prior to submission to me. The goal was to make use of the circus (peer) groups as an electronic forum for experimenting with new collaborative practices that have academic and social value.

Positive views were reported about the use of a “circus” structure for this course. Students described this approach as “successful,” “creative,” and “helpful as starting point for and intersection of all assignments.” As one person summed up, “The circus theme helped us to individualize our topics and think flexibly, not unlike the circus itself—a multifaceted happening!”

Doctoral students have different needs from those studying at earlier levels of education. In educational leadership programs, the focus on scholar-practitioner writing development makes isolation more likely. With this understanding, I designed the course so that it would emphasize not only individual progress and “becoming” but also reciprocity with peers around a common focus. Students were responsible for completing all assignments but they were in control of determining the actual percentages and weights. The idea was for everyone to integrate all completed assignments (e.g., reviews of articles and data displays) into a unified whole, in such a way that would enable connections to be generated and material developed for the dissertation.

The requirements for this course supported the scholarly development, practical knowledge, and mentoring of the doctoral group. Students were not only expected to develop working relationships with peers but also to solicit input from their major professors, and to meet with stakeholders about their evolving research. Other assignments included warm up activities, reviews of articles, concept-and-data displays, outlines of the evaluation paper and a complete product (literature review or case study). Also, self-and-peer assessments were completed. Students commented that these reflective activities built upon one another and supported goals:

The assignments were useful since they provided an opportunity for me to think about research and relevant literature. The readings focused my attention on mentoring and teacher evaluation in my current school situation. The paper was probably my greatest learning experience. Since I was allowed to explore beyond a strictly evaluation topic, I expanded my knowledge in an area that has helped me tremendously. My dissertation is a challenge that lies before me, but what I gained in this class has put me well ahead of where I expected to be. I’ve finally met with my major professor about my writing.

To encourage writing of high quality, all assignments, including the learning contract itself, were submitted as drafts. The value of revision for doctoral students not only supports the art of reflection and analysis but also provides opportunities for adding critical components to a work over time (Mullen, 2001). The evaluation paper was submitted not less than four times from most students, with instruments included and improvements highlighted. Developmental writing in the form of revision was a requirement for completion and success in this course.

The learning contract became even more crucial for this online course than I had experienced in traditional courses. Devising a contract enhanced the focus and productivity of the entire class and made transparent the responsibility of individuals for their own learning. It was used to encourage students to individualize the course to meet their own needs, while operating within a specified but flexible conceptual framework. They were being asked to both guide and assess their own competence as self-directed inquirers, and to thereby engage in a model of personal accountability considered essential to professional development (Gottesman, 2000).

Knowles’ (1975) classic scaffold for adult learners focuses on the contract. Modifying this existing structure was necessary for engaging a group of educational leaders with varying degrees of productivity and readiness. Students used the contract template to reflect on major tasks to be completed: assignments and allocations, research, evidence for the learning (e.g., verbal/written communication), and evidence of accomplishments (validation/authentication from individuals and groups). As one person summarized, “Gaining feedback from multiple constituencies, including principals, was vital for my work. This piloted activity proved ambitious but, because it had been noted as an authentication measure, I received course credit for it.”

Participants found the contract exercise to be groundbreaking, as it forced the issue of responsibility to their own learning and documentation of the work they undertook:

When we developed the contract, I really did not understand where it would take me. Now I understand that it was our personal timeline. Because we developed our own contract it had meaning. The professor developed the syllabus or roadmap but we decided on the grading system and the direction(s) for our work. This constructivist approach to learning proved to be a successful way for learning and assessing at the graduate level.

The learning contract motivated me to constantly revisit my learning objective and to fulfill all my obligations. More than this, I’ve been challenged to assess the traditional way that I have been evaluating teachers as an assistant principal in a Title 1 school.

Methods of Summative Assessment

For this article, assessment of the instruction and learning that resulted from the course is discussed in some detail. This analysis draws primarily upon the anonymous online survey data that was generated and, as secondary data sources, the identifying assessment data (student self-assessment, peer assessment, and electronic correspondence). Results are based on a 100% return rate (25 surveys), averaging 10 typed pages each of anonymous data, in addition to other assessment data, which were assignment based. In total, 100 completed assessments were reviewed. The use of multiple instruments helped me to become as informed as possible about the quality of experience for learners, with an aim toward improving my pedagogy of online courses.

Circumstances converged, influencing my decision to develop a final assessment dubbed a “marathon survey” by students. My College of Education had not yet prepared a survey for online courses (personal communications, associate dean, March 2002), and the traditional forms of evaluation were not appropriate. Also, surveys created for online courses proved difficult to locate. Thus I developed an instrument of my own and shared it with my college.

The culture that supports alternative assessment is still in the early stages of development (Clark, 2000). Much confusion exists as to how best to identify and measure indicators for outcomes (Hargreaves, et al., 2002). Before circulating the exploratory instrument I first attempted to validate it, using three sources of support:

1. Verma and Mallick’s (1999) tips for developing instruments that address pedagogical goals

2. Learning objectives (e.g., “explore the role of instructional leaders in improving the current culture of evaluation”) specified in my course syllabus

3. An instructional technology expert and online instructor with a background in assessment

With this professor I shared that my roles as instructor and assessor might bias the instrument or be perceived as doing this. This individual, who had a lack of involvement in the course and stake in its success, suggested adjustments that reflected an equal emphasis on positive and negative feedback in all critical areas of the instrument. Later, this expert helped to improve upon my analysis; in response, I undertook a holistic analysis of the data in addition to the thematic reading. Subsequently, a bigger picture evolved; this encompassed contradictions and possibilities of online learning that would have been missed by the thematic analysis alone.

The survey is an open-ended, summative assessment covering eight major areas with specific questions. Spaces were provided for writing. As Clark (2000) reminds us, questions asked inevitably inform only those areas identified for feedback. Consistent with Clark’s typology for evaluation, the survey items I developed addressed learning and outcomes, as well as instruction and technology. My summary of these areas follows:

· Overall: distance learning (WebCT) experience

· Organization of the technology-delivered course

· Nature of contact with course leader and support expert

· Evaluation of course on selected dimensions

· Personal and professional growth outcomes

· Comparison to other courses

· Transfer of learning to the work environment

· Recommendations regarding instructor and course

A graduate student (not in the course) circulated the online survey subsequent to the submission of grades. Having a student collect the data meant that I could maintain objectivity and participants could be honest, without fear of penalty. They were informed that their sharing would be anonymous and that the returned surveys would be forwarded to me without identifiers.

The Doctoral Cohort’s Profile: A Data-Generated Perspective Information obtained from the online survey was used to produce a profile of this group of 25 doctoral students. Areas included demographics, stage of coursework, and academic/ professional goals. The cohort was completing coursework for the doctorate in educational leadership and anticipating the exam stage leading to doctoral candidacy. Individuals ranged from 26 to over 55 years old. Like other distance education groups, this one was older and more established than traditional classes; responsibilities extended to family and career. This mixed-gendered group was multilingual and ethically diverse (White, African American, and Latin).

Working full-time in the West Central Florida region, all were school practitioners and most former teachers. These lead teachers, program coordinators, assistant principals, principals, and (the minority) assistant superintendents and superintendents represented the elementary, middle, and secondary levels. Professional goals focused on the principalship for most and on the superintendency for a few. Several, who expressed a secondary interest in university teaching, hoped to write a book one day. Dissertation topics ranged but emphasized issues of mentoring and diversity in contexts of high poverty, low-performing schools and related policy contexts.

All valued high-quality professional development for teachers, and caring and appropriate pedagogy. A shared belief was that school reform should focus on improving the quality of education and life for students. The doctorate was viewed as an opportunity to offer the public education system something worthwhile beyond their current accomplishments.

A Summative Analysis through Doctoral Students’ Eyes

Value of Online Interaction

Mixed reviews about the value of online learning for doctoral students emerged from the assessment data. Students expressed desire for greater teacher involvement despite the face-to-face seminars, meetings with the professor, and daily access and feedback. Some problematic situations had been resolved at the end of the course but certainly not all. For example, the expectations for student-directed learning and professor-led instruction had not been fully determined, and the internal “insider/outsider” experience of some groups remained unsettled. Such complex issues go beyond the parameters of this online course and of electronic learning.

Issues of interaction and learning emerged as broad themes from the data. More specific issues concerned whole-group interaction, peer-based reciprocity, and student-teacher reciprocity.

Quality of interaction is a complex issue that needs more study if online learning is to be strengthened. Toward this end, Roberts (2002) offers differentiated views of interaction: as reciprocity between parties, as discourse in which ideas are engaged, and as action in the world. All three levels of interaction were evident in the data and will be discussed.

The literature on distance education is much more positive than my results: For example, cooperative and enriched learning are presented as outcomes of Internet-based courses (Fuks, Gerosa, De Lucena, 2002). However, the many unresolved challenges associated with nontraditional course delivery are also recognized. Notably, rewards have yet to be established, as part of the tenure and promotion process, for faculty using computer technology in teaching and scholarship, especially in innovative ways (Culp, Riffee, Starrett, Sarin, & Abrahamsen, 2001).

In the world of cyberspace learning, interactive patterns alter. Change in leadership and learner roles follow for teachers and students (Boyer, 2001). Palloff and Pratt (2001) encourage this new view of interactivity, asking faculty to relinquish control so that students can take the lead in learning. However, this philosophy should not be adopted as a rationale for abandoning students online but rather for facilitating their academic development and mentoring mosaics.

Whole-group interaction. Although the students were completely in support of the freedom that comes with web-enhanced delivery, all reported having missed the interpersonal dimension that being physically with others brings. This aspect of interaction—whole-group interactivity—was viewed as the weakest dimension of the online environment and hence the least satisfying aspect of the course. Students expressed their desire for live interaction, including face-to-face contact, bodily performance, and interchange with dynamic professors:

The most negative aspect of the WebCT experience was the lack of face-to-face contact with the professor. She has proven in the past to be dynamic and effective. Web-based instruction made it impossible for students to benefit from her enthusiasm and energy to the degree that a traditional model would allow.

WebCT course delivery is not conducive to performing “live.” What I didn’t like was the absence of body language/vocal tone, which lets me emotionally/intellectually feed off the immediacy and energy of others.

Interestingly, the most critical point made about the course was also the most positive, for it clarified the students’ vision and values. At least 30% of the students asserted that online learning should be reserved for “professors who are not particularly dynamic—the best professors should be in the “real” classroom with us!” Dialogue that transpires in a fully embodied, human context was missed by all as well as opportunities for pedagogical performance and mentorship and for “sparking ideas.” Students together provided support for the deeper cultural performances and rituals of learning that the electronic classroom probably has difficulty emulating.

Peer-based reciprocity. Peer learning was, overall, considered more beneficial than whole group interaction, although it also received mixed reviews. Not all members understood the value of learning from peers from the outset: “Initially, I did not realize how important it would be to communicate so closely with my group members. They were all reliable and provided continuous feedback by asking probing questions and responding to my paper.”

In my experiences with educational leadership groups, learning actively with others and sharing ideas and resources cannot be assumed. In competitive cultures, this is a learned capacity that not all adults bring to the higher education setting. Accordingly, the students documented their experiences of social learning as a critical dimension of academic socialization for leaders today. Although many had learned the value of collaboration in academia, they nonetheless struggled with transferring this learning to the online environment and their e-writing groups:

Group connections are important in a web-based class. The cohort group entered with those connections already made, so it was easier for us to select from among our friends. But then I’m glad for my membership with the eclectic “Prancing Ponies” that absorbed a newcomer. Ultimately, I appreciate the broader support base provided by this course.

Assessments of the peer-based learning communities were discussed at length. Reports focused on the concerted effort involved in forming scholarly networks as well as the rewards:

Trying to connect with my group took the most time, but I've grown in new ways and made friendships that extend my network of professional colleagues to other schools.

This class really helped me focus on a dissertation topic. It forced me to meet with my peers, develop skills together, and consider feedback for furthering my ideas.

However, the negative aspects of cooperative group work were cited as least as frequently as the positive. The issue of communicating from a distance was the main reason given for this difficulty. Compounding this problem were issues involving the newcomers’ lack of familiarity with the cohort, and the limited capacity of the cohort members to accommodate the outsiders. The quality of engagement for the whole membership seemed partly colored by the rapid emergence of two “groups.” One group, the newcomers, felt ignored and largely hinted at experiences of frustration, misunderstanding, and even exclusion:

I received an email from one cohort member claiming I wasn’t “chatting” enough online—it came off as negative. I had emailed her for responses but had not heard back. All of this bothered me. In contrast, other responses from the group seemed too positive. It was as though I entered the process as a struggling outsider and remained that way. I really would have like more critiquing to improve on the process of scholarly writing.

What seems interesting is that for even the most problematic experiences described, some positive elements seemed to have occurred:

I made telephone contact with two group members and this proved to be very productive, with the chance to talk freely about my research questions. I took notes and ended up with great ideas for my draft writing and data collection plans, including instruments.

Barriers to communication seemed accentuated for the new students. Unlike the members of the cohort, the newcomers (8) may have experienced an outsider’s status. While supported in some respects, most thought the feedback they received from peers was shallow or cautious:

Collaborating from a distance is very difficult. We were not united by friendship or topic, as I was new. However, my circus group was willing to help as needed. But then again, while I enjoyed the contact, the comments on my work were polite, not substantive.

Anticipating this potential issue involving the integration of the newcomers, I had worked closely with the class during the first meeting so that everyone’s academic and professional interests would become known. The newcomers were then absorbed into small groups using similar dissertation topics as a major criterion for selection and as cerebral glue. However, the new members felt intellectually behind as they had not previously worked on dissertation ideas.

The newcomers all felt at a disadvantage. Despite feelings of inadequacy, the assistance extended by the instructor and some peers, as well as the developmental expectations for successful completion, helped these individuals to persevere. However, the agenda for interaction at the retreat, which featured hands-on collaborative exercises, had not unfolded as a platform for integration:

I needed more rapport with my team before going to just an online exchange. I think the cohort had an advantage in using the online course. They had worked together for so long and had already written in-depth, scholarly papers for this professor’s other class.

I had hoped that my group would provide a means of support, collaboration, and foundation. The excitement and level of contact between us diminished quickly, after one month. However, we did accomplish the warm-up activities, discussions of the syllabus, and the learning contract together, which helped me to establish my own agenda.

The cohort’s comments on their interactions with the newcomers were similarly mixed. Tensions, for some, persisted because they had not experienced “a pre-established comfort level” with these “virtual strangers.” Few invitations for in-person meetings had been extended to the newcomers.

The level of discomfort expressed about the integration of newcomers raises issues for the e-learning environment more generally: What is the potential of distance education for bringing together “virtual” strangers who have not met and will not during the online experience? And how might electronic communities deal with marginality or the exclusion of individuals who do not “fit” some familiar context—in this case, a cohort bonded with a preexisting history? As one “outsider” commented, “I would have liked to have been able to ‘chat’ online. I visited the site only to find ‘empty rooms.’” Issues of belonging and not belonging need serious attention in the world of distance education. By listening to students, more of value can be uncovered.

Cohort and noncohort members alike suggested that more whole-class meetings would have helped to build bridges and resolve internal tensions. The strain of communicative difficulty was most apparent for the newcomers, yet both groups made similar recommendations: “A combination of online with more face-to-face meetings would have proved productive. A few class sessions throughout the semester could have helped with building rapport with my team.”

Although I value this feedback and will act upon it in the future, I am also left wondering why the small groups did not take more initiative to meet on their own. Despite my encouragement to do so, this occurred only where individuals happened to be taking another class together. Students seemed to share only a moderate desire to develop their learning communities, partly as a consequence of their busy schedules: “Feedback from my peer group was limited. I participated very little due to the pressures of my job, which left little room for discussion via the Internet. I basically scrambled just to keep up with the individual writing and my contract.”

These results plant the doubt that web-based learning, at least at the doctoral level, can simply be an extension of earlier online degree programs. Can the e-learning context substitute for intermittent, whole-group meetings for bringing together doctoral students into viable partnerships of inquiry? Based on my own experience, it seems that the view perpetuated by distance education that the onus is on the learner to do the learning and take initiative for learning may be missing the reality of how people learn best. One would naturally expect that doctoral students, especially those embarking on their dissertations, could move ahead with minimal face-to-face contact, but this is not necessarily the case. As the participants themselves testified, ongoing interaction and stimulation is needed to help keep doctoral students (at least in educational leadership) focused on their goals, disciplined, and savvy to the academic process.

Finally, remarks about self-discipline varied and even represented extremes. Some students indicated that they had a “problem with self-motivation and self-starter issues” while others reported being “very self-disciplined” but having “to pace myself against others in completing and reviewing assignments in a timely manner.”

Student-teacher reciprocity. Generally, students claimed to have made heavier use of my feedback than that of their peers. The regular, detailed attention I paid to everyone’s writing was appreciated. Intensive writing-and-research activity formed the backbone of this course, and individual and group success was determined by how well expectations in this area were met. Students all reported having felt guided in ways consistent with their academic goals and dissertation topics. The feedback coincided with the area where I had spent the bulk of my time:

The professor provided detailed feedback on each draft submittal. The responses began with praise and followed with specific criticism. Her comments were designed to challenge and stretch my writing, and she always asks for more—which is what I need.

I felt fortunate that the instructor seemed totally immersed in my project—and we all felt this way—yet she remained objective. She was critical of details involving issues of confidentiality that could get me into trouble if I should fail to exercise caution, and she made lots of suggestions for helping me to think as a researcher.

Related support seemed to have occurred in the area of professional development. New connections had been forged between the professional world and the academic enterprise:

The instructor helped me to understand how my paper can be used to initiate professional contacts for the research needed, in my case with leading school officials, and she suggested resources for inclusion in my work.

I now see that my professional development is an integral part of my academic goals. I actually made some great contacts in the region, got access to school district data files through the professor’s communications with superintendents on my behalf, and even launched my interviews with this stakeholder group.

The context for student-teacher reciprocity was obviously the online learning environment, with some face-to-face contact. The professor’s role seemed more significant and central than I had at first imagined for a group of doctoral students. As typically reported:

I don't think I've had an instructor that showed more insight and concern, especially considering the remote delivery model and its constant challenges.

I grew the most in learning how advantageous a distance learning program could become as long as the professor supplied timely and appropriate feedback along with additional resources—in which case, we were fortunate to experience both.

Consistent with this feedback were statements favoring more personal contact with the professor.

A minority of students would have appreciated greater clarity of the course assignments. This concern probably emanated from the newcomers, as they had neither had the opportunities for clarity built into the previous course, nor were they accustomed to my working style or that of the cohort’s. In contrast, the majority (probably members from the cohort) felt that the retreat, syllabus, and ongoing contact with me provided them with the very grounding they sought:

The syllabus was very detailed and helpful in outlining expectations for the new course while building upon the previous course. The professor continued to be accessible 7/24, following through on all requests for clarity and dialogue. She also kept up an active email communications system, and so we all learned from each other’s questions and the threaded discussions she wove around key topics.

The regular personal meetings that I held with individuals and groups throughout the semester, and during flexible office hours, were only fleetingly mentioned. This had struck me as odd, as there were ample opportunities for providing feedback to this effect on the survey. I speculate that the students construed face-to-face contact as whole-group instruction only, because for them this represented a lost opportunity to operate as part of the larger whole. Feedback that led me to make this inference emphasized the importance of intervening sessions:

“I would have responded better to the entire experience if we had maybe had ‘live’ interim meetings,” and “I missed the interaction with Dr. X and wanted to be able to interact more with the instructor and my peers at the same time.”

Understanding the deeper dynamics that lie behind the readiness of learners for distance learning needs exploration. While doctoral students’ demands for flexibility and self-directed learning is certainly being heard by the distance education movement, one has to wonder about another need that my study reinforces. The traditional classroom experience appears to offer unique qualities that cannot, at this time, be substituted through online delivery. Strongly valued is whole-group instruction within “live” settings. What cannot be replaced is the inspiration and motivation that synergistically occurs as well as the human dynamics of knowledgeable instructors and peers, in addition to the richness, clarity, and depth of the dialogic event.

Discussion: Contradictions and Possibilities

The experience of web-enhanced learning that my student group reflected upon represents a mixed bag of contradictions and possibilities. Both of these dynamics—contradictions and possibilities—permeated the entire data set of multiple assessments. By treating the data more holistically, I came to see how tensions abound for students, not in relation to electronic communications per se but rather in relation to online course delivery at the doctoral level. The deeper contradictions that had been identified through this study did not exist apart from possibilities for proceeding more knowledgeably.

Contradictions between the desire for distance learning and human interaction—two irreconcilable worldviews at their extreme—were paramount in the data. The revolution that is occurring outside us in the world of distance education is being simultaneously reflected from within. The new mindset of distance learning, which necessitates a disruption to the old mindset, has been characterized as an occasion for personal “becoming” (Pass, 2002). While this journey of becoming for universities challenges educators to think and behave in new ways, this pathway also needs to be monitored. Alternative modes of learning enrich in some respects, and only poorly substitute in others, depending on the student population and the degree program.

Benefits of this E-Learning Experiment

The doctoral class sought the convenience of an e-learning option and, in the process of engagement, derived far greater gains. Benefits accrued were reflected in these major areas, quoted verbatim from the data: “not having to travel to class,” “working at my own pace,” “thinking deeply about my research questions,” “having more time and space to engage in reflective inquiry with myself and others,” “becoming a collaborative writing community at a more rapid and complex rate,” “ongoing and continuous excellent feedback from the instructor,” “gaining a significant amount of knowledge that’s useful for the day-to-day role as a school administrator,” “learning to use technology as a means of communicating in ways conducive with the growing demands of a paperless and global society,” and “learning how the individualization of a course to our needs models what we should be doing better as administrators (when evaluating teachers and developing professional growth opportunities).”

However, the gains listed pale in comparison to the distance literature that assumes too many benefits. Studies that best reflect the advantages I have identified focus on electronic writing communities that attempt to promote intellectual and social learning (e.g., Tannacito, 2002). Another parallel finding is that the effective incorporation of electronic technology into pedagogy offers multiple means of learning as well as student choice and critique (e.g., Myers, 2002). In addition, electronic classrooms that meaningfully engage school communities and stakeholder groups is highly valued (e.g., Myers, 2002).

Online educational leadership courses are expected to assist school administrators in developing ideas and skills for reculturing organizations for contemporary times (e.g., Boyer, 2001). It was in this critical and transformative context that my course was aligned with national policy advocating the increased use of technology for school leaders (e.g., Mullen, et al., 2002).

Drawbacks of this E-Learning Experiment

This doctoral group clearly missed the experience of being part of something larger than themselves—the sense of completeness that comes from the physical togetherness enabled by traditional seminar settings. Without the benefit of face-to-face contact within the context of whole-group discussion in a real setting, the electronic classroom proved less welcoming and hence deficient by comparison. Most participants experienced dissatisfaction with the online learning option because it was a poor substitution for interpersonal, sensory learning.

Major drawbacks were categorized as follows: “little face-to-face communication with classmates or professor,” “difficulty finding time to develop and hone group writing projects and to read others’ work,” “reconsider integrating non-cohort members into cohort groups in the future, as this wasn’t easy for some of us,” “members of the group understood the assignments in different ways and needed more professor input,” “need for an increase in class members’ contact through topics/questions that involve the whole class,” and “use of the WebCT email system was so erratic it required use of our regular email.”

Although it is assumed that “virtual classrooms … are meant to serve disciplined adult learners” (Matthews, 1999, p. 2), major adjustment will be required for success. Many adults simply are not ready for this change because they are not always as self-managed as one might expect or as apt to work collaboratively as one might assume. Also, they would need to have been psychologically prepared. My own group had not discussed the ways in which online learning at the doctoral level differs from the norm. For us, this conversation would have had to focus on emotional and intellectual adjustment. Such limits on discourse with live groups are a perennial problem inherited by hybrid courses, and certainly distance education.

It is believed that “a new mindset” will enable learners to take greater control over their own learning. Walker (2002) argues that the academic community needs to “move from the belief that face-to face is best to the belief that various environments support high quality learning” (p. 1). This assumption needs to be examined, especially as it applies to doctoral learning. That is one key issue. The other is that unless this newly evolving mindset is supported with structures in higher education that promote a transformation in consciousness through such mechanisms as support for faculty and student development; whole community dialogue; and realistic, problem-solving opportunities for learners, many mature student groups and tenure-earning faculty will struggle unnecessarily. Also, some doctoral students may view, as mine did, the online learning opportunity as a lesser form of education, especially considering their complex needs.

Institutions and faculty will need to develop mechanisms for systematically and patiently listening to students (Walker, 2002). This feedback will help us to both monitor and improve upon this revolution in e-learning (renamed “e-pedagogy” by Pass (2002) that is sweeping education. The importance of bringing more of a human face to the experience of e-learning, especially for noninstructional technology courses like my own, should not be lost.

Another Mixed Bag: “Virtually” Inclusive?

Cooperative learning using the circus groups was acknowledged by some of the students as viable and inclusive. But the newcomers and a minority of the cohort members reported otherwise. The view of some cohort students that individuals unknown to them should not have been incorporated into their working groups is disturbing. The transference from an online course to the everyday work of professionals is essential where leaders will need to exchange knowledge and build partnerships with multiple constituencies (Kankaanranta 2001), as well as persons of diverse backgrounds. Dynamics of exclusion require more scrutiny than ever as they can take subtle forms within the seemingly inclusive electronic community.

Based on my teaching experiences, the doctoral cohort culture can form an impenetrable enclave, partly out of the sheer need for survival and loyalty. This fear can translate into resistance towards all newcomers, making integration more trying than is necessary. The view of school leaders who rationalize exclusion based on personal comfort only serves to amplify the need for deeper engagement with such professional groups. Perhaps others can learn from this insider/outsider phenomenon. This cautionary tale underscores the need for diversity readiness awareness within our professional leadership ranks if online learning is to become truly transformative.

Recommendations for Improving Web-Enhanced Instruction

E-learning is here to stay as educational institutions forge ahead with unprecedented speed in this area of change. Faculty who publish the results of the electronic communities they develop are also changing the directions of many academic journals that now welcome computer-based teaching innovations in education. Many of us who teach in nontechnology disciplines such as educational leadership will find ourselves drawn into and accountable to this new realm of teaching and scholarship. Despite any concerns or even mixed reviews that we may have, the climate supports moving ahead and problem solving one’s way to solutions as a matter of course.

The doctorate in education, once a safe haven of tradition and elitism, has come to mean something different over time and will probably be redefining itself more fundamentally once again. Electronic delivery of the doctoral program has already been tested in one pocket of my university where it attempted to produce more broadly inclusive and diverse communities at the global level of higher education. But issues of student and faculty readiness cannot be overlooked in this zeal for change, nor can empirical studies that examine what is gained and what is lost.

Beyond my own context, the doctorate in educational leadership has already been altered by technology. Courses have been converted to the electronic medium and entire programs launched for distance (Boyer, 2001). As a novice distance educator attempting to examine my own mindset through an out-of-the-box experience, I now offer summaries through two sets of lenses—that of faculty and student. Only some of these perspectives overlap, as my students’ concerns did not typically extend to the instructor’s role in distance learning. Further, at times what students ask for contradicts with the very survival of tenure-earning faculty members. As a glaring example, e-learning is very time and labor intensive: Students expect complete availability and accessibility on the professor’s part, which sets the tone for the doctor being on call—day and night, weekends and holidays. I rose to this expectation because I believe that effective mentoring must extend to the distance medium.

The ideas I conclude with build upon the insights gained through this study, but they also echo the suggestions distance educators have made for assisting new online instructors.

Professor’s Recommendations

Novice distance educators can benefit from others’ advice for undertaking online ventures in higher education, specifically doctoral education. Just as faculty do not want to be viewed as “Luddites”— rioters who fought the displacement of factory workers by machinery in the 18^th Century ( Webster’s Dictionary)—we do not want to thoughtlessly “jump onto the band wagon” of distance education. It is with this goal of seeking a more balanced perspective during these e-revolutionary times that my suggestions are being offered. In no particular order:

· Be prepared to rethink such commonplace issues as preparation time, teaching load, class size, and contact hours for instructional delivery (Karlen, 2001).

· Obtain the institutional support necessary for the electronic delivery of courses (e.g., training, preparation time that includes reading distance education literature and course assessments, technical assistance, release time, stipends) as well as recognition for additional time spent mentoring and advising doctoral groups.

· Develop strategies for this labor-intensive enterprise by soliciting the help of competent individuals and various support groups (e.g., computer experts; student leaders; faculty advisors; rank, promotion, and tenure committees).

· Encourage changes in policy that govern faculty performance and evaluation: “Policies related to faculty evaluation may need reconsideration to include teaching and scholarship related to electronic initiatives” (Karlen, 2001, p. 3).

· Be highly flexible with your student groups but also set clear boundaries on your availability, as specified on the syllabus (account for days, evenings, weekends, and holidays) and reinforce these boundaries through consistent and fair action.

· At least two instructional technology experts should assist you in designing and delivering web-based courses, despite scarce resources, because “creating and maintaining a Web-based course is certainly not a one-person effort. There are many tasks complex enough to be assigned to specific people with the required technical skills” (Hsu, Marques, Hamza, & Alhalabi, 1999, p. 3).

· Monitor undesirable student enrolment numbers and section loading for single courses.

· Facilitate student growth beyond academic learning that is beneficial to the students’ skills development and professional goals.

· Make the online environment appealing, colorful, and alive with constant updates and include highlights of exemplary student work. Also consider using a metaphorical overlap (e.g., the circus) that is meaningful and structurally consistent.

· Disseminate the best student products produced in the online environment.

· Conduct a formal study of the online environment and pursue publication of it.

Students’ Recommendations

Novice distance learners, especially at the doctoral level, need suggestions for making their own online ventures as beneficial as possible. The recommendations that follow were solicited from my students for students more generally, and are represented from that perspective:

· Assume you will experience increased stress and a heavier workload than in traditional courses, as web-enhanced contexts are more challenging than traditional learning.

· Mix online interactions with face-to-face meetings, both in student groups and as a whole class. These forms of traditional interaction are necessary, or else small working groups may not fare well; rapport must be developed before moving to strictly online platforms.

· If the electronic system your university is using is static (like WebCT), then work via your personal email accounts with others, and submit assignments to the instructor through his or her personal email account.

· Develop topics for using the chat room option that are directly relevant to your interests in order to obtain a high return on investment. If the instructor utilizes this option, a once-weekly chat room requirement should constitute a percentage of the final grade.

· Carefully consider how your small online groups will be constituted and decide, with the instructor, the criteria for belonging (e.g., similar dissertation topics).

· Develop a learning contract that clarifies your learning goals and expectations for the course and if you make any changes, let the instructor know.

· Highlight your changes on all draft writing so that others can identify what has been added—respect others’ time! Also, where relevant, write a blurb outlining what suggested changes have been made. This will help prepare you for communicating the revisions you make on papers submitted to instructors, major professors, and editors.

· View your coursework as potential material for publication and, with the professor’s support, prepare papers for review with journals in your discipline.

· Be sure to fill out the IRB (Institutional Review Board) forms before you collect any information on human participants, or else you will be prevented from using the data and results for your dissertation.

· Utilize the technical support systems that the instructor has developed for you and be prepared for some uneven communications, especially with the converting and downloading of electronic files, from different computer platforms.

· Importantly, stay in close contact with the instructor who is your lifeline in the e-learning world, and openly share frustrations and agree on solutions. Communication is key to success, especially in the online environment, so do not repress your concerns.

· Discuss/negotiate tasks and workloads at the outset of every group assignment with all members; “step up to the plate” and be responsible for monitoring the quality of work before submitting even draft products to the instructor. Everyone’s contribution to a collective product reflects on everyone else. Toward this goal, do not hesitate to use all means to contact your group members, as not everyone checks regularly for email messages. But they will still expect to be included in the work!

· Remember that because we are in a pioneering era, frustrations are inevitable. As broadband/streaming applications become mainstream, e-learning could become exciting, but currently the technology is too under-developed for that experience to be routine.

Finally, what we learn as novice distance educators should be shared in a more official capacity. This way, the standards set for the expectation and assessment of teaching in higher education might become more realistic and fair. While e-learning may in fact be a mixed bag of contradictions and possibilities for the doctorate and the entire education profession at this time, we have yet to pull together as a problem-solving community. One way to proceed is to listen carefully to each other as well to our students and nontechnology faculty—those with fresh eyes whose learning has value.

Acknowledgements

The doctoral students who participated in this e-learning assessment study are much appreciated for their contributions. They granted permission for the anonymous use of the assessment data.

References

Ainley, J., Banks, D., & Fleming, M. (2002, April). The influence of information technologies on teaching and learning: Perspectives from five Australian schools. Paper presented at the annual conference of the American Educational Research Association, New Orleans.

Bothel, R., & Enfinger, J. (1999, September). You don’t have to go the whole distance. Technological Horizons in Education, 27(2), 1-5. [On-line] Available: http://www.thejournal.com.

Bowers, C. A. (1998). Toward a balanced perspective on the educational uses of computers: Advantages, myths, and the responsibilities of educators. International Journal of Leadership in Education, 1(1), 75-83.

Boyer, N. (2001). Building online learning: System insights into group learning in an online environment. Unpublished doctoral dissertation, University of South Florida, Florida.

Cisco Systems/Princeton Survey Research Associates. (2001). Education, innovation, & the Internet: Nobel Laureates look to the future. [report] [On-line] Available: www.cisco.com/nobel

Clark, R. E. (2000). Evaluating distance education: Strategies and cautions. The Quarterly Review of Distance Education, 1(1), 3-16.

Creanor, L. (2002). A tale of two courses: A comparative study of tutoring online. Open Learning, 17(1), 57-68.

Culp, G., Riffee, W., Starrett, D., Sarin, S., & Abrahamsen, H. (2001, January). Faculty rewards in digital instructional environments. Syllabus, 11-14. [report] [On-line] Available: www.syllabus.com.

Diem, R. A. (2002). Lessons for technological infusion: Preparing social studies teachers to use technology. CCSjournal, 1-3. [On-line] Available: http://www.cssjournal.com/archives.

Dominquez, P. S., & Ridley, D. (1999, September). Reassessing the assessment of distance education courses. Technological Horizons in Education, 27(2), 1-6. [On-line] Available: http://www.thejournal.com/magazine.

Fuks, H., Gerosa, M. A. & de Lucena, C. J. P. (2002). The development and application of distance learning courses on the Internet. Open Learning, 17(1), 23-38.

Gottesman, B. (2000). Peer coaching for educators (2^nd ed.). Lanham, MD: Scarecrow.

Hammonds, K. H. (2001, December). Circus acts. Fast Take, 53, 66-68. [On-line] Available: http://www.fastcompany.com/online/53/circus.

Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform. American Educational Research Journal, 39(1), 69-95.

Hsu, S., Marques, O., Hamza, M. K., & Alhalabi, B. (1999, September). How to design a virtual classroom: 10 easy steps to follow. Technological Horizons in Education, 27(2), [On-line] Available: http://www.thejournal.com/magazine.

Kankaanranta, M. (2001). Constructing digital portfolios: Teachers evolving capabilities in the use of information and communication technology. Teacher Development, 5(2), 259-275.

Karlen, J. M. (2001, Winter). Accreditation and assessment in distance learning. Academic Leadership, 1(4), 1-6. [On-line] Available: http://www.academicleadership.org.

Knowles, M. (1975). Self-directed learning. Chicago: Follett Publishing.

Lanahan, L. (2002, April). Beyond school-level Internet access: Support for instructional use of technology. [Brief report from the National Center for Education Statistics). Washington, DC: U.S. Department of Education Office of Educational Research and Development. [On-line] Available: http://nces.ed.gov/pubsearch.

Matthews, D. (1999, September). The origins of distance education and its use in the United States. Technological Horizons in Education, 27(2), 1-12. [On-line] Available: http://www.thejournal.com/magazine.

McNabb, M., Hawkes, M., & Rouk, U. (1999). Critical issues in evaluating the effectiveness of technology, 1-12. [On-line] Available: http://www.ed.gov/Technology.

Meehan, S., Holmes, B., & Tangney, B. (2001). Who wants to be a teacher? An exploration of the theory of communal constructivism at the chalk face. Teacher Development, 5

(2), 177-190.

Mendis, P. (2001, Winter). Teaching in the information age: Leadership aspects of integrated learning with technology in democratic environments, Academic Leadership, 1(2), 1-5. [On-line] Available: http://www.academicleadership.org

Mullen, C. A. (2001). The need for a curricular writing model for graduate students. Journal of Further and Higher Education, 25(1), 117-126.

Mullen, C. A. (with Stover, L., & Corley, B.) (2001). School accreditation and teacher empowerment: An Alabama Case. Teacher Development: An International Journal of Teachers’ Professional Development, 5(1), 121-137.

Mullen, C. A., Gordon, S. P., Greenlee, B., & Anderson, R. H. (2002). Capacities for school leadership: Emerging trends in the literature. International Journal of Educational Reform, 11(2).

Mullen, C. A. (2005). Fire and ice: Igniting and channelling passion in new qualitative researchers. New York: Peter Lang.

National Policy Board for Educational Administers for the Educational Leadership Constituent Council. (1995). NCATE program standards: Advanced programs in educational leadership for principals, superintendents, curriculum directors, and supervisors. Alexandria, VA: Educational Leadership Constituent Council. [On-line] Available: http://www.ncate.org/standard.

Nyquist, J. D., & Woodford, B. J. (2000). Re-envisioning the Ph.D.: What concerns do we have? Seattle, WA: Center for Instructional Development and Research, and University of Washington.

Palloff, R. M., & Pratt, K. (2001). Lessons from the cyberspace classroom: The realities of online teaching. San Francisco, CA: Jossey-Bass.

Project Team of Bruce Landon. (2002, March 26). Online educational delivery applications: A web tool for comparative analysis. [On-line] Available: http://www.c2t2.ca.

Roberts, B. (2002). Interaction, reflection and learning at a distance. Open Learning, 17(1), 39-55.

Seifter, H., Orpheus Chamber Orchestra, & Economy, P. (2001). Leadership ensemble: Lessons in collaborative management. New York: Henry Holt and Company.

Serwatka, J. A. (2002, April). Improving student performance in distance learning courses. Technological Horizons in Education, 29(9), 1-8. [On-line] Available: http://www.thejournal.com/magazine.

Simonson, M. (2000). Personal innovativeness, perceived organizational innovativeness, and computer anxiety: Updated scales. The Quarterly Review of Distance Education, 1(1), 69-76.

Sorensen, C., & Baylen, D. (2000). Perception versus reality: Views of interaction in distance education. The Quarterly Review of Distance Education, 1(1), 45-58.

Wallace, S. O., Acker-Hocevar, M., & & Sweatt, O. (2001). Developing an open systems view for assessing educational leadership. Journal of School Leadership, 11(3), 241-257.

Wheeler, S. (2000). Instructional design in distance education through telematics. The Quarterly Review of Distance Education, 1(1), 31-44.