Methods of Summative Assessment

For this paper, assessment of the instruction and learning that resulted from the course is discussed in some detail. This analysis draws primarily upon the anonymous online survey data that was generated and, as secondary data sources, the identifying assessment data (student self-assessment, peer assessment, and electronic correspondence). Results are based on a 100% return rate (25 surveys), averaging 10 typed pages each of anonymous data, in addition to other assessment data, which were assignment based. In total, 100 completed assessments were reviewed. The use of multiple instruments helped me to become as informed as possible about the quality of experience for learners, with an aim toward improving my pedagogy of online courses.

Circumstances converged, influencing my decision to develop a final assessment dubbed a “marathon survey” by students. My College of Education had not yet prepared a survey for online courses (personal communications, associate dean, March 2002), and the traditional forms of evaluation were not appropriate. Also, surveys created for online courses proved difficult to locate. Thus I developed an instrument of my own and shared it with my college.

The culture that supports alternative assessment is still in the early stages of development (Clark, 2000). Much confusion exists as to how best to identify and measure indicators for outcomes (Hargreaves, et al., 2002). Before circulating the exploratory instrument I first attempted to validate it, using three sources of support:

1.       Verma and Mallick’s (1999) tips for developing instruments that address pedagogical goals

2.       Learning objectives (e.g., “explore the role of instructional leaders in improving the current culture of evaluation”) specified in my course syllabus

3.       An instructional technology expert and online instructor with a background in assessment

With this professor I shared that my roles as instructor and assessor might bias the instrument or be perceived as doing this. This individual, who had a lack of involvement in the course and stake in its success, suggested adjustments that reflected an equal emphasis on positive and negative feedback in all critical areas of the instrument. Later, this expert helped to improve upon my analysis; in response, I undertook a holistic analysis of the data in addition to the thematic reading. Subsequently, a bigger picture evolved; this encompassed contradictions and possibilities of online learning that would have been missed by the thematic analysis alone.

The survey is an open-ended, summative assessment covering eight major areas with specific questions. Spaces were provided for writing. As Clark (2000) reminds us, questions asked inevitably inform only those areas identified for feedback. Consistent with Clark’s typology for evaluation, the survey items I developed addressed learning and outcomes, as well as instruction and technology. My summary of these areas follows:

        · Overall: distance learning (WebCT) experience

        · Organization of the technology-delivered course

        · Nature of contact with course leader and support expert

        · Evaluation of course on selected dimensions

        · Personal and professional growth outcomes

        · Comparison to other courses

        · Transfer of learning to the work environment

        · Recommendations regarding instructor and course

A graduate student (not in the course) circulated the online survey subsequent to the submission of grades. Having a student collect the data meant that I could maintain objectivity and participants could be honest, without fear of penalty. They were informed that their sharing would be anonymous and that the returned surveys would be forwarded to me without identifiers.

The Doctoral Cohort’s Profile: A Data-Generated Perspective

Information obtained from the online survey was used to produce a profile of this group of 25 doctoral students. Areas included demographics, stage of coursework, and academic/ professional goals. The cohort was completing coursework for the doctorate in educational leadership and anticipating the exam stage leading to doctoral candidacy. Individuals ranged in age from 26 to over 55 years old. Like other distance education groups, this one was older and more established than traditional classes; responsibilities extended to family and career. This mixed-gendered group was multilingual and ethically diverse (White, African American, and Latin).

Working full-time in the West Central Florida region, all were school practitioners and most former teachers. These lead teachers, program coordinators, assistant principals, principals, and (the minority) assistant superintendents and superintendents represented the elementary, middle, and secondary levels. Professional goals focused on the principalship for most and on the superintendency for a few. Several, who expressed a secondary interest in university teaching, hoped to write a book one day. Dissertation topics ranged but emphasized issues of mentoring and diversity in contexts of high poverty, low-performing schools and related policy contexts.

All valued high-quality professional development for teachers, and caring and appropriate pedagogy. A shared belief was that school reform should focus on improving the quality of education and life for students. The doctorate was viewed as an opportunity to offer the public education system something worthwhile beyond their current accomplishments.

A Summative Analysis through Doctoral Students’ Eyes

Value of Online Interaction

Mixed reviews about the value of online learning for doctoral students emerged from the assessment data. Students expressed desire for greater teacher involvement despite the face-to-face seminars, meetings with the professor, and daily access and feedback. Some problematic situations had been resolved at the end of the course but certainly not all. For example, the expectations for student-directed learning and professor-led instruction had not been fully determined, and the internal “insider/outsider” experience of some groups remained unsettled. Such complex issues go beyond the parameters of this online course and of electronic learning.

Issues of interaction and learning emerged as broad themes from the data. More specific issues concerned whole-group interaction, peer-based reciprocity, and student-teacher reciprocity.

Quality of interaction is a complex issue that needs more study if online learning is to be strengthened. Toward this end, Roberts (2002) offers differentiated views of interaction as reciprocity between parties, as discourse in which ideas are engaged, and as action in the world. All three levels of interaction were evident in the data and will be discussed.

The literature on distance education is much more positive than my results. For example, cooperative and enriched learning is presented as outcomes of Internet-based courses (Fuks, et al., 2002). However, the many unresolved challenges associated with nontraditional course delivery are also recognized. Notably, rewards have yet to be established, as part of the tenure and promotion process, for faculty using computer technology in teaching and scholarship, especially in innovative ways (Culp, Riffee, Starrett, Sarin, & Abrahamsen, 2001).

In the world of cyberspace learning, interactive patterns alter. Change in leadership and learner roles follow for teachers and students (Boyer, 2001). Palloff and Pratt (2001) encourage this new view of interactivity, asking faculty to relinquish control so that students can take the lead in learning. However, this philosophy should not be adopted as a rationale for abandoning students online but rather for facilitating their academic development and mentoring mosaics.

Whole-group interaction. Although the students were completely in support of the freedom that comes with web-enhanced delivery, all reported having missed the interpersonal dimension that being physically with others brings. This aspect of interaction—whole-group interactivity—was viewed as the weakest dimension of the online environment and hence the least satisfying aspect of the course. Students expressed their desire for live interaction, including face-to-face contact, bodily performance, and interchange with dynamic professors.

The most negative aspect of the WebCT experience was the lack of face-to-face contact with the professor. Web-based instruction made it impossible for us to benefit from her intellect, enthusiasm, and energy to the degree that a traditional model would allow.

WebCT course delivery is not conducive to performing “live.” What I didn’t like was the absence of body language/vocal tone, which lets me emotionally/intellectually feed off the immediacy and energy of others.

Interestingly, the most critical point made about the course was also the most positive, for it clarified the students’ vision and values. At least 30% of the students asserted that online learning should be reserved for “professors who are not particularly dynamic—the best professors should be in the “real” classroom with us!” Dialogue that transpires in a fully embodied, human context was missed by all as well as opportunities for pedagogical performance and mentorship and for “sparking ideas.” Students together provided support for the deeper cultural performances and rituals of learning that the electronic classroom probably has difficulty emulating.

Peer-based reciprocity. Peer learning was, overall, considered more beneficial than whole group interaction, although it also received mixed reviews. Not all members understood the value of learning from peers from the outset. “Initially, I did not realize how important it would be to communicate so closely with my group members. They were all reliable and provided continuous feedback by asking probing questions and responding to my paper.”

In my experiences with educational leadership groups, learning actively with others and sharing ideas and resources cannot be assumed. In competitive cultures, this is a learned capacity that not all adults bring to the higher education setting. Accordingly, the students documented their experiences of social learning as a critical dimension of academic socialization for leaders today. Although many had learned the value of collaboration in academia, they nonetheless struggled with transferring this learning to the online environment and their e-writing groups.

Group connections are important in a web-based class. The cohort group entered with those connections already made, so it was easier for us to select from among our friends. But then I’m glad for my membership with the eclectic “Prancing Ponies” that absorbed a newcomer. Ultimately, I appreciate the broader support base provided by this course.

Assessments of the peer-based learning communities were discussed at length.

Reports focused on the concerted effort involved in forming scholarly networks as well as the rewards.

Trying to connect with my group took the most time, but I've grown in new ways and made friendships that extend my network of professional colleagues to other schools.

This class really helped me focus on a dissertation topic. It forced me to meet with my peers, develop skills together, and consider feedback for furthering my ideas.

However, the negative aspects of cooperative group work were cited as least as frequently as the positive. The issue of communicating from a distance was the main reason given for this difficulty. Compounding this problem were issues involving the newcomers’ lack of familiarity with the cohort, and the limited capacity of the cohort members to accommodate the outsiders. The quality of engagement for the whole membership seemed partly colored by the rapid emergence of two “groups.” One group, the newcomers, felt ignored and largely hinted at experiences of frustration, misunderstanding, and even exclusion:

I received an email from one cohort member claiming I wasn’t “chatting” enough online—it came off as negative. I had emailed her for responses but had not heard back. All of this bothered me. In contrast, other responses from the group seemed too positive. It was as though I entered the process as a struggling outsider and remained that way. I really would have like more critiquing to improve on the process of scholarly writing.

What seems interesting is that for even the most problematic experiences described, some positive elements seemed to have occurred:

I made telephone contact with two group members and this proved to be very productive, with the chance to talk freely about my research questions. I took notes and ended up with great ideas for my draft writing and data collection plans, including instruments.

Barriers to communication seemed accentuated for the new students. Unlike the members of the cohort, the newcomers (8) may have experienced an outsider’s status. While supported in some respects, most thought the feedback they received from peers was shallow or cautious:

Collaborating from a distance is very difficult. We were not united by friendship or topic, as I was new. However, my circus group was willing to help as needed. But then again, while I enjoyed the contact, the comments on my work were polite, not substantive.

Anticipating this potential issue involving the integration of the newcomers, I had worked closely with the class during the first meeting so that everyone’s academic and professional interests would become known. The newcomers were then absorbed into small groups using similar dissertation topics as a major criterion for selection and as cerebral glue. However, the new members felt intellectually behind as they had not previously worked on dissertation ideas.

The newcomers all felt at a disadvantage. Despite feelings of inadequacy, the assistance extended by the instructor and some peers, as well as the developmental expectations for successful completion, helped these individuals to persevere. However, the agenda for interaction at the retreat, which featured hands-on collaborative exercises, had not unfolded as a platform for integration.

I needed more rapport with my team before going to just an online exchange. I think the cohort had an advantage in using the online course. They had worked together for so long and had already written in-depth, scholarly papers for this professor’s other class.

I had hoped that my group would provide a means of support, collaboration, and foundation. The excitement and level of contact between us diminished quickly after one month. However, we did accomplish the warm-up activities, discussions of the syllabus, and the learning contract together, which helped me to establish my own agenda.

The cohort’s comments on their interactions with the newcomers were similarly mixed. Tensions, for some, persisted because they had not experienced “a pre-established comfort level” with these “virtual strangers.” Few invitations for in-person meetings had been extended to the newcomers.

The level of discomfort expressed about the integration of newcomers raises issues for the e-learning environment more generally: What is the potential of distance education for bringing together “virtual” strangers who have not met and will not meet during the online experience? And how might electronic communities deal with marginality or the exclusion of individuals who do not “fit” some familiar context—in this case, a cohort bonded with a pre-existing history? As one “outsider” commented, “I would have liked to have been able to ‘chat’ online. I visited the site only to find ‘empty rooms.’” Issues of belonging and not belonging need serious attention in the world of distance education. By listening to students, more of value can be uncovered.

Cohort and noncohort members alike suggested that more whole-class meetings would have helped to build bridges and resolve internal tensions. The strain of communicative difficulty was most apparent for the newcomers, yet both groups made similar recommendations. “A combination of online with more face-to-face meetings would have proved productive. A few class sessions throughout the semester could have helped with building rapport with my team.”

Although I value this feedback and will act upon it in the future, I am also left wondering why the small groups did not take more initiative to meet on their own. Despite my encouragement to do so, this occurred only where individuals happened to be taking another class together. Students seemed to share only a moderate desire to develop their learning communities, partly as a consequence of their busy schedules: “Feedback from my peer group was limited. I participated very little due to the pressures of my job, which left little room for discussion via the Internet. I basically scrambled just to keep up with the individual writing and my contract.”

These results plant the doubt that web-based learning, at least at the doctoral level, can simply be an extension of earlier online degree programs. Can the e-learning context substitute for intermittent, whole-group meetings for bringing together doctoral students into viable partnerships of inquiry? Based on my own experience, it seems that the view perpetuated by distance education - that the onus is on the learner to do the learning and take initiative for learning - may be missing the reality of how people learn best. One would naturally expect that doctoral students, especially those embarking on their dissertations, could move ahead with minimal face-to-face contact, but this is not necessarily the case. As the participants themselves testified, ongoing interaction and stimulation are needed to help keep doctoral students (at least in educational leadership) focused on their goals, disciplined, and savvy to the academic process.

Finally, remarks about self-discipline varied and even represented extremes. Some students indicated that they had a “problem with self-motivation and self-starter issues” while others reported being “very self-disciplined” but having “to pace myself against others in completing and reviewing assignments in a timely manner.”

Student-teacher reciprocity. Generally, students claimed to have made heavier use of my feedback than that of their peers.The regular, detailed attention I paid to everyone’s writing was appreciated. Intensive writing-and-research activity formed the backbone of this course, and individual and group successes were determined by how well expectations in this area were met. Students all reported having felt guided in ways consistent with their academic goals and dissertation topics. The feedback coincided with the area where I had spent the bulk of my time.

The professor provided detailed feedback on each draft submittal. The responses began with praise and followed with specific criticism. Her comments were designed to challenge and stretch my writing, and she always asks for more—which is what I need.

I felt fortunate that the instructor seemed totally immersed in my project—and we all felt this way—yet she remained objective. She was critical of details involving issues of confidentiality that could get me into trouble if I should fail to exercise caution, and she made lots of suggestions for helping me to think as a researcher.

Related support seemed to have occurred in the area of professional development. New connections had been forged between the professional world and the academic enterprise:

The instructor helped me to understand how my paper can be used to initiate professional contacts for the research needed, in my case with leading school officials, and she suggested resources for inclusion in my work.

I now see that my professional development is an integral part of my academic goals. I actually made some great contacts in the region, got access to school district data files through the professor’s communications with superintendents on my behalf, and even launched my interviews with this stakeholder group.

The context for student-teacher reciprocity was obviously the online learning environment, with some face-to-face contact. The professor’s role seemed more significant and central than I had at first imagined for a group of doctoral students. As typically reported:

I don't think I've had an instructor who showed more insight and concern, especially considering the remote delivery model and its constant challenges.

I grew the most in learning how advantageous a distance learning program could become as long as the professor supplied timely and appropriate feedback along with additional resources—in which case, we were fortunate to experience both.

Consistent with this feedback were statements favoring more personal contact with the professor.

A minority of students would have appreciated greater clarity of the course assignments. This concern probably emanated from the newcomers, as they had neither had the opportunities for clarity built into the previous course, nor were they accustomed to my working style or that of the cohort’s. In contrast, the majority (probably members from the cohort) felt that the retreat, syllabus, and ongoing contact with me provided them with the very grounding they sought:

The syllabus was very detailed and helpful in outlining expectations for the new course while building upon the previous course. The professor continued to be accessible 7/24, following through on all requests for clarity and dialogue. She also kept up an active email communications system, and so we all learned from each other’s questions and the threaded discussions she wove around key topics.

The regular personal meetings that I held with individuals and groups throughout the semester, and during flexible office hours, were only fleetingly mentioned. This struck me as odd, as there were ample opportunities for providing feedback to this effect on the survey. I speculate that the students construed face-to-face contact as whole-group instruction only, because for them this represented a lost opportunity to operate as part of the larger whole. Feedback that led me to make this inference emphasized the importance of intervening sessions:

“I would have responded better to the entire experience if we had maybe had ‘live’ interim meetings,” and “I missed the interaction with Dr. X and wanted to be able to interact more with the instructor and my peers at the same time.”

Understanding the deeper dynamics that lie behind the readiness of learners for distance learning needs exploration. While doctoral students’ demands for flexibility and self-directed learning is certainly being heard by the distance education movement, one has to wonder about another need that my study reinforces. The traditional classroom experience appears to offer unique qualities that cannot, at this time, be substituted through online delivery. Strongly valued is whole-group instruction within “live” settings. What cannot be replaced is the inspiration and motivation that synergistically occurs as well as the human dynamics of knowledgeable instructors and peers, in addition to the richness, clarity, and depth of the dialogic event.

Discussion: Contradictions and Possibilities

The experience of web-enhanced learning that my student group reflected upon represents a mixed bag of contradictions and possibilities. Both of these dynamics—contradictions and possibilities—permeated the entire data set of multiple assessments. By treating the data more holistically, I came to see how tensions abound for students, not in relation to electronic communications per se but rather in relation to online course delivery at the doctoral level. The deeper contradictions that had been identified through this study did not exist apart from possibilities for proceeding more knowledgeably.

Contradictions between the desire for distance learning and human interaction—two irreconcilable worldviews at their extreme—were paramount in the data. The revolution that is occurring outside us in the world of distance education is being simultaneously reflected from within. The new mindset of distance learning, which necessitates a disruption to the old mindset, has been characterized as an occasion for personal “becoming” (Pass, 2002). While this journey of becoming for universities challenges educators to think and behave in new ways, this pathway also needs to be monitored. Alternative modes of learning enrich in some respects, and only poorly substitute in others, depending on the student population and the degree program.

Benefits of this E-Learning Experiment

The doctoral class sought the convenience of an e-learning option and, in the process of engagement, derived far greater gains. Benefits accrued were reflected in these major areas, quoted verbatim from the data: “not having to travel to class,” “working at my own pace,” “thinking deeply about my research questions,” “having more time and space to engage in reflective inquiry with myself and others,” “becoming a collaborative writing community at a more rapid and complex rate,” “ongoing and continuous excellent feedback from the instructor,” “gaining a significant amount of knowledge that’s useful for the day-to-day role as a school administrator,” “learning to use technology as a means of communicating in ways conducive with the growing demands of a paperless and global society,” and “learning how the individualization of a course to our needs models what we should be doing better as administrators (when evaluating teachers and developing professional growth opportunities).”

However, the gains listed pale in comparison to the distance literature that assumes too many benefits. Studies that best reflect the advantages I have identified focus on electronic writing communities that attempt to promote intellectual and social learning (e.g., Tannacito, 2002). Another parallel finding is that the effective incorporation of electronic technology into pedagogy offers multiple means of learning as well as student choice and critique (e.g., Myers, 2002). In addition, electronic classrooms that meaningfully engage school communities and stakeholder groups is highly valued (e.g., Myers, 2002).

Online educational leadership courses are expected to assist school administrators in developing ideas and skills for reculturing organizations for contemporary times (e.g., Boyer, 2001). It was in this critical and transformative context that my course was aligned with national policy advocating the increased use of technology for school leaders (e.g., Mullen, et al., 2002).

Drawbacks of this E-Learning Experiment

This doctoral group clearly missed the experience of being part of something larger than themselves—the sense of completeness that comes from the physical togetherness enabled by traditional seminar settings. Without the benefit of face-to-face contact within the context of whole-group discussion in a real setting, the electronic classroom proved less welcoming and hence deficient by comparison. Most participants experienced dissatisfaction with the online learning option because it was a poor substitution for interpersonal, sensory learning.

Major drawbacks were categorized as follows: “little face-to-face communication with classmates or professor,” “difficulty finding time to develop and hone group writing projects and to read others’ work,” “reconsider integrating non-cohort members into cohort groups in the future, as this wasn’t easy for some of us,” “members of the group understood the assignments in different ways and needed more professor input,” “need for an increase in class members’ contact through topics/questions that involve the whole class,” and “use of the WebCT email system was so erratic it required use of our regular email.”

Although it is assumed that “virtual classrooms … are meant to serve disciplined adult learners” (Matthews, 1999, p. 2), major adjustment will be required for success. Many adults simply are not ready for this change because they are not always as self-managed as one might expect or as apt to work collaboratively as one might assume. Also, they would need to have been psychologically prepared. My own group had not discussed the ways in which online learning at the doctoral level differs from the norm. Such limits on discourse with live groups are a perennial problem inherited by hybrid courses, and certainly distance education.

It is believed that “a new mindset” will enable learners to take greater control over their own learning..” Walker (2002) argues that the academic community needs to “move from the belief that face-to face is best to the belief that various environments support high quality learning” (p. 1). This assumption needs to be examined, especially as it applies to doctoral learning. That is one key issue. The other is that unless this newly evolving mindset is supported with structures in higher education that promote a transformation in consciousness ( through such mechanisms as support for faculty and student development, whole community dialogue, and realistic, problem-solving opportunities for learners), many mature student groups and tenure-earning faculty will struggle unnecessarily. Also, some doctoral students may view, as mine did, the online learning opportunity as a lesser form of education, especially considering their complex needs.

Institutions and faculty will need to develop mechanisms for systematically and patiently listening to students (Walker, 2002). This feedback will help us to both monitor and improve upon this revolution in e-learning (renamed “e-pedagogy” by Pass (2002) that is sweeping education. The importance of bringing a more human face to the experience of e-learning, especially for noninstructional technology courses like my own, should not be lost.

Another Mixed Bag: “Virtually” Inclusive?

Cooperative learning using the circus groups was acknowledged by some of the students as viable and inclusive. But the newcomers and a minority of the cohort members reported otherwise. The view of some cohort students that individuals unknown to them should not have been incorporated into their working groups is disturbing. The transference from an online course to the everyday work of professionals is essential where leaders will need to exchange knowledge and build partnerships with multiple constituencies (Kankaanranta, 2001), as well as persons of diverse backgrounds. Dynamics of exclusion require more scrutiny than ever as they can take subtle forms within the seemingly inclusive electronic community.

Based on my teaching experiences, the doctoral cohort culture can form an impenetrable enclave, partly out of the sheer need for survival and loyalty. This fear can translate into resistance towards all newcomers, making integration more trying than is necessary. The tendency to rationalize exclusion based on familiarity only serves to amplify the need for deeper engagement with professional groups. Perhaps others can learn from the insider/outsider phenomenon that occurred. This cautionary tale underscores the need for diversity awareness within our professional leadership ranks if online learning is to become truly transformative.

Recommendations for Improving Web-Enhanced Instruction

E-learning is here to stay as educational institutions forge ahead with unprecedented speed in this area of change. Faculty who publish the results of electronic community development are changing the directions of academic journals; many now welcome computer-based teaching innovations in education. Those of us who teach in nontechnology disciplines such as educational leadership will find ourselves drawn into and accountable to this new realm of teaching and scholarship. Despite any concerns that we may have, the climate supports moving ahead and problem solving one’s way to solutions. However, ample time for reflection is needed.

The doctorate in education, once a safe haven of tradition and elitism, is now being fundamentally redefined. Electronic delivery of the doctoral program has already been tested in one pocket of my university, but the attempt to produce a broadly inclusive community at the global level faltered. In other places, the doctorate has been altered by technology but the results are unknown at this time. In some instances, entire doctoral programs have been launched for distance without sufficient assessment of student needs (Boyer, 2001). Although much remains uncertain at this time, pedagogical studies that examine gains and losses are clearly needed.

Reflecting on my learning as a novice distance educator, I now offer summaries through the lenses of faculty and student. Only some of these views overlap, as my students’ concerns did not typically extend to the instructor’s role in distance learning. Further, what students sometimes asked for contradicts the very survival of tenure-earning faculty. As a glaring example, e-learning is very time-and-labor intensive: Students expect complete availability and accessibility, which requires that the “the doctor” be “on call” day and night, weekends and holidays. I rose to this expectation because I believe that effective mentoring must extend to the distance medium.

The ideas I conclude with build upon the insights gained through this study and echo the researchers’ suggestions for assisting new online instructors.

Professor’s Recommendations

Novice distance educators can benefit from advice for undertaking online ventures in higher education, specifically doctoral education. Just as faculty do not want to be viewed as “Luddites”— rioters who fought the displacement of factory workers by machinery in the 18^th Century ( Webster’s Dictionary)—we do not want to thoughtlessly “jump onto the band wagon” of distance education. It is with this goal of seeking a more balanced perspective during these e-revolutionary times that my suggestions are being offered. In no particular order:

·         Be prepared to rethink such commonplace issues as preparation time, teaching load, class size, and contact hours for instructional delivery (Karlen, 2001).

·         Obtain the institutional support necessary for the electronic delivery of courses (e.g., training, preparation time that includes reading distance education literature and online course assessments, technical assistance, release time, stipends).

·         Seek recognition for additional time spent mentoring and advising doctoral groups.

·         Develop strategies for this labor-intensive enterprise by soliciting the help of competent individuals and various support groups (e.g., computer experts, student leaders).

·         Encourage changes in policy that govern faculty performance and evaluation: “Policies related to faculty evaluation may need reconsideration to include teaching and scholarship related to electronic initiatives” (Karlen, 2001, p. 3).

·         Be flexible but also set expectations for your availability, as specified on the syllabus (including weekends); reinforce the boundaries through consistent and fair action.

·         Utilize at least two computer experts because “creating and maintaining a Web-based course is certainly not a one-person effort. There are many tasks complex enough to be assigned to specific people with the required technical skills” (Hsu, et al., 1999, p. 3).

·         Monitor undesirable student enrolment numbers and section loading for single courses.

·         Extend academic learning to incorporate skills development and professional goals.

·         Make the online environment colorful and appealing with constant updates; incorporate exemplary student samples. Also consider using a metaphorical overlap (e.g., the circus) that is meaningful and structurally consistent.

·         Disseminate the best student products produced in the online environment.

·         Conduct a formal study of the online environment and pursue publication of it.

Students’ Recommendations

Novice distance learners, especially at the doctoral level, need suggestions for making their own online ventures as beneficial as possible.

·         Assume you will experience increased stress and a heavier workload than in traditional courses, as web-enhanced contexts are more challenging than traditional learning.

·         Mix online interactions with face-to-face meetings, both in student groups and as a whole class. Rapport must be developed before moving to strictly online platforms.

·         If the electronic system your university is using is static (like WebCT), then work via your personal email accounts with others, and submit assignments to the instructor through his or her personal email account.

·         Develop topics for using the chat room option that are directly relevant to your interests. If the instructor utilizes this option, it should count as a percentage of your final grade.

·         Carefully consider how your small online groups will be constituted and perhaps changed. Negotiate the criteria for belonging (e.g., similar dissertation topics).

·         Develop a learning contract that clarifies your learning goals and expectations for the course and, if you make changes, let the instructor know.

·         Highlight your changes on all draft writing so that others can identify what has been added. Also, write blurbs outlining the changes you have made. This will help prepare you for communicating the revisions you make on papers submitted to others.

·         View your coursework as potential material for publication and, with the support of professors, prepare papers for review with journals in your discipline.

·         Complete the IRB (Institutional Review Board) forms before you collect any data on human participants, or else you will not be able to use the information later.

·         Utilize the technical support systems that the instructor has developed for you and be prepared for some uneven communications due to different computer platforms.

·         Importantly, communicate regularly with the instructor—your lifeline in the e-learning world—and share frustrations and find solutions together. Do not repress your concerns.

·         In your peer groups, negotiate tasks for assignments. Closely monitor the quality of work being produced. Everyone’s contribution reflects on yours.

·         Remember that because we are in a pioneering era, frustrations are inevitable. As broadband/streaming applications become mainstream, e-learning could become exciting.

Finally, what we learn as novice distance educators should be shared in a more official capacity, at the level of policy-making. This way, the standards set for the expectation and assessment of teaching in higher education can become more realistic and fair. While e-learning may in fact be a mixed bag of contradictions and possibilities for the doctorate and the entire education enterprise at this time, we have yet to pull together as a problem-solving community. One way to proceed is to listen carefully to those with fresh eyes whose learning has value.

Acknowledgements

The author wishes to thank the doctoral students who participated in this e-learning study and appreciates the permission granted for the anonymous use of the assessment data.

References

Ainley, J., Banks, D., & Fleming, M. (2002, April). The influence of information technologies on teaching and learning: Perspectives from five Australian schools. Paper presented at the annual conference of the American Educational Research Association, New Orleans.

Bothel, R., & Enfinger, J. (1999, September). You don’t have to go the whole distance. Technological Horizons in Education, 27(2), 1-5. [On-line] Available: www.thejournal.com.

Bowers, C. A. (1998). Toward a balanced perspective on the educational uses of computers: Advantages, myths, and the responsibilities of educators. International Journal of Leadership in Education, 1(1), 75-83.

Boyer, N. (2001). Building online learning: System insights into group learning in an online environment. Unpublished doctoral dissertation, University of South Florida, Florida.

Cisco Systems/Princeton Survey Research Associates. (2001). Education, innovation, & the Internet: Nobel Laureates look to the future. [report] [On-line] Available: www.cisco.com/nobel.

Clark, R. E. (2000). Evaluating distance education: Strategies and cautions. The Quarterly Review of Distance Education, 1(1), 3-16.

Creanor, L. (2002). A tale of two courses: A comparative study of tutoring online. Open Learning, 17(1), 57-68.

Culp, G., Riffee, W., Starrett, D., Sarin, S., & Abrahamsen, H. (2001, January). Faculty rewards in digital instructional environments. Syllabus, 11-14. [report] [On-line] Available: www.syllabus.com.

Diem, R. A. (2002). Lessons for technological infusion: Preparing social studies teachers to use technology. CCSjournal, 1-3. [On-line] Available: www.cssjournal.com/archives.

Fuks, H., Gerosa, M. A. & de Lucena, C. J. P. (2002). The development and application of distance learning courses on the Internet. Open Learning, 17(1), 23-38.

Gottesman, B. (2000). Peer coaching for educators (2^nd ed.). Lanham, MD: Scarecrow.

Hammonds, K. H. (2001, December). Circus acts. Fast Take, 53, 66-68. [On-line] Available: www.fastcompany.com/online/53/circus.

Hargreaves, A., Earl, L., & Schmidt, M. (2002). Perspectives on alternative assessment reform. American Educational Research Journal, 39(1), 69-95.

Hsu, S., Marques, O., Hamza, M. K., & Alhalabi, B. (1999, September). How to design a virtual classroom: 10 easy steps to follow. Technological Horizons in Education, 27(2), [On-line] Available: www.thejournal.com/magazine.

Kankaanranta, M. (2001). Constructing digital portfolios: Teachers evolving capabilities in the use of information and communication technology. Teacher Development, 5(2), 259-275.

Karlen, J. M. (2001, Winter). Accreditation and assessment in distance learning. Online Journal of Academic Leadership, 1(4), 1-6. [On-line] Available: www.academicleadership.org.

Knowles, M. (1975). Self-directed learning. Chicago: Follett Publishing.

Lanahan, L. (2002, April). Beyond school-level Internet access: Support for instructional use of technology. [Brief report from the National Center for Education Statistics). Washington, DC: U.S. Department of Education Office of Educational Research and Development. [On-line] Available: www.nces.ed.gov/pubsearch.

Matthews, D. (1999, September). The origins of distance education and its use in the United States. Technological Horizons in Education, 27(2), 1-12. [On-line] Available: www.thejournal.com/magazine.

McNabb, M., Hawkes, M., & Rouk, U. (1999). Critical issues in evaluating the effectiveness of technology, 1-12. [On-line] Available: www.ed.gov/technology.

Mendis, P. (2001, Winter). Teaching in the information age: Leadership aspects of integrated learning with technology in democratic environments. Online Journal of Academic Leadership, 1(2), 1-5. [On-line] Available: www.academicleadership.org.

Mullen, C. A. (2001). The need for a curricular writing model for graduate students. Journal of Further and Higher Education, 25(1), 117-126.

Mullen, C. A., Gordon, S. P., Greenlee, B., & Anderson, R. H. (2002). Capacities for school leadership: Emerging trends in the literature. International Journal of Educational Reform, 11(2), 158-198.

National Policy Board for Educational Administration. (1995). NCATE program standards: Advanced programs in educational leadership for principals, superintendents, curriculum directors, and supervisors. Alexandria, VA: Educational Leadership Constituent Council. [On-line] Available: www.ncate.org.

Nyquist, J. D., & Woodford, B. J. (2000). Re-envisioning the Ph.D.: What concerns do we have ? Seattle, WA: Center for Instructional Development and Research.

Palloff, R. M., & Pratt, K. (2001). Lessons from the cyberspace classroom: The realities of online teaching. San Francisco, CA: Jossey-Bass.

Pass, E. (2002). E-pedagogy: Deleuze and Guattari in the web-design class. Kairos: A journal for teachers of writing in webbed environments, 1(6), 1-6. [On-line] Available: www.english.ttu.edu.

Project Team of Bruce Landon. (2002, March 26). Online educational delivery applications: A web tool for comparative analysis. [On-line] Available: www.c2t2.ca.

Roberts, B. (2002). Interaction, reflection and learning at a distance. Open Learning, 17(1), 39-55.

Seifter, H., Orpheus Chamber Orchestra, & Economy, P. (2001). Leadership ensemble: Lessons in collaborative management. New York: Henry Holt and Company.

Serwatka, J. A. (2002, April). Improving student performance in distance learning courses. Technological Horizons in Education, 29(9), 1-8. [On-line] Available: www.thejournal.com.

Simonson, M. (2000). Personal innovativeness, perceived organizational innovativeness, and computer anxiety: Updated scales. The Quarterly Review of Distance Education, 1(1), 69-76.

Sorensen, C., & Baylen, D. (2000). Perception versus reality: Views of interaction in distance education. The Quarterly Review of Distance Education, 1(1), 45-58.

Twale, D. J., & Kochan, F. K. (2000). Assessment of an alternative cohort model for part-time students in an educational leadership program. Journal of School Leadership, 10, 188-208.

Verma, G. K., & Mallick, K. (1999). Researching education: Perspectives and techniques. London: Falmer.

Wallace, S. O., Acker-Hocevar, M., & Sweatt, O. (2001). Developing an open systems view for assessing educational leadership. Journal of School Leadership, 11(3), 241-257.

Wheeler, S. (2000). Instructional design in distance education through telematics. The Quarterly Review of Distance Education, 1(1), 31-44.