Empirical Research
CONDUCTING EPIDEMIOLOGICAL RESEARCH IN SOUTH AFRICA: CHALLENGES IN THE DATA COLLECTION PROCESS
By Yegan Pillay
Feb 13, 2007 - 4:38:38 PM

Yegan Pillay Ph.D.

Department of Psychiatry and Mental Health

University of Cape Town

Athens, OH-45701

Alan J. Flisher Ph.D., F.C. Psych (S.A.)

Department of Psychiatry and Mental Health

University of Cape Town

Crick Lund , M.A.

Department of Psychiatry and Mental Health

University of Cape Town

Abstract

This article addresses the challenges faced by research supervisors and research assistants while collecting data to explore the prevalence of psychiatric disorders among children and adolescents in South Africa. The article focuses on budgetary constraints, interviewee expectations, moral issues, limited confidentiality, fatigue factors and racial issues that threatened the completion of the project. These challenges are examined from the emic-etic theoretical paradigm and the interactions between the research supervisors, the research assistants and the research participants are analyzed. The authors propose recommendations that can serve to prepare researchers who embark on conducting research in similar socio-cultural environments.

KEY WORDS: Epidemiological, challenges, Xhosa, emic, etic

Introduction

The use of structured diagnostic interview schedules is rapidly becoming a popular methodological strategy to explore the prevalence of similar psychiatric disorders across dissimilar cultures (Bird, 1996; Bravo, Canino, Ribio-Stipec, & Woodbury-Farina, 1991; Canino & Rubio-Stipec, 1993; Robins, Heltzer, et.al., 1981). Researchers using various versions of the diagnostic interview schedules report that the instrument is potentially useful for conducting epidemiological research especially in settings such as Africa where resources and the availability of skilled clinicians are limited (Robertson et al, 1999).

However, the majority of the research material documenting the use of the structured diagnostic interview schedules do not focus on the ?behind the scenes? activities that facilitate the data collection process. The primary focus of this paper is to present a narrative of the unforeseen challenges faced by the researchers and research assistants during a study to explore the prevalence of psychiatric disorders among the youth in Khayelitsha, South Africa.

Culturally sensitive researchers have to continually adapt the methodological designs so that the process of inquiry is synthesized with the cultural characteristics of the group that is being studied. Jahoda (1995) asserts that cross-cultural studies focus on the theoretical and methodological aspects of research and tend to ignore the influence of emic and etic factors on the outcome of research. This anomaly prevails despite the introduction of the emic-etic dichotomy into the discipline of psychology as early as the 1960?s. With this assertion in mind, the emic-etic theoretical paradigm will be used as a point of departure to describe the research process and to facilitate an understanding of some of the challenges that plagued the project.

Emic and Etic Model

The emic-etic distinction emerged in the late 1940?s, and has become part anthropological and cross cultural literature as a result of linguist Kenneth Pike?s search for a syntactical counterpart to the phonological and morphological terms ?phoneme? and ?morpheme? (Edwards, 1997). Pike (1954) introduced the term ?emic? extracted from ?phonemic? and ?etic? from ?phonetic? to characterize linguistically based cultural phenomena.

The emic approach can be defined as a structural one whereby the investigator assumes that human behavior is patterned even though the members of the society being studied may not be aware of many units of the structuring (French, 1963). In other words emic constructs are an insider?s perspective that defines conceptual schemes and categories that are considered to be meaningful and appropriate for individuals operating within a given cultural paradigm.

Etic constructs on the other hand are conceptual schemes and categories that are expressed and defined by that which is considered to be meaningful and appropriate to the observer. In other words items of behavior are not examined in the systems that they occur in but by the criteria imposed on the system by the observer (French, 1963).

The role players in this research project can be divided into three distinct sub-systems. They are depicted in Figure 1 as three concentric circles. The core represents the research participants, the second layer represents the research assistants and the outer layer represents the research supervisors. The researcher supervisors exert influence on the research assistants, who in turn influence the research participants. There is also the reciprocal influence of the research participants on the research assistants who influence the research supervisors. It was evident that a synergistic cyclical interaction occurred in the various sub-group permutations during this study.

Figure 1 Research Sub-system (Figure not included)

Background

The study described in this paper was conducted by the Department of Psychiatry and Mental Health at the University of Cape Town in South Africa. The primary objective of the study was to translate and adapt the Diagnostic Interview Schedule for Children Version Four (DISC-IV) to make it suitable for use among Xhosa speaking South Africans. Moreover the DISC-IV was examined for its validity and its test-retest reliability. The DISC-IV is a structured psychiatric interview for use in clinical and epidemiological studies and is organized into six diagnostic sections: Anxiety Disorders, Mood Disorders, Disruptive Disorders, Substance Use Disorders, Schizophrenia and Miscellaneous Disorders. Furthermore there are parallel parent (DISC-P) and child (DISC-Y) components of the instrument.

The respondents in this study comprised 105 parent/care giver and child pairs who resided in Khayelitsha, South Africa. Khayelitsha is an informal settlement that lies on the outskirts of Cape Town with a population of approximately 300,000. Khayelitsha, similar to other South African informal settlements, is characterized by high levels of crime, over-crowding, poor transportation, limited infrastructure and a dire shortage of funding and skilled clinicians for much needed mental health services. Approximately 50% of the participants were recruited randomly from the community and the remainder were patients receiving services at the Empilweni project. This project is based in the Department of Psychiatry and Mental Health at the University of Cape Town and provides community based mental health services for children and adolescents in Khayelitsha.

Data Collection

Six research assistants were responsible for obtaining data from interviews with parents/caregivers and children. They were recruited through advertisements in the local media. Selection was conducted through interviews and the successful candidates were exposed to a short but intensive training course. A major consideration for selection was fluency in both English and Xhosa. Xhosa is the home language of almost all Khayelitsha residents. The final data collection team comprised 2 males and 4 females. Four of the recruits were undergraduate students and two were unemployed post-graduates. The research assistants worked in pairs, one interviewed the parent and the other interviewed the child. The same researchers returned after approximately two weeks to conduct the second interviews.

Challenges with the Data Collection

Interviewee Expectations

Many participants in this study perceived the research assistants to be representatives from the Western Cape social support services, despite being informed that they were part of a research project. Patients who were referred by the Empilweni Clinic proved to be a greater challenge than the community sample. Their willingness to participate in the study was motivated primarily by the misperception that that the research assistants were providing follow up to an initial consultation at the clinic. The research assistants were oblivious to the dynamics of the participant?s previous interaction with the clinic staff and were ill prepared to answer questions such as ?when will my social assistance come through?? ?Does my child qualify for state assistance?? ?Do you have my medication?? ?Which school will my child attend??

Parents were also inquisitive about the diagnoses of their children and specifically their prognoses. This was motivated by the misperceived ?expert? status of the research assistants. Since the research assistants were not mental health practitioners, they became frustrated because they felt out of their depth when these questions were posed. Parents also questioned the objective of the interviews because no tangible benefits were immediately discernable to them. Research assistants reported that they would sometimes ?embellish? responses to encourage continued participation in the project. This ?underhanded? strategy was used because research assistants were compensated per interview, and since it was a logistical challenge to access compliant participants, every effort was engaged in to reduce attrition for the follow up interview. Some of the research participants were promised that they would be receiving individualized reports of the findings of the study to avoid attempting to answer some of the pertinent questions. Provision of individualized reports was not part of the research design and no such reports were in fact provided.

Participants defined the interaction with the research assistants differently to the expectations of research assistants as well as the research supervisors. From the emic-etic theoretical framework, the insider/outsider dichotomy became clearly evident. The main challenge faced by the research assistants (observers) was the assumption by the research participants (observed) that the interviewers were social workers or government officials who could provide the much needed resources. It was important for the respondents to attend to their priorities such as taking care of their children, preparing dinner, or watching television while the priorities of the research assistant remained centered on the data collection. This insidious dissonance manifested as frustration on the part of the research assistants as well as the research participants because what was considered meaningful and appropriate for one sub-system (Figure 1) took on a different dimension for the other subsystem.

Research Assistant Issues

Interpersonal and intrapersonal factors played a significant role in the dynamics within the research team. It became obvious as the data collection progressed that the financial incentive was the driving force behind research assistant participation in this study. The relatively competitive remuneration rate in a province with high unemployment, attracted a variety of applicants with differing levels of expertise and backgrounds. This diversity among the research assistants contributed significantly to their varying experiences in the field.

During the weekly supervision meetings it was noted that the research assistants spent an inordinate amount of time addressing the social plight of the study participants. It was evident that this had a profound impact on the data collection. The social situation of the interviewees influenced the research assistants to the extent that some, especially the social science majors, addressed the immediate needs of the community members rather than collected data. A research pair reported that they arrived one evening with the intention of beginning with the interview only to realize that the children in the family had not eaten all day because there was no money to buy food. Despite being unemployed herself one member of the team went out to buy food for the family before proceeding with the interview. On the other hand some research assistants were able to divorce themselves from the social conditions and focus on the mechanical task of collecting data. This group attained higher productivity with the accompanying higher monetary reward than their colleagues. This resulted in discord between different members of the research team as some felt that they were working harder but were not being compensated for their extra effort.

Since the research assistants had to work in pairs and the research team comprised both students and unemployed individuals, scheduling interviews posed a problem. The unemployed research assistants were available during the day whereas the students were available in the afternoons and early evenings. The situation was exacerbated further when a male member entered into romantic relationships with two female members. This love triangle created tension in the team and limited the pairing permutations of the research assistants. The research supervisors were challenged to diffuse the volatile atmosphere that permeated the weekly supervisory sessions. The meetings became an opportunity for emotional catharsis rather than to take care of administrative matters for which it was initially intended. It was fortuitous that the research supervisors were psychologists and used the ?administrative? sessions to process the experiences of the research assistants.

Etic and emic principles could be applied to the research supervisors/research assistant dyad. The research assistants accused the research supervisors of being aloof and not emphatic to experiences that were appropriate and meaningful to them. Moreover the research assistants inferred that the research supervisors were oblivious to the interaction that they had with the research participants, eg. compensating research participants or purchasing food to motivate participation. This perceived insensitivity challenged the efficacy of the data collection and threatened to jeopardize the project.

Financial Issues

Remuneration of research assistants was an issue that frequently plagued the project. The research supervisor had decided at the outset that research assistants would be paid in advance for the data collected. This strategy was employed because the cash strapped students and the unemployed post?graduates did not have funds at the beginning of the project to pay for their travel and other incidental expenses to conduct the interviews. This meant that the research supervisor had to estimate the number of interviews that would be conducted for the month for individual assistants. This posed a challenge especially since this was a seminal study and the diversity among the research team made accurate projections difficult. Initially all research assistants started with the same target of completed interviews. During the next month the actual number of questionnaires completed was compared with the projected estimate. The necessary adjustments were then made by either compensating the research assistants for questionnaires completed that were not accounted for in the previous projection or adjusting their remuneration to account for shortfalls in the actual versus the projected figures. The constant fluctuation in the output for each research assistant proved to be not only an administrative challenge but also fueled the discord that prevailed in the group because remuneration was based on productivity.

During the latter phase of the project, some research assistant salaries were relatively low when compared to their colleagues due to overestimation or reduced productivity. At one stage, two of the research assistants owed money to the Department of Psychiatry and Mental Health for the advances received that did not match their productivity in the following pay period. The financial situation and other contributory factors addressed later in this paper had a cumulative effect on the group morale to the extent that the completion of the data collection was placed in jeopardy. The research assistants contested the accuracy of the computation and challenged the formula used to calculate their remuneration. Most felt that the Department of Psychiatry and Mental Health and the University of Cape Town were using underhanded tactics to deny them fair compensation for their work.

The perceived underhandedness was exacerbated when income tax was levied on their salaries. An assumption was made by the research manager that the research assistants understood that income tax would be deducted from the salaries as it done for any employee of the university. This matter was not addressed during the orientation session and the research assistants assumed that they would receive ninety South African rands (approximately 11 US dollars) for each completed questionnaire. Research assistants with higher levels of productivity were adversely affected because a larger proportion was appropriated from their salaries in taxes than their colleagues who completed fewer interviews. These individuals in particular questioned the need for increased productivity and whether it was of any value to continue to collect data. It was the intervention of the project supervisors who promised that they would be contacting the personnel office to see what can be done about the taxes that preempted a possible stoppage in the data collection.

Research assistants were compensated on a weekly basis for telephone and travelling expenses that were incurred during the data collection process. The accounts department at the university required the furnishing of receipts prior to reimbursement. This meant that the research assistants had to initially pay for these expenses and claim them back at a later date. This placed an added financial burden on the students and unemployed research assistants. Furthermore any assistance provided by the research assistants either altruistically or to motivate participation in the study could not be compensated from petty cash. This resulted in the inflation of telephone and travelling costs to offset this expenditure. There was no provision in the system to detect ?phantom? claims. Since the data collection had reached a crucial stage and financial matters became a sensitive issue within the group, the project supervisors would at times turn a blind eye to obvious exorbitant claims that did not match the productivity for the week. These unforeseen expenses exerted strain on the limited budget for the project.

Moreover the real time to complete the interviews exceeded the projected time frame. This resulted in a huge burden being placed on the funding for the payment of research assistants and the project supervisor. Additional funding had to be secured to ensure the completion of the project.

The issues related to the financial challenges that were experienced by the research assistants can also be examined from the emic-etic theoretical perspective. The observers in this instance were the research supervisors and the observed were the assistants. The assumption was that the assistants understood the manner in which remuneration was computed, including the tax deductions. However, from an emic perspective, the research assistants deduced the actions of the research supervisors as being underhanded. Moreover, the research supervisor sub-system focused on the data collection and the budgetary constraints whereas for the research assistant sub-system the plight of the assistants as well as the suffering of the respondents in this study was meaningful.

Moral Dilemma

Several research assistants referred to the intra-psychic conflict that ensued as a result of their quest to complete as many questionnaires as possible to enhance their salaries. The moral issue of completing the demands of the research project and turning a blind eye to the suffering of people adversely affected some of the research assistants. In many instances they felt helpless in addressing the priorities of the people and felt that they were taking and not putting anything back into the community. Factors that contributed to this sense of helplessness were an apparent limitation in the training of the research assistants with regard to their knowledge of community referral sources as well as a general lack of expertise in mental health related issues. A sentiment expressed was that there would have been a lesser degree of guilt if they could advise the participants as to what channels to follow to alleviate their strife. Research assistants were uncomfortable about using ?deceitful? strategies such as incorrectly informing participants that they would receive individualized reports for their children so as to encourage participation in the first and second interviews, yet felt that they had no option but to complete the demands of the study.

When applying the emic-etic principles to the moral dilemma experienced by the research assistants it becomes apparent that the intra-psychic conflict that affected the research assistants (insiders) was outside the awareness of the research supervisors (outsiders). The research assistants had ethical and moral questions about not attending to the social distress of the respondents in this study whereas the research supervisors were focused on budgetary issues and the completion of the data collection. Initially both the research supervisor and research assistant sub-systems were unified by a common goal of collecting the data. The research assistants, however, became transformed and influenced by being immersed in the research participant sub-system. This resulted in the divergence of motivation toward what was previously a common goal and the ethics of the data collection process was challenged. The ensuing dissonance between the sub-system placed a severe strain on all the role players involved and threatened to impact the research process.

Fatigue

During the latter stages of the project fatigue experienced by the research assistant became clearly evident. Second interviews became a formidable challenge because the interviewees reflected on the approximately two hours that it took to complete the first interview and were reluctant to agree to second interviews. In most cases the potential participants could not see the direct benefit of participating in the study because no tangible rewards were immediately discernible, especially after the experiences of the first interview. However, many seized the opportunity to give vent to their frustrations about the lack of support services and other social issues because to them the research assistants were perceived to be social workers.

Since most interviews were conducted in the late afternoon or early evening it was not uncommon for participants to temporarily abort the interview and attend to chores such as cooking dinner, taking care of the children, or watching their favorite television soap opera. The misperception that they were social workers and that interviews were conducted during the peak evening period often resulted in the research assistants spending more than the allocated time for which they were compensated to complete each questionnaire.,

Furthermore, informal transportation in the township added to the hardship faced by the research assistants. Minibus taxis which carry an average of approximately 10 passengers is the most popular vehicle used by informal operators. Afternoon travelling was not only hazardous but was also a period when the minibus taxis were overloaded. This resulted in extreme levels of exhaustion even before interviews began. High levels of crime prevalent in Khayelitsha contributed to the anxiety experienced by the research assistants, especially when interviews took longer than was anticipated, to the extent that they would abort interviews and have to return to the same parent/child pair to complete the interview. These factors not only contributed significantly to the high degree of frustration and fatigue experienced but also to the dissatisfaction with the financial reward for the additional time expended to complete the task.

Besides providing an opportunity for catharsis to occur during the weekly supervision, little could be done by the research supervisors regarding moral conflicts or factors that contributed to the exhaustion experienced by the research assistants. This exacerbated the perception of the research assistants that the research supervisors remained dissociated from the data collection process and assumed the role of merely observing and not being empathic to the experiences of the research assistant sub-group. From the emic-etic paradigm the experiences of fatigue by the research assistants as a result of the factors delineated appeared to be outside the realm of the awareness of the research supervisors. This is likely to have contributed to the state of aloofness from the research process and contributed significantly to the discontent of the research assistants.

Racial Dynamic

All the research assistants were Black and the managers of the project were Indian or White. Some of the research assistants perceived the plight of the community in Khayelitsha to be replicated to a certain extent within the research team. The majority of the research assistants reported that they were unfairly treated especially with regard to remuneration and were not informed beforehand that the tax structure applicable to full time and contract staff would be applicable to them.

According to the research assistants, the manner in which the payment for completed questionnaires was computed was not made clear during the orientation. They felt that the research managers were exploiting them and were not emphatic to their experiences. There was a perception among some that the managers of the project had a primary motive of completing the project and engaged in minimizing the problems faced by the research assistants. A research team member described the experiences of the research assistants as a microcosm of experiences of the community in Khayelitsha where the emphasis was on the data collection with little emphasis on what could be done to alleviate the distress of the interviewees.

In examining the research supervisor and research assistant sub-group interaction from the insider/outsider perspective, it is clear that there was a lack of congruency as to how each sub-group interpreted the situation. The research supervisors did not attend to the racial composition of the sub-groups nor did they consider it to be a significant factor. The research assistants, however, attributed their negative experiences to the racial composition and considered this to be a significant factor with regard to the attitude of the research supervisors toward their experiences. They attributed the lack of empathy on the part of the research supervisors to racism. This suggests that the ?apartheid baggage? that many South Africans carry continues to permeate many facets of the society to which the research team was not immune. A contributory factor to the dissonance between the research assistant and research supervisor sub-groups was that what appeared to be palpable to one subsystem was not perceptible by the other. This manifested as a situation of distrust of the research supervisors by the research assistants and posed a formidable challenge to the research process.

Confidentiality

Overcrowding is a major problem in Khayelitsha because of the severe housing shortage in the Western Cape. It is not uncommon for many individuals from the same extended family to occupy one dwelling. It proved to be a major challenge for the research assistants to find a suitable location to conduct the interviews. It was imperative for both parent/caregiver and child to be interviewed and in most cases these interviews had to be conducted simultaneously for logistical expediency. Because space was at a premium, interviews with the parent/care giver and child were often conducted in the same room or within earshot. The close proximity of the parent/caregiver and the child hindered the interview process. The parent/caregiver would often focus on the questions asked of the child and the child?s responses rather than questions that were asked of them and their responses despite being advised to the contrary. The main concern of the parent/caregiver was to provide ?correct? responses so that they will receive the much needed state assistance. This proved to be very frustrating for the research assistants and became a time consuming exercise. The frustration experienced by the research assistants may be interpreted from the emic-etic perspective as a lack of awareness on the part of the research assistants as to that which was important and meaningful to those being interviewed. For the parent/caregivers it was a priority to ensure that the interviewers were given the ?correct? answers. Hence the interruptions when the child was asked to respond.

It is evident from the different sub-system permutations that the research supervisor sub-system exerted influence on the research assistant sub-system which, in turn, had a significant impact on the research participants. The respondents influenced the research assistants. The experiences of the research assistant affected the supervisor in that exorbitant claims for incidental expenses were not investigated for fear that the research assistants would abort the data collection process. The cyclical nature of the experiences of the various role players and the ensuing activities had an insidious yet profound impact on the research process and contributed significantly to the challenges faced by the research team. Although the data collection for this study was accomplished it is apparent that the process was challenging. To prepare researchers who intend to conduct research with similar variables and to avoid the pitfalls that this research team encountered, the following recommendations are posited.

Recommendations

First, the research participants have to be given explicit information detailing the research process in its entirety. A major challenge was the obvious breakdown in communication, especially with the clinic sample. The research team assumed that the potential participants were briefed by the clinic staff. The clinic staff assumed that they were only required to provide a list of possible participants. It is recommended that when conducting research with this or a similar sample, research assistants conduct an initial screening interview to ascertain the whether the potential respondent is appropriate for the study. It is also important to determine the literacy level of the participants and to use vernacular familiar to the respondents that is distinct from literary or research language. This is likely to contribute to a comprehensive understanding of the research process. In this study participants gave consent to participate. But it became clear that informed consent was not obtained. Therefore a respondent?s signature is necessary but not a sufficient requirement. The advantages and risks associated with the provision of the information have to be clearly delineated at the outset so that the participant can weigh the pros and cons of involvement. This strategy will reduce the unrealistic expectations of the participants, enhance motivation and reduce the level of discontent experienced by the research assistants as well as the respondents.

Second, the data gatherers are crucial to any study. It is of strategic importance that

research assistants are fully conversant with their responsibilities, the risks involved and the benefits of being part of a research team. It was discernible that the varying degrees of experience, qualifications, maturity and motivation among the research assistants had a significant impact on the data collection process. Rather than filling these positions randomly from the pool of job seekers in a region that is characterized by high levels of unemployment, it behooves the research supervisors to employ a homogenous field team to alleviate some of the challenges alluded to in this paper. It is recommended that research assistants be recruited from groups that have similar research interests, eg. a class of research methodology students, and that research activities be linked to a course curriculum. Monetary reward could be a fringe benefit but ought not to be the driving force behind involvement in the research. This strategy will eliminate some of the challenges related to scheduling appointments and enhance working together because the research assistants will be motivated primarily by a common academic goal.

Third, the experience of research assistants highlights the need for effective orientation. It is imperative that the research assistants be aware of the exigencies of their respective positions prior to the onset of data collection. If there is compensation either in the monetary form or academic credit this must be specified in a written contract so that both parties have recourse in the event of any queries. All deductions must be discussed prior to the finalization of the contract. The system of advanced payment has inherent flaws and is strongly discouraged.

A concern that was common to all research assistants in this study was the state of being helpless when interviewees asked questions related to their psychological as well as their social plight. Research assistants reported that if they knew where to refer these individuals it would have helped reduce some of the intra-psychic distress that ensued. In light of this assertion, orientation to community referral sources ought to be an integral part of the training of research assistants. It will be important for representatives from the various providers serving the geographical area in which the study is being conducted to address the research teams so that they can be apprised of services in the area and be better equipped to make referrals when necessary.

Finally, issues of confidentiality during the interviews tested the frustration threshold of the research assistants and may have affected the validity of the data. Conducting the interviews in over-crowded dwellings in Khayelitsha minimized the degree of control that research assistants had over the behavior of the respondents, especially with parents/caregivers ?assisting? the child with questions, attending to dinner, and diverting their attention to the television screen rather than attending to the interviewer. In similar socio-economic conditions it is recommended that alternative locations such as caravans or mobile interviewing booths be provided. This will reduce the incidence of disruptions and give credence to the data collection process.

Conclusion

It is evident that conducting research in post?apartheid South Africa presents unique challenges. Since this was a seminal study, the research team was unprepared for some of the unforeseen events that unfolded as the data collection progressed. A major impediment was the manifest dissonance that occurred between the three sub-groups

(research supervisors, research assistants, and research participants) and the reciprocal nature of the relationship between the groups. Majority of the respondents did not fully understand the rationale for the interviews and what was going to be done with the data. Moreover, the respondents did not see the relationship between the interviews and the alleviation of their living conditions. This led to resistance, non-compliance, and in some instances hostility toward the research assistants. Research assistants, on the other hand, attributed their negative experiences to the lack of empathy shown by the research supervisors to their hardships. This perception led to mistrust, dissent, frustration and the accompanying lack of motivation. On several occasions the data collection process was jeopardized.

From the emic-etic theoretical exposition, the challenges experienced ?behind the scenes? may be interpreted as a lack of understanding of that which is meaningful and appropriate to each of the subgroups from the perspective of the ?outsiders?. In other words, each sub-group focused on their specific agendas which undermined the essence of the data collection. It is important for researchers embarking on similar research projects to consider some of the recommendations posited in this paper so that similar impediments are minimized and that all the role players can appreciate and be united by the benefits of community research.

References

Bird, H. (1996). Epidemiology of childhood disorders in a cross-cultural context. Journal of Child Psychology and Psychiatry, 37, 35-49.

Bravo, M., Canino, G. J., Rubio-Stipec, M., & Woodbury-Farina, M. (1991). A cultural adaptation of a psychiatric epidemiological instrument: The Diagnostic Interview Schedule?s adaptation in Puerto Rico. Culture Medicine and Psychiatry, 15, 1-18.

Canino, G. J., Rubio-Stipec, M. (1993) The Spanish translation and cultural adaptation of the Diagnostic Interview Schedule for Children (DISC) in Puerto Rico. Culture Medicine and Psychiatry, 17, 329-344.

Edwards, B. L. (1997). Pike?s tagmemic linguistic theory. 17 March, 2001 < http://personal.bgsu.edu/~edwards/tag3.html>.

French, D. (1963). The relationship of anthropology to studies in perception and cognition. In S. Koch (Ed.), Psychology: A Study of Science, Vol. 6 (pp.384- 398). New York: McGraw-Hill.

Jahoda, G. (1995). In pursuit of the emic?etic distinction: Can we ever capture it? In N. R. Goldberger & J. B. Veroff (Eds.), The Culture and Psychology Reader (pp. 128-138). New York: New York University Press.

Pike, K. L. (1954). Language in relation to a unified theory of the structure of human behavior. The Hague: Mouton.

Robertson, B.A., Ensink, K., Parry, C. D. H., Chalton, D. (1999). Performance of the Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3) in an informal settlement area in South Africa . Journal of the American Academy of Child and Adolescent Psychiatry 38, 1156-1164.

Robins, L. N., Helzer, J. E., Croughan, J., & Ratcliff, K. S. (1981). The NIHM Diagnostic Interview Schedule: Its history characteristics and validity . Archives of General Psychiatry, 38, 381-389.