Surveying mobile populations: Lessons from recent longitudinal surveys of Indigenous Australians

Author/editor: Hunter, B, Smith, D
Year published: 2000
Issue no.: 203

Abstract

Geographically mobile populations are notoriously difficult to survey, especially in a cross-cultural context. In broad terms, it is difficult to ensure that respondents are representative of the underlying population and that data obtained are relevant to them. At a practical level, the problem can be as basic as not having any well-formed notion of what defines a household. Consequently, the resulting analysis of households is at best imprecise and, at worst, conceptually confused.

Longitudinal data add a time dimension to surveys and the resulting analysis is potentially sensitive to the initial experience of individual respondents. This paper documents the lessons for the design and conduct of longitudinal data collection from three recent surveys of an exceptionally mobile population, Indigenous Australians. Since high levels of mobility characterise many unemployed and younger Australians, the lessons described here have wider application for general longitudinal surveys.

The CAEPR household survey project

In 1998, the Centre for Aboriginal Economic Policy Research (CAEPR) was contracted to investigate, at the community level, the policy and service delivery effectiveness and appropriateness of the payment of welfare transfers to Indigenous families for the care of their children. The research agenda is to be carried out over a three to four year period, during which two Indigenous communities (Yuendumu in central Australia and Kuranda near Cairns) will be revisited each year and particular households within them surveyed. In the first of these visits, in 1999, lengthy face-to-face interviews were carried out with a key reference person in each household. Genealogies covering all household members were recorded with these respondents (Smith 2000).

Mobility is a crucial factor in the composition and formation of Indigenous households. Adults and children in both Yuendumu and Kuranda are extremely mobile, travelling between a set of usual home bases within and across different communities. Children travel with, and without, their parents and siblings, and this flow is unpredictable. However, the nature of mobility and resulting dynamic developmental cycles of households are not ad hoc phenomena, but subject to the regulating influence of social relatedness and systems of kinship. Key economic relations tend to extend over the boundaries of a single dwelling and residents shift between these places, creating kin-related clusters of households.

In Yuendumu, censuses were taken nightly of all persons staying overnight in households over a twelve-month period. The dynamic nature of Indigenous household composition can be illustrated in one of the four-bedroom houses surveyed. A total of 27 different adults and 15 different children slept at the house over the fortnight, totalling 42 different persons. Out of this flow of 42, a core of 11 persons (7 adults and 4 children) slept at the house for the whole two-week period (Musharbash 2000).

The Barriers to Work Project

The second survey examined is the Barriers to Work Project (BWP), conducted by the Manguri Corporation, which investigated the employment experiences of 25 young Indigenous people (aged 20-30 years old) during 1997 and 1998. This Project showed that the same mobility which causes poor response rates among Indigenous surveys makes it difficult to keep the same Indigenous interviewers over time.

The DEWRSB longitudinal survey of Indigenous job seekers

The final survey examined is the Department of Employment, Workplace Relations, and Small Business's (DEWRSB's) longitudinal survey of Indigenous job seekers collected between early 1996 and late 1997. Face-to-face interviews, predominantly administered by Indigenous interviewers, were conducted in a range of urban areas, large rural centres, and remote centres. Two main issues arise in the analysis of DEWRSB's data: the implications for the analysis of the low response rates among mobile people and the effect of mobility of Indigenous interviewers.

The initial sample contained 7221 names of Indigenous job seekers. Of these, 2503 were successfully interviewed at the first wave, representing a 35 per cent response rate. Once a person had responded to the first wave the chance of being reinterviewed was somewhat higher: the people who could be interviewed were less mobile than those who did not respond.

Indigenous interviewers were either relatively mobile or harder to replace than non-Indigenous counterparts. As a result, there were considerable fluctuations in the Indigenous composition of the interview workforce, especially in non-metropolitan areas (e.g. Alice Springs).

Surveying mobile populations in a longitudinal context

Some key factors underlying the process of mobility in Indigenous households include access to resources ('demand sharing'), availability and quality of housing, overcrowding, conflict, the impact of death, and 'visiting' patterns. The experience of CAEPR's community-level household survey suggests the need for a multi-dimensional, nested set of definitions of 'household'. Minimally, 'household' should be defined using a combination of levels, which are increasingly inclusive: for example, incorporating the ABS standard and several alternative definitions which cover all persons staying in a particular location (including visitors) overnight or in the previous four weeks. There may be a recall problem for retrospective questions on mobility.

There appears to be a trade-off between data quality, response rates, and survey costs. The use of Indigenous interviewers does not, in itself, guarantee that response rates will be acceptable. While the use of such interviewers enhances our confidence in the quality of the data as it relates to Indigenous people's lives, more resources may need to be devoted to following up non-respondents and collecting information.

A more subtle data quality problem arises from the fluctuation in the proportion of interviewers who are Indigenous as a result of mobility, cultural-specific factors, and the fact that it is simply harder to replace interviewers from a population which is relatively rare (in a statistical sense). If the response to Indigenous interviewers is qualitatively different, then large changes in the number of Indigenous interviewers in successive waves of a longitudinal survey will lead to large apparent changes in answers to questions. These interviewer-induced changes in response can fundamentally change the results of analysis that uses longitudinal techniques (which are exceptionally sensitive to any measurement error).

In the mid 1970s, the Henderson Poverty Inquiry in Brisbane secured high response rates using two-person interview teams comprising one Indigenous and one non-Indigenous person. The training and monitoring of interviewers made this a costly process. The CAEPR survey project in 1999 also used interviewing teams, but these were differently constituted from those of the Henderson Inquiry. The CAEPR project researchers worked closely with Aboriginal research facilitators, who introduced the project interviewers to potential respondents, helped explain the nature of the research to them, and acted as translators during the interview. They did not, however, administer the questionnaire itself; this was consistently done by the project researchers. This approach was relatively cost-effective and secured similar response rates to those in the Brisbane component of the Henderson Inquiry.

Large-scale longitudinal surveys are a relatively recent phenomenon in Australia and the recent spate of such social surveys is largely driven by relatively new technology-Computer Aided Telephone Interviewing (CATI). Such surveys are cost effective because they combine interviewing and data entry tasks, but telephone interviews have limitations in the surveying of Indigenous Australians and thus it is inappropriate to rely solely on this methodology. At the very least, telephone techniques should only be relied upon after a relationship has been established with a respondent through face-to-face interviews. Ideally, if cost constraints are not at issue, face-to-face interviews should be used at every stage.

ISBN: 0 7315 2638 4

ISSN:1036 1774

Updated:  27 May 2009/Responsible Officer:  Centre Director/Page Contact:  CASS Marketing & Communications