Abstract
There is potential for measurement problems in both retrospective and panel microdata. In this paper, we compare results on basic indicators related to labor markets and their dynamics from retrospective and panel survey data in Egypt, in order to determine the conditions under which results are similar or different. Specifically, we (1) assess the consistency of reporting of time-invariant characteristics in different waves of the panel, (2) compare the retrospective and panel data results on past labor market statuses, (3) assess the consistency of estimates of labor market transition rates across two specific dates by comparing panel and retrospective data, (4) assess the consistency of estimates of the level and trends of annual labor market transition rates across retrospective data from different waves of the survey, and (5) assess whether retrospective data can provide accurate trends of labor market aggregates, such as unemployment rates. We find that it is possible to garner useful information on labor market dynamics from retrospective data, but one must be cautious about which information to trust and at what level of detail. We conclude with a discussion of implications for future research as well as future survey design.
JEL Classification: C83, C81, J01, J62, J64
Similar content being viewed by others
1 Introduction
The analysis of labor market dynamics requires the availability of data about the same individuals at multiple points in time. This kind of data allows for the examination of flows between different labor market states rather than simply assessing labor market stocks over time, which is what is usually possible with cross-sectional data. Data about the same individuals over time can either be in the form of panel data, where individuals are visited and interviewed multiple times over the course of several months or years, or retrospective data, where individuals are asked about their past labor market trajectories at one point in time. Although both methods of data collection suffer from different kinds of measurement errors, panel data are often deemed superior because they minimize recall error, which could be substantial in retrospective data (Mathiowetz and Duncan 1988; Magnac and Visser 1999; Artola and Bell 2001; Bound et al. 2001; Pina-Sánchez et al. 2014). Panel data, however, are expensive and difficult to collect and are, therefore, rarely available to researchers in developing countries. If available, they are generally not collected frequently enough to observe complete labor market trajectories and transitions (Blossfeld et al. 2007) and suffer from sample attrition and a variety of measurement errors (Artola and Bell 2001; Feng and Hu 2013). It is therefore useful to examine how well retrospective data perform in assessing labor market dynamics and the extent to which analyses that depend on them conform to results obtained from panel data.
Due to potential problems with both retrospective and panel data, it is worthwhile to compare results on basic indicators related to labor market dynamics from retrospective and panel data on the same sample of individuals, in order to determine the conditions under which they provide similar or substantially different results. This paper takes advantage of a unique opportunity to undertake such a comparison, where both panel and retrospective data are available for the same individuals in the 1998, 2006, and 2012 Egypt Labor Market Panel Surveys (ELMPSs). Not only do the reference periods of the retrospective data overlap with the dates of the previous waves of the survey, but also the retrospective periods from different waves of the survey overlap with each other as well. This allows for comparisons of retrospective and panel data at the same point in time, as well as comparisons of recalled events from one wave with the same events recalled in another wave.
In this paper, we (i) assess the consistency of reporting of time-invariant characteristics in different waves of the panel, (ii) compare the retrospective and panel data results on reported past labor market statuses and transitions, (iii) assess the consistency of estimates of labor market transition rates across two specific dates by comparing panel and retrospective data, (iv) assess the consistency of estimates of the level and trends of annual labor market transition rates across retrospective data from two different waves of the survey, and (v) assess whether retrospective data can provide accurate trends of labor market aggregates, such as employment-to-population ratios and unemployment rates. For a number of the comparisons, we estimate multivariate models of the determinants of alignment of the different characteristics, states, or labor market dynamics between the two data sources, i.e., retrospective and panel data.
The rest of the paper is organized as follows. Section 2 reviews the theory and past evidence on measurement error in contemporaneous and recalled data. Section 3 includes a discussion of our data source and methods of analysis. Section 4 lays out all the findings on the various comparisons we make and Section 5 concludes with recommendations as to what kinds of information can be reliably collected using retrospective questions, how to improve retrospective data collection strategies to obtain more reliable information, and potential methods for correcting measurement errors.
2 Theories and past evidence on measurement problems in current and recalled data
The literature on measurement error suggests a wide variety of issues that might contribute to measurement errors in both current (contemporaneous panel) and recalled (retrospective) data. The implications of measurement error depend substantially on the nature of the problems. Truly random errors in continuous variables will not bias estimates of key statistics such as means or estimates of linear regression models when serving as the dependent variable (Bound et al. 2001). Random errors in an explanatory variable, x, will downward-bias or attenuate the estimated coefficient. Random errors in categorical or binary variables are more problematic as they bias model estimates and descriptive statistics.
When measurement errors are systematic, they will bias both basic statistics and regression coefficients in complex ways (Bound et al. 2001). For instance, when studying the incomes of the self-employed, individuals with more education may keep accounting books and be able to more accurately report their incomes. If less educated individuals systematically under-report their incomes, this will systematically bias a regression estimating the relationship between years of education and income. Particularly for topics that relate to behaviors or states that have strong connotations of social (un)desirability, such as the intention to send children to school or the receipt of charity, respondents may intentionally misreport. Under-reporting will occur for socially undesirable phenomena and over-reporting for desirable phenomena, generating “social desirability bias” (Bound et al. 2001).
A large body of literature focuses on the recall or retrieval process and the nature of errors in recall. These are particularly likely to be affected by the recall period. That is, the longer the recall period, the more likely that respondents will report with error, although the extent to which this is a problem varies substantially over studies of different outcomes (Bound et al. 2001). Studies of panel data on dates have identified what is commonly referred to as a “seam effect,” i.e., excessive numbers of changes at the “seam” between one study period and the next (Bound et al. 2001).
The “salience” or importance of events may affect the accuracy with which they are reported (Bound et al. 2001; Judge and Schechter 2009). For instance, unemployment spells of only a few weeks may be of lower salience than unemployment spells that last a year and therefore be more likely to be forgotten. If individuals do remember events, they may not readily remember the exact timing of events. This leads to measurement errors such as “heaping,” where individuals tend to report certain numbers as responses (Roberts and Brewer 2001). For instance, respondents often report adult ages in years in multiples of 5 or child ages in months rounded to the nearest year or half year (Heitjan and Rubin 1990; Roberts and Brewer 2001). Question and questionnaire design can play an important role in whether respondent errors occur. Identifying the best respondent within a household, deciding on the level of aggregation for data, and asking for information in the most appropriate units and for the most appropriate reference period are important elements of design that will affect the accuracy of measurement (Puetz 1993; Grosh and Glewwe 2000).
A study on measurement error from the Malaysian Family Life Survey (MFLS) panel illustrates some of the issues that may occur in developing country data (Beckett et al. 2001). The findings demonstrate not only that substantial errors can occur but also that reporting of retrospective events can be quite accurate. For instance, while 95% of the ever-married sample reported being currently married in the first wave of the survey in 1976, 12 years later, only 84% of panel respondents reported in 1988 that they had been married in 1976. However, the same rate of mortality for children born prior to 1976 results from both the 1976 and 1988 interviews (Beckett et al. 2001). The level of detail in the question affected the accuracy of reporting as well; for instance, agreement was much higher in reporting whether a child was ever breastfed than the duration of breastfeeding. The salience of events also mattered; women reported inter-district moves (a more substantial move) more consistently in 1976 compared to 1988 than intra-district moves. Different reporting errors with the MFLS were related to respondent characteristics.
A number of studies have also been conducted on measurement of income, assets, and consumption. Respondents tend to resort to inference, i.e., reporting mean income, as recall periods lengthen (de Nicola and Giné 2014). Agricultural data show relatively little recall bias, although more salient events may be reported more accurately (Beegle et al. 2012). Asking about the dates of major purchases directly elicits responses of similar quality to asking in relation to time cues (anchoring) important to the respondent; using unimportant time cues generates substantially worse results (de Nicola and Giné 2014). Compared to a benchmark of personal diary use, other diary and retrospective approaches have lower recall, with particularly acute problems for poorer, larger, and less educated households (Beegle et al. 2012). Questions on total rather than categorical expenditure suffer less recall bias (Hiroyuki et al. 2010). On the dynamic side, poverty mobility may suffer from (1) inaccurate measures of income or consumption, (2) price deflation, and (3) mismatching of households over survey waves (Dercon and Shapiro 2007).
3 Data and methods
3.1 Data sources
The Egypt Labor Market Panel Survey (ELMPS) provides a unique opportunity to assess data issues. With waves in 1998, 2006, and 2012, it is possible to use the ELMPS to compare retrospective and panel data over multiple periods (OAMDI 2016). The ELMPS is a nationally representative household survey with detailed modules on current and past labor market statuses. Of the original 23,997 individuals interviewed in 1998, 13,218 (55.1%) were re-interviewed in both 2006 and 2012. Of the 37,140 individuals interviewed in 2006, 28,770 (77.5%) were re-interviewed in 2012.Footnote 1 A retrospective panel of annual statuses is constructed from retrospective data in each wave and compared to panel and retrospective data from previous waves. Reporting of time invariant information, such as parent’s education, is also compared based on reports in different waves of the survey.
A particularly important element of our analyses relies on the labor market history section of the ELMPS surveys, which is administered to all individuals 15 and older who ever worked. In 2012, this section asks chronologically for the start dates (year, month) and characteristics of the first four labor market statuses (be it employment, unemployment, or out of the labor force) lasting 6 months or moreFootnote 2 from the time the individual exited school.Footnote 3 If the individual is employed in that status, she or he is asked about the details of such employment. Moreover, the total number of employment spells lasting more than 6 months and their start and end dates can be obtained from the life events calendar section of the questionnaire.
It is important to note that in the preceding waves of the ELMPS survey (1998 and 2006), the labor market history questions were sequenced differently. Specifically, these waves of the survey used a reverse chronological order (starting with the current status and moving backward) in eliciting labor market trajectories as compared to the chronological method (starting with the first status and moving forward) used in 2012. In addition, in 1998 and 2006, information was collected in a separate part of the questionnaire about the first job in which the individual was engaged for a period of more than 6 months. Unlike the 2012 wave, the 1998 and 2006 waves did not contain a life events calendar and therefore no information on the total number of primary jobs the individual engaged in over his/her lifetime. This questionnaire design implies that initial unemployment and out of labor force states could be missed, as well as employment states between the first job and the pre-previous status.
A potential concern for our analyses is the pattern of attrition across waves of the survey. This issue was discussed in some detail in Assaad and Roushdy (2009), for attrition between 1998 and 2006, and in Assaad and Krafft (2013), for attrition from 2006 to 2012. Two potential types of attrition are possible. First, households from the previous round may not be found at all (type I attrition) and individuals who split from their original households may not be found (type II attrition). The type I attrition rate was found to be 23.5% between 1998 and 2006 and 17.3% between 2006 and 2012. The type II attrition rates were 15.4% for 1998 to 2006 and 30.3% for 2006 to 2012. Analyses of attrition showed that it was not purely random. Generally, between 1998 and 2006 as well as between 2006 and 2012, households in Greater Cairo with younger household heads were more likely to experience type I attrition. Analyses of attrition for 1998 to 2006 showed no systematic differences between the two samples in terms of type II attrition. For 2006 to 2012, individuals who split from their original households were less likely to be found if they were from Greater Cairo and were male, older, and married. Panel data weights were created based on the probability of the two types of attrition predicted on the basis of observables in the previous wave. These weights were used to correct all estimates generated from the panel data in this paper. While this accounts for attrition based on observables, it does not correct for any attrition based on unobservables.
3.2 Methods
To compare retrospective and panel data, the retrospective data were mapped onto panel data from previous waves in such a way that retrospective and contemporaneous information is available for the same individual at the same point in time. We then draw on the econometric literature on measurement error to assess and compare the data sources and suggest possible corrections to account for measurement error (Fuller 1987; Magnac and Visser 1999; Black et al. 2000; Bound et al. 2001; Carroll et al. 2012).
As a first check on the accuracy of the panel data, we begin by comparing the consistency of time invariant information across different waves of the panel. We do this for own education for adults, father’s employment status and sector of work when the individual was 15 years of age, and recalled costs of marriage. We then compare labor market statuses at a given point in time (1998 and 2006) across retrospective and panel data to assess the accuracy of recall and identify statuses that are particularly prone to erroneous recall. We subsequently estimate multivariate models of the probability of alignment in own education, father’s employment status and sector, and own labor market status between the two kinds of data as a function of individual characteristics and whether the information was consistently elicited from the individual him/herself or whether one or both responses were from a proxy respondent. In the labor market status models, we include the nature of the past labor market status itself, and the contemporaneous labor market status in 2012, as regressors.
The next step is to assess the consistency of reporting of labor market transitions in retrospective and panel data. To do this, we convert the retrospective data into an annual retrospective panel, which contains information about the main labor market variables every year. Using this retrospective panel, we calculate the rate of change in labor market status from 1998 to 2006 using the waves of the panel for those dates to the rates of change over the same period as reported by the 2012 retrospective data. We also estimate multivariate models to examine the correlates of misalignment in reporting about transitions in the panel data (1998 to 2006) and in the 2012 retrospective data referring to 1998 and 2006. We finally move to comparing annual transition rates derived from the retrospective data in different waves of the survey.
In examining annual labor market transition rates, we examine two types of transitions of particular interest to the study of labor market dynamics: (1) transitions among employment, unemployment, and out-of-labor-force states and (2) job-to-job transitions among the employed. Within the first type, we include job-finding rates for the unemployed and those out of the labor force and separation rates from employment to either unemployment or out of the labor force. The second type includes transitions across different types of jobs, such as public and private employment, and wage and non-wage work. We examine how different waves of the retrospective data generate transition rates, by type of transition. Finally, we revisit the question of whether the levels and trends in important labor market measures, such as the employment-to-population ratio and the unemployment rate, can be accurately assessed using the retrospective data, by comparing different waves of retrospective data and contemporaneous sources of these data, such as the official labor force survey.
4 Findings
4.1 Consistency of reporting of time-invariant information across different waves of a panel survey
4.1.1 Own education for adults
The accuracy of the characteristics individuals report in any survey, such as their age, education, or labor market characteristics, plays an important role in researchers’ ability to accurately describe economies and labor markets. Because the ELMPS is a panel, we can compare characteristics that should remain largely unchanged, such as education (for adults) as reported in different waves of the survey in order to assess their accuracy. Figure 1 compares the reporting of education in 1998 with that in 2006Footnote 4 for individuals aged 30–54Footnote 5 in 1998. It is important to keep in mind that either the 1998 or the 2006 response could be inaccurate when they disagree or both could be consistent (but still inaccurate). Overall, 79% of responses are consistent over time, but there is substantial variation in terms of which education categories are reported consistently. For instance, 90% of those who reported being illiterate in 1998 also report being illiterate in 2006. The remainder primarily reported being able to read and write but having no education certificate (7%), which could be a genuine change in literacy, or reported having primary education (3%). The ability to read and write appears to be the most poorly reported education status, with only 34% of those reporting they could read and write but having no education certificate in 1998 reporting the same status in 2006. The most common response in 2006 for this group was “illiterate,” which may represent a genuine decay in reading and writing ability. In general, when reporting is different, the reported alternative is usually a proximate level of education. For instance, 20% of those who reported general secondary in 1998 then reported that they attended vocational secondary in 2006.Footnote 6 Creating less finely aggregated categories, such as only four education levels, leads to more consistency in reporting. When the respondent is the individual in question in both rounds as opposed to a proxy respondent, there is a slight (but not dramatic) improvement in the consistency of the data.Footnote 7
Education (eight categories) as reported in 1998 versus 2006, aged 30–54 in 1998 (percentage). Source: Authors’ calculations based on ELMPS 1998 and ELMPS 2006
4.1.2 Father’s sector of work and employment status
The ELMPS collects information on father’s characteristics, referring to father’s characteristics at the time the respondent’s age was 15 when the father is not present in the household.Footnote 8 Figure 2 shows the consistency of responses over time (2006 versus 2012) for a variable that combines information on the father’s sector of work (government/public enterprise/private) and employment status (wage/non-wage) among respondents aged 30–54Footnote 9 whose father was not present in 2006 or 2012. Additional file 1: Figure S1 shows analyses comparing 1998 vs. 2006, which are substantively similar to those for 2006 vs. 2012 shown in Fig. 2. As shown in Fig. 2, approximately 70% of respondents who reported that their father worked as a government wage worker in 2006 reported the same status for their father in 2012. Private sector wage work on the part of the father is not as consistently reported (55% were consistent from 2006 to 2012). Of those who reported in 2006 that their fathers were non-wage workers when they were 15, 32% were identified as private sector wage workers in 2012. Consistency is most problematic for fathers who were reported to work in public enterprises. Only 38% of individuals who reported that their father worked as a public enterprise wage worker in 2006 reported the same status in 2012. Instead, 38% reported their father worked as a government wage worker and 18% reported their father worked as a private sector wage worker. Relatively few (6%) said their father was a non-wage worker. These results suggest that respondents are sometimes inferring or reconstructing aspects of their father’s sector of work. For instance, individuals may know that their father works in a utility but may not know the sector of employment and reconstruct it. Analysis according to the identity of the respondent shows that only in regard to public enterprise work does having the individual in question be the respondent appreciably improve consistency. Given the high rate of confusion differentiating public enterprise work from government work, distinguishing simply between public sector and private sector may be preferable. Additional guidance on distinguishing between private sector wage and non-wage work when fielding may also be helpful for improving the accuracy of this measure.
Father’s sector of work and employment status when individual was age 15, as reported in 2006 versus 2012, father not in household in 2006 or 2012, aged 30–54 in 2006 (percentage). Source: Authors’ calculations based on ELMPS 2006 and ELMPS 2012
4.1.3 Recalled costs of marriage
Understanding the investments individuals have made over time often requires asking about past outlays of expenditure. Individuals are expected to report the value of expenditures and investments at the time these outlays occurred. However, especially when individuals are inferring or reconstructing the value of an investment, for instance inferring the cost of their housing at the time of marriage based on their current cost or value of housing, this can cause problems in assessing trends over time. Essentially, individuals may (fully or partially) update past expenditures from nominal to real terms, creating artificial declines in costs over time. Figure 3 shows the trends in the total costs of marriage over time for women who were married in 2012 and present in both the 2006 and 2012 waves (they may not have yet been married in 2006).Footnote 10 Marriage is an enormous investment for young people and their families, and the cost of marriage and its trends in Egypt and in the Middle East and North Africa (MENA) region are the subject of substantial concern and discussion (Singerman and Ibrahim 2003; Singerman 2007; Assaad and Ramadan 2009; Dhillon and Yousef 2009; Salem 2014, 2015, 2016; Assaad and Krafft 2015a, b). The figure shows both the nominal (reported) costs and the real costs (inflated to 2012 LE based on the CPI) by year for 2000–2012, as reported in both 2006 and 2012.
Total nominal and real marriage costs (in LE) for 2000–2012, as reported in 2006 versus 2012, individuals in both waves, answering marriage section in 2012 (may not yet have been married in 2006). Source: Authors’ calculations based on ELMPS 2006 and ELMPS 2012
Focusing first on the period for which we have two sources, 2000–2006, nominal marriage costs are clearly rising over time using both the 2006 and 2012 data. The data also show that 2012 reports are consistently higher than those of 2006. Real marriage costs appear to be flat or falling over time in both sources; however, the data again show that reports from 2012 are consistently higher. Using the 2006 data, it appears that from 2000 to 2006, real marriage costs averaged around 60,000 to 70,000 LE. Using the 2012 data looking at the same respondents’ real marriage costs from 2000 to 2006, it appears marriage costs averaged around 90,000 LE. This is inconsistent with what was reported in 2006. It appears individuals are partially (but not fully) updating nominal expenditures into real terms. Further investigation suggests different elements of marriage costs are updated differentially, likely related to how easy they are to recall or reconstruct.
Continuing to examine the 2012 data out to 2012 in real terms, it appears that marriage costs have fallen substantially over time, from around 90,000 in 2000–2006 based on the 2012 data to around 60,000–70,000 by 2012. This implies the real cost of marriage over the 2000–2012 period decreased almost a third. However, looking back at real marriage costs as reported in 2001–2006 from the 2006 data, marriage costs have remained essentially constant, in the 60,000–70,000 range. The decline in real costs implied by looking over time with the 2012 data is therefore an artifact of measurement error. This is evidence that, particularly when asked about events that are a number of years in the past, individuals may be inferring their value or inflating into current terms. This suggests that retrospective data should not be used to assess time trends for financial outlays; repeated cross sections or panel data are required for that. Comparing investments in the few years preceding a survey wave to investments in the few years preceding different survey waves will be more accurate for such data. Alternatively, measures that remove potential inflation, such as asking “how many multiples of your annual income were your marriage costs?” may generate superior results. This is an important area for future research.
4.2 Comparing labor market statuses across retrospective versus panel data
4.2.1 Alignment of labor market statuses in general
Detecting the labor market statuses of individuals, namely whether they are out of the labor force, unemployed, or employed, and if employed, their employment status, is at the heart of labor market surveys. This section first assesses whether aggregate statistics vary across retrospective and panel data and then whether the same individuals report consistently across contemporaneous and retrospective data. Figure 4 presents aggregate labor market statistics by gender for 1998 and 2006 using both contemporaneous statistics from the waves of the panel and retrospective reports from 2012 for those years. Additional file 2: Figure S2 presents a similar comparison using 2006 retrospective data to generate 1998 statuses and compare to 1998 contemporaneous data. The results are generally equivalent, although the 2006 retrospective data does a slightly better job capturing women who ever worked and therefore has a more consistent estimate of being out of the labor force. Individuals 20–44 in the base of the pair of years are used in order to capture prime-age transitions and a consistent set of dynamics. Notably, for males, the aggregates from both the retrospective and contemporaneous data are quite similar, with a few exceptions. Reporting of public sector work, private formal and informal regular wage work, and self-employment are fairly similar. Irregular wage work is differentially reported in the retrospective data, which is likely because hours of work fluctuate over time; individuals may remain at the same job over time but report that it is irregular in 2012 and map that back onto their status in previous years. Changes in regularity within the same job are also not captured within the labor market history and are difficult to detect. Regularity of employment should therefore not be assessed from retrospective data. Being an employer was more likely to be reported in the contemporaneous than in retrospective data. This may be because whether a self-employed worker hires other workers or not is a varying, time-bound status that is not easily recalled. Unemployed statuses appear to be under-reported in the retrospective data. For instance, while 5–6% of males were unemployed contemporaneously, in the retrospective data, only 2% of males report being unemployed at that date. Likewise, in the contemporaneous data, more individuals report being out of the labor force. This could be due to a misunderstanding of the meaning of status in the retrospective labor market history modules, where some individuals interpreted the term to mean jobs. It is therefore necessary to clearly distinguish between questions about previous jobs and previous non-employment states in such retrospective modules.
Labor market status, as reported contemporaneously for 1998 and 2006 and as reported retrospectively for those years from 2012 data, by sex, individuals aged 20–44 in 1998 or 2006 and present in both waves (percentage). Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006 and ELMPS 2012
There are so few females in a number of labor market statuses that our assessment for women focuses primarily on the public sector, unpaid family work, unemployment, and being out of the labor force. Public sector work is quite consistently reported in the aggregates, which may be due in part to the stability of this employment status. Unpaid family work, which includes subsistence work, is much more frequently reported in the contemporaneous data (5–10% across years) than in retrospective data (3–4%). This is due to the well-documented difficulty in detecting these forms of employment for women in Egypt, an issue we return to in more detail below. Unemployment is again more frequently reported in the contemporaneous data than in the retrospective data. This is likely due to the fact that many women who search for work never end up working (Assaad and Krafft 2014) and thus are not asked the questions in the labor market history. As a result of these patterns in reporting employment, being out of the labor force is higher in the retrospective than contemporaneous data for women.
Some labor market statuses are particularly prone to a greater degree of misreporting over time. Figure 5 presents the distribution of retrospective statuses reported in 2012 for 1998 and 2006 by the status reported contemporaneously in 1998 and 2006, for individuals 20–44 in 1998 or 2006, by sex. Additional file 3: Figure S3 shows an equivalent comparison using 2006 retrospective data for 1998. Reports are more consistent across 2006 retrospective data for 1998 and 1998 contemporaneous data than for 2012 retrospective data for 1998 and 1998 contemporaneous data. This finding suggests that more recent dates and statuses are likely to be reported more accurately. The analysis in Fig. 5 essentially breaks down the aggregates shown in Fig. 4 by origin state. While public sector employment tends to be reported quite consistently, other labor market statuses are frequently not reported consistently. Formal private sector work tends to be more consistently reported than informal work. Most reporting of wage work is consistent, but the type of wage work is not consistently reported. Distinctions between self-employment and being employers are likewise blurred. A higher degree of inconsistency is also apparent for those who were unemployed. Some of this may be because the duration of these statuses is shorter, so the contemporaneous status may be off relative to the status that is measured as the predominant status for the year in the retrospective data. Females have a much higher probability than males of reporting that they are out of the labor force (which is their predominant status). Less formal forms of employment in the contemporaneous data for women, such as being an employer, self-employed, or an unpaid family worker are particularly likely to be reported as being out of the labor force in the retrospective data. Being unemployed suffers from a similar problem, likely, as mentioned earlier, due to the large share of women who start out as unemployed but ultimately never find work.
Labor market status, as reported in 1998 or 2006 versus 2012 retrospective data for 1998 or 2006, by sex, individuals aged 20–44 in 1998 or 2006 and present in both waves (percentage). Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006 and ELMPS 2012
The problems associated with detecting employment even contemporaneously among marginally employed women in agriculture and animal husbandry in Egypt are well known (Anker and Anker 1995; Assaad 1997; Langsten and Salem 2008). These problems result from the fact that women in such employment consider themselves not to be working and respond to direct questions about their work status in the negative. Only when probed with a large number of keyword questions is it determined that they are working according to international definitions of what constitutes economic activity. Predictably, the difficulty in detecting women’s correct employment status for marginally employed women is compounded when the question refers to a reference period well in the past. Further examination of the data demonstrates that a key problem is detecting whether women who are currently not working ever worked at all. Among the women examined, not even two thirds (64%) of those who were identified in 2006 as engaging in market work reported that they ever worked in 2012. This was not a problem for men (99% of those working in 2006 reported that they ever worked in 2012). Among women who were not working in 2012 but were working in 2006, only 17% reported ever working in 2012. Only those who report ever working are asked the labor market history, and thus, these women are considered to have never worked and no labor market history data are collected for them.
Further analysis of the data shows that reporting whether women ever worked at all varied substantially by whether the individual was consistently responding for him or herself or if a proxy respondent was used. Among the women examined where a proxy respondent was providing data, just 51% of those who were identified in 2006 as engaging in market work reported that they ever worked in 2012, compared to 69% when the respondent was consistently the individual herself. Among women who were not working in 2012 but were working in 2006, only 10% reported ever working in 2012 when it was not consistently the individual in question reporting and 20% when it was the individual reporting. While both illustrate extremely low rates of reporting work, having the individual in question as the respondent did lead to increased accuracy in regard to ever working.
These patterns, as with education, suggest a number of issues for analyzing labor market statuses and dynamics. For instance, a category of private wage work, incorporating regular formal and informal and irregular workers, would be more consistently reported than the disaggregated categories, and transitions between regular/irregular and, to a lesser extent, formal/informal may be poorly reported over time. Self-employment and being an employer are also often mixed up and might be better combined into a single category. For females, retrospective data should be treated with particular caution, as women may not report ever working when they have done so or report being out of the labor force when they were in fact self-employed or unpaid family workers.
We had initially expected substantially more consistent reporting of labor market statuses when the individual was consistently responding for him/herself than if a proxy respondent was responding on their behalf. However, that was not always the case. The lack of higher consistency when the individual is reporting for him or herself could be due to a variety of reasons. It may be that proxy respondents are less accurate, but still consistent, if they consistently simplify the labor market trajectories of the individuals for whom they are reporting.
4.2.2 Recalling past unemployment spells
While the aggregate labor market statistics are not substantially different, the inconsistency of individuals’ responses over time is troubling. This section attempts to analyze some of the patterns and sources of disagreement in the data sources, focusing on the case of unemployment, the occurrence and duration of which is of particular interest within the Egyptian and MENA labor markets (Assaad and Krafft 2014; Kherfi 2015). The inconsistencies between contemporaneous unemployment and retrospective unemployment reporting could be occurring for a variety of reasons. Because only individuals who ever worked are asked the retrospective questions, excluding women who sought but never began work, this section focuses solely on the unemployment dynamics of individuals who ever worked and examines several different questions essentially revolving around the issue of why there are inconsistencies across the data sources. Do individuals report unemployment in their retrospective histories, but just during a different year? Are shorter spells of unemployment more likely to be forgotten with time?
Since the primary concern is that unemployment is under-reported in the retrospective data, in Table 1, for those who reported unemployment in the 2006 or 1998 waves,Footnote 11 we examine the reports of unemployment in the 2012 retrospective data and a number of characteristics, including the mean current unemployment duration at the time of survey, and the percentage of individuals who experienced short (less than 6 months) current unemployment durations as of the time when they were surveyed. Notably, for those unemployed in the contemporaneous data for 1998, just 9% of labor market statuses in the 2012 retrospective data for 1998 included a report of unemployment. The alignment was slightly better in 2006, when 13% were aligned. This could again be due to the potential confusion between past “statuses” and past “jobs” in the job history modules, something that can be ameliorated with separate questions for jobs and non-employment states. Additional file 4: Table S1, which performs equivalent comparisons using 2006 retrospective data, suggests that, as with other results, consistency decays with time. Using the 2006 retrospective data, 16% of 1998 unemployment statuses were consistent across the contemporaneous and retrospective data. In Table 1, individuals who were unemployed in 2006 were more likely to report unemployment within 1 year (5%) or 2 to 5 years (12%) than those unemployed in 1998 (1% reported unemployment within 1 year and 7% within 2 to 5 years). More individuals reported being unemployed at some point more than 5 years out in 1998 (11%) than in 2006 (7%). Reporting of unemployment is less accurate, both in terms of reporting at all and the timing of unemployment, going further back in time.
Notably, 71% of individuals who were contemporaneously unemployed in 1998 did not ever report being unemployed in the labor market histories. Because the labor market histories in 2012 go forward in time, it is possible that unemployment occurred after the fourth status (the last status asked in the labor market history). Therefore, those with a fourth status are separated out but comprise a small share of the distributions (6% for those unemployed contemporaneously in 1998 and 4% of those in 2006).
The characteristics of unemployment, specifically its duration to date as of the contemporaneous status reported in 1998 or 2006, are related to the probability of accurately reporting. Those whose reporting aligned had, on average, shorter durations of unemployment to date, 26 months in 1998 and 31 months in 2006. Those who reported their unemployment, but with imprecise timing, tended to have shorter durations of unemployment than the average. Those who never reported being unemployed in the retrospective data had slightly longer than average unemployment durations if unemployed in 1998, but not in 2006. However, those who never reported being unemployed were slightly more likely than average to have had unemployment durations of less than 6 months to date, which, if they ended before reaching the 6-month mark, would not appear by definition in the labor market history. Overall, it appears that gathering data on historical patterns of unemployment, even among those who ever worked, is likely to produce substantially different results than using contemporaneous data. It seems likely that retrospective data will both under-report past unemployment and distort its characteristics. Having the individual in question report for him or herself does not substantially improve the reporting of unemployment. Asking specific questions about unemployment in the retrospective data rather than relying on general questions about past labor market statuses could potentially improve the reporting of unemployment in retrospective data.
4.2.3 Multivariate models of alignment between retrospective and panel data
Particularly concerning in assessing measurement error is whether errors are systematic (related to covariates). Such relationships will bias any attempts to examine the relationship between covariates and potentially mismeasured outcomes. To assess whether there are systematic patterns, in terms of observables, of misreporting, we run probit models for alignment (consistency) in a number of outcomes across data sources.
We first examine the consistency of reporting time invariant data in the panel. Since the education status questions were not asked in 2012 unless an individual’s education status actually changed, as with the descriptive analysis, we limit the education comparisons to 1998 and 2006 only. Reporting is deemed aligned when the value of the education categorical variableFootnote 12 is the same in both (1998 and 2006) waves of the panel. We also examine consistency of reporting the combined (four category) father’s sector and employment status variable. Reporting is considered aligned when the value of this variable is the same across a pair of waves in the panel. The father’s employment sector and status analyses are restricted to those whose fathers were not members of the household in both waves, as in the descriptive analysis. For both education and father’s employment sector and status, models are restricted to those who were 30–54 in the base period (1998 or 2006), as is the case in the descriptive analysis. Table 2 reports the marginal effects from the probit model for the probability of alignment of own education across 1998–2006 (Col. 1), father’s employment sector and status across 2006–2012 (Col. 2), 1998–2006 (Col. 3), and 1998–2012 (Col. 4).
The probability of alignment for the reference case is reported in the first row of the table. The reference case is a 30–34-year-old individual in the base period (1998 or 2006), who is university educated, residing in Greater Cairo, did not consistently respond for him or herself (i.e., a proxy responded on his or her behalf in at least one wave), and was a public wage worker in the base period. The probability of alignment of educational status between the 2006 retrospective data and the 1998 contemporaneous data is quite high for the reference case, 91%. It is somewhat lower for father’s employment sector and status, where it is 71% for consistency of reporting between 2006 and 2012 and 75% for consistency between 1998 and 2006 and between 1998 and 2012.
As we have seen in the descriptive analysis above, the education states that are subject to the most error in reporting relative to university education are “read and write” and general secondary, followed by post-secondary, preparatory, and primary (see Col. 1). Women are slightly better than men in terms of consistency in reporting education, but no other covariates are significant predictors of the probability of alignment.
For father’s employment sector and status (Cols. 2–4), we can also see that irrespective of the period being considered, the least consistently reported statuses are “public enterprise worker” and “private wage worker” as we reported above. Non-wage work is slightly less well reported than government work, but the difference is only statistically significant in the 2006–2012 period. The only other covariates associated with the consistency of reporting of father’s employment sector and status are own education. Individuals with less education and with vocational secondary education have less consistency than those in other educational categories. However, these differences are only statistically significant over the 2006–2012 and the 1998–2012 periods. Having the individual himself be the respondent in both periods improves consistency only slightly and only significantly in the 1998–2012 comparison. Age, gender, region of residence, and contemporaneous own labor market status in base year are not associated with any appreciable differences in the consistency of reporting father’s sector/employment status. We can thus conclude that other than the value of the variable in question in the base year, the errors in reporting of own education and father’s sector/employment status in the retrospective data are mostly random, with some weak association with education in the case of father’s sector and employment status.
We move next to a multivariate analysis of the consistency of reporting one’s own labor market status, by sex, as shown in Table 3. The outcome we examine is alignment, namely whether labor market status is consistently reported between the contemporaneous panel data and the retrospective data. We define labor market status as the 9-category variable shown in Fig. 4. We examine the consistency between the 2012 retrospective data and the 1998 contemporaneous panel data (Cols. 1–2), the consistency between the 2012 retrospective data and the 2006 contemporaneous panel data (Col. 3–4), and the consistency between the 2006 retrospective data and the 1998 contemporaneous panel data (Cols. 5–6). Education (aggregated to four categories), age group, region, and whether the individual in question was consistently the respondent are included as covariates. We also include among the covariates the individual’s own labor market status as contemporaneously measured in the base year. Unlike own education or father’s employment, which should not change much over time, labor market status does change over time, which may affect reporting. We therefore include the labor market status in the end year of the pair,Footnote 13 since the descriptive results highlighted these as potential factors in the consistency of reporting. Our analyses are restricted to those who were 20–44 years old in the base year of the pair. The reference case probabilities of alignment, which are for a public sector worker in the base year, are quite high, ranging from 87% in 2006–2012 to 93% in 1998–2006 for males and from 80% in 2006–2012 to 95% in 1998–2006 for females.
As we have established from the descriptive analysis, different labor market statuses in the base year have very different probabilities of alignment. The worst alignment is for those who were unemployed in the base year, most of whom did not report that they were unemployed at that time in the retrospective data. The next worst alignment is for all the non-wage statuses (employer, self-employed, and unpaid family worker). The probability of misalignment is even higher for women in these categories than for men. This result is due to the difficulty in measuring these employment states for women, since they generally do not consider such work to be employment. Irregular wage work, as discussed above, also has poor alignment, because of the shifting nature of such a status and the difficulty in detecting when such shifts occur. Formal and informal private wage work also have a higher probability of misalignment relative to public sector work, again, more so for women than for men. Being out of the labor force is a base state that results in more misalignment for men and no misalignment for women. This difference is because it is a rare state for adult males and the most predominant state for women.
Contemporaneous statuses in the end year of the pair are also related to misalignment, primarily for men. Those not employed in the end year have more alignment, possibly because they are then more consistently reporting a non-employed status. Informality and irregularity in the end year are sometimes significantly and usually negatively associated with alignment. A more irregular and informal work trajectory may be more difficult to recall.
As with education and father’s employment (sector and status), age and region are rarely associated with the probability of misalignment of the labor market status for either men or women. However, at least for males, education is associated with alignment. Men with basic education have between 6 and 15 percentage points lower probability of alignment than men with university education, depending on the period being examined. Similarly, illiterates and those who can only read and write have between 9 and 11 percentage points lower probability of alignment and those with secondary degrees have between 7 and 12 percentage points lower probability. There are very few significant differences by education for women.
4.3 Comparing labor market transition rates across retrospective and panel data
An important use of retrospective and panel data is to measure transition rates between different labor market statuses in order to assess labor market dynamics. We have demonstrated not only that there could be substantial misalignment between contemporaneously measured statuses and ones measured by means of retrospective questions but also that the overall distribution of statuses is fairly similar (Fig. 4). If the measurement errors are primarily random in the reporting of the timing of statuses, measures of labor market transition rates could still be fairly accurate. However, if the entire statuses are lost in the retrospective data (as appears to be the case for unemployment), then measures of labor market dynamics will be understated and will point to a more rigid labor market than is actually the case. Because the ELMPS contains three panel waves, it is actually possible to assess labor market transition rates by using either purely retrospective or purely panel data. This section specifically compares transition rates, by initial status, from 1998 to 2006, based first on the 1998 and 2006 panel data and, second, on the retrospective data collected in 2012 for the transition from 1998 to 2006. This analysis is performed only for individuals who appear in all three waves and who were 20–44 in 1998. The status used for classification purposes comes from either the retrospective or the panel data, depending on which data are being used to calculate the transition rates.
There are some key points to keep in mind when considering this comparison. The contemporaneous status is (as is the case throughout this paper) the “usual” status in the 3-month period preceding the survey. In the retrospective data coming from the labor market history module of the survey, statuses have to be at least 6 months long to be reported. It is therefore likely that in the panel data, some of the transitions that are detected relate to statuses that lasted less than 6 months and that would not be observed by definition in the retrospective data. This would tend to inflate panel data transition rates upward, but probably not by much. We know from the 2012 contemporaneous data that only 1.2% of employed individuals have a different primary job in the reference week than in the reference 3 months, suggesting that short-term transitions are rare. Transition rates in the panel data are therefore only likely to be inflated by a few percentage points at most. Although the distribution of labor market statuses and sectors across panel and retrospective data is fairly similar (Fig. 4), the differences that do exist are going to affect the measurement of transition rates as well.
Figure 6 shows the rates of transition from each labor market status as measured by panel and retrospective data sources. Notably, transition rates are understated by about half in the retrospective data relative to the panel data for males (25 versus 52%) and by more than two thirds for females (8 versus 30%). Looking across statuses, every employment status in 1998 suffers from under-reporting of transitions in the retrospective data, but to varying degrees. For males, transitions out of unemployment and OLF statuses are fairly comparable, but this is not the case for females.
Rates of status change (percentage) in panel data for 1998 to 2006 versus rates of status change in retrospective data from 2012 for changes from 1998 to 2006 by sex and status in 1998, aged 20–44 in 1998. Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006 and ELMPS 2012. Notes: Based only on individuals in all three waves. Status in 1998 is from either retrospective or panel data depending on whether transition rates are being examined for retrospective or panel data
Besides differential rates of transition, there are differential patterns in terms of which transitions are detected or not detected (not shown). More subtle transitions, such as transitions from informal to formal private wage work, or from employer to self-employed and vice versa, are more likely to be missed in the retrospective data. More distinctive transitions—such as those between public and private sector jobs and between wage and non-wage work—are also somewhat under-reported in the retrospective data, but to a lesser extent. Particularly for women, the retrospective data is less able to detect transitions into and out of the labor force, a problem related to the issue we discussed earlier about the difficulty in detecting women’s self-employment and unpaid family labor in the Egyptian context. Women in the public sector are much more likely to report being employed in the past. Since they typically have low transition rates, this tends to understate overall transition rates for women.
4.3.1 A multivariate analysis of probability of alignment in reporting transitions across panel and retrospective data
We report in Table 4 the marginal effects from probit models of the alignment in the reporting of transitions in the panel data (1998 to 2006) and in the 2012 retrospective data referring to 1998 and 2006. This analysis parallels the above descriptive analysis presented in Fig. 6. The analysis is similarly restricted to individuals 20–44 in the base (1998) year who had a transition in the panel data from 1998 to 2006. We define alignment quite conservatively. A transition is only aligned when the (nine category) panel and retrospective labor market statuses are completely consistent across 1998 and 2006, that is, when the exact same transition is detected. Misalignment can therefore occur when the retrospective data have no transition, a different origin state, or a different end state. We include as covariates age, sex, region, and the panel statuses in 1998 and 2006, as we know certain transitions and statuses are differentially reported.
We report the probability of alignment for the reference case (a 20–24-year-old university educated male residing in Greater Cairo and working in the public sector in 1998 and 2006, with not consistently the respondent him or herself reporting). The reference probability of transition alignment is relatively high, 71%. Recall from Fig. 6 that even in the aggregate, half of transitions are not reported.
The most important determinant of alignment in most cases is the origin and destination labor market status. Compared to the reference public sector case, every other status in 1998 or 2006 predicts worse alignment of transitions, almost always significantly so. Only for those who are OLF in 1998 is the difference insignificant. As with reporting of contemporaneous statuses, unemployment does poorly, and also especially if an individual was an irregular wage worker in 2006.
The differences in the probabilities of transition alignment across the other covariates are smaller than by status but appreciable. Lower levels of education predict slightly lower alignment, usually significantly so, compared to university-educated individuals. Compared to young persons 20–24 (the reference category), older groups have lower, usually significantly, probabilities of alignment in reporting transitions. Region of residence does not systematically affect the probability of alignment. Remarkably, having the individuals consistently be the respondent to the questionnaire does not significantly improve the probability of alignment in all three types of transitions. This may be because individuals reporting for themselves report more complex but less precise histories. After accounting for destination and origin states, there are not significant differences in reporting by sex.
4.4 Comparing the levels and trends of annualized labor market transition rates across retrospective data from different waves of the survey
4.4.1 Measuring annualized transition rates from retrospective data
To further investigate the extent to which the ELMPS retrospective data suffer from measurement problems, we compare the transition probabilities obtained from the retrospective data for the same time period as assessed by different waves of the survey.Footnote 14 The dynamics we focus on are primarily the job-finding (f) and separation rates (s), which can be defined as the share of employed, E, and non-employed, NE, changing states over time, t;
4.4.2 Separation rates
Using the data from three waves of the survey, in Fig. 7, we overlap the separation rates calculated over the years.Footnote 15 This is done for male workers between 15 and 54 years of age in year t. An increase in the separation rates approaching the year of each survey is observed.Footnote 16 This tends to be true for both men and women (not shown) but is much more pronounced in the male workers’ trends. The ELMPS 2012, most likely due to the chronological design of the retrospective accounts, is doing a better job than the ELMPS 2006 in capturing the transitions of individuals that are further back in time. Still, if we take more recent rates as true, both are underestimating the employment to non-employment transitions.
Employment to non-employment separation rate (percentage) by wave, males aged 15–54, 1990–2010. Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006, and ELMPS 2012
As a further investigation into how employment to non-employment transitions are under-estimated using retrospective panels, we repeat the above exercise making a distinction between unemployment and inactivity states. We replot the separation rate time series, first using employment to unemployment transitions and second using employment to inactivity transitions. Figures 8 and 9 show that the increase in the separation rates continues to appear for both types of separations. While the underestimation is much attenuated for the employment to unemployment transitions because we have excluded from the contemporaneous reports the unemployment statuses that lasted less than 6 months, this cannot be done for the employment to inactivity transitions, where the date of start of the status is not available and hence assumed to be at the beginning of the year. In fact, the retrospective accounts of the ELMPS 2012 seem to capture well the employment to unemployment separation rates. The underestimation of the separation rates observed in Fig. 7 for the ELMPS 2006 and 2012 seems to result mainly from including inactivity statuses lasting less than 6 months. The increase in the employment-to-inactivity separations seem to occur gradually over the 2 years closest to the survey.
Employment to unemployment separation rate (percentage) by wave, males aged 15–54, 1990–2010. Source: Author’s calculations based on ELMPS 1998, ELMPS 2006, and ELMPS 2012
Employment to inactivity separation rate (percentage) by wave, males aged 15–54, 1990–2010. Source: Author’s calculations based on ELMPS 1998, ELMPS 2006, and ELMPS 2012
4.4.3 Job-finding rates
Moving now to the dynamics of the job-finding process, recall that these rates are the flow of workers from non-employment to employment relative to the stock of non-employed. As was true for the job separation rates time series, we again suspect an underestimation of the non-employed as we go back in time. This time, the job-finding rates calculated from our retrospective panels are over-estimated. The most reliable point, in terms of the level of the job-finding rate, is likely to be the last point approaching the year of the survey. Figure 10 shows the overlap of the finding rates time series. We note likely over-estimation in the job-finding rates mirroring the under-estimation we noted above in the separation rates. If we set aside the level issue, and focus on the trend of both job-finding and separation rates, we note that there has been a very slight increase in the job-finding rates over time for males and almost no substantial change for females. A slight increase in the separation rates over time is also observed for both male and female workers (Fig. 7).
Non-employment to employment job-finding rate (percentage) by sex and wave, aged 15–54, 1990–2010. Source: Author’s calculations based on ELMPS 1998, ELMPS 2006, and ELMPS 2012
4.4.4 Job-to-job transitions
Having examined states and transitions between employment and non-employment, we now examine job-to-job transitions among the employed. The comparisons of retrospective and panel data show that more aggregated employment statuses are likely to be more consistently reported. For instance, it appears that respondents have difficulty distinguishing between informal and formal employment states as well as regular and irregular work. Therefore, we limit the analysis of the retrospective transition rates to three broad employment sectors, namely public wage work, private wage work, and non-wage work.
In Fig. 11, we overlap the job-to-job transition rates calculated using the ELMPS 1998, ELMPS 2006, and ELMPS 2012 retrospective panels. These rates are obtained by dividing the number of workers transitioning from one sector to another between years t and t + 1 by the number of workers employed in the origin sector in year t. Generally, we observe a close overlap of the job-to-job transition rates obtained using the different retrospective panel datasets. This finding suggests that using retrospective accounts give consistent conclusions about the trends of job-to-job transition rates over time, especially when these trends are smoothed. However, it is crucial to note that the levels of these transition rates are under-estimated given what we saw earlier in Fig. 6.
a–f Job-to-job transitions by wave for 1998, 2006, and 2012, male workers, aged 15–54, four period moving averages, 1990–2010. Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006 and ELMPS 2012
4.5 Do retrospective data provide accurate trends of past labor market aggregates?
The problems we observe in assessing labor market dynamics using retrospective data also present challenges to assessing stocks over time. This section examines the stocks derived from the retrospective data for two specific statistics: the unemployment rate and the employment to population ratio (employment rate). Figure 12 illustrates the evolution of employment to population ratio. The pattern suggests that as we go back in time, we only retain the employment states of our sample and lose track of their non-employment history. We obtain as a result a decreasing employment-to-population ratio, which is not consistent with patterns observed contemporaneously in the panel (Assaad and Krafft 2015c). The magnitude of the decrease differs from one survey to the other. The ELMPS 2012 seems to have less of a decreasing trend than the ELMPS 2006. The most likely explanation for this observation is the different structure of the ELMPS 2012 questionnaire. Asking individuals about their past statuses in a chronological order, starting with the first status, rather than backward in time as was the case in 1998 and 2006, may have increased consistency of employment trends. However, none of these approaches recovers the pattern of employment observed in the panel contemporaneous statuses, which is an increase in employment rates over time for men and a rise and fall in employment rates for women.
Employment-to-population ratio (percentage) by sex and wave, aged 15–54, 1990–2010. Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006, and ELMPS 2012
Superimposing the retrospective data and the unemployment rates from the official Labor Force Survey (LFS) further illustrates how the estimates of different labor market states get distorted if one uses retrospective data. Figure 13 shows that the retrospective data does not align with the evolution of Egypt’s unemployment rate over the past two decades as reported in official statistics. Unemployment is increasingly under-estimated relative to official statistics using retrospective data that are further back in time.
Unemployment rate (percentage), males, aged 15–64, by wave and compared to those reported in the LFSS, 1990–2010. Source: Authors’ calculations based on ELMPS 1998, ELMPS 2006, ELMPS 2012, and LFSS data. Based on CAPMAS’s bulletin of the Labor Force Sample Survey for 1990-2010
5 Conclusions
The primary objective of this paper is to assess the accuracy of labor market dynamics using retrospective data. We conclude that it is possible to garner useful information on labor market dynamics from retrospective data, but one must be cautious about which information to trust and at what level of detail. One of our most basic conclusions is that information on past employment collected using retrospective data can be fairly reliable, so long as fine distinctions between employment states are not made. For instance, the distinctions between employer and self-employed, public enterprise and government, formal and informal wage work, or regular and irregular wage work are not easily made using retrospective data.
In the case of women engaged in self-employment, whether in agriculture or outside agriculture, the distinction between being employed and not employed is hard enough to make in contemporaneous data, let alone in retrospective data. In Egypt, women in this kind of employment typically do not consider themselves to be employed and may move frequently between employment and non-employment states, as defined by international labor statisticians. To assess their current status accurately, researchers must use complex keyword-based questions that inquire about a large number of activities, and even this detailed approach often fails to elicit reliable estimates of female participation in home-based self-employment and unpaid family labor (Anker and Anker 1995; Assaad 1997; Langsten and Salem 2008; Assaad and El-Hamidi 2009). It is impossible to ask questions at this level of detail about a retrospective period, casting doubt on the employment transitions obtained from retrospective data for women in self-employment. Conversely, transitions across well-defined employment states, such as between public and private wage work or between public wage work and non-wage work, can be captured fairly reliably using retrospective data.
Spells of non-employment interspersed between employment spells are usually hard to recall, whether they are unemployment spells or spells outside the labor force altogether. For instance, 71% of those observed as unemployed in 1998 never reported any unemployment at any time in the past in the retrospective data obtained from them in the 2012 wave. Thus, transitions from non-employment to employment and vice versa will be understated in retrospective data. This result has important implications for the accurate reporting of separation rates for the employed and job-finding rates for either the unemployed or those outside the labor force, and the stock of unemployed in past dates. Generally, these rates will be understated, and possibly increasingly so as we go back in time, confounding any measurement of trends. In contrast, trends describing job-to-job transitions can be captured more reliably using retrospective data. A re-design of the retrospective modules to separately ask about past employment and non-employment states as suggested below could improve recall of past non-employment states.
Another conclusion we derived from analyzing the reporting of recalled marriage costs is that retrospective questions eliciting monetary amounts are unreliable at best. Even when asked to report the nominal amount paid at the time, at least some respondents tend to inflate the amount to their equivalent value at the time of the survey. It thus becomes impossible to ascertain monetary trends over time when some of the data is inflated and some of it is not.
Finally, this experience has allowed us to derive some important lessons on how to improve questionnaire design to collect more accurate retrospective data. First, in comparing the retrospective data from 2012 to the data from previous rounds, we determined it is preferable to ask questions about the individual’s labor market trajectory in chronological rather than in reverse chronological order. It elicits better information about labor market entry and in particular about any initial unemployment spells prior to first employment. Second, we suspect that many respondents (and possibly interviewers) interpreted status to mean job, contributing to the underreporting of non-employment spells. In future versions of these labor market panel surveys, we will test using separate questions for non-employment spells and for employment spells. The questionnaire should elicit first information about the non-employment spell just after exit from school, if any, and determine whether it was an unemployment spell or an out of the labor force spell. This would be followed by questions about the first employment spell and its characteristics, the next non-employment spell, if any, and so on. Another improvement to the questionnaire will be to ask those who have never worked for a period of more than 6 months prior to the interview and are currently inactive about whether they have ever sought employment and about the timing and length of the spell in which they were seeking employment, at least for the first time. Even though these changes will not eliminate recall bias, they could potentially reduce bias that results from questionnaire design.
Given budgetary and data availability constraints, the retrospective panels are currently the primary source of data in the MENA region that allow researchers to study labor market dynamics. Having discussed the errors encountered in retrospective data, it is important to note that it is possible to use some remedies to attenuate these measurement errors and eventually produce less biased (or possibly unbiased) results. A possible solution would be to match biased estimates obtained from retrospective data with more accurate estimates obtained from auxiliary contemporaneous cross-sectional data. Of course, this could be obtained from the same dataset or an external data source, so long as comparability between the data sets is verified. In this case, one assumes that the information obtained from the contemporaneous data is the most accurate. Assumptions about the (functional) form of the “forgetting rate” or information loss in the retrospective data would also be required. Yassin and Langot (2017) correct the ELMPS aggregate labor market transition rates between employment, unemployment, and inactivity states, obtained from the retrospective panels, using this methodology. This approach can even be extended to make use of the micro-level information available about the labor market transitions. Using the aggregate measurement errors estimated for the different types of transitions, one could distribute these errors in the form of weights to the individuals in the survey (Yassin 2016). Again, assumptions need to be made on how to attribute weights to the individuals. Another possible solution, with a different assumption, would be to estimate the alignment rate, possibly the rate of telling the truth, and eventually creating a weight such that individuals who report the truth have higher weights. This requires however the availability of both micro-level contemporaneous and retrospective information for the same individuals. In our case, it could be applied to the ELMPS but not to other datasets, for instance the Jordan Labor Market Panel Survey (JLMPS) and the Tunisia Labor Market Panel Survey (TLMPS), where for the time being, only one wave of the panel is available. Drawbacks of how representative the sample becomes after the creation of such weights need to be also discussed.
To conclude, we believe that panel data with retrospective modules to fill in the gaps between waves of the panel are the best data we can realistically hope for to study labor market dynamics in developing country contexts. Some advanced countries have moved to continuous administrative data to study such phenomena. However, given the low administrative capacity of most developing countries and the high rates of labor market informality, such methods are unlikely to become practical in developing country contexts very soon. In the absence of such panel data, a great deal can be learned from properly designed retrospective questions, so long as researchers are aware of the limitations of these data. As a general rule, distinctions that are hard to make in contemporaneous data, like differences between regular and irregular employment, formality and informality, illiteracy and literacy, and non-employment and home-based self-employment for women, are going to be even harder to make retrospectively. Shorter spells and more frequent events in an individual’s labor market trajectory are more likely to be forgotten. We have attempted in this work to highlight some of these problems, but we are in no way suggesting that analyses based on retrospective data are worthless. We are simply advising that proper caution needs to be exercised in interpretation and have provided some pointers as to where the potential pitfalls might lie.
Notes
Panel attrition is discussed in more detail below.
Statuses of less than 6 months are dropped, and if four statuses are not enough to reach the current status, the fifth and later statuses are also dropped.
For individuals who never went to school, the retrospective period starts at age 6.
Not everyone was asked the education questions again in 2012; if the respondent reported that his/her educational status did not change since 2006, the 2006 data was re-used in 2012. The alignment of reported own education statuses was therefore only examined between 1998 and 2006.
We choose this age group since individuals at younger ages could still be acquiring additional education.
General secondary is typically a very small category in Egypt because most of those who achieve this level of education are eligible for higher education and typically proceed to higher levels. Smaller categories may suffer from more misreporting due to their size. For instance, if random typos are uniformly distributed across the categories, more responses in the smallest categories are likely to be errors.
When individuals were not consistently responding for themselves, it is possible that the same individual was responding in their place in both waves (e.g., a spouse) but the data does not allow us to determine whether this is so.
When the father is present, it is usually possible to use the retrospective information on the father to obtain the father’s characteristics when the individual in question was 15.
Fathers are likely to be still present in the household for respondents younger than 30 years old.
Only women were asked in 2006, so we restrict our analyses in 2012 to women’s reports. We do not restrict by age so as to not confound timing of marriage shifts with time trends. Cost of marriage was not asked in 1998.
Data are not separated by gender or restricted by age so as to ensure an adequate sample size.
As defined in the descriptive section above.
We use less than the full set of nine labor market statuses as regressors here—relying on the three key characteristics of employed or not, informal or not, and irregular or not—to ensure results are estimable for both men and women.
See Assaad et al. (2016) for a discussion of how different approaches to data inclusion affect estimation of dynamics.
Given that the survey was designed to capture only retrospective labor market statuses that last for more than 6 months, we reclassify, for comparability, those short unemployment spells as employment lags.
We exclude from our analyses 2011, since we cannot disentangle reporting issues from the effects of the 2011 Uprising in Egypt.
References
Anker R, Anker M. Measuring female labour force with emphasis on Egypt. In: Khoury NF, Moghadam VM (eds) Gender and development in the Arab World: women’s economic participation: patterns and policies. Helsinki: United Nations University Press; 1995. pp 148–176.
Artola C, Bell U-L. Identifying labour market dynamics using labour force survey data. Mannheim; 2001.
Assaad R. The employment crisis in Egypt: current trends and future prospects. Res Middle East Econ. 1997;2:39–66.
Assaad R, El-Hamidi F. Women in the Egyptian labor market: an analysis of developments, 1988-2006. In: Assaad R, editor. The Egyptian labor market revisited. Cairo: The American University in Cairo Press; 2009. p. 219–58.
Assaad R, Krafft C. The Egypt labor market panel survey: introducing the 2012 round. IZA J Labor Dev. 2013;2:1–30.
Assaad R, Krafft C. Youth transitions in Egypt: school, work, and family formation in an era of changing opportunities. Doha: Silatech; 2014.
Assaad R, Krafft C. An empirical analysis of the economics of marriage in Egypt, Morocco, and Tunisia. In: Monga C, Lin JY, editors. The Oxford handbook of Africa and economics: policies and practices. Oxford: Oxford University Press; 2015a.
Assaad R, Krafft C. The economics of marriage in North Africa: a unifying theoretical framework. In: Monga C, Lin JY, editors. The Oxford handbook of Africa and economics: contexts and concepts. Oxford: Oxford University Press; 2015b.
Assaad R, Krafft C. The structure and evolution of employment in Egypt: 1998-2012. In: Assaad R, Krafft C, editors. The Egyptian labor market in an era of revolution. Oxford: Oxford University Press; 2015c. p. 27–51.
Assaad R, Krafft C, Yassin S Comparing retrospective and panel data collection methods to assess labor market dynamics. Cairo, Egypt 2016.
Assaad R, Ramadan M. Did housing policy reforms curb the delay in marriage among young men in Egypt? Washington, DC: Wolfensohn Center for Development; 2009.
Assaad R, Roushdy R. Methodological appendix 3: an analysis of sample attrition in the Egypt labor market panel survey 2006. In: Assaad R, editor. The Egyptian labor market revisited. Cairo: American University in Cairo Press; 2009. p. 303–16.
Beckett M, Da Vanzo J, Sastry N, et al. The quality of retrospective data: an examination of long-term recall in a developing country. J Hum Resour. 2001;36:593–625.
Beegle K, Carletto C, Himelein K. Reliability of recall in agricultural data. J Dev Econ. 2012;98:34–41. https://doi.org/10.1016/j.jdeveco.2011.09.005.
Beegle K, De Weerdt J, Friedman J, Gibson J. Methods of household consumption measurement through surveys: experimental results from Tanzania. J Dev Econ. 2012;98:3–18. https://doi.org/10.1016/j.jdeveco.2011.11.001.
Black D, Berger M, Scott F. Bounding parameter estimates with nonclassical measurement error. J Am Stat Assoc. 2000;95:739–48.
Blossfeld H-P, Golsch K, Rohwer G. Event history analysis with Stata. Mahwah: Lawrence Erlbaum Associates; 2007.
Bound J, Brown C, Mathiowetz N. Measurement error in survey data. Handb Econom. 2001;5:3705–843.
Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu CM. Measurement error in nonlinear models: a modern perspective. 2nd: CRC Press; 2012.
de Nicola F, Giné X. How accurate are recall data? Evidence from coastal India. J Dev Econ. 2014;106:52–65. https://doi.org/10.1016/j.jdeveco.2013.08.008.
Dercon S, Shapiro JS. Moving on, staying behind, getting lost: lessons on poverty mobility from longitudinal data. 2007.
Dhillon N, Yousef T. Generation in waiting: the unfulfilled promise of young people in the Middle East. Washington, DC: Brookings Institution Press; 2009.
Feng S, Hu Y. Misclassification errors and the underestimation of U. S. unemployment rates. Am Econ Rev. 2013;103:1–40. https://doi.org/10.1257/aer.103.2.1054.
Fuller WA. Measurement error models. New York: John Wiley & Sons; 1987.
Grosh M, Glewwe P, editors. Designing household survey questionnaires for developing countries: lessons from 15 years of the living standards measurement study. Washington, DC: The World Bank; 2000.
Heitjan D, Rubin DB. Inference from coarse data via multiple imputation with application to age heaping. J Am Stat Assoc. 1990;85:304–14.
Hiroyuki N, Sawada Y, Mari T. Asking retrospective questions in household surveys: evidence from Vietnam. 2010.
Judge G, Schechter L. Detecting problems in survey data using Benford’s law. J Hum Resour. 2009;44:1–24.
Kherfi S. Determinants of unemployment duration in Egypt. In: Assaad R, Krafft C, editors. The Egyptian labor market in an era of revolution. Oxford: Oxford University Press; 2015. p. 90–107.
Langsten R, Salem R. Two approaches to measuring women’s work in developing countries: a comparison of survey data from Egypt. Popul Dev Rev. 2008;34:283–305.
Magnac T, Visser M. Transition models with measurement errors. Rev Econ Stat. 1999;81:466–74.
Mathiowetz NA, Duncan GJ. Out of work, out of mind: response errors in retrospective reports of unemployment. J Bus Econ Stat. 1988;6:221–9.
OAMDI. Labor market panel surveys (LMPS). Version 2.2 of licensed data files; ELMPS 2012 2016.
Pina-Sánchez J, Koskinen J, Plewis I. Measurement error in retrospective work histories. Surv Res Methods. 2014;8:43–55.
Puetz D. Improving data quality in household surveys. In: Von Braun J, Puetz D, editors. Data needs for food policy in developing countries: new directions for household surveys. Washington, DC: IFPRI; 1993. p. 173–85.
Roberts JM, Brewer DD. Measures and tests of heaping in discrete quantitative distributions. J Appl Stat. 2001;28:887–96. https://doi.org/10.1080/02664760120074960.
Salem R Trends and differentials in Jordanian marriage behavior: timing, spousal characteristics, household structure and matrimonial expenditures. In: Assaad R (ed) The Jordanian labour market in the new millennium. Oxford University Press 2014, pp 189–217.
Salem R. Changes in the institution of marriage in Egypt from 1998 to 2012. In: Assaad R, Krafft C, editors. The Egyptian labor market in an era of revolution. Oxford: Oxford University Press; 2015. p. 162–81.
Salem R. Imagined crises: assessing evidence of delayed marriage and never-marriage in contemporary Egypt. In: Celello K, Kholoussy H, editors. Domestic tensions, national anxieties: global perspectives on marriage crisis. Oxford: Oxford University Press; 2016. p. 231–54.
Singerman D. The economic imperatives of marriage: emerging practices and identities among youth in the Middle East. 2007.
Singerman D, Ibrahim B. The costs of marriage in Egypt: a hidden variable in the new Arab demography. In: Hopkins NS, editor. Cairo papers in social science, volume 24 numbers 1/2: the new Arab family. Cairo: American University in Cairo Press; 2003. p. 80–166.
Yassin S. Constructing labor market transitions recall weights in retrospective data: an application to Egypt and Jordan. Cairo: Economic Research Forum Working Paper Series; 2016.
Yassin S, Langot F. Correcting Measurement Errors in Transition Models Based on Retrospective Panel Data. Neuchâtel: IRENE Working Papers 17-04, IRENE Institute of Economic Research; 2017.
Acknowledgements
We would like to thank the anonymous referee and the editor for the useful remarks.
Responsible editor: Hartmut F. Lehmann
Funding
This research has benefited from funding from the Economic Research Forum, Cairo, Egypt ( www.erf.org.eg ), under the auspices of a research project entitled Labor Market Dynamics in the Middle East and North Africa.
Availability of data and materials
The data used for this paper are publicly available through the Open Access Microdata Initiative of the Economic Research Forum ( erfdataportal.com ).
Author information
Authors and Affiliations
Contributions
All three authors contributed equally to this manuscript.
Corresponding author
Ethics declarations
Competing interests
The IZA Journal of Development and Migration is committed to the IZA Guiding Principles of Research Integrity. The authors declare that they have observed these principles.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional files
Additional file 1:
Figure S1. Father’s sector of work and employment status when individual was age 15, as reported in 1998 versus 2006, father not in household in 1998 or 2006, aged 30–54 in 1998 (percentage). Source: Authors’ calculations based on ELMPS 1998 and ELMPS 2006. (DOCX 16 kb)
Additional file 2:
Figure S2. Labor market status, as reported contemporaneously for 1998 and as reported retrospectively for 1998 from 2006 retrospective data, by sex, individuals aged 20–44 in 1998 and present in both waves (percentage). Source: Authors’ calculations based on ELMPS 1998 and ELMPS 2006. (DOCX 23 kb)
Additional file 3:
Figure S3. Labor market status, as reported in 1998 versus 2006 retrospective data for 1998, by sex, individuals aged 20–44 in 1998 and present in both waves (percentage). Source: Authors’ calculations based on ELMPS 1998 and ELMPS 2006. (DOCX 24 kb)
Additional file 4:
Table S1. Patterns of unemployment reporting as reported in 1998 versus 2006 retrospective data for 1998, individuals reporting contemporaneous unemployment in 1998 and present in 2006. (DOCX 14 kb)
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Assaad, R., Krafft, C. & Yassin, S. Comparing retrospective and panel data collection methods to assess labor market dynamics. IZA J Develop Migration 8, 17 (2018). https://doi.org/10.1186/s40176-018-0125-7
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40176-018-0125-7