Advertisement

Reporting of Cultural Factors in Autism Research Publications in Sweden: Application of the GAP-REACH Checklist

  • Rano Zakirova EngstrandEmail author
  • Nina Klang
  • Tatja Hirvikoski
  • Mara Westling Allodi
  • Lise Roll-Pettersson
Open Access
Review Paper

Abstract

Systematic reporting of cultural variables in research publications is important to address disparities in diagnostics and treatment for children with ASD from diverse backgrounds. The present review examined reporting of cultural factors in ASD publications in the Swedish research context by using the GAP-REACH checklist developed by the Cultural Committee of the Group for the Advancement of Psychiatry. Thirty peer-reviewed articles published in English between 2013 and 2015 met inclusion criteria. Depending on research designs, 46% of the reviewed studies defined cultural factors using various proxies for ethnicity to describe study participants; none of the studies used the “race” variable; 23.3% provided rationale for inclusion of cultural factors. The checklist in its modified form is applicable within the Swedish context.

Keywords

Autism publications Cultural factors GAP-REACH checklist 

Introduction

Recent epidemiological studies on global prevalence of autism spectrum disorder (ASD) revealed that the estimate increased over time with a median prevalence of 62/10,000, indicating that worldwide one in 160 children has ASD (Elsabbagh et al. 2012; World Health Organization 2017). These children and their families risk facing social stigma, isolation, and discrimination (World Health Organization 2014). Research findings from several multicultural societies, e.g., the USA and Canada, have revealed existing disparities between minority and majority populations with ASD in accessing high-quality health care services (Bishop-Fitzpatrick and Kind 2017), or getting eligibility for special education services (Tincani et al. 2009). In response, the 67th World Health Assembly (WHA67) adopted a resolution on Comprehensive and Coordinated Efforts for the Management of Autism Spectrum Disorders, where Member States are urged “to identify and address disparities in access to services for persons with autism spectrum disorders and other developmental disorders” (World Health Organization 2014, p.3).

The concerns of the WHO are well justified as studies have shown that cultural factors can potentially lead to health disparities in recognition, diagnostic assessment, and treatment for children with ASD especially regarding ethnic minority communities (Interagency Autism Coordinating Committee (IACC) 2014; Mandell et al. 2002, 2009; National Autism Center 2015; Pierce et al. 2014). Mandell and Novak (2005) argue that investigation of cultural factors in ASD research should focus on studying families’ cultural views and beliefs about causes of their child’s disability, existing barriers to ASD diagnosis as well as parents’ treatment preferences. Most recent developments in diagnostic assessment, e.g., the Cultural Formulation Interview (CFI) included into the latest edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5, American Psychiatric Association 2013), now allow exploring these cultural views and beliefs held by children and/or their caregivers to facilitate comprehensive assessment and access to quality healthcare, social and educational services for ethnically and culturally diverse populations in multicultural environments.

Sweden is a multicultural society and host country for many refugees and immigrants (Bäärnhielm and Scarpinati Rosso 2009), where the prevalence rate of ASD in children has been estimated to be almost 1% (Nygren et al. 2012). This fact has gained paramount attention by the research community placing Sweden on the 10th place of the top 25 countries that published ASD research between 1980 and 2010 (Office of Autism Research Coordination [OARC] 2012). Moreover, Sweden-based ASD researchers were among the first in Europe and in Nordic countries who began studying possible associations between ethnicity, immigration, and potential risks for ASD among children of immigrant parents (Keen et al. 2010), with the scientific report published as early as 1987 (Gillberg et al. 1987). This emphasis on cultural variables points to researchers’ recognition of the importance of studying these variables for the development of comprehensive supports and services for culturally diverse children with ASD and their families in Sweden (Barnevik-Olsson et al. 2008). However, it is less clear if collecting and reporting of cultural variables are common research practices in autism research conducted in Sweden.

The CFI (American Psychiatric Association 2013) presents a practical tool to be used by researchers and practitioners for collecting data on cultural and contextual factors to inform clinical practice (Hinton and Hinton 2016). As Sweden-based researchers have begun using the DSM-5 in their work regarding diagnostic assessment of ASD, especially for young children (Zander 2015), their awareness of the CFI has probably increased. Bäärnhielm et al. (2016) reported on experiences of clinical use of the CFI’s preceding framework, the Outline for Cultural Formulation (OCF), included into the DSM-IV (American Psychiatric Association 1994). Practitioners expressed their interest in cultural formulation, which, as Bäärnhielm et al. (2016) argue, can be explained partly by the increased diversity nationwide, and partly by limitations of the International Classification of Diseases (ICD-10, World Health Organization n.d.)—the official diagnostic system in Sweden—in addressing cultural variations in mental health and mental healthcare. However, the key factor in increased attention to cultural formulation is that the core CFI is now an integral part of the DSM diagnostic system, as in Sweden DSM is often used in research and clinical practice in parallel with ICD, significantly influencing professional and public debates on mental health and mental health care (Bäärnhielm et al. 2016). To promote the use of the core CFI and to increase cultural awareness among practitioners, the CFI was translated into Swedish and is now available to practitioners free of charge. Moreover, the information about the CFI is included in guidelines for healthcare, in recent medical textbooks, and on official psychiatric Web sites (Bäärnhielm et al. 2016). Although the CFI is currently not a part of the teacher and special teacher training programs in Sweden, diagnostic assessments using DSM-5 for identification of ASD in children and youth will certainly have impact on educational supports and services in the long-term perspective, especially for students with immigrant background. According to the Swedish National Board of Health and Welfare (2013), children who have at least one parent born outside Sweden constitute one fifth of all children in Sweden. A study by Cederlund et al. (2014) showed that out of the 33 young children diagnosed with ASD who participated in the study 6% had one parent born in another European country, 6% had one parent born in another part of the world, and 49% both parents were born outside Sweden, reflecting the current situation in the country regarding ethnic and cultural diversity of this population. In this context, it would be instructive to investigate if the Swedish researchers have become more attentive to reporting cultural factors in their original empirical research after the DSM-5 appeared first in English in 2013, and later in Swedish in October 2014.

The GAP-REACH Checklist to Assess Reporting of Cultural Factors

Systematic documenting of factors related to ethnicity and culture in scientific publications can help understand and reduce culture-related disparities in service provision affecting diverse ethnic and cultural populations (Salway et al. 2009, 2011; Lewis-Fernández et al. 2013; Mir et al. 2012). Research evidence plays a vital role in informing service development, shaping policy, and promoting debates to help reduce existing inequalities for minority ethnic groups (Mir et al. 2012).

Underreporting of cultural factors in the field of ASD is, however, well documented (Broder-Fingert et al. 2017; Kistner and Robbins 1986; National Autism Center 2015; Pierce et al. 2014; West et al. 2016; Wong et al. 2014, 2015). For instance, Pierce et al. (2014) evaluated methodological practices for reporting ethnicity for study participants in ASD studies published in three autism-related academic journals. They found that 72% of the reviewed articles did not include ethnicity or other descriptors (e.g., racial or cultural differences) for study participants. It was also revealed that when ethnicity was reported, 54% of these studies did not include ethnicity or race variable into data analysis. The reviews of intervention research reporting on evidence-based practices (EBPs) for children and youth with ASD (National Autism Center 2015; Wong et al. 2014, 2015) also showed that most efficacy studies did not systematically include information on cultural demographics of their participants. More specifically, these studies did not describe ethnicity, nationality, or race of the participants other than White, Caucasian, or European-American (West et al. 2016). Inadequate reporting of cultural factors presents a number of methodological issues affecting both research and practice. For instance, it can hinder conducting subsequent analyses to understand reasons for cross-cultural differences in ASD prevalence, symptomatology, and treatment preferences (Chung et al. 2014); it can also present threats to replicability and generalizability of study results (Kistner and Robbins 1986), especially, for the proposed evidence-based interventions. As West et al. (2016) noted the absence of scientifically sound interventions with culturally diverse students might make teachers feel skeptical about the efficacy of the recommended EBPs in educational settings as these EBPs may conflict with beliefs and values of these students and their families.

On the other hand, researchers’ clear reporting of culture-related factors can ensure rigor of their findings; inclusion of cultural variables into analyses can provide viable outcome data to compare, replicate, or extend across different or similar ethnic groups (Pierce et al. 2014), as well as can help establish external validity for EBPs for diverse individuals with ASD (West et al. 2016). This may have direct implications for practice as it allows to (1) understand mechanisms underlying disproportionate diagnosis and access to services for ethnic minorities with ASD (Pierce et al. 2014), (2) increase practitioners’ awareness on impact of cultural factors on treatment outcomes (Chung et al. 2014), and (3) design culturally competent, individualized interventions, tailored to meet unique needs of each person with ASD and his/her family members (Stahmer et al. 2011). Those in turn can help eliminate existing disparities, and in combination with preventive interventions, may potentially lead to improved quality of life and life expectancy for individuals with ASD (Bishop-Fitzpatrick and Kind 2017).

To assess the scope of reporting cultural factors in psychiatric research publications, the Cultural Committee of the Group for the Advancement of Psychiatry (GAP) developed a checklist GAP-REACH (Race, Ethnicity, And Culture in Health; Lewis-Fernández et al. 2013). The GAP-REACH checklist was developed based on expert consensus, empirical review of publication content, and item-based reliability assessment. The purpose of the checklist is descriptive, to be used by researchers, journal editors and reviewers. It is operationalized as a list of 16 domains and allows calculating a total score for an article to evaluate whether the study results reported in the article reflected adequate attention to cultural factors to guide planning of appropriate interventions for culturally and ethnically diverse populations. At the same time, application of the checklist is flexible as it allows omitting certain items if not applicable to the research methodology. There is, though, an existing agreement among the scholars engaged into work on developing standards for reporting research on ethnicity and health about what domains are to be considered as the most important when reporting cultural characteristics of research samples. These domains include (a) clear definition of ethnicity and cultural variables, (b) rationale for including these in the study, (c) methods for their ascertainment, and (d) their inclusion in data analysis and interpretation (Lewis-Fernández et al. 2013). As of date, only one study reported using the GAP-REACH checklist. Geary (2015) reviewed publications on well-being of American Indian and Alaska Native children and their families found in 22 US national databases. The GAP-REACH criteria were adapted to assess quality of reporting cultural demographic variables in 33 selected studies on a scale “low-medium-high.” Although informative, this study has an important limitation: it did not report data on inter-rater reliability for individual items of the checklist. Besides, the study restricted its focus on describing a particular ethnic population, and it was not autism-specific. To our knowledge, the present study is the first to use the GAP-REACH criteria to review literature in the field of ASD research.

Aims and Research Questions

The main aim of the present study was to assess the scope of reporting ethnicity and other cultural factors in research publications by Swedish scholars involved in empirical research in ASD in children and youth by using the checklist GAP-REACH (Lewis-Fernández et al. 2013). An additional aim of this study was to test the utility of the checklist for assessing ASD research publications within the Swedish context. Based on the existing agreement on the key domains for comprehensive reporting of cultural variables (Lewis-Fernández et al. 2013), we asked the following research questions:
  1. 1.

    How do Sweden-based researchers define or conceptualize cultural factors in their studies?

     
  2. 2.

    How do they describe cultural characteristics of the study participants?

     
  3. 3.

    Do the researchers provide rationale for inclusion of cultural factors in the studies?

     

We chose to limit our study to the Swedish research context only, as the existing review studies that investigated a similar concept in the field of ASD were predominantly conducted in the USA (e.g., Broder-Fingert et al. 2017; Pierce et al. 2014; West et al. 2016). Another reason for examining only Swedish ASD publications was a practical one: we are a group of researchers who are involved into intervention research for children and youth with ASD and other developmental disabilities in various educational and clinical settings in Sweden. It is our hope that findings from the present study will inform researchers and practitioners in our country as well as the colleagues from other culturally and ethnically diverse societies in Europe and worldwide.

Method

We conducted a review based on a comprehensive search of scientific literature published online between the 13 May 2013 (the DSM-5 publication date in the USA) and 30 November 2015. For the search and selection strategies as well as for reporting and discussing results, we used Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards (Moher et al. 2009) where applicable (please see Supplemental online material for the PRISMA checklist).

Inclusion Criteria

Studies included in the review were to meet the following criteria: (a) it was an original empirical research in autism spectrum disorders, (b) it was conducted in Sweden, (c) participants involved children 0–18 years diagnosed with ASD and/or their family members, and (d) it was published in English. Studies excluded were literature reviews, meta-analyses, editorials, conceptual/theoretical papers, and letters to the editors.

Search Process

Research articles published in peer-reviewed academic journals were obtained through search in the following databases: Educational Resource Information Center (ERIC), Cumulative Index to Nursing and Allied Health Literature (CINAHL), MEDLINE, Scopus, Sociological Abstracts, and Social Services Abstracts. The following search terms were used: “autism spectrum disorders in children” OR “ASD” OR “Asperger’s syndrome” OR “pervasive disorders” AND “Sweden” OR “Swedish.” Three limiters were applied: geographic location—Sweden, children 0–18 years, and English as a publication language. The reference management system used was RefWorks. Initial search yielded 1331 articles. After removing duplicate articles retrieved from the databases, the search yielded 1173 articles. The next step in selecting articles was to read through titles to remove articles not fulfilling the inclusion criteria. The screening of the titles yielded 523 articles. The next round of the screening was to examine abstracts to determine if the study was conducted in Sweden. If the abstracts did not provide sufficient information, the full texts were retrieved, where the sections participants and setting were screened to get information on where recruitment for the study took place. This screening round yielded 32 potential articles. The full texts of these articles were retrieved, saved in PDF format for further analysis. Initial reading of the full texts excluded two articles based on the inclusion criteria: participants in one study were adults; another article presented a conceptual paper (see Fig. 1).
Fig. 1

Review process of articles included into analysis

Analysis

All identified articles (n = 30) were assessed for reporting ethnicity and cultural factors of research participants using the checklist GAP-REACH (Lewis-Fernández et al. 2013). The checklist consists of 16 item domains and guiding representative questions for each domain. The domains are grouped around traditional format for original articles: Introduction, Methods, Results, and Discussion. The reported inter-rater reliability (IRR) of the checklist is κ = 0.91, and the internal consistency is α = 0.885 (Lewis-Fernández et al. 2013; see Table 1 for description of domains).
Table 1

The GAP-REACH checklist: domains and items

Item

Overall REC use

 1. At least one REC term used in the title or abstract (YES/NO)

 2. At least one REC term used in the article text (YES/NO)

 3. Definition/conceptualization of REC term(s) provided (YES/NO)

Introduction/Background

 4. Rationale for study question or design discussed in terms of REC factors (YES/NO)

Methods

 5. REC included in sampling procedure (YES/NO)

 6. Sample described in terms of REC characteristics (YES/NO)

 7. Methods described for assessing REC characteristics of participants (YES/NO)

 8. Language proficiency of participants specified (YES/NO or N/A)

 9. Relevance of REC characteristics of interviewers and participants discussed (YES/NO or N/A)

 10. Relevance of language characteristics of interviewers and participants discussed (YES/NO or N/A)

 11. Language match between participants and instruments is appropriately described (YES/NO or N/A)

 12. Measurement equivalence of instruments described for all REC groups in the study (YES/NO or N/A)

Results

 13. Effect of REC factors on study outcome(s) tested (YES/NO or N/A)

 14. REC factors included in data analysis (YES/NO)

Discussion

 15. REC emphasized in data interpretation (YES/NO)

 16. Study limitations discussed in REC terms (YES/NO)

Adapted from Lewis-Fernández et al. (2013)

REC race, ethnicity, and culture

Guided by representative questions for each item domain of the checklist, every article was read through with close attention paid to information in each section and then coded following the coding guidelines. For each item, only one answer was selected. Items 1–7 and 14–16 had yes or no answers (i.e., scored either positively or negatively); items 8–13 had yes, no, and an additional option not applicable (N/A). In the original checklist, the items are coded as N/A if they were not applicable to research methodology, e.g., studies that did not involve participant interviews, and therefore, it was not possible to assess participants’ language proficiency (item 8), there was no need for translation of study instruments (item 11), or measurement equivalence (item 12) (Lewis-Fernández et al. 2013). The GAP-REACH total score was then calculated by adding the items scored yes and not applicable, dividing the sum by the total number of items (16), and multiplying by 100%. Thus, each reviewed study obtained the total GAP-REACH score in percentage, indicating how well cultural factors were addressed in the article (i.e., coded yes) or did not apply to the research methodology (Lewis-Fernández et al. 2013). To answer the research questions, during the data extraction phase special attention was paid to determine if the items 3, 4, and 6 were applicable throughout the texts or the tables where the findings were presented. The identified articles were also screened for research areas and categorized according to Milner and Cho (2014) as (a) basic (e.g., biomedical studies, risk factors), (b) descriptive (diagnosis and prevalence studies), (c) treatment/intervention, and (d) other (e.g., quality of life, etc.).

Inter-Rater Reliability

The first author coded and analyzed all identified articles (n = 30) using the checklist’s scoring guidelines developed by Lewis-Fernández et al. (2013). Permission to use the scoring guidelines for this study was obtained by the first author from R. Lewis-Fernández (personal communication, December 9, 2015). The second author (a PhD-level researcher) coded and analyzed seven (n = 7) randomly selected articles (≈ 25% of all identified articles). Coders followed the scoring guidelines for each item independently of each other. Prior to initiating the coding process, the second coder studied the scoring guidelines of the GAP-REACH checklist as well as the original article on the development and the use of the checklist. Furthermore, the second author consulted with the first author on the use of scoring guidelines. To calculate IRR for each item both percentage of agreement and Cohen’s kappa statistics for two raters were used, first, by hand calculation, and then by using computer software (IBM SPSS Statistics for Windows, version 22) to verify the analyses. The percentage of agreement was calculated according to the following formula: the number of agreements divided by the total number of agreements and disagreements, multiplied by 100. Confidence intervals (CIs) for obtained kappa for each item were hand calculated by using the following generic formula: κ ± 1.96SE. The main reason for the combined use of two measures of reliability and agreement (i.e., kappa statistics and percentage of agreement) was to give a detailed picture of the degree of the reliability and agreement as each single measure provides only limited information (Kottner et al. 2011). Percentage of agreement allows researchers to identify variables that may be problematic (McHugh 2012). This decision laid ground for discussion of disagreements between two coders. After scoring and coding was completed by both coders, disagreements were examined and discussed following the checklist’s scoring guidelines. After discussion, 100% agreement was achieved across the articles between coders. Following the results of reached agreements between two coders, the complete sample of the articles (n = 30) was re-coded by the first coder.

Results

Table 2 provides a list of reviewed articles with information on (a) authors, (b) publication year, (c) article title, (d) journal, (e) diagnostic classification/criteria used, and (f) type of research (basic, descriptive, treatment/intervention, other). The majority of the reviewed studies (n = 18, 60%) were categorized as “basic,” six studies (20%) as “descriptive,” three studies (10%) as “treatment/intervention,” and three (10%) as “other.”
Table 2

Selected articles for review

No.

Article authors

Publication year

Article title

Journal

Diagnostic classification/criteria used

Type of study (basic/intervention)

1

Ahlström and Wentz

2014

Difficulties in everyday life: young persons with attention-deficit/hyperactivity disorder and autism spectrum disorders perspectives. A chat-log analysis.

International Journal of Qualitative Studies on Health and Well-being

DSM-IV

Other (qualitative study design)

2

Anclair and Hiltunen

2014

Cognitive behavioral therapy for stress-related problems: single-case studies of parents of children with disabilities.

Clinical Case Studies

n/a

Treatment/intervention

3

Atladottir et al.

2015

The increasing prevalence of reported diagnoses for childhood psychiatric disorders: a descriptive multinational comparison.

European Child & Adolescent Psychiatry

ICD-8; ICD-9; ICD-10; DSM-IV

Descriptive (prevalence)

4

Bölte et al.

2014

The roots of autism and ADHD twin study in Sweden (RATSS).

Twin Research and Human Genetics

ICD-10; DSM-5

Basic (etiology; twin study)

5

Butwicka et al.

2015

Hypospadias and increased risk for neurodevelopmental disorders.

Journal of Child Psychology and Psychiatry

ICD-8, ICD-9, ICD-10

Basic (risk factors)

6

Callenmark et al.

2014

Explicit versus implicit social cognition testing in autism spectrum disorder.

Autism

DSM-IV-TR

Basic (social cognition)

7

Cederlund et al.

2014

Pre-schoolchildren with autism spectrum disorders are rarely macrocephalic: a population study

Research in Developmental Disabilities

DSM-IV

Basic (risk factors)

8

De Bildt et al.

2015

Autism Diagnostic Interview-Revised (ADI-R) algorithms for toddlers and young preschoolers: application in a non-US sample of 1104 children

Journal of Autism and Developmental Disorders

DSM-5 (mentioned)

Descriptive (diagnosis)

9

Domellöf et al.

2014

Health-related quality of life of children and adolescents with functional disabilities in a northern Swedish county

Quality of Life Research

n/a (not reported)

Other (quality of life)

10

Falck-Ytter, Thorup & Bölte

2015

Brief report: lack of processing bias for the objects other people attend to in 3-year olds with autism

Journal of Autism and Developmental Disorders

DSM-IV

Basic

11

Fernell et al.

2015

Autism spectrum disorder and low vitamin D at birth: a sibling control study

Molecular Autism

n/a (not reported)

Basic (risk factors)

12

Gardner et al.

2015

Maternal body mass index during early pregnancy, gestational weight gain, and risk of autism spectrum disorders: results from a Swedish total population and discordant sibling study

International Journal of Epidemiology

ICD-9, ICD-10, DSM-IV

Basic (risk factors)

13

Idring et al.

2015

Changes in prevalence of autism spectrum disorders in 2001–2011: findings from the Stockholm youth cohort

Journal of Autism and Developmental Disorders

ICD-7, ICD-9, ICD-10, DSM-IV

Descriptive (prevalence)

14

Idring et al.

2014

Parental age and the risk of autism spectrum disorders: findings from a Swedish population-based cohort

International Journal of Epidemiology

ICD-9, ICD-10, DSM-IV

Basic (risk factors)

15

Janeslätt et al.

2013

Evaluating interventions using time aids in children with disabilities

Scandinavian Journal of Occupational Therapy

n/a (not reported)

Treatment/Intervention

16

Kerekes et al.

2015

Neurodevelopmental problems and extremes in BMI

PeerJ

DSM-IV

Basic (risk factors)

17

Lee et al.

2015

Maternal hospitalization with infection during pregnancy and risk of autism spectrum disorders

Brain, Behavior, and Immunity

n/a

Basic (risk factors)

18

Löfkvist et al.

2014

Lexical and semantic ability in groups of children with cochlear implants, language impairment and autism spectrum disorder

International Journal of Pediatric Otorhinolaryngology

DSM-IV-R

Basic

19

Lundström et al.

2015a

Autism phenotype versus registered diagnosis in Swedish children: prevalence trends over 10 years in general population samples

The BMJ

ICD-9, ICD-10, DSM-IV

Descriptive (prevalence; twin study)

20

Lundström, et al.

2015b

Autism spectrum disorders and coexisting disorders in a nationwide Swedish twin study

Journal of Child Psychology and Psychiatry

DSM-IV

Basic (comorbidity)

21

Lundström et al.

2014

Childhood neurodevelopmental disorders and violent criminality: a sibling cohort study

Journal of Autism and Developmental Disorders

DSM-IV

ICD-8,9,10

Basic (risk factors)

22

McEvilly et al.

2015

Sick leave and work participation among parents of children with autism spectrum disorder in the Stockholm youth cohort: a register linkage study in Stockholm, Sweden

Journal of Autism and Developmental Disorders

DSM-IV

ICD-9, 10

Other (family well-being)

23

Miniscalco et al.

2014

Imitation (rather than core language) predicts pragmatic development in young children with ASD: a preliminary longitudinal study using CDI parental reports

International Journal of Language & Communication Disorders

DSM-IV

Basic

24

Pahnke et al.

2014

Outcomes of an acceptance and commitment therapy-based skills training group for students with high-functioning autism spectrum disorder: a quasi-experimental study

Autism

DSM-IV

Treatment/intervention

25

Palmqvist et al.

2013

Screening for substance use disorders in neurodevelopmental disorders: a clinical routine?

European Child & Adolescent Psychiatry

DSM-5

Descriptive (screening)

26

Ronald et al.

2014

Symptoms of autism and ADHD: a Swedish twin study examining their overlap

Journal of Abnormal Psychology

DSM-IV

Basic

27

Sandin et al.

2014

The familial risk of autism

JAMA

ICD-7-10

Basic (genetic risk factors)

28

Selten et al.

2015

Risks for nonaffective psychotic disorder and bipolar disorder in young people with autism spectrum disorder. A population-based study

JAMA Psychiatry

DSM-IV, ICD-9-10

Basic (risk factors)

29

Törn et al.

2015

Childhood neurodevelopmental problems and adolescent bully victimization: population-based, prospective twin study in Sweden

European Child & Adolescent Psychiatry

DSM-IV-TR

Basic

30

Zander et al.

2015

The added value of the combined use of the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule: diagnostic validity in a clinical Swedish sample of toddlers and young preschoolers

Autism

DSM-IV-TR

Descriptive (diagnosis)

Definitions and Conceptualizations of Cultural Factors

The analysis revealed that the authors of 14 out of the 30 reviewed articles (46%) provided various definitions and conceptualizations of ethnicity and cultural factors (Table 3). These definitions and conceptualizations are briefly described in Table 4 for each identified study.
Table 3

Percentages of articles that provided definitions and descriptions of cultural factors as well as rationale for inclusion of these factors in the reviewed studies (n = 30)

Results

N (%)

Definitions and conceptualizations of cultural factors

14 (46)

 Region of birth as proxy for ethnicity/culture

10 (33.3)

 Language as proxy for ethnicity/culture.

1 (3.3)

 Race as a proxy for ethnicity/culture

0 (0)

Rationale for inclusion of cultural factors in the studies

7 (23.3)

Table 4

Results of the analysis using the GAP-REACH checklist

Author(s)

Definition of cultural variablesa

Rationale for inclusion of cultural variablesa

Sample described in terms of cultural characteristicsa

Anclair and Hiltunen (2014)

Yes

No

Geographic region of birth

Butwicka et al. (2015)

Yes

No

Parental country of birth categorized as (a) Sweden, (b) Nordic countries, (c) outside Nordic countries

Cederlund et al. (2014)

Yes

Yes

Parental country of birth categorized as (a) both parents born in Sweden, (b) one parent born in another European country, (c) one parent born in another part of the world, (d) both parents born outside Sweden (either European country or other parts of the world).

De Bildt et al. (2015)

No

Yes

No (= not described)

Domellöf et al. (2014)

No

Yes

No (= not described)

Fernell et al. (2015)

Yes

Yes

The 1st cohort group: “the Stockholm Somali group”; the 2nd cohort group based in Gothenburg categorized as (a) Swedish, (b) miscellaneous (non-Scandinavian, South America, East Africa), (c) African/Middle East.

Gardner et al. (2015)

Yes

No

Maternal country of birth categorized as (a) mother born in Sweden, (b) mother born outside Sweden

Idring et al. (2015)

Yes

No

Maternal country of birth categorized as (a) mother born in Sweden, (b) mother born outside Sweden with low/high Human Development Index (HDI)

Idring et al. (2014)

Yes

Yes

Maternal country of birth categorized as (a) mother born in Sweden, (b) mother born in Europe outside Sweden, (c) mother born outside Europe

Lee et al. (2015)

Yes

No

Maternal country of birth categorized as (a) mother born in Sweden, (b) mother born in Europe outside Sweden, (c) mother born outside Europe

Löfkvist et al. (2014)

Yes

Yes

Swedish language proficiency

Lundström et al. (2015b)

Yes

No

No (= not described)

McEvilly et al. (2015)

Yes

No

No (= not described)

Selten et al. (2015)

Yes

No

Personal or parental history of migration

Törn et al. (2015)

Yes

No

No (= not described)

Zander et al. (2015)

Yes

Yes

Maternal country of origin

aCoding based on the results reported by the first author/coder

Region of Birth as Proxy for Ethnicity/Culture

Ten of all reviewed studies (33.3%) used geographic region of birth as a proxy for ethnicity to describe samples’ cultural characteristics (Tables 3 and 4). The majority of these studies employed population-based, longitudinal register designs. Parental country of birth was the most commonly used proxy for ethnicity although the authors used different operational definitions. For instance, Butwicka et al. (2015) categorized regions into groups: Sweden, other Nordic countries, and outside Nordic countries; similarly, Törn et al. (2015) used the term Scandinavian countries. Cederlund et al. (2014) operationalized parental country of birth in three different ways: firstly, the authors categorized regions into groups: both parents born in Sweden; one parent born in another European country; one parent born in another part of the world; both parents born outside Sweden (either European country, or other parts of the world). Secondly, the dichotomies were used: children born to immigrant parents vs. children born to Swedish parents, or children born to native and non-native Swedish parents. Another term used to define ethnicity in children in this study was non-Caucasian. Based on the hypothesis tested, Fernell et al. (2015) categorized data on ethnicity into three origin groups: Swedish, Miscellaneous (including non-Scandinavian, South America, East Asia), and African/Middle East. Other studies used a more specific term the maternal country of origin with various categorizations (Gardner et al. 2015; Idring et al. 2014; Lee et al. 2015; Zander et al. 2015; see Table 4).

Language as a Proxy for Ethnicity/Culture

A study by Löfkvist et al. (2014) was the only one (3.3%) that used language as a proxy for ethnicity to describe their sample’s cultural characteristics (Tables 3 and 4). Knowledge of Swedish was a requirement for participation in the study for three different clinical groups: children with ASD, with cochlear implants (CI), and with language impairment. For instance, the authors noted that the children diagnosed with ASD were Swedish-speaking children and attended either regular or special classes in mainstream schools, “where spoken Swedish was the main language of education,” and that “all of the children used spoken Swedish as their main communication mode” (p. 255). Good command of Swedish was required for children’s participation in the control group as well, as illustrated by the authors’ statement made: “They were recruited, with help from the school nurse, from two different schools in a small town with a relatively low percentage of immigrants and are therefore considered to be representative of Swedish-speaking children in general” (p. 256).

Race as a Proxy for Ethnicity/Culture

The application of the GAP-REACH checklist revealed that a race variable could not be coded or scored as it was not found in any of the articles under review (Table 3).

Rationale for Inclusion of Cultural Factors in the Studies

The analysis of Introduction/Background sections of the reviewed articles showed that seven studies (23.3%) described the rationale of including cultural factors into their research design (Tables 3 and 4). For example, Cederlund et al. (2014) referred to several comparative studies examining association between large head size and ASD conducted on clinical samples of children with various ethnic backgrounds described as, e.g., Caucasian, Chinese, South Asian, Indian, African, North American, South America, and European. The investigators chose to examine the same association in population representative sample within the Swedish context reflecting Sweden’s current diversity in terms of ethnic and cultural background of the population. The authors compared data obtained for children whose parents were native Swedish with children whose parents were born outside Sweden. De Bildt et al. (2015) conducted a multi-national study to investigate algorithms of the Autism Diagnostic Interview-Revised (ADI-R) for toddlers and young preschool children in nine European countries, including Sweden. The rationale for conducting the study was to replicate the US study and to contribute to evidence base on the earliest signs of autism and generating European practice guidelines on early identification and intervention. Domellöf et al. (2014) explored health-related quality of life in children and adolescents with various types of disabilities including ASD in a cultural setting of northern Sweden. The rationale was to investigate how quality of life of these children would be influenced by cultural differences and socioeconomic factors found in one specific geographic area of Sweden. The study by Fernell et al. (2015) included children diagnosed with ASD born to immigrants with Somali origin to test their hypothesis of low level of vitamin D associated with the risk of ASD. In this study, ethnicity was considered as one of the contributing factors alongside birth order and season of birth. In providing the rationale for the study, the researchers referred to several epidemiological findings that had showed associations between deficiency in vitamin D in dark-skinned immigrants and risk for ASD in children. Zander et al. (2015) examined diagnostic validity of the combined ADR-I and ADOS use for the clinical Swedish sample. The rationale for testing the validity of these tools in the Swedish context was, as the authors argued, that the majority of similar studies were done primarily in the USA and in few other cultural contexts, such as Australia and the Netherlands.

Inter-Rater Reliability

Table 5 presents scores and IRR for each item and the total GAP-REACH score resulting from randomly selected articles (n = 7) before discussion of disagreements between two independent coders. Kappa coefficient for the items ranged from 0.00 to 1.00, and percentage of agreement ranged from 42 to 100%. Kappa estimates for 4 of the 16 items were 1.0, while percentage of agreement for 11 of the 16 items were 85% and higher. The results show disagreements in items 1, 2, 3, and 7 with both low levels of the percentage of agreement and kappa coefficient, with latter demonstrating slight strength of agreement between two coders as suggested by Landis and Koch (1977). Items 10, 15, and 16 got 100 percentage of agreement, and there was perfect agreement on kappa estimate (κ = 1.0). Item 5 showed higher level on the percentage of agreement, whereas kappa coefficient showed slight agreement (κ = 0.00). Item 13 showed 71% percentage of agreement with a κ coefficient of 0.36.
Table 5

Scores from testing of the GAP-REACH checklist and IRR for each item in a random sample of ASD research articles (n = 7) by two independent coders before discussion of disagreements

Article (N)

Coder (n = 2)

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

Item 7

Item 8

Item 9

Item 10

Item 11

Item 12

Item 13

Item 14

Item 15

Item 16

Total score (%)

1

C1

No

No

Yes

No

No

No

Yes

N/a

N/a

N/a

N/a

N/a

Yes

Yes

No

Yes

62.5

C2

No

No

No

No

No

No

No

N/a

N/a

N/a

N/a

N/a

Yes

Yes

No

Yes

50

2

C1

Yes

Yes

No

No

No

No

No

No

No

No

No

No

N/a

No

No

No

18.75

C2

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

No

0

3

C1

Yes

Yes

Yes

No

No

Yes

Yes

N/a

N/a

N/a

N/a

N/a

No

No

No

No

62.5

C2

No

No

No

No

No

No

Yes

N/a

N/a

N/a

N/a

N/a

No

No

No

No

37.5

4

C1

No

Yes

Yes

Yes

No

Yes

Yes

No

No

No

Yes

No

Yes

Yes

Yes

No

56.25

C2

No

Yes

No

Yes

No

Yes

No

No

No

No

No

No

Yes

Yes

Yes

No

37.5

5

C1

Yes

Yes

No

No

No

No

No

N/a

N/a

N/a

N/a

N/a

N/a

No

No

No

50

C2

No

No

No

No

No

No

No

N/a

N/a

N/a

N/a

N/a

N/a

No

No

No

37.5

6

C1

No

No

No

No

No

No

No

No

No

N/a

N/a

N/a

N/a

No

No

No

25

C2

No

No

No

No

No

No

No

N/a

N/a

N/a

N/a

N/a

N/a

No

No

No

37.5

7

C1

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

No

Yes

Yes

N/a

No

No

No

62.5

C2

No

No

No

No

No

Yes

No

N/a

No

No

N/a

No

No

No

No

No

18.75

% of agreement

57%

42%

42%

85%

85%

85%

57%

85%

85%

100%

85%

85%

71%

100%

100%

100%

 

κ

0.00

0.12

0.00

0.59

0.00

0.69

0.22

0.69

0.72

1.00

0.58

0.69

0.36

1.00

1.00

1.00

 

95% CI

0.000 to 0.000

− 0.147 to 0.397

0.000 to 0.000

− 0.009 to 1.268

0.000 to 0.000

0.691 to 0.701

− 0.195 to 0.639

0.171 to 1.221

0.232 to 1.208

1.00 to 1.00

− 0.092 to 1.268

0.171 to 1.221

− .21 to 0.938

1.00 to 1.00

1.00 to 1.00

1.00 to 1.00

 
As Table 5 shows, the first and the second coders had several disagreements concerning coding, especially with regard to items 1, 2, and 3. Other coding disagreements were related to items 8, 9, and 11–13. During the consensus coding, the second coder agreed with the first coder on all items for the articles numbered 1–4 and 7; and for the articles numbered 5 and 6, the first coder agreed with the second coder on all items (see Table 5 for specific items). Table 6 shows the number of items scored as “yes” and “N/A” as well as the total GAP-REACH score for each reviewed article based on scoring by the first coder before and after discussion of disagreements.
Table 6

The total GAP-REACH score in percentage for each reviewed article

 

Before discussion of disagreements

After discussion of disagreements

Author(s)

Items scored “yes” (n)

Items scored “n/a” (n)

GAP-REACH total score (%)

Items scored “yes” (n)

Items scored “n/a” (n)

GAP-REACH total score (%)

Ahlström and Wentz (2014)

0

1

6.25

0

1

6.25

Anclair and Hiltunen (2014)

4

1

31.25

4

1

31.25

Atladottir et al. (2015)a

2

6

50

0

6

37.5

Bölte et al. (2014)

1

1

12.5

2

1

18.75

Butwicka et al. (2015)

5

6

68.75

5

6

68.75

Callenmark et al. (2014)

5

0

31.25

5

0

31.25

Cederlund et al. (2014)a

9

0

56.25

9

0

56.25

De Bildt et al. (2015)

5

3

50

5

3

50

Domellöf et al. (2014)

4

1

31.25

5

1

37.5

Falck-Ytter et al. (2015)a

0

4

25

0

6

37.5

Fernell et al. (2015)

11

5

100

11

5

100

Gardner et al. (2015)

8

5

81.25

10

5

93.75

Idring et al. (2015)

7

5

75

7

5

75

Idring et al. (2014)a

5

5

62.5

5

5

62.5

Janeslätt et al. (2013)

1

0

6.25

1

0

6.25

Kerekes et al. (2015)

1

1

12.6

1

1

12.5

Lee et al. (2015)

7

5

75

8

5

81.25

Löfkvist et al. (2014)a

9

1

62.5

9

1

62.5

Lundström et al. (2015a)

3

5

50

4

5

56.25

Lundström et al. (2015b)

4

1

31.25

4

1

31.25

Lundström et al. (2014)

1

4

31.25

2

4

37.5

McEvilly et al. (2015)a

5

5

62.5

5

5

62.5

Miniscalco et al. (2014)

3

0

18.75

3

0

18.75

Pahnke et al. (2014)

2

1

18.75

2

1

18.75

Palmqvist et al. (2013)

1

3

25

1

3

25

Ronald et al. (2014)a

2

1

18.75

2

1

18.75

Sandin et al. (2014)

2

6

50

2

6

50

Selten et al. (2015)

7

5

75

7

5

75

Törn et al. (2015)

3

0

12.5

3

0

12.5

Zander et al. (2015)

10

1

68.75

10

1

68.75

Scores are presented based on scoring performed by the first coder before and after discussion of disagreements

aArticles randomly selected for IRR

The re-coding of the full sample of the articles by the first coder revealed that items 1 or 2 scored initially as negative were coded as positive for three studies (Bölte et al. 2014; Gardner et al. 2015; Lundström et al. 2015a); item 4—from negative to positive for two studies (Lee et al. 2015; Lundström et al. 2014), and item 11—from negative to positive for one study (Domellöf et al. 2014). These changes in coding resulted in higher total GAP-REACH scores (Table 6).

Discussion

The purpose of this literature review was to describe methodological practices of reporting culture-related factors in ASD research publications published by Sweden-based researchers. Thirty peer-reviewed articles published in English between May 2013 and November 2015 were analyzed. To assess the scope of reporting of cultural factors in the selected articles, the checklist GAP-REACH was used.

Scope of Reporting Cultural Factors

The results of the study demonstrate that the Sweden-based researchers involved in ASD research, generally tend to consider factors such as ethnicity to describe their study samples. Informed by Swedish national and regional health registers, such as the Swedish Medical Birth Register, the Multigenerational register, the National Patient register, the Swedish Twin register, and the Stockholm Youth Cohort (SYC), the researchers chose using various proxies for ethnicity, the most common of which is parental country or area of origin/birth. In this study the majority of the articles that defined and conceptualized ethnicity were population-based register studies. As Hollander (2013) states, “Swedish registers do not record race, culture, or ethnicity for ethical reasons. However, they do register the immigrants’ country of origin… [that]… can sometimes be used as a proxy for ethnicity” (p.19). Indeed, the Swedish Personal Data Act (PUL 1998) did not allow registering sensitive personal information such as race, ethnicity or other culture-related data such as religion. Therefore, the use of geographic area or country of origin as proxy for ethnicity by the researchers shown in this study justifies this approach. In the current review, the researchers used various ways to operationalize parental country of origin. These findings are consistent with Hollander’s (2013) description of different ways of operationalizing the immigrants’ country of origin in her research on social inequalities of mental health and mortality among refugees and other immigrants to Sweden: (1) the simplest dichotomization of native/immigrant, (2) the immigrant’s income level in their country of origin (native, OECD-country, non-OECD-country), and (3) different clusters of country or area of origin.

Yet one study (Löfkvist et al. 2014) used language as a proxy for ethnicity to describe the demographic characteristics of the study participants. This finding could be partly explained by the design of the study—not based on national population registers—where a good command of Swedish was a requirement to test lexical and semantical ability in young children. Hollander (2013) maintains that it is difficult to define ethnicity as a variable in an epidemiological empirical study. Therefore, it could be argued that different definitions, conceptualizations and operationalization of the ethnicity variable as demonstrated in the present review could be indicative of the contemporary view of culture as multidimensional and dynamic, where ethnicity is just one of the several dimensions (Kagawa Singer et al. 2014, 2016). In social research, for instance, different conceptualizations of the term “ethnicity” can accentuate different characteristics of social or group identity: some focus on cultural commonality, such as collective beliefs and behaviors, other focus on geographic regions and shared biological traits, and some other—on socio-political aspects viewing ethnic groups hierarchically with the emphasis on discrimination and disadvantage (Salway et al. 2009). Within the Swedish context and, possibly, within broader Nordic context, it might be feasible for ASD researchers to use a number of additional descriptors to report cultural characteristics of their participants, as the proxies for ethnicity used in the reviewed studies provide only limited information and, therefore, may not always accurately reflect the complexity of cultural identities of individuals. These descriptors may include participants’ tribal affiliation, geographical backgrounds within their home countries; language dialects; religious, spiritual beliefs and/or moral traditions (Gellerman and Lu 2016; Groen et al. 2016); and in some instances, one may even consider participants’ worldviews (F. Lu, personal communication, December 2, 2016). The Swedish legislation such as the Act concerning the Ethical Review of Research Involving Humans (2003:460), as well as the European Union’s General Data Protection Regulation (GDPR 2016/679) that substituted the Personal Data Act (1998:204) in May 2018, would allow collecting and processing such personal information for research purposes.

The results of the present review show that a study by Domellöf et al. (2014) stands out as an interesting case, when cultural variables were mentioned (e.g., “cultural setting,” “cultural differences”) generating positive scores for items 1 and 2; however, these variables were not defined. Neither were cultural characteristics of the study participants described: a short demographic survey collected data on age, gender, diagnosis, and parental educational background. Yet, a closer examination of the introduction, methods, and references sections of the study report, points to several important aspects that could explain these findings. Firstly, the study aimed to explore health-related quality of life (HRQoL) in children and adolescents in relation to their disability using the Swedish child-friendly version of the instrument—the EuroQoL Five Dimensions Health Questionnaire (EQ-5D-Y)—introduced and tested by the EuroQol Group (Burström et al. 2010; Ravens-Sieberer et al. 2010). In this regard, the study by Domellöf et al. (2014) can be viewed as an example of the cross-cultural application of the EQ-5D-Y in the cultural context of the Swedish health-care system, which explains why the REC terms were mentioned in the study. Secondly, as surveys were administered in Swedish, the high non-response rate (70.2%) to surveys could be, partially, due to possible unwillingness of non-Swedish-speaking respondents to participate in the study (E. Domellöf, personal communication, January 29, 2018). As Dömellöf pointed out, initially all children and adolescents enrolled into the child habilitation center (disability services) in that county were asked to participate in the study regardless of their ethnic or cultural backgrounds; however, only a smaller proportion of participants actually responded to the surveys.

One of the important findings of the present study was that the “race” variable was not found in the reviewed articles, and therefore, it was not coded or scored. There are two explanations to this finding. Firstly, as it has been mentioned earlier, the Personal Data Act (1998:204) prohibited collecting information on race. Secondly, the concept of ‘race’ is not used in Sweden, unlike some other countries, e.g., the USA, as it is seen as socially constructed (Bäärnhielm 2014). Interestingly, Yudell et al. (2016) describe a longstanding debate in science about the role of concept of race that is frequently used as a proxy for studying human genetic diversity. In discussing this debate, the authors argue that in contemporary science, population genetic methods do not support existing classifications of races as these commonly defined racial groups are genetically mixed without clear-cut boundaries. Yudell et al. (2016), therefore, recommend that

Scientific journals and professional societies should encourage use of terms like ‘ancestry’ or ‘population’ to describe human groupings in genetic studies and should require authors clearly define how they are using such variables. It is preferable to refer to geographic ancestry, culture, socioeconomic status, and language, among other variables, depending on the questions being addressed, to untangle the complicated relationship between humans, their evolutionary history, and their health. (p. 565).

In the light of this perspective, the inclusion of the concept race in the DSM-5 and its cultural formulation should probably be further discussed and problematized.

The present study’s findings demonstrate that the study by Zander et al. (2015) was the only one that described how data were collected during the diagnostic assessment procedure for parents of children with ASD whose Swedish was not native. The authors explicitly pointed out that certified interpreters assisted administration of these tools (item 10 in the GAP-REACH checklist: discussion of relevance of language characteristics of interviewers and participants). Furthermore, the study by Idring et al. (2015) stands alone in addressing disparities in access to services and diagnosis for children with ASD (with or without intellectual disability) of immigrant background. Disparities in access to diagnosis and interventions based on ethnicity continue to be a significant challenge (Interagency Autism Coordinating Committee (IACC) 2014). For this reason, the findings reported by Idring et al. (2015) and their discussion in the article can be viewed as an important step forward in raising researchers’, practitioners’, and general public’s awareness on these issues.

The results of the present study also showed that the majority (60%) of research designs covered topics related to etiology and risk factors of ASD (coded as basic research), while intervention studies and studies investigating quality of life and family well-being topics covered only 10% in each research area. On the one hand, these findings reflect the current global trend in autism research supporting data reported by several authors and research groups (Graff et al. 2014; Interagency Autism Coordinating Committee (IACC) 2014; Milner and Cho 2014). For instance, the IACC (2014) described that 55% of the total number of 2010 ASD primary research publications were dedicated to biomedical research, 19%—to treatments and interventions, and only 10%—to services and life-span issues. As Milner and Cho (2014) suggest, this emphasis on identifying causal factors of ASD will likely prevail in the future, which is not surprising as “ASD has only recently become a common condition, and there are currently no pharmacological interventions for the core symptoms of ASD” (p. 49). On the other hand, given the aims of the present study, these results could be explained by methodological choices made by the researchers. Biomedical studies using observational research methods that constitute the largest proportion of the reviewed studies tend to report cultural variables of their participants based on existing operational definitions suggested by the population registers. The use of population databases is also a cost effective way to study research phenomena as they can be easily accessed, are less time consuming and relatively inexpensive (Mann 2003). This is in contrast to employing clinical and/or intervention study designs where involvement of culturally and linguistically diverse children with ASD and their family members might require additional resources to be spent, for instance, for translation and cross-cultural validation of assessment and intervention instruments; cultural adaptation of interventions, use of simultaneous interpreters, or participation of culturally competent practitioners and researchers. Therefore, it is not surprising that in this study the results demonstrate that only a handful of clinical and/or interventions studies reported involving participants with other cultural backgrounds than Swedish, and using interpreters to communicate with them. This finding is a reason for concern as underreporting of ethnicity and cultural factors in research and other related methodological issues may affect not only scientific rigor of research concerning replicability and generalizability of study results (Kistner and Robbins 1986; Pierce et al. 2014), but also may affect effectiveness of interventions for culturally diverse individuals with ASD and their family members (National Autism Center 2015). This may in turn have direct implications for practice as a larger number of children from immigrant families in Sweden might enter special educational and healthcare services for ASD diagnostics and treatment.

Assessing the Utility of the GAP-REACH Checklist within the Swedish Context

The results of the study show that items that allow N/A scoring could be applied to one third of the reviewed articles, which resulted in generating a higher total score for those articles, e.g., 50 and higher (Table 6). Consistent with the results of the checklist development reported by Lewis-Fernández et al. (2013), inapplicability of certain items was possible due to methodological choices made by the researchers. For instance, it was not possible to code language proficiency of participants, or language match between participants and instruments in the studies that employed national population-based health registers. This was also reflected in IRR estimates. As shown in Table 5, item-based analysis of the checklist between two independent coders revealed moderate to excellent IRR reliability (based on κ coefficient) for items 8–12 when both coders independently agreed on choosing the option N/A.

The results show low kappa value for items 5 and 7, which is consistent with findings reported by Lewis-Fernández et al. (2013). However, in the present study percent of agreement for item 5 was 85% with a high level of negative agreements from both coders. This result can be cautiously interpreted as a paradox of κ described well in the literature when agreement of positive ratings is compared to agreement of negative ratings (Feistein and Cicchetti 1990; Viera and Garrett 2005). For item 7, describing methods for assessing cultural characteristics of study participants, the results show low IRR estimates and high level of disagreements between coders. To avoid future disagreements in coding, it is suggested that coders closely follow the scoring instructions where methods for assessing cultural characteristics of study participants are well described.

The application of the GAP-REACH checklist revealed some limitations of its use in the Swedish context. For instance, none of the reviewed articles used the concept of race to describe the study participants. As the concept of “race” is not used in Sweden, the GAP-REACH checklist may be used in its modified form by simply excluding this concept during the coding process. Another concern regarded the coding of terms found in some of the reviewed articles, e.g., Scandinavian data (Atladottir et al. 2015, p.176) or Scandinavian countries (Törn et al. 2015, p. 1049). The question arises here: Can these terms be used as definitions of culture to describe certain commonalities in registry systems and service provision found in Nordic countries, which may be distinct from other geographic places in the world? For example, Atladottir et al. (2015) reported that data on diagnoses from outpatient visits could be obtained only from the public sectors in Denmark, Finland, and Sweden, while in Western Australia, these data could be obtained from both public and private sectors. Interestingly, in the present review, the difficulties in defining cultural factors were reflected in disagreements between two coders on items 1, 2, and 3. These results are inconsistent with the findings reported by Lewis-Fernández et al. (2013) for items 1 and 2 that showed the highest IRR based on kappa statistics. Lewis-Fernández et al. (2013) argue that these two items require less judgment by the coders as this judgment depends on the presence or absence of specific terms either in the title, in the abstract, or in the article text. This was not the case in the present study, when the first three items required substantial judgment in coding. The disagreements were due to different understandings of how ethnicity was reported in the studies. For instance, concerning items 1 and 2, the second coder limited the interpretation of the REC terms only to minority cultures within the majority Swedish culture, thus, coding terms “Swedish” or “Swedish-speaking” found in titles or abstracts as negative; whereas the first coder interpreted these terms as REC factors and coded as positive. The disagreements between two coders regarding item 3 may reflect the different understandings of the term ethnicity. As Sweden-based researchers often report ethnicity by using various proxies, the second coder did not consider these proxies as clear definitions of REC factors in the reviewed studies and, therefore, coded this item as negative. The subsequent discussion of disagreements by both coders resulted in agreement about which terms could be defined and coded as cultural factors in the analyzed articles. For future studies, it is suggested that difficulties in coding of specific terms and definitions of cultural factors be resolved by undertaking several actions prior to initiating coding procedures: (1) all research team members should be involved into discussion and agreed on definitions and operationalization of cultural factors; the investigators could also take advice from experts in the field; (2) based on the agreed operational definitions of cultural factors, the scoring instructions for using the GAP-REACH checklist should be revised with adjustments made to the Swedish context. These results indicate that high quality training for coders before conducting coding is essential and should be based on the revised checklist and revised scoring instructions. Discussing the checklist’s items among coders prior to initiating the coding process can assist in minimizing coding disagreements. In the present study, possible reasons for disagreements between the coders and low IRR estimates for some items were due to somewhat insufficient training of the coders, lack of detailed discussions of specific items to harmonize, for instance, definitions of REC terms, as well as joint pilot scoring on a sample article before the actual coding procedure took place.

Discussion of disagreements between two coders and the subsequent re-coding of the full sample of articles by the first coder revealed some important findings to be mentioned here. It became evident that recording information on translation of study instruments could be occasionally missed during the coding process. For instance, during the initial coding round, item 11 was scored as negative by the first coder for the study by Domellöf et al. (2014). During the re-coding round, the scoring was changed into positive, which also generated a higher total GAP-REACH score (Table 6). Similarly, the second coder did not record information on translation of instruments for two studies included into IRR and, therefore, initially coded this item as negative and N/A, correspondingly. Future coders may want to pay closer attention to methods section of research reports to record any relevant information on translations of study instruments. Likewise, coders are advised to thoroughly follow scoring guidelines for item 4 not to miss recording REC-related terms used to describe research questions, study designs or background literature. In the present study, the scoring for item 4 was changed from negative to positive for two studies (Lee et al. 2015; Lundström et al. 2014) during the re-coding round.

Generally, the analysis of the reviewed articles demonstrated potential advantages of using the GAP-REACH checklist to assess the scope of reporting of culture-related factors in ASD publications within the Swedish context. The checklist can serve as an important reminder for reporting these factors in manuscript before publication. If research methodology does not permit including cultural factors when reporting research findings, the researchers still could use some of the checklist’s items, e.g., discussion of study limitations in terms of ethnicity and culture. The GAP-REACH can also serve as an aid tool for editors and reviewers of local scientific journals in evaluating the quality of reporting of cultural factors in submitted manuscripts.

Implications for Practice

The results of the present study indicate that scientific publications on autism in the Swedish research context tend to describe cultural characteristics of their samples, especially in population-based register studies. However, only a few publications discussed clinical implications of the findings. Under-representation of culturally diverse samples in research may not only limit generalizability of findings but also may affect intervention effectiveness for individuals with ASD in the long run. Therefore, it is important that practitioners critically assess research findings in the light of study limitations, paying closer attention to how study participants are described, especially in intervention studies. The results of this review suggest that certain items of the GAP-REACH checklist can assist practitioners to make such assessments.

Limitations

This study has several limitations. It was conducted in the Swedish context, which can limit its generalizability. However, as Sweden has become a multicultural society due to immigrations of people from various ethnic and religious backgrounds, the results of this study could be generalized to other high-income western European countries with similar population heterogeneity. Another limitation is that PsycINFO—the commonly used database for literature search related to the psychology or psychiatry subject areas—was not included into the present study. However, we used Scopus, one of the largest multidisciplinary databases of peer-reviewed academic literature which is broad in scope (Ware and Mabe 2015), and has a high degree of reported overlap of journal titles between several scientific databases including PsycINFO (Gavel and Iselid 2008). Other limitations are inclusion criteria for the review: articles were published in English after May 2013, the publication month of the DSM-5 that includes the CFI. However, the Swedish translation of the DSM-5 appeared in October 2014. We are aware that the publication process may take longer time for researchers, and it is plausible that not all researchers were informed by the CFI included in the DSM-5 at the time of data collection and writing of manuscripts. Indeed, as Table 3 shows, the majority of publications reported involving participants who had been diagnosed with ASD using the diagnostic criteria of either the ICD or the DSM-IV, and only three included studies used the DSM-5 for data collection. Nevertheless, the inclusion criteria applied in this review allowed us to rapidly examine the extent of the research problem as well to test the potential utility of the GAP-REACH checklist in a different cultural context such as Sweden.

A limitation of this study is that IRR for the checklist’s items were calculated for only 25% of randomly selected articles which could affect confidence intervals estimates for the obtained kappa, and the precision of IRR for each item. Future testing of the checklist should include larger samples of articles. Another limitation is a lack of common agreement on operational definitions of cultural factors prior to the coding procedure that affected IRR estimates and number of agreements on some items between two coders. Yet, these preliminary results provide useful information for future studies and can be viewed as pre-testing.

Conclusion

For research designs not employing population-based registers and databases, e.g., intervention studies, it could be advantageous to report ethnicity or other culture-related demographic characteristics of research participants to address external validity of study results and to inform practitioners. Swedish legislation – the Personal Data Act (1998:204), and the EU Regulation on General Data Protection (GDPR 2016/679) that substituted the Personal Data Act as of the 25th May 2018, permit collecting and processing information on ethnicity and culture for research purposes if the researchers clearly describe in their research proposals to regional ethics review boards how they would protect identities of their research participants, what benefits potential studies would bring, and how possible risks could be minimized (the Act concerning the Ethical Review of Research Involving Humans 2003:460; Swedish Data Inspection Board 2008). Reporting ethnicity or other cultural characteristics of research participants could not only contribute to methodological rigor of research but also could reduce possible disparities in access to high-quality interventions and services for ethnically and culturally diverse children with ASD and their families in Sweden. In sum, the GAP-REACH checklist would be a useful tool for comprehensive reporting of culture-related factors if modified by the Swedish autism researchers.

Notes

Acknowledgments

The authors wish to extend their gratitude to Prof. Roberto Lewis-Fernández from Columbia University, USA, for his support and providing permission for us to use the GAP-REACH checklist in this study, and to extend special thanks to Dr. Sofie Bäärnhielm for providing comments to the manuscript regarding the use of Cultural Formulation Interview in Sweden. The authors would like to express sincere gratitude to the anonymous reviewer for the insightful suggestions that helped improve our manuscript.

Authors’ Contributions

RZE conceived and designed the study, performed data analysis and interpreted the results, drafted the manuscript, and addressed the reviewer comments. NK participated in data analysis and contributed to manuscript development. TH, MAW, and LR-P provided recommendations to enhance stringency, feedback, assisted in interpreting the results, reviewed, and edited the final version of the manuscript. All authors read and approved the final manuscript.

Funding

This study was funded by a PhD student scholarship awarded to the first author by the Department of Special Education, Stockholm University, Sweden.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Informed Consent

Not applicable.

Supplementary material

40489_2018_147_MOESM1_ESM.doc (63 kb)
ESM 1 (DOC 63 kb)

References

  1. Act concerning the Ethical Review of Research Involving Humans. (2003:460). Retrieved from: https://www.epn.se/media/2348/the_ethical_review_act.pdf.
  2. Ahlström, B., & Wentz, E. (2014). Difficulties in everyday life: young persons with attention-deficit/hyperactivity disorder and autism spectrum disorders perspectives. A chat-log analysis. International Journal of Qualitative Studies on Health and Well-being, 9.  https://doi.org/10.3402/qhw.v9.23376.CrossRefGoogle Scholar
  3. American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.Google Scholar
  4. American Psychiatric Association. (2013). Cultural formulation. In: Diagnostic and statistical manual of mental disorders (5th ed.pp. 749–759). Washington, DC: American Psychiatric Association.CrossRefGoogle Scholar
  5. Anclair, M., & Hiltunen, A. (2014). Cognitive behavioral therapy for stress-related problems: single-case studies of parents of children with disabilities. Clinical Case Studies, 13(6), 472–486.CrossRefGoogle Scholar
  6. Atladottir, H., Gyllenberg, D., Langridge, A., Sandin, S., Hansen, S. N., Leonard, H., et al. (2015). The increasing prevalence of reported diagnoses of childhood psychiatric disorders: a descriptive multinational comparison. European Child & Adolescent Psychiatry, 24(2), 173–183.CrossRefGoogle Scholar
  7. Bäärnhielm, S. (2014). Transkulturell psykiatri. [Transcultural psychiatry]. Stockholm: Natur & Kultur.Google Scholar
  8. Bäärnhielm, S., & Scarpinati Rosso, M. (2009). The cultural formulation: a model to combine nosology and patient’s life context in psychiatric diagnostic practice. Transcultural Psychiatry, 46(3), 406–428.CrossRefGoogle Scholar
  9. Bäärnhielm, S., Ndetei, D., Rohlof, H., Musa Misani, K., et al. (2016). Application of the Cultural Formulation Interview in international settings. In R. Lewis-Fernández, N. K. Aggarwal, L. Hinton, D. E. Hinton, & L. J. Kirmayer (Eds.), DSM-5. Handbook on the cultural formulation interview (pp. 237–251). Arlington: American Psychiatric Association.Google Scholar
  10. Barnevik-Olsson, M. D., Gillberg, C., & Fernell, E. (2008). Prevalence of autism in children born to Somali parents living in Sweden: a brief report. Developmental Medicine and Child Neurology, 50, 598–601.  https://doi.org/10.1111/j.1469-8749.2008.03036.x.CrossRefPubMedGoogle Scholar
  11. Bishop-Fitzpatrick, L., & Kind, A. J. H. (2017). A scoping review of health disparities in autism spectrum disorder. Journal of Autism and Developmental Disorders, 47, 3380–3391.  https://doi.org/10.1007/s10803-017-3251-9.CrossRefPubMedPubMedCentralGoogle Scholar
  12. Bölte, S., Willfors, C., Berggren, S., Norberg, J., Poltrago, L., Mevel, K., et al. (2014). The roots of autism and ADHD twin study in Sweden (RATSS). Twin Research and Human Genetics, 17(3), 164–176.CrossRefGoogle Scholar
  13. Broder-Fingert, S., Silva, C., Silverstein, M., & Feinberg, E. (2017). Participant characteristics in autism intervention studies. Autism, 1–2.  https://doi.org/10.1177/1362361317722306.
  14. Burström, K., Svartengren, M., & Egmar, A.-C. (2010). Testing a Swedish child-friendly pilot version of the EQ-5D instrument – initial results. European Journal of Public Health, 21(2), 178–183.CrossRefGoogle Scholar
  15. Butwicka, A., Lichtenstein, P., Landén, M., Nordenvall, A., Nordenström, A., Nordenskjöld, A., & Frisén, L. (2015). Hypospadias and increased risk for neurodevelopmental disorders. Journal of Child Psychology and Psychiatry, 52(6), 155–161.CrossRefGoogle Scholar
  16. Callenmark, B., Kjellin, L., Rönnqvist, L., & Bölte, S. (2014). Explicit versus implicit social cognition testing in autism spectrum disorder. Autism, 18(6), 684–693.CrossRefGoogle Scholar
  17. Cederlund, M., Miniscalco, C., & Gillberg, C. (2014). Pre-school children with autism spectrum disorder are rarely macrocephalic: a population study. Research in Developmental Disabilities, 35(5), 992–998.CrossRefGoogle Scholar
  18. Chung, K.-M., Jang, J., & Adams, H. L. (2014). Multicultural and minority issues. In J. Tarbox & D. P. Dixon (Eds.), Handbook of early intervention for autism spectrum disorders (pp. 737–650). Autism and Child Psychopathology Series. New York: Springer.  https://doi.org/10.1007/978-1-4939-0401-3_29.CrossRefGoogle Scholar
  19. De Bildt, A., Sytema, S., Zander, E., Bölte, S., Sturm, H., Yirmiya, N., et al. (2015). Autism Diagnostic Interview-Revised (ADI-R) algorithms for toddlers and young preschoolers: application in a non-US sample of 1,104 children. Journal of Autism and Developmental Disorders, 45, 2076–2091.CrossRefGoogle Scholar
  20. Domellöf, E., Hedlund, L., & Ödman, P. (2014). Health-related quality of life of children and adolescents with functional disabilities in a northern Swedish county. Quality of Life Research, 23, 1877–1882.CrossRefGoogle Scholar
  21. Elsabbagh, M., Divan, G., Koh, Y.-J., Kim, Y. S., Kauchali, S., Marcín, C., et al. (2012). Global prevalence of autism and other pervasive developmental disorders. Autism Research, 5, 160–172.CrossRefGoogle Scholar
  22. Falck-Ytter, T., Thorup, E., & Bölte, S. (2015). Brief report: lack of processing bias for the objects other people attend to in 3-year olds with autism. Journal of Autism and Developmental Disorders, 45(6), 1897–1904.  https://doi.org/10.1007/s10803-014-2278-4.CrossRefPubMedGoogle Scholar
  23. Feistein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6), 543–549.CrossRefGoogle Scholar
  24. Fernell, E., Bejerot, S., Westerlund, J., Miniscalco, C., Simila, H., Eyles, D., et al. (2015). Autism spectrum disorder and low vitamin D at birth: a sibling control study. Molecular Autism, 6(3), 1–9.Google Scholar
  25. Gardner, R., Lee, B., Magnusson, C., Rai, D., Frisell, T., Karlsson, H., et al. (2015). Maternal body mass index during early pregnancy, gestational weight gain, and risk of autism spectrum disorders: results from a Swedish total population and discordant sibling study. International Journal of Epidemiology, 870–883.  https://doi.org/10.1093/ij/dyv081.
  26. Gavel, Y., & Iselid, L. (2008). Web of Science and Scopus: a journal title overlap study. Online Information Review, 32(1), 8–21.  https://doi.org/10.1108/14684520810865958.CrossRefGoogle Scholar
  27. Geary, E. M. (2015). American Indian child and family well-being: a systematic review of research from twenty-two national datasets. (Doctoral Dissertation, College of Social Work, University of Utah). Retrieved from: http://cdmbuntu.lib.utah.edu/utils/getfile/collection/etd3/id/3734/filename/3718.pdf.
  28. Gellerman, D. M., & Lu, F. G. (2016). Aspects of cultural identity related to spirituality, religion, and moral traditions. In R. Lewis-Fernández, N. K. Aggarwal, L. Hinton, D. E. Hinton, & L. J. Kirmayer (Eds.), Handbook on the cultural formulation interview (pp. 118–122). Arlington: American Psychiatric Association.Google Scholar
  29. General Data Protection Regulation (GDPR 2016/679). Official Journal of the European Union, L119, pp 1–88. Retrieved here: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016R0679&qid=1473069680789&from=SV.
  30. Gillberg, C., Steffenberg, S., Börjesson, B., & Andersson, L. (1987). Infantile autism in children of immigrant parents. A population-based from Göteborg, Sweden. The British Journal of Psychiatry, 150, 316–321.CrossRefGoogle Scholar
  31. Graff, H. J., Berkeley, S., Evmenova, A. S., & Park, K. L. (2014). Trends in autism research: a systematic journal analysis. Exceptionality, 22, 158–172.CrossRefGoogle Scholar
  32. Groen, S., Rohlof, H., & Jadhav, S. (2016). Aspects of cultural identity related to national, ethnic, and racial background; languages; and migration. In R. Lewis-Fernández, N. K. Aggarwal, L. Hinton, D. E. Hinton, & L. J. Kirmayer (Eds.), DSM-5 handbook on the cultural formulation interview (pp. 107–117). Arlington: American Psychiatric Association.Google Scholar
  33. Hinton, D. E., & Hinton, L. (2016). In R. Lewis-Fernández, N. K. Aggarwal, L. Hinton, D. E. Hinton, & L. J. Kirmayer (Eds.), DSM-5 handbook on the cultural formulation interview (pp. 45–55). Arlington: American Psychiatric Association.Google Scholar
  34. Hollander, A.-C. (2013). Social inequalities in mental health and mortality among refugees and other immigrants to Sweden. Epidemiological studies of register data. (Doctoral Dissertation, Karolinska Institutet). Retrieved from: https://openarchive.ki.se/xmlui/handle/10616/41325.
  35. Idring, S., Magnusson, C., Lundberg, M., Ek, M., Rai, D., Svensson, A. C., et al. (2014). Parental age and the risk of autism spectrum disorders: findings from a Swedish population-based cohort. International Journal of Epidemiology, 43(1), 107–115.  https://doi.org/10.1093/ije/dyt262.CrossRefPubMedGoogle Scholar
  36. Idring, S., Lundberg, M., Sturm, H., Dalman, C., Gumpert, C., Rai, D., et al. (2015). Changes in prevalence of autism spectrum disorders in 2001–2011: findings from the Stockholm youth cohort. Journal of Autism and Developmental Disorders, 45, 1766–1773.CrossRefGoogle Scholar
  37. Interagency Autism Coordinating Committee (IACC). (2014). IACC strategic plan for autism spectrum disorder research—2013 update. Accessed on the 13 August 2016 from: https://iacc.hhs.gov/publications/strategic-plan/2013/.
  38. Janeslätt, G., Kottorp, A., & Granlund, M. (2013). Evaluating interventions using time aids in children with disabilities. Scandinavian Journal of Occupational Therapy.  https://doi.org/10.3109/11038128.2013.870225.CrossRefGoogle Scholar
  39. Kagawa Singer, M., Dressler, W. W., George, S. M., & Elwood, W. N. (2014). The cultural framework for health: an integrative approach for research and program design and evaluation. Retrieved from: https://obssr-archive.od.nih.gov/pdf/cultural_framework_for_health.pdf.
  40. Kagawa Singer, M., Dressler, W., George, S., & NIH Expert Panel. (2016). Culture: the missing link in health research. Social Science and Medicine.  https://doi.org/10.1016/j.socscimed.2016.07.015.CrossRefGoogle Scholar
  41. Keen, D. V., Reid, F. D., & Arnone, D. (2010). Autism, ethnicity and maternal immigration. The British Journal of Psychiatry, 196, 274–281.  https://doi.org/10.1192/bjp.bp.109.065490.CrossRefPubMedGoogle Scholar
  42. Kerekes, N., Tajnia, A., Lichtenstein, P., Lundström, S., Anckarsäter, H., Nilson, T., & Råstam, M. (2015). Neurodevelopmental problems and extremes in BMI. PeerJ.  https://doi.org/10.7717/peerj.1024.CrossRefGoogle Scholar
  43. Kistner, J., & Robbins, F. (1986). Brief report: characteristics of methods of subject selection and description in research in autism. Journal of Autism and Developmental Disorders, 16(1), 77–82.CrossRefGoogle Scholar
  44. Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., et al. (2011). Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Journal of Clinical Epidemiology, 64, 96–106.CrossRefGoogle Scholar
  45. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.CrossRefGoogle Scholar
  46. Lee, B., Magnusson, C., Gardner, R. M., Blomström, Å., Newschaffer, C. J., et al. (2015). Maternal hospitalization with infection during pregnancy and risk of autism spectrum disorders. Brain, Behavior, and Immunity, 44, 100–105.CrossRefGoogle Scholar
  47. Lewis-Fernández, R., Raggio, G., Gorritz, M., Duan, N., Marcus, S., Cabassa, L. J., et al. (2013). GAP-REACH: a checklist to assess comprehensive reporting of race, ethnicity, and culture in psychiatric publication. Journal of Neurology and Mental Disorders, 201(10), 860–871.CrossRefGoogle Scholar
  48. Löfkvist, U., Almkvist, O., Lyxell, B., & Tallberg, I.-M. (2014). Lexical and semantic ability in groups of children with cochlear implants, language impairment and autism spectrum disorder. International Journal of Pediatric Otorhinolaryngology, 78(2), 253–263.CrossRefGoogle Scholar
  49. Lundström, S., Forsman, M., Larsson, H., Kerekes, N., Serlachius, E., Långström, N., & Lichtenstein, P. (2014). Childhood neurodevelopmental disorders and violent criminality: a sibling cohort study. Journal of Autism and Developmental Disorders, 44, 2707–2716.CrossRefGoogle Scholar
  50. Lundström, S., Reichenberg, A., Anckarsäter, H., Lichtenstein, P., & Gillberg, C. (2015a). Autism phenotype versus registered diagnosis in Swedish children: prevalence trends over 10 years in general population samples. BMJ.  https://doi.org/10.1136/bmj.h1961.CrossRefGoogle Scholar
  51. Lundström, S., Reichenberg, A., Melke, J., Råstam, M., et al. (2015b). Autism spectrum disorders and coexisting disorders in a nationwide Swedish twin study. Journal of Child Psychology and Psychiatry, 56(6), 702–710.CrossRefGoogle Scholar
  52. Mandell, D., & Novak, M. (2005). The role of culture in families’ treatment decisions for children with autism spectrum disorders. Mental Retardation and Developmental Disabilities Research Reviews, 11, 110–115.CrossRefGoogle Scholar
  53. Mandell, D., Listerud, J., Levy, S. E., & Pinto-Martin, J. A. (2002). Race differences in the age at diagnosis among Medicaid-eligible children with autism. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1447–1453.CrossRefGoogle Scholar
  54. Mandell, D. S., Wiggins, L. D., Arnstein Carpenter, L., Daniels, J., DiGuiseppi, C., Durkin, M. S., et al. (2009). Racial/ethnic disparities in the identification of children with autism spectrum disorders. American Journal of Public Health, 99(3), 493–498.CrossRefGoogle Scholar
  55. Mann, C. J. (2003). Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emergency Medical Journal, 20, 54–60 Accessed here on the 15th May 2017: http://emj.bmj.com/content/emermed/20/1/54.full.pdf.CrossRefGoogle Scholar
  56. McEvilly, M., Wicks, S., & Dalman, C. (2015). Sick leave and work participation among parents of children with autism spectrum disorder in the Stockholm youth cohort: a register linkage study in Stockholm, Sweden. Journal of Autism and Developmental Disorders, 45(7), 2157–2167.CrossRefGoogle Scholar
  57. McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3), 276–282.CrossRefGoogle Scholar
  58. Milner, L., & Cho, M. (2014). Focusing on cause or cure? Priorities and stakeholder presence in childhood psychiatry research. AJOB Empirical Bioethics, 5(1), 44–55.CrossRefGoogle Scholar
  59. Miniscalco, C., Rudling, M., Råstam, M., Gillberg, C., & Åsberg Johnels, J. (2014). Imitation (rather than core language) predicts pragmatic development in young children with ASD: a preliminary longitudinal study using CDI parental reports. International Journal of Language & Communication Disorders, 49(3), 369–375.CrossRefGoogle Scholar
  60. Mir, G., Salway, S., Kai, J., Karlsen, S., Bhopal, R., Ellison, G., & Sheikh, A. (2012). European Journal of Public Health, 23(3), 504–510.CrossRefGoogle Scholar
  61. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097.CrossRefGoogle Scholar
  62. National Autism Center. (2015). Findings and conclusions: National standards project, phase 2. Randolph: Author.Google Scholar
  63. Nygren, G., Sandberg, E., Gillstedt, F., Ekeroth, G., Arvidsson, T., & Gillberg, C. (2012). A new screening programme for autism in a general population of Swedish toddlers. Research in Developmental Disabilities, 33, 1200–1210.CrossRefGoogle Scholar
  64. Office of Autism Research Coordination (OARC), National Institute of Mental Health and Thomson Reuters, Inc. on behalf of the Interagency Autism Coordinating Committee (IACC). (2012). IACC/OARC autism spectrum disorder research publications analysis report: The global landscape of autism research. Retrieved from: http://iacc.hhs.gov/publications-analysis/july2012/index.shtml.
  65. Pahnke, J., Lundgren, T., Hursti, T., & Hirvikoski, T. (2014). Outcomes of an acceptance and commitment therapy-based skills training group for students with high-functioning autism spectrum disorder: a quasi-experimental study. Autism, 18(8), 953–964.CrossRefGoogle Scholar
  66. Palmqvist, M., Edman, G., & Bölte, S. (2013). Screening for substance use disorders in neurodevelopmental disorders: a clinical routine? European Child & Adolescent Psychiatry, 23, 365–368.CrossRefGoogle Scholar
  67. Personal Data Act (1998:204). Retrieved from: http://www.government.se/content/1/c6/01/55/42/b451922d.pdf.
  68. Pierce, N. P., O’Reiley, M. F., Sorrels, A. M., Fragale, C. L., White, P. J., Aguilar, J. M., & Cole, H. A. (2014). Ethnicity reporting practices for empirical research in three autism-related journals. Journal of Autism and Developmental Disorders, 44, 1507–1519.  https://doi.org/10.1007/s10803-014-2041-x.CrossRefPubMedGoogle Scholar
  69. Ravens-Sieberer, U., Wille, N., Badia, X., Bonsel, G., Burstrom, K., Cavrini, G., Devlin, N., Egmar, A. C., Gusi, N., & Herdman, M. J. (2010). Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Quality of Life Research, 19(6), 887–897.CrossRefGoogle Scholar
  70. Ronald, A., Anckarsäter, H., Larsson, H., & Lichtenstein, P. (2014). Symptoms of autism and ADHD: a Swedish twin study examining their overlap. Journal of Abnormal Psychology, 123(2), 440–451.CrossRefGoogle Scholar
  71. Salway, S., Allmark, P., Barley, R., Higgingbottom, G., Gerrish, K., & Ellison, G. (2009). Researching ethnic inequalities. Social Sciences Research Update, 58.Google Scholar
  72. Salway, S., Barley, R., Allmark, P., Gerrish, K., Higgingbottom, G., Johnson, G., & Ellison, G. (2011). Enhancing the quality of published research on ethnicity and health: is journal guidance feasible and useful? Diversity in Health and Care, 8(3), 155–165.Google Scholar
  73. Sandin, S., Lichtenstein, P., Kuja-Halkola, R., Larsson, H., Hultman, C. M., & Reichenberg, A. (2014). The familial risk of autism. JAMA, 311(17), 1770–1777.CrossRefGoogle Scholar
  74. Selten, J.-P., Lundberg, M., Rai, D., & Magnusson, C. (2015). Risks for nonaffective psychotic disorder and bipolar disorder in young people with autism spectrum disorder. A population-based study. JAMA Psychiatry, 72(5), 483–489.CrossRefGoogle Scholar
  75. Stahmer, A., Schreibman, L., & Cunningham, A. (2011). Toward a technology of treatment individualization for young children with autism spectrum disorders. Brain Research, 1380, 229–239.CrossRefGoogle Scholar
  76. Swedish Data Inspection Board. (2008). Datainspektionens allmänna råd. Säkerhet för personuppgifter. Stockholm. Retreived from: http://www.datainspektionen.se/Documents/faktabroschyr-allmannarad-sakerhet.pdf.
  77. Swedish National Board of Health and Welfare. (2013). Psykologutredning i skolan. [Psychological assessment at school]. Socialstyrelsen. Retrieved from: http://www.socialstyrelsen.se/Lists/Artikelkatalog/Attachments/19164/2013-6-39.pdf.
  78. Tincani, M., Travers, J., & Boutot, A. (2009). Race, culture, and autism spectrum disorder: understanding the role of diversity in successful educational interventions. Research & Practice for Persons with Severe Disabilities, 34(3–4), 81–90.CrossRefGoogle Scholar
  79. Törn, P., Pettersson, E., Lichtenstein, P., Anckarsäter, H., Lundström, S., Gumpert, C. H., et al. (2015). Childhood neurodevelopmental problems and adolescent bully victimization: population-based, prospective twin study in Sweden. European Child & Adolescent Psychiatry, 24, 1049–1059.CrossRefGoogle Scholar
  80. Viera, A. J., & Garrett, J. M. (2005). Understanding interobserver agreement: the kappa statistic. Family Medicine, 37(5), 360–363.PubMedGoogle Scholar
  81. Ware, M., & Mabe, M. (2015). The STM report: an overview of scientific and scholarly journal publishing (4th ed.). The Hague: International Association of Scientific, Technical and Medical Publishers Retrieved from: http://www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf.Google Scholar
  82. West, E. A., Travers, J. C., Kemper, T. D., Liberty, L. M., Cote, D. L., McCollow, M. M., & Stansberry Brusnahan, L. L. (2016). Racial and ethnic diversity of participants in research supporting evidence-based practices for learners with autism spectrum disorder. The Journal of Special Education, 50(3), 151–163.  https://doi.org/10.1177/0022466916632495.CrossRefGoogle Scholar
  83. Wong, C., Odom, S., Hume, K., Cox, A., Fettig, A., Kucharczyk, S., et al. (2014). Evidence-based practices for children, youth, and young adults with autism spectrum disorder. Chapel Hill: The University of North Carolina, Frank Porter Graham, Child Development Institute.Google Scholar
  84. Wong, C., Odom, S., Hume, K., Cox, A., Fettig, A., Kucharczyk, S., et al. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. Journal of Autism and Developmental Disorders, 45(7), 1951–1966.  https://doi.org/10.1080/10459881003785506.CrossRefPubMedGoogle Scholar
  85. World Health Organization. (2014). Resolution on autism “Comprehensive and Coordinated Efforts for the Management of Autism Spectrum Disorders”. Geneva: WHO Retrieved from: http://www.who.int/mental_health/maternal-child/WHA67.8_resolution_autism.pdf.Google Scholar
  86. World Health Organization. (2017). Autism spectrum disorder. Fact sheet. Retrieved from: http://www.who.int/mediacentre/factsheets/autism-spectrum-disorders/en/.
  87. World Health Organization. (n.d.). International Classification of Diseases and Related Problems, Tenth revision. Retrieved from: http://www.who.int/classifications/icd/en/.
  88. Yudell, M., Roberts, D., DeSalle, R., & Tishkoff, S. (2016). Taking race out of human genetics. Engaging a century-long debate about the role of race in science. Science, 351(6273), 564–565.CrossRefGoogle Scholar
  89. Zander, E. (2015). Standardizing diagnostic assessment of autism spectrum disorder—assets and challenges. Doctoral dissertation. Stockholm: Karolinska Institutet.Google Scholar
  90. Zander, E., Sturm, H., & Bölte, S. (2015). The added value of the combined use of the Autism Diagnostic Interview-Revised and the Autism Diagnostic Observation Schedule: diagnostic validity in a clinical Swedish sample of toddlers and young pre-schoolers. Autism, 19(2), 187–199.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.Department of Special EducationStockholm UniversityStockholmSweden
  2. 2.Department of EducationUppsala UniversityUppsalaSweden
  3. 3.Department of Women’s and Children’s Health, Pediatric Neuropsychiatry Unit, Center for Neurodevelopmental Disorders at Karolinska Institutet (KIND)Karolinska InstitutetStockholmSweden
  4. 4.Habilitation & Health, Stockholm County CouncilStockholmSweden

Personalised recommendations