Reliability of job satisfaction measures

Abstract

In this paper, we evaluate consistency in answers to subjective questions about job satisfaction and explore the implications of any inconsistencies. We do this by analyzing a cross-country data set for 6 EU countries where respondents were posed the same question about overall job satisfaction twice within the same questionnaire. We find that, on a 0–10 point ordered utility scale, 80% either classify themselves identically or in the immediate adjacent and that the differences in classification are symmetric around zero. Furthermore, we find that highly satisfied workers report most consistently. When job satisfaction is used as an explanatory variable, we show how OLS-parameter estimates provide a lower bound and IV-estimates an upper bound of the true estimate – and that the bounds are fairly tight. When job satisfaction is used as dependent variable, we generally find high consistency when parameters are highly significant in both models, while less significant or insignificant parameter estimates may change considerably. This indicates that higher significance standards may be advisable in analyses with satisfaction measures as dependent variable compared to more traditional models that are not based on subjective data.

Appendices

Appendix A

Conjoint analysis

The introductory text reads: Imagine that, for some reason, you had to stop with your current job and had to look for a new one. Imagine that after a short time you get several offers. We will list them on the following screen. These listed job offers do not differ from your current job except from some points we specifically mention. Can you please evaluate these offers on a scale from 0 to 10, where 0 means the worst possible and 10 the best possible offer? And indicate if they are acceptable?

Example of a vignette (each respondent received five vignettes and there were 19 different clusters of five vignettes):

Type of conätract	Perämaänent with a risk of losäing the job and then receive unemäployäment benäeäfits	More
Numäber of work hours	50 h per week	More
Influäence on own work	Nobody but you decide over your work	More
Orgaäniäsaätion of the work	The job entails work in difäferäent teams	More
Start/end time	The employer decides on work hours (not night shifts) and can change this on a monthly basis	More
Eduäcaätion and trainäing	The employer will not offer you a speäcific eduäcaätion
Intenäsity	The job is very demandäing, which means that you need to stick to tight deadälines most of the time
Pension age	This firm has no early retireäment plan	More
Akäerlof theäory	Same workäing conädiätions as in other comäpaänies Loyäalty from both sides No posäsiäbiläiäties for shirkäing	More
Net wage	10% less per hour than your curärent job
Hold the mouse over “More” to gain addiätional inforämaätion

How would you rate this offer? Can you please evaluate this offer on a scale from 0 to 10?

Where 0 means the worse possible and 10 the best possible

q 0 – Worst posäsiäble
q 1
q 2
q 3
q 4
q 5
q 6
q 7
q 8
q 9
q 10 – Best posäsiäble
q 11 – Don’t know

Would this job offer be acceptable?

Yes

No

Don’t know

Appendix B

Country-specific histograms

Figure B1. Difference in job satisfaction categorization, (JS1 – JS2), by country.

Appendix C

Table C1 Cross tabulation of JS1 and JS2

Cross-tabulation of replies to the two job satisfaction questions

JS2
	0	1	2	3	4	5	6	7	8	9	10	Total
0	3	4	7	11	5	6	2	0	1	1	1	41
1	0	3	7	8	7	3	2	3	3	1	0	37
2	0	4	10	16	16	12	9	4	5	0	1	77
3	0	1	4	33	38	31	27	12	7	2	1	156
4	0	0	2	9	31	31	42	12	6	1	0	134
JS1 5	0	0	4	8	33	87	66	50	28	6	3	285
6	0	1	1	2	7	49	132	134	47	9	4	386
7	0	1	0	2	10	30	121	425	194	37	14	834
8	0	0	1	1	3	13	44	247	609	153	35	1,106
9	0	1	2	1	0	4	8	44	167	204	32	463
10	0	0	0	1	1	2	3	15	63	86	149	320
Total	3	15	38	92	151	268	456	946	1,130	500	240	3,839

Appendix D

Table D1 Descriptive statistics

	Obs	Mean	SD
Female	4321	0.47	0.50
Pers. monthly income/1000	4321	1.39	2.18
Age	4321	37.76	10.86
Age2	4319	1544.36	857.11
Hours per week	4321	37.74	10.57
Comämutäing time	4321	24.81	22.29
Union memäber	4321	0.39	0.49
Perämaänent conätract	4321	0.83	0.37
Tenäure less than one year	4321	0.13	0.34
Tenäure 1–3 years	4321	0.28	0.45
Firm size < 10 employäees	4321	0.20	0.40
Firm size 10–24 employäees	4321	0.16	0.37

Appendix E

Table E1 Probit model for probability of change in reported job satisfaction

	Coef.	SE
Job satäisäfacätion (ref = 10)
JS1 = 0	1.264	0.276 **
JS1 = 1	1.078	0.260 **
JS1 = 2	1.094	0.193 **
JS1 = 3	0.724	0.132 **
JS1 = 4	0.639	0.135 **
JS1 = 5	0.459	0.102 **
JS1 = 6	0.341	0.095 **
JS1 = 7	−0.061	0.082
JS1 = 8	−0.157	0.078 *
JS1 = 9	0.058	0.088
Indiävidäual charäacäteräisätics
Female	0.008	0.043
Age	0.004	0.014
Age2	0.000	0.000
Cohab/maräried	−0.010	0.046
Low eduäcaätion	0.080	0.060
Tenäure < 1 year	0.037	0.067
Tenäure 1–3 years	−0.068	0.050
Income (euro)/1000	0.011	0.010
Work hours	−0.012	0.010
Work hours2	0.000	0.000
Counätry dumämies (ref = Nl)
Denämark	0.187	0.066 **
Finäland	0.129	0.088
Greece	−0.008	0.071
Spain	0.213	0.095 *
UK	0.344	0.069 **
Occuäpaätion (ref = serävice)
Manäager	−0.015	0.133
Proäfesäsional	−0.028	0.115
Clerk	−0.073	0.069
Craft	0.168	0.134
Pers. serävice	−0.061	0.103
Minäing	−0.216	0.104 *
Sales	−0.005	0.079
Machine	−0.123	0.104
Arms	0.050	0.158
Other	0.034	0.071
Vignette group	Yes	(All insigänifäiäcant)
Conästant	Yes	Insigänifäiäcant
Numäber of obserävaätions	4,232
Log likeäliähood	2,728