Data mining for unemployment rate prediction using search engine query data

Abstract

Unemployment rate prediction has become critically significant, because it can help government to make decision and design policies. In previous studies, traditional univariate time series models and econometric methods for unemployment rate prediction have attracted much attention from governments, organizations, research institutes, and scholars. Recently, novel methods using search engine query data were proposed to forecast unemployment rate. In this paper, a data mining framework using search engine query data for unemployment rate prediction is presented. Under the framework, a set of data mining tools including neural networks (NNs) and support vector regressions (SVRs) is developed to forecast unemployment trend. In the proposed method, search engine query data related to employment activities is firstly extracted. Secondly, feature selection model is suggested to reduce the dimension of the query data. Thirdly, various NNs and SVRs are employed to model the relationship between unemployment rate data and query data, and genetic algorithm is used to optimize the parameters and refine the features simultaneously. Fourthly, an appropriate data mining method is selected as the selective predictor by using the cross-validation method. Finally, the selective predictor with the best feature subset and proper parameters is used to forecast unemployment trend. The empirical results show that the proposed framework clearly outperforms the traditional forecasting approaches, and support vector regression with radical basis function (RBF) kernel is dominant for the unemployment rate prediction. These findings imply that the data mining framework is efficient for unemployment rate prediction, and it can strengthen government’s quick responses and service capability.

Appendix: The top 100 search engine query data

No.	Key words	No.	Key words
1	filing unemployment	51	ohio unemployment rate
2	unemployment filing for	52	unemployment ny
3	unemployment office	53	unemployment compensation
4	file for unemployment	54	unemployment in az
5	unemployment file for	55	to apply for unemployment
6	unemployment state	56	unemployment insurance claim
7	state of unemployment	57	unemployment department of labor
8	insurance unemployment	58	department of labor unemployment
9	washington unemployment	59	labor department unemployment
10	unemployment file	60	unemployment check
11	unemployment insurance	61	unemployment for mn
12	unemployment apply	62	unemployment in indiana
13	department of unemployment	63	unemployment in california
14	unemployment website	64	snag a job
15	unemployment application	65	unemployment grants
16	unemployment new york	66	unemployment in pennsylvania
17	washington state unemployment	67	unemployment benefit insurance
18	Wisconsinunemployment benefits	68	claim unemployment benefit
19	insurance for unemployment	69	part time unemployment
20	apply for unemployment	70	security jobs
21	unemployment claims	71	new york unemployment benefit
22	unemployment apply for	72	unemployment insurance benefit
23	apply for unemployment	73	unemployment dol
24	unemployment ca	74	unemployment info
25	unemployment services	75	unemployment commission
26	unemployment security	76	michigan unemployment benefits
27	unemployment	77	weekly unemployment insurance
28	to file unemployment	78	weekly unemployment benefits
29	unemployment benefits	79	nyc unemployment benefits
30	file for unemployment online	80	green jobs
31	ohio unemployment benefits	81	how to claim unemployment
32	unemployment file claims	82	unemployment rate
33	to file for unemployment	83	unemployment insurance benefits
34	unemployment benefits pa	84	unemployment weekly benefits
35	unemployment benefit	85	online unemployment application
36	nys dept labor	86	unemployment rate ny
37	state unemployment benefit	87	jobs in usa
38	connecticut unemployment benefits	88	new york unemployment benefits
39	dept of unemployment	89	benefits for unemployment
40	nys dept of labor	90	police jobs
41	for unemployment benefits	91	dc unemployment
42	uimn.org	92	unemployment in kansas
43	unemployment in michigan	93	mass unemployment benefits
44	unemployment benefit claim	94	unemployment online
45	unemployment payment	95	unemployment in florida
46	unemployment in colorado	96	eligible for unemployment
47	apply for unemployment online	97	benefits of unemployment insurance
48	unemployment benefits insurance	98	unemployment eligibility
49	application for unemployment	99	construction jobs
50	benefits unemployment insurance	100	unemployment rate recession