1 Introduction

Measurement of innovation has been gaining interest, as innovation is an important source of human welfare and economic growth, while it is diffused across the complex economic structure, producing innumerable better, faster and cheaper alternatives. To understand better the dynamics of economic growth and enact facilitative policies, we must design improved measures of innovation. At the same time, firms would benefit as to implement best practices in innovation management and services.

For the sake of the subsequent discussion, we rely on the following definition of innovation:Footnote 1 The design, invention, development and/or implementation of new or altered products, services, process, systems, organizational structures, or business models for the purpose of creating new value for customers and financial returns for the firm. It is rather obvious from this perspective that while innovation is a change in the performance of the supply side of the economy, the value induced is well measured from the demand side.

As a better, faster and cheaper alternative enhances the value of innovation, it is essential that we consider innovations by price deflation and performance improvement, perhaps independently. In general, the former is called process innovation while the latter is called product innovation. To do so, we must evaluate the qualitative changes in commodities with relevance to the overall innovation gain, while controlling and identifying how much performance bettered and how much price halved, in effect. The problem is, thus, to figure out how to measure the value of qualitative as well as quantitative changes in a differentiated market in different periods.

One way would be to apply a hedonic approach, whereby regressing characteristics of a commodity on its price. By means of time dummy variables we can estimate the marginal prices of the attributes, and at the same time retrieve the changes in CPI (consumer price index) which is equivalent to the quality-adjusted price change.Footnote 2 Although CPI is one demand side assessment of innovation, hedonic approach itself is based on both consumers’ and producers’ optimizing behavior. If hedonic regression is not weighted by sales volume for each commodity, which often is the case, the index would be more or less a supply side measurement.

Hedonic function and related price indices can make relative comparison between the levels of innovation but are not compatible with the absolute terms for welfare change. On that regard, discrete choice methods, which are based on the consumer’s intrinsic behavior, are eligible for estimating the change in the consumer surplus, through structural econometric investigations. Discrete choice models are widely applied in a variety of situations, including welfare measures estimation for cereals using time dummy variables by Nevo (2003). Mason and Quigley (1990) and Cropper et al. (1993) found that discrete choice models yield better estimates of non-marginal changes compared to those obtained using hedonic methods through their simulation experiments. Recent advancement includes various techniques concerning the IIA property of the logistic regression, omitted variables, endogeneity, etc. (Berry 1994; Berry et al. 1995; Petrin and Train 2010).

Under the discrete choice framework, an utility-controlled price index can be sought, for which they are called the COL (cost of living) index. Incidentally, CPI is taken as an approximation to the COL index (see Fisher and Shell 1998, p. 3). A true COL index is based on the theory of consumer demand which indicates how much money a consumer would need in later period, relative to the amount of money he needed in the base period to attain the same level of utility as in the base period (Jonker 2001). The COL index, however, is by far less illuminating than the welfare measurements that we combine to obtain that index. On the other hand, the utility-constant valuation of attributes by way of discrete choice methods, allows us to project all qualitative characteristics along with prices of goods onto the iso-consumer surplus plane, so that not only the welfare gain from innovation can we assess in terms of compensated variation, but also can we separate it into the innovative components that we consider.Footnote 3 In this paper we thereby disaggregate the welfare gain into the qualitative and the budgetary terms.

As for our analyses, we collected weekly market data through an internet shopping mall called Kakaku.com in Japan for LCD (Liquid Crystal Display) TVs that amount to the retail price and counts of page views which we use as a proxy for transaction counts for each model that prevailed within the market during the period October 2005–October 2007. LCD TV has received considerable attention in recent years, partly by the digitization of the terrestrial broadcasts in Japan, let alone the recent prominent innovations in LCD High Definition TV’s. Another feature is that TVs were the largest expended items (larger than PCs) in the educational and entertainment durable goods category for households expenditure, according to the Family Income and Expenditure Survey 2006–2008 of the Ministry of Internal Affairs and Communications Japan.Footnote 4

The attributes for the entire set of models were obtained from the website where the basic information for each model is available. These data were used to estimate a discrete choice model based on a nested logistic regression, and the quality measures by using the estimated parameters on the attributes. The performance attributes, in this regard, were selected in a same manner as the case for the hedonic attributes. We first estimate a hedonic function and were able to compare the estimates with those obtained by the discrete choice model. Transition of price and quality relationship is then assessed by the utility constant indifference line and its shift in time, while we eliminate the supply side of the market by assuming inelastic supply curve; in discrete choice framework, utilities will be measured relatively in terms of differences in consumers surplus, as is dealt in Trajtenberg (1989).

Accordingly, and assuming that the indifference lines are all parallel, we are able to assess the price changes that are fair to the qualitative improvements in the commodities. The overall innovation gain, therefore, will coincide with the budget gain that eliminates the value of qualitative improvements, plus that quality gain which we designate fair assessment of the qualitative improvements, in monetary terms. Although we will not be discovering index figures , innovation gain, if any, will be measured as the quality normalized price deflation, or, what is known by the term process innovation.

The rest of the paper is organized as follows. In Section 2 we briefly summarize the LCD TV market in Japan during the sample time span. Section 3 outlines the theoretical foundations of discrete choice models and elaborates on the measurement that is consistent with the consumers welfare change. The data and performance attributes selection along with hedonic regressions are discussed in Section 4. Value estimates of the performance attributes for the discrete choice model via nested logit regression, and the main result of price-quality relationship is presented in this section as well. Section 5 concludes.

2 Overview of the LCD TV market 2005–2007

The Japanese Economy has gradually recovered from 2005 to 2007 and the household expenditure has stable growth. The global LCD TV market has rapidly grown during the same period and the Japanese market has also expanded (Figs. 1 and 2) Global LCD TV (TFT) shipment has increased at 54.3% per annum, while the color LCD TV shipment for the Japanese market has grown at 28.2% p.a.. In 2005, the color LCD TV shipment has overtaken the color CRT TV as the most popular TV in Japan. Behind the trend lies the decrease of LCD panel price and fierce domestic and international competition among manufacturers.

Fig. 1
figure 1

Global LCD TV (TFT) shipment (1000 units). Source: Fuji Chimera Research Institute, Inc.

Fig. 2
figure 2

TV shipment for the Japanese market (1000 units). Source: the Japan Electronics and Information Technology Industries Association “Domestic Shipments of Major Consumer Electronic Equipment”

The suppliers of LCD panel have expanded their production capacities and then the unit price of a-Si TFT panel as predominant raw material of LCD TV has declined dramatically (Fig. 3). The annual change rate of panel price is approximately − 20 to − 30% from 2005 to 2007, particularly, the price of 50–59 in. panel has changed at − 45.9% per year. Japanese manufacturers face intensified competition with foreign firms such as Samsung and Philips in the international LCD TV market except for Japan. On the other side, they compete with each other in the Japanese market because the market is dominated by Japanese manufacturersFootnote 5 (Fig. 4). Sharp consistently has the largest Japanese market share in both under 32 in. and over 32 in. LCD TV from 2005 to 2007. Panasonic (Matsushita) has developed its market share for under 32 in. LCD TV, however has lost market share for over 32 in. LCD TV to Toshiba through low-price competition.

Fig. 3
figure 3

Unit price of a-Si TFT panel by size (1000 Yen/unit). Source: Fuji Chimera Research Institute, Inc.

Fig. 4
figure 4

Top 3 in the Japanese market share for LCD TV (%). Source: BCN “BCN Award”

3 Econometric framework

3.1 Linear nested logit representation

Discrete choice models are based on the random utility models. For the time being, we will focus on a single choice situation, omitting all superscript t for simplicity. We nevertheless assume that there is a single representative consumer.Footnote 6 The specification of the model posits that the level of utility that consumer n enjoys from a given product j is a function of both a vector of individual characteristics y n and a vector of product characteristics (X j , ξ j , p j ), where p j represents the price, and X j  = (x j1, ⋯ , x jz , ⋯ , x jZ )′ and ξ j are, respectively, observed and unobserved (by the econometrician) product attributes, of the product j.

The utility derived by consumer n from consuming product j is given by the scalar value U nj  = U(y n , p j , X j , ξ j ; θ) where θ is a vector of parameters that are indifferent in choice situations, to be estimated. The consumer chooses good j if U j  > U i  ∀ j ≠ i. We assume that there is an alternative j = 0, or the outside good, which represents the option of not purchasing any of the alternatives j = 1, ⋯ ,J and allocating all expenditures to this good. The consumer’s utility is decomposed as U j  = V j  + ε j where ε j captures the factors that affect utility but are not included in V j , part of utility that the researcher captures. The researcher does not know ε j and therefore treats these terms as random. By assuming each ε j distributed iid type I extreme value, the probability of the representative consumer choosing j is evaluated by the standard conditional logit formula, such that \(s_{j} = {e^{V_{j}}}/{\sum_{i} e^{V_{i}}}\), with s j being the market share for j. Standard logit models, however, exhibit the IIA (independence from irrelevant alternatives) property, which implies proportional substitution across alternatives.Footnote 7

Meanwhile, nested logit models are designed to disallow IIA for alternatives in different nests while allowing for those within each nest, whose formula is shown as below.

$$ s_{j} = \frac{e^{V_{j}/\lambda_g} \left( \sum_{j \in g} e^{V_j/\lambda_g} \right)^{\lambda_g - 1}}{ \sum_{k \in G} \left( \sum_{j \in k} e^{V_{j}/\lambda_k} \right)^{\lambda_k} } $$
(1)

Here, g is a member of a nonoverlapping set of nests G spanned in the whole set of alternatives J + 1, while parameter \(\lambda_g \in \left[ 0, \, 1 \right]\) is a measure of the degree of independence in unobserved utility among the alternatives in the nests. A value of λ g  = 1 indicates complete independence within the nests for which case the nested logit model reduces to the standard logit model. Nested logit formula 1 can be decomposed into two logits such that,

$$ s_{j} = \frac{e^{V_j/\lambda_g}}{\sum_{j \in g} e^{V_j/\lambda_g}} \cdot \frac{ \left( \sum_{j \in g} e^{V_j/\lambda_g} \right)^{\lambda_g}}{ \sum_{k \in G} \left( \sum_{j \in k} e^{V_{j}/\lambda_k} \right)^{\lambda_k} } = s_{j/g} \cdot s_g $$
(2)

where, s g and s j/g are the share of the nest g within G, and the share of good j within the nest g, respectively. Notice that if we recognize y as income of the representative consumer, and assume linear utility function such as

$$ V_j = \alpha y + \delta_j = \alpha (y - p_j) + \boldsymbol{\beta} X_j + \xi_j, $$
(3)

with \(\delta_j = - \alpha p_j + \boldsymbol{\beta} X_j + \xi_j\) being the partial utility, we can replace V j with δ j in the formula 2, and by taking logs for s g and s j/g we obtain the linearized configuration as follows.

$$ \ln s_g = \lambda_g \ln \left( \sum\limits_{j \in g} e^{{\delta_j}/{\lambda_g}} \right) - \ln \left( \sum\limits_{k \in G} \left( \sum\limits_{j \in k} e^{{\delta_j}/{\lambda_k}} \right)^{\lambda_k} \right) $$
(4)
$$ \ln s_{j/g} = \delta_j/\lambda_g - \ln \left( \sum\limits_{j \in g} e^{{\delta_j}/{\lambda_g}} \right) $$
(5)

With the outside good j = 0 as the only member of the nest g = 0 and with δ 0 = 0, and by taking Eqs. 4 and 5 into account, we have

$$ \ln s_g - \ln s_0 = - \alpha p_j + \boldsymbol{\beta} X_j + \xi_j - \lambda_g \ln s_{j/g}. $$
(6)

Upon estimating the parameters of Eq. 6, we are concerned with endogeneity problems, however. In our specification the unobserved error term ξ j reflects the unquantified aspects of style, prestige, reputation, etc., of the product and thus it is likely to be correlated with the price of products p j . For instrumental variables, we look for those that are relevant to prices while uncorrelated with the demand features, such as the cost side variables.

3.2 Cost of living index

Cost of living index measures the ratio of the minimum expenditures required in the current period and the base period, to allow each consumer to attain a particular utility level. Formally, a representative consumer’s COL index is defined as the ratio of Hicksian expenditures such that

$$ I^t = m\left(p^{t}, u^{{0}}\right)/ m\left(p^{{0}}, u^{{0}}\right) $$

where, m(p, u) denotes the (total) expenditure function at the price vector p and the utility level u. Superscript t denotes the corresponding time period with different choice situations. In terms of a random utility framework, the utility the representative consumer receives in the choice situation t, is the greatest utility one can receive from the set of alternatives available at that time, J t, which, by definition is the representative consumer surplus expressed as below.

$$ \textstyle u^t = \max_{j \in J^t} U_j^t = \max_{j \in J^t} V_j^t + \varepsilon_j^t $$

Under the logit assumptions the expected consumer surplus associated with a set of alternatives takes a closed form such that,

$$ E \left[ u^t \right] = \ln \left( \sum_{j \in J^t} e^{V_j^t } \right) = \ln \left( \sum_{j \in J^t} e^{a \left( y^t - p^t_j \right) + \boldsymbol{b} X^t_j } \right) $$
(7)

where, a and \(\boldsymbol{b}\) are the estimated parameters for α and \(\boldsymbol{\beta}\), respectively.Footnote 8 Now, the expenditure required for obtaining utility level u 0 with the alternatives available at t will be the υ t for the following equation.

$$ E \left[ u^0 \right] = \ln \left( \sum_{j \in J^0} e^{a \left(y^0 - p^0_j \right) + \boldsymbol{b} X^0_j } \right) = \ln \left( \sum_{j \in J^t} e^{a \left(\upsilon^{t} - p^t_j \right) + \boldsymbol{b} X^t_j } \right) $$
(8)

To obtain COL index we solve Eq. 8 for υ t to have

$$ I^t = \frac{\upsilon^t}{y^0} = 1 - \frac{\ln \sum_{j \in J^t}e^{-a p^t_j + \boldsymbol{b} X^t_j } - \ln \sum_{j \in J^0}e^{-a p^0_j + \boldsymbol{b} X^0_j }}{a y^0} $$

This index can be a measure of innovation, but it depends on the initial total expenditure of a representative consumer, y 0, which is not available to our analysis. In what follows, we measure expenditures in terms of differences, instead of ratios, in assessing the gained welfare from innovations.

3.3 Measurement of innovations

For convenience’s sake, we first introduce the following figure:

$$ \pi^t = \frac{1}{a} \ln \left( \frac{\sum_{j \in J^0} e^{\boldsymbol{b} X^0_j }}{\sum_{j \in J^t} e^{-a p^t_j + \boldsymbol{b} X^t_j }} \right) $$
(9)

Then the difference in the Hicksian expenditures up to time t, which is obtainable via Eq. 8, can be measured by using π t in the following manner. Notice that this is the expected consumer surplus evaluated in monetary terms.

$$ y^0 - \upsilon^t = \frac{1}{a} \ln \left( {\textstyle \sum_{j \in J^t} e^{-a p^t_j + \boldsymbol{b} X^t_j }} \right) - \frac{1}{a} \ln \left( {\textstyle \sum_{j \in J^0} e^{-a p^0_j + \boldsymbol{b} X^0_j }} \right) = \pi^0 - \pi^t $$
(10)

We call this amount the welfare gain obtained by the occurrence of innovation.

For the characteristic change, we measure it by h 0 − h t of the following reconstruction of Eq. 8.

$$ \textstyle E \left[ u^0 \right] = \ln \left( \sum_{j \in J^0} e^{a h^0 + \boldsymbol{b} X^0_j } \right) = \ln \left( \sum_{j \in J^t} e^{a h^t + \boldsymbol{b} X^t_j } \right) $$
(11)

Once again, for convenience’s sake, we introduce the following figure which we call the average price at t:

$$ p^t = \frac{1}{a} \ln \left( \frac{\sum_{j \in J^t} e^{\boldsymbol{b} X^t_j }}{\sum_{j \in J^t} e^{-a p^t_j + \boldsymbol{b} X^t_j }} \right) $$
(12)

Then, the characteristic change is evaluated by the following quality gain i.e.,

$$ h^0 - h^t = \frac{1}{a} \ln \left( {\textstyle \sum_{j \in J^t} e^{\boldsymbol{b} X^t_j }} \right) - \frac{1}{a} \ln \left( {\textstyle \sum_{j \in J^0} e^{\boldsymbol{b} X^0_j }} \right) = p^t - \pi^t $$
(13)

Notice that quality gain measures qualitative change in the attributes but independently of any price change.

Thus, the residual p 0 − p t be the budget gain, which equals to the change in the average price, and this will look as below.

$${p}^0 - p^t = \frac{1}{a} \ln \left( \frac{\sum_{j \in J^0} e^{\boldsymbol{b} X^0_j }}{\sum_{j \in J^0} e^{-a p^0_j + \boldsymbol{b} X^0_j }} \right) - \frac{1}{a} \ln \left( \frac{\sum_{j \in J^t} e^{\boldsymbol{b} X^t_j }}{\sum_{j \in J^t} e^{-a p^t_j + \boldsymbol{b} X^t_j }} \right) $$
(14)

Figure 5 illustrates the procedures for measurement of innovation, in a single choice case, which is otherwise called the repackaging case (Fisher and Shell 1998); there exist only one good for the two periods 0 and t, and their quality is represented by the quantity of the good. According to Eq. 7, the representative consumer’s utility will be indifferent for the pair (p,X) at each period, in the following equations. We may perceive these equations to represent indifferent technologies of the corresponding periods. These are the two lines drawn in Fig. 5.

$$ u^0 = a \left(y^0 - p^0\right) + {b} X^0 = a \left(y^0 - p\right) + {b} X $$
(15)
$$ u^t = a \left(y^t - p^t\right) + {b} X^t = a \left(y^t - p\right) + {b} X $$
(16)

By substituting X 0 for the X in Eq. 16 we are allowed to find π t, which is compatible with the formula 9, as follows.

$$ \pi^t = \frac{{b} X^0 - (-a p^t + b X^t)}{a} $$
Fig. 5
figure 5

Measurement of innovation gain

Observe that the innovation between (p 0, X 0) and (p t, X t) is evaluated as the difference in the quality adjusted prices using old and new technologies, that is, the vertical distance between the reference price p 0 = π 0 and π t, while the utilities are indifferent in each technology. In such ways, innovation is measured as welfare gain p 0 − π t. And because qualitative change is evaluated indifferent within the constant utility line, qualitative gain is evaluated accordingly in a single good case, which is compatible with the case of multiple goods i.e., Eq. 13, such that:

$$ p^t - \pi^t = \frac{{b} \left( X^t - X^0 \right)}{a} $$

To summarize the framework, innovation gain is measured by π 0 − π t, which equals the quality gain p t − π t plus the budget gain π 0 − p t.

4 Estimation and results

4.1 The data

The data used in our study was obtained from Kakaku.com, an internet price comparison service in Japan. Kakaku.com is not a retailer but a website (or an electronic bulletin board) that collects the retail prices of many registered retailers for possibly each and every model of commodities that ranges from automobiles to cosmetics. Retailers’ merit to register in Kakaku.com is its direct link to retailers’ own websites where customers can proceed to make payment transactions. Kakaku.com’s website also provides the basic specification of each model in a common table for qualitative comparison. Many reviews and comments if any are also attached to each item.

As is common with many internet advertising models, Kakaku.com charges money to the advertiser (retailer) according to the counts of clicks (CC) on retailer’s advertisement that brings visitors to retailer’s website. The amount of the charge is accounted for by the CPC (cost per click) and CC. Customers, before visiting the retailer’s website, will compare between different models according to their specifications, reviews, evaluations, reputations, and retail prices. Kakaku.com offers such information for each model. The number of customer visits to this information website, is called the counts of page views (PV) for a certain model.

We were able to obtain weekly PV data from the first week of October 2005 to the last week of October 2007.Footnote 9 To avoid ambiguity, we let \(N^t_j\) denote the aggregated counts of PVs for model j within an unit time span identified by the median calendar time t. Our data is that of weekly time span identified by the date of Wednesdays. The average figure of \(\sum_{i=1}^{J^t} N_{i}^t\) was about 831K views per week. There were 109 weeks altogether so t = 0, ⋯ , T where T = 108. We use the relative counts of PVs as a proxy to represent the share of transactions for each model i.e., \(s_{j}^t \approx {N_{j}^t}/{\sum_{i=1}^{J^t} N_{i}^t}\). The retail price, in accordance, were aggregated into weekly data. We denote \(p_j^t\) the weekly unweighted averaged lowest price tagged on model j at date t.

In Table 1 we show the aggregated market shares of major brand names in our data. Table 1a shows the brand aggregated shares of models that existed in the spanned market,Footnote 10 whereas Table 1b shows the PV weighted shares of models which reflect the consumers’ votes. Obviously people voted for models with named brands rather than others. Figure 4 shows that the market shares of the POS (point of sale) data of 50 retailing firms in Japan, and the PV weighted shares, therefore, show certain properties in common; perhaps the difference is the share of OTHERS brand the composition of which we are not aware, besides those for our data. SHARP was the leading brand in both markets of different sizes. SONY became about even with PANASONIC in smaller size TVs with regard to weighted shares while surpassed PANASONIC in larger size TVs.

Table 1 Brand shares in different sizes (%)

The specifications of all models prevailed in the whole time span of obtained data were retrieved from the Kakaku.com website by hand. There were 720 models in total. The summary statistics are provided in Table 2. Along with these specifications, we attached vintage, denoted \(\tau^0_j\), and age, denoted \(\tau^t_j\), that specifies the date that a model j was introduced to the market, and the time distance between \(\tau^0_j\) and the current date t, respectively. Hence, the candidates for the performance attributes are time independent except for the age, and so we write them as \(X_t^j\) anyway.

Table 2 Summary statistics of main specifications

4.2 Hedonic function and variable selection

We first estimate consumers price index via a log-linear hedonic function. Empirical application of the hedonic approach takes the form of regression with observed prices as dependent and performance attributes as explanatory variables, typically,

$$ \ln p_{j}^t = \alpha + \boldsymbol{\beta} X_{j}^t + \boldsymbol{\gamma} D^{t} + \mu_{j}^t $$
(17)

where, \(X_{j}^t = (x_{j1}^t, \cdots, x_{jZ}^t )'\)denote the characteristics of a model j at date t. D t is a vector of time dummy variables such that \(D^{t} = ( d_{1}^t, \cdots, d_k^t, \cdots, d_T^t )' \), where, \(d_{k}^t =1\) if k = t, and \(d_{k}^t =0\) otherwise. \(\boldsymbol{\gamma} = (\gamma^1, \cdots, \gamma^t, \cdots, \gamma^T )\) is the parameter to be estimated. We note, by Eq. 17, that quality-adjusted price index I 0t , in period t relative to base period 0 (I 00 = 1), can be evaluated by the following formula:

$$ I_{0t} = \exp \left({\gamma^t} \right) $$
(18)

As the performance attributes include not only the time invariant specifications but also the time dependent, age, we must note here that quality adjusted price changes are measured as if the age, as well as the specifications, are the same in different periods.

Although the attributes listed in Table 2 represent important characteristics in LCD TVs, there is obvious multicollinearity among the variables. Indeed, the full hedonic model, with all the variables in Table 2 included as regressors and the log of PRICEFootnote 11 (pooled) as the regressand, turned out to have very large VIFs (variance inflation factor) with mean VIF = 23.12. We thus looked into the correlation between the variables and found that there was a highly correlated group of variables including SIZE, HORIZONTAL, VERTICAL, WIDTH, HEIGHT, WATTS, and WEIGHT. Another correlated group consists of FULL, DIGITAL, BS, CS, DIGITAL_TUNER, and NETWORK. We decided to use SIZE and DIGITAL for representing these two groups, as these variables are expected to be more important for choice decision than others.

As for the explanatory variables we introduced AGE which reflects the time span of each model j being in the market until certain date t, measured in weeks.Footnote 12 Another variable we introduce is POWER, which is the ratio of WATTS with respect to WEIGHT. LCD TV is expected to perform better if power dissipation is larger for every unit scale, in our case measured in weights. So, UPOWER is expected to indicate some kind of performance of the model. CONTRAST and SPEED is dropped due to lack of observations instead. We hence specified the attributes for estimating the hedonic function, which we summarize in Table 3. Marginal values in terms of price is also presented.Footnote 13

Table 3 Estimated hedonic coefficients of performance attributes

We verified the collinearity using VIF after the log linear hedonic function regression with these performance attributes, and obtained low VIFs all less than 5 and 2.33 for average VIF. We also performed stepwise regression with 5% significance level, while treating time and brand dummies as groups, and all variables of Table 3 turned out to be significant. The relevance of the log-linear functional form was also checked by the Box-Cox test. As a result all the three functional forms, namely, linear, log-linear and double log functions, were rejected with very low levels of significance, while the least significant (with smallest χ 2) form was log-linear.

The result of the hedonic regression 17 and the corresponding price index via Eq. 18 are presented in Table 3 and in Fig. 6, respectively. Only the weighted regression by the PV counts (the proxy to the number of transactions) is presented. Number of observations, adjusted R2 and standard errors are presented at the bottom of the table. The estimation fit the data very well. All variables for the weighted model, including time dummies were very significant (not shown here). Hedonic price indices are different for the two cases. Weighted model shows about 50% decline in the price index while unweighted model shows about 40%.

Fig. 6
figure 6

Innovation measured in time

4.3 Nested logit regression

The set of categories G is classified into six unoverlapping categories by SIZE. g 1 consists of sizes under 10 (in.), g 2 of those between 10–25 (in.) except the model j = 242, g 3 of those between 25–33 (in.), g 4 of those between 33–50 (in.), and g 5 consists of sizes over 50 (in.). For aligning the valuations in different dates, an outside good g 0 must be chosen. We found one model j = 242, i.e., Sharp’s model LC-13S4, that had been in the market (PV > 0) for the whole sample time span and with very small price fluctuations. Thusly, we calculated LNSG, LNS0 and LNSJG, which are equivalent to ln s g , ln s 0 and ln s j/g , respectively, with time variations.

A nested logit model of the formula 6 is estimated with a pooled sample. We use the performance attributes specified in the previous section. Notice that the only time variant variable in the linear representative utility function is AGE, so by a pooled sample regression, we are assuming that the marginal valuations on the performance attributes are time invariant, while the decay in the utility is captured by AGE. As is discussed in the previous section, we instrumented PRICE by the cost side variables i.e., HEIGHT and WIDTH. The corresponding instrumental variables GMM (generalized method of moments) estimators are presented in Table 4. The results shown are those with heteroskedasticity-robust standard errors.

Table 4 Robust GMM estimators of the log-linear nested logit model

We note that our endogeneity test rejects the null hypothesis that the specified endogenous regressors can actually be treated as exogenous. The instruments are highly relevant, so that excluded instruments are correlated with the endogenous regressors; the regression clears the underidentification test, with a large Kleibergen–Paap rk LM statistic. Instruments are strong enough as we have a large Kleibergen–Paap Wald F statistic. And we have a small J statistic of Hansen’s for which we do not reject the null of validity, that the instruments are uncorrelated with the error term.

4.4 Quality and budget gains

In Fig. 6 we illustrate the quality gain (Eq. 13) over the time line in monetary figures along with that of the overall welfare gain (Eq. 10). Also, the budget gain (Eq. 14) is illustrated in the same frame, but in this case the negative of the budget gain is illustrated, since p t has increased throughout the time line. We can see several jumps in these two lines, indicating that there had been leaps in the performance of LCD TVs, however, as well in the compensation (or, negative budget gain) at the same time. The welfare gain, therefore, has been rather smooth, by negating the two effects. We also see that, at the end of the period, 380,664 (JPY) gained in the quality, 94,012 (JPY) lost in the budget, whereby earning worth of 286,652 (JPY) in the overall welfare per consumer in these two years.

In Fig. 7 we put quality change on the horizontal axis instead of time. Hence, the 45 degree line is the price change had it not been for any budget gain with respect to the actual quality change. The dim line is the negative budget gain with the quality change, so the difference between these two lines, illustrated by the solid line, will be the overall welfare gain. From this figure we see that quality had improved while price held somehow unchanged in the former half, while budget gain could not fully catch up with the quality improvement in the latter half. As we can see for this figure, high performance quality models would hit the scene from time to time, nevertheless with legitimate but high price tags. It is in the aftermath of the qualitative improvements when innovation comes to place. The corresponding innovation is the gradual decay in price (dim line), where the innovation actually gained.

Fig. 7
figure 7

Innovation gain

5 Concluding remarks

Our purpose in this paper, was to explore the possibility to measure how much an innovative progress, whether if the price has fallen or if the quality has increased or if there is a little bit of both, took place in a certain period of time, from the consumer’s point of view. We may then be able to probe further the inputs and outputs of production by taking the quality into account. And in such ways, structural analysis of innovation may come into focus. Innovation, in terms of price reduction while holding the quality constant, may be measured by way of hedonic function estimation with time dummy variables, although this measure includes both kinds of innovations. The quality (product) innovation, nevertheless, can be measured because the quality of a product may be quantified by the aid of values of attributes that we estimate.

For attribute value estimation, besides the hedonic approach, we employed discrete choice model based on the intrinsic valuation and behavior of consumers, while avoiding the identification problem inherent in the hedonic approach. As to draw a comparison between the two approaches we estimated hedonic function as well. Also, the performance attributes employed for estimating the discrete choice model were selected using the process of hedonic regression. Further, in regard to the welfare gain from innovation, expressed in formula 10, being equivalent to the expected consumer surplus, we found that the qualitative gain will be evaluated by formula 13 and that the budget gain by formula 14 fill in the gap between the qualitative change and the welfare gain, all based on the discrete choice framework.

For our empirical study we used market data for LCD TVs in Japan obtained from Kakaku.com, a large internet price comparison service. We use weekly data of page view counts as well as the retail prices. Specifications of attributes for each model were taken from the website as well. Price indices were estimated by hedonic regression both weighted and unweighted by the proxy of transactions i.e., page view counts. Discrete choice model that amounts, in our case, to a log-linear nested logit instrumental variables regression, measures the attributes’ marginal monetary values, and thus, they were used for the quality measurement of LCD TVs that have been available in each period of time.

The estimation result shows that frequency weighted hedonic regression estimated about 50% in the quality adjusted price index decline during the sample time span. The instrumented nested logit regression estimated the innovation to gain as much as 287 (KJPY), while the initial average price being at 236 (KJPY). Qualitative advancement was estimated to be as much as 381 (KJPY), while the average price increased by 94 (KJPY). We found, in our example, that the welfare gain from innovation had been quality-oriented, and that the average price did not fall below the reservation price for that bettered performances. Instead, we saw the average price being consistently above the initial state, leaping every once in a while with the synchronized quality leap, followed by the piecemeal average price decline, perhaps reflecting the process of innovation.