Measuring Innovation part 1: Frequently Used Indicators

http://www.innovationmanagement.se/2013/02/15/measuring-innovation-part-1-frequently-used-indicators/?goback=%2Egde_1817279_member_214551365  

One of the most common questions people ask me is how I measure innovation when conducting my research. The question echoes an underlying concern about how innovation can be captured and adequately measured. In this article I delineate the most frequently used innovation indicators, their strengths, and their drawbacks.

PUBLISHED: FEBRUARY 15, 2013
Written by Leif Denti

This article is the first part of a two part series on how to measure innovation in organizations. But let’s make it clear from the beginning. It is not easy to measure innovation since virtually all indicators risk missing some aspects of the process. For instance, patents or patent applications indicate some kind of technological progress but patents are seldom transformed into tangible products. In fact it looks bad for patents. 90 to 95 percent of all patents lack any market relevance and 99 percent fail to bring any profit to the firm (Stevens & Burley, 1997). Still, patenting is often necessary for many firms to keep their competitive advantage so clearly patents indicate something about the innovativeness of a firm. What about financial figures such as revenue? These risk being too broad since many other influences affect revenue besides innovation, including luck.

Below I summarize the most frequently used measures of innovation including their main advantages and drawbacks. For better clarity I divide them into three categories. Product/technology measures, financial measures, and subjective measures.

Product/technology measures

These indicators measure aspects of the innovation process which have the distal goal of transforming new ideas and technology into tangible products (goods or services).

Measure	Example study	Main advantage	Main drawback
New products or product improvements	Elenkov & Manev (2009)	Measures actual implementation	All products are not guaranteed to succeed
Patents or patent applications	Jung et al. (2008)	Measures technological progress	Patents are seldom realized
Patent citations	Makri & Scandura (2010)	Measures importance of patents	Patents may be self-cited
Invention disclosures or suggestions	Axtell et al. (2000)	Measures rate of idea generation	Ideas are seldom realized
Process innovations	West et al. (2003)	Measures improvements in processes and methods	Too much focus on processes is the ‘innovators dilemma’

Financial/market measures

These indicators measure aspects of the financial performance of an organization. Mainly in relationship to R&D spending and sales of new products (goods or services).

Measure	Example study	Main advantage	Main drawback
Ratio of sales of new products to total sales	Czarnitzki & Kraft (2004)	Indicator of success on market	Very broad, other factors confounds the measure
Ratio of sales of new products to R&D expenditures	Gumusluoglu & Ilsev (2009)	Indicator of R&D efficiency	Difficult to establish a valid baseline
Total R&D spending	García-Morales et al. (2008)	Easy to obtain	Does not indicate innovation efficiency
Number of employees in R&D	García-Morales et al. (2008)	Easy to obtain	Does not indicate innovation efficiency
New markets entered	Elenkov & Manev (2009)	Indicator of radical innovation	Roughly 60 percent of new products succeed

Subjective measures

Although the indicators above are among the most common when measuring innovation in firms, many of the activities that can be characterized as innovative risk being overlooked if innovation is measured solely using the broad searchlight of these quantitative measures. Activities like these can be labeled ‘dark innovation’ (Martin, 2012).  Examples include activities that are incremental (e.g., improvements in quality), involve little formal R&D (e.g. the concept of ‘skunk work’), and is seldom patented. One method to capture ‘dark innovation’ is to use subjective assessments.

Measure	Example study	Main advantage	Main drawback
Innovative work behavior (IWB)	De Jong & Den Hartog (2010)	Flexible, can measure any innovation activity	IWB’s does not unequivocally lead to tangible outcomes
Team innovativeness	Hurley & Hult (1998)	Flexible, can measure any innovation activity	Low correlations with number of implemented innovations
Organizational innovation	Chen et al. (2006)	Holistic assessment of the organization	Difficult to establish a valid baseline

Selection of measure is highly specific to the organization and depends on a number of factors. The relevance of indicators varies with technological domain and also depends on the specific product offering of the firm. Moreover, R&D functions of organizations may differ in degree of formalization which affects the degree to which ‘dark innovation’ takes place.

In my research which primarily concerns the innovation of individuals and teams in industrial (mainly automotive) R&D environments we have chosen a combination of quantitative indicators of innovation and subjective assessments of individuals’ innovative behaviors. This method covers outcomes of innovation as well as ‘dark innovation’ activities. In my next article we will look more closely at the elusive nature of ‘dark innovation’, as well as subjective assessments of innovation including what they measure and how they correlate with tangible outcomes.

By Leif Denti