Measuring Innovation part 1: Frequently Used Indicators
http://www.innovationmanagement.se/2013/02/15/measuring-innovation-part-1-frequently-used-indicators/?goback=%2Egde_1817279_member_214551365One of the most common questions people ask me is how I measure innovation when conducting my research. The question echoes an underlying concern about how innovation can be captured and adequately measured. In this article I delineate the most frequently used innovation indicators, their strengths, and their drawbacks.
PUBLISHED: FEBRUARY 15, 2013
Written by Leif Denti
This article is the first part of a two part series on how to measure innovation in organizations. But let’s make it clear from the beginning. It is not easy to measure innovation since virtually all indicators risk missing some aspects of the process. For instance, patents or patent applications indicate some kind of technological progress but patents are seldom transformed into tangible products. In fact it looks bad for patents. 90 to 95 percent of all patents lack any market relevance and 99 percent fail to bring any profit to the firm (Stevens & Burley, 1997). Still, patenting is often necessary for many firms to keep their competitive advantage so clearly patents indicate something about the innovativeness of a firm. What about financial figures such as revenue? These risk being too broad since many other influences affect revenue besides innovation, including luck.
Below I summarize the most frequently used measures of innovation including their main advantages and drawbacks. For better clarity I divide them into three categories. Product/technology measures, financial measures, and subjective measures.
Product/technology measures
These indicators measure aspects of the innovation process which have the distal goal of transforming new ideas and technology into tangible products (goods or services).
Measure
|
Example study
|
Main advantage
|
Main drawback
|
New products or product improvements
| Elenkov & Manev (2009) |
Measures actual implementation
|
All products are not guaranteed to succeed
|
Patents or patent applications
|
Jung et al. (2008)
|
Measures technological progress
|
Patents are seldom realized
|
Patent citations
|
Makri & Scandura (2010)
|
Measures importance of patents
|
Patents may be self-cited
|
Invention disclosures or suggestions
|
Axtell et al. (2000)
|
Measures rate of idea generation
|
Ideas are seldom realized
|
Process innovations
|
West et al. (2003)
|
Measures improvements in processes and methods
|
Too much focus on processes is the ‘innovators dilemma’
|
Financial/market measures
These indicators measure aspects of the financial performance of an organization. Mainly in relationship to R&D spending and sales of new products (goods or services).
Measure
|
Example study
|
Main advantage
|
Main drawback
|
Ratio of sales of new products to total sales
| Czarnitzki & Kraft (2004) |
Indicator of success on market
|
Very broad, other factors confounds the measure
|
Ratio of sales of new products to R&D expenditures
|
Gumusluoglu & Ilsev (2009)
|
Indicator of R&D efficiency
|
Difficult to establish a valid baseline
|
Total R&D spending
|
García-Morales et al. (2008)
|
Easy to obtain
|
Does not indicate innovation efficiency
|
Number of employees in R&D
|
García-Morales et al. (2008)
|
Easy to obtain
|
Does not indicate innovation efficiency
|
New markets entered
|
Elenkov & Manev (2009)
|
Indicator of radical innovation
|
Roughly 60 percent of new products succeed
|
Subjective measures
Although the indicators above are among the most common when measuring innovation in firms, many of the activities that can be characterized as innovative risk being overlooked if innovation is measured solely using the broad searchlight of these quantitative measures. Activities like these can be labeled ‘dark innovation’ (Martin, 2012). Examples include activities that are incremental (e.g., improvements in quality), involve little formal R&D (e.g. the concept of ‘skunk work’), and is seldom patented. One method to capture ‘dark innovation’ is to use subjective assessments.
Measure
|
Example study
|
Main advantage
|
Main drawback
|
Innovative work behavior (IWB)
| De Jong & Den Hartog (2010) |
Flexible, can measure any innovation activity
|
IWB’s does not unequivocally lead to tangible outcomes
|
Team innovativeness
|
Hurley & Hult (1998)
|
Flexible, can measure any innovation activity
|
Low correlations with number of implemented innovations
|
Organizational innovation
|
Chen et al. (2006)
|
Holistic assessment of the organization
|
Difficult to establish a valid baseline
|
Selection of measure is highly specific to the organization and depends on a number of factors. The relevance of indicators varies with technological domain and also depends on the specific product offering of the firm. Moreover, R&D functions of organizations may differ in degree of formalization which affects the degree to which ‘dark innovation’ takes place.
In my research which primarily concerns the innovation of individuals and teams in industrial (mainly automotive) R&D environments we have chosen a combination of quantitative indicators of innovation and subjective assessments of individuals’ innovative behaviors. This method covers outcomes of innovation as well as ‘dark innovation’ activities. In my next article we will look more closely at the elusive nature of ‘dark innovation’, as well as subjective assessments of innovation including what they measure and how they correlate with tangible outcomes.
By Leif Denti