Problems with measuring the "knowledge economy"

The previous posting on More on the productivity paradox highlighted the problem of the measurement of the modern, new or "knowledge economy". Much time and effort is expended by many national and international organisations in an attempt to measure the economy or economies of the world. While the measuring of the ‘standard’ economy is funny enough, when we move to the measurement of the ‘knowledge economy’ measurement goes from the mildly humorous to the outright hilarious. Most attempts to measure, or even define, the information or knowledge economy border on the farcical: the movie version should be called, "Mr Bean(counter) Measures the Economy".

There are substantial challenges to be overcome in any attempt to measure the knowledge economy. These are at both the theoretical and the method level. A more consistent set of definitions are required as are more robust measures that are derived from theory rather than from whatever data is currently or conveniently available. In order to identify the size and composition of the knowledge based economy one inevitably faces the issue of quantifying its extent and composition. Economists and national statistical organisations are naturally drawn to the workhorse of the ‘System of National Accounts’ as a source of such data. Introduced during World War II as a measure of wartime production capacity, the change in (real) Gross Domestic Product (GDP) has become widely used as a measure of economic growth. However, GDP has significant difficulties in interpretation and usage (especially as a measure of wellbeing) which has led to the development of both ‘satellite accounts’ - additions to the original system to handle issues such as the ‘tourism sector’; ‘transitional economies’ and the ‘not-for-profit sector’ - and alternative measures, for example, the Human Development Indicator and Gross National Happiness. GDP is simply a gross tally of products and services bought and sold, with no distinctions between transactions that add to wellbeing, and those that diminish it. It assumes that every monetary transaction adds to wellbeing, by definition. Organisations like the Australian Bureau of Statistics and the OECD have adopted certain implicit/explicit definitions, typically of the Information Economy-type, and mapped these ideas into a strong emphasis on impacts and consequences of ICTs. The website (http://www.oecd.org/sti/information-economy) for the OECD’s Information Economy Unit states that it:

“[...] examines the economic and social implications of the development, diffusion and use of ICTs, the Internet and e-business. It analyses ICT policy frameworks shaping economic growth productivity, employment and business performance. In particular, the Working Party on the Information Economy (WPIE) focuses on digital content, ICT diffusion to business, global value chains, ICT-enabled off shoring, ICT skills and employment and the publication of the OECD Information Technology Outlook.”

Furthermore, the OECD’s Working Party on Indicators for the Information Society has

“[...] agreed on a number of standards for measuring ICT. They cover the definition of industries producing ICT goods and services (the “ICT sector”), a classification for ICT goods, the definitions of electronic commerce and Internet transactions, and model questionnaires and methodologies for measuring ICT use and e-commerce by businesses, households and individuals. All the standards have been brought together in the 2005 publication, Guide to Measuring the Information Society [ . . . ]” (http://www.oecd.org/document/22/0,3343,en_2649_201185_34508886_1_1_1_1,00.html).

The whole emphasis is on ICTs. For example, the OECD’s “Guide to Measuring the Information Society” has chapter headings that show that their major concern is with ICTs. Chapter 2 covers ICT products; Chapter 3 deals with ICT infrastructure; Chapter 4 concerns ICT supply; Chapter 5 looks at ICT demand by businesses; while Chapter 6 covers ICT demand by households and individuals.

As will be shown below several authors have discussed the requirements for, and problems with, the measurement of the knowledge/information economy. As noted above most of the data on which the measures of the knowledge economy are based comes from the national accounts of the various countries involved. This does raise the question as to whether or not the said accounts are suitably designed for this purpose. There are a number of authors who suggest that in fact the national accounts are not the appropriate vehicle for this task. Peter Howitt argues that:

“[...] the theoretical foundation on which national income accounting is based is one in which knowledge is fixed and common, where only prices and quantities of commodities need to be measured. Likewise, we have no generally accepted empirical measures of such key theoretical concepts as the stock of technological knowledge, human capital, the resource cost of knowledge acquisition, the rate of innovation or the rate of obsolescence of old knowledge.” (Howitt 1996: 10).

Howitt goes on to make the case that because we can not measure correctly the input to and the output of, the creation and use of knowledge, our traditional measure of GDP and productivity give a misleading picture of the state of the economy. Howitt further claims that the failure to develop a separate investment account for knowledge, in much the same manner as we do for physical capital, results in much of the economy’s output being missed by the national income accounts.

In Carter (1996) six problems in measuring the knowledge economy are identified:

1. The properties of knowledge itself make measuring it difficult,
2. Qualitative changes in conventional goods: the knowledge component of a good or service can change making it difficult to evaluate their ‘levels of output’ over time,
3. Changing boundaries of producing units: for firms within a knowledge economy, the boundaries between firms and markets are becoming harder to distinguish,
4. Changing externalities and the externalities of change: spillovers are increasingly important in an knowledge economy
5. Distinguishing ‘meta-investments’ from the current account: some investments are general purpose investments in the sense that they allow all employees to be more efficient
6. Creative destruction and the ‘useful life’ of capital: knowledge can become obsolete very quickly and as it does so the value of the old stock drops to zero.

Carter argues that these issues result in it being problematic to measure knowledge at the level of the individual firm. This results in it being difficult to measure knowledge at the national level as well since the individual firms’ accounts are the basis for the aggregate statistics and thus any inaccuracies in the firms’ accounts will compromise the national accounts.

Haltiwanger and Jarmin (2000) examine the data requirements for the better measurement of the information economy. They point out that changes are needed in the statistical accounts which countries use if we are to deal with the information/knowledge economy. They begin by noting that improved measurement of many “traditional” items in the national accounts is crucial if we are to understand fully Information Technology’s (IT’s) impact on the economy. It is only by relating changes in traditional measures such as productivity and wages to the quality and use of IT that a comprehensive assessment of IT’s economic impact can be made. For them, three main areas related to the information economy require attention:

The investigation of the impact of IT on key indicators of aggregate activity, such as productivity and living standards,

1. The impact of IT on labour markets and income distribution and
2. The impact of IT on firm and on industry structures.

Haltiwanger and Jarmin outline five areas where good data are needed:

1. Measures of the IT infrastructure,
2. Measures of e-commerce,
3. Measures of firm and industry organisation,
4. Demographic and labour market characteristics of individuals using IT, and
5. Price behaviour.

In Moulton (2000) the question is asked as to what improvements we can make to the measurement of the information economy. In Moulton’s view additional effort is needed on price indices and better concepts and measures of output are needed for financial and insurance services and other “hard-to-measure” services. Just as serious are the problems of measuring changes in real output and prices of the industries that intensively use computer services. In some cases output, even if defined, is not directly priced and sold but takes the form of implicit services which at best have to be indirectly measured and valued. How to do so is not obvious. In the information economy, additional problems arise. The provision of information is a service which in some situations is provided at little or no cost via media such as the web. Thus on the web there may be less of a connection between information provision and business sales. The dividing line between goods and services becomes fuzzier in the case of e-commerce. When Internet prices differ from those of brick-and-mortar stores do we need different price indices for the different outlets? Also the information economy may affect the growth of Business-to-Consumer sales, new business formation and in cross-border trade. Standard government surveys may not fully capture these phenomena. Meanwhile the availability of IT hardware and software results in the variety and nature of products being provided changing rapidly. Moulton also argues that the measures of the capital stock used need to be strengthened, especially for high-tech equipment. He notes that one issue with measuring the effects of IT on the economy is that IT enters the production process often in the form of capital equipment. Much of the data entering inventory and cost calculations are rather meagre and needs to be expanded to improve capital stock estimates. Yet another issue with the capital stock measure is that a number of the components of capital are not completely captured by current methods, an obvious example being intellectual property. Also research and development and other intellectual property should be treated as capital investment though they currently are not. In addition to all this Moulton argues that the increased importance of electronic commerce means that the economic surveys used to capture its effects need to be expanded and updated.

In Peter Howitt’s view there are four main measurement problems for the knowledge economy:

1. The “knowledge-input problem”. That is, the resources devoted to the creation of knowledge are underestimated by standard measures.
2. The “knowledge-investment problem”. The output of knowledge resulting from formal and informal R&D activities is typically not measured.
3. The “quality improvement problem”. Quality improvements go unmeasured.
4. The “obsolescence problem”. No account is taken of the depreciation of the stock of knowledge (and physical capital) due to the creation of new knowledge.

To deal with these problems Howitt makes a call for better data. But it’s not clear that better data alone is the answer, to both Howitt’s problems and the other issues outlined here. Without a better theory of what the “knowledge economy” is and the use of this theory to guide changes to the whole national accounting framework, it is far from obvious that much improvement can be expected in the current situation.

One simple, theoretical, question is, To which industry or industries and/or sector or sectors of the economy can we tie knowledge/information production? When considering this question several problems arise. One is that the “technology” of information creation, transmission and communication pervades all human activities so cannot fit easily into the national accounts categories. It is language, art, shared thought, and so on. It is not just production of a given quantifiable commodity. Another issue is that because ICT exists along several different quantitative and qualitative dimensions production can not be added up. In addition if much of the knowledge in society is tacit, known only to individuals, then it may not be possible to measure in any meaningful way. Also if knowledge is embedded in an organisation via organisational routines then again it may not be measurable. Organisational routines may allow the knowledge of individual agents to be efficiently aggregated, much like markets aggregate information, even though no one person has a detailed understanding of the entire operation. In this sense, the organisation “possesses” knowledge which may not exist at the level of the individual member of the organisation. Indeed if, as Hayek can be interpreted as saying, much of the individual knowledge used by the organisation is tacit, it may not even be possible for one person to obtain the knowledge embodied in a large corporation.

As noted above Carter (1996) emphasises that it is problematic to measure knowledge at the national level in part because it is difficult to measure knowledge at the level of the individual firm. Part of the reason for this is that none of the orthodox theories of the firm offer us a theory of the “knowledge firm” which is needed to to guide our measurement.

Thus many of the measurement problems of the "knowledge economy" are rooted in the fact that we don't have a good theory of the "knowledge economy" or the "knowledge firm". Without such theories calls for better data are wasted, they miss the point. "Better" data collection alone is not going to improve the measurement of the "knowledge economy".