“You get what you measure.” Or not? - Challenges for fact-based quality management

The old saying “You get what you measure” is used in many contexts all over the world, and it has influenced on reinforcing interests in measurements. We got, for instance, 1090000 (20.12.2012) hits by Google for this statement. Most of the found references seemed to agree this statement but still, is this saying really right. Do you really get what you measure? What are the most significant driving forces for business achievements?

   Measurements should not be seen as an isolated issue within business management. It is very essential to consider also why measurements are needed, what is their purpose, and what should be done with the results of the measurements in order to get the benefits of these results. The key question is what is the role of the measurements in the context of business management as a whole.

   Conceptually, the measurement topic is challenging. Metrology is the science of measurement and its application, and the vocabulary of metrology covers the generally accepted terms and definitions for the whole topic.  These general basic concepts may be used also in business management measurements. Measurement means experimentally obtaining one or more quantity values that can reasonably be attributed to a quantity of the item as the object of the measurement.

   Business related measurements are not any isolated activities, they should always be considered with the other business management methodologies.

   Tools for the fact-based management are often called “quality tools” and they have a long history in the development of quality expertise. Traditional quality tools are strongly related with data processing and management of business operations including statistical process control. Assortments of the methods in the different references may be seen confusing because tools have been collected and arranged differently. There are basic quality tools (7 tools) and new management and planning tools (7 tools), standardized statistical process control (SPC) tools (84 tools), an A-Z quality toolkit (28 tools), a quality toolbox (135 tools), Kaizen tools (e.g. 14 tools), SixSigma tools, etc. An organization that is systematically approaching the fact-based management and related measurements should select and create its own effective toolbox based on its factual business needs.

   Analytical methodology is needed to understand measured data in the relevant business context. Many of the quality tools are for data analyses. Analyses refer to activities to extract larger meaning from data and information to support evaluation, decision making, testing, improvement, and innovation. Analysis entails using data to determine trends, projections, and cause and effect that might not otherwise be evident.

   When considering the fact-based management one should clearly understand the related concepts of facts, data, information, knowledge, and wisdom, and recognize their roles and relationships (see figure).

Figure. Organization's actual business situation is composed of facts in the business processes. Management's decisions and interventions are based on information from these facts combined with the management's previous information and tacit knowledge (understanding). Managerial knowledge develops mainly through the collaborate learning. "Ba" is a Japanese name for collaborative knowledge creating and learning environment.

   Knowledge is built on theory. Theory is a window into the world. Interpretation of data from observations, tests, or experiments will depend largely on knowledge of the subject matter. Knowledge of somebody always depends on the paradigm that he/she follows intuitively or consciously. There are three major paradigms of knowledge theories:

• Heuristic (true knowledge is of one’s internal comprehension)
• Empirical (true knowledge is of external observation and experience)
• Pragmatic (knowledge is true if and only if it works satisfactorily)

   It is important that even practical business people recognize, which paradigm they follow. However, very often they don’t see significant differences of these different approaches, which means that they don’t have a profound and clear basis for their management knowledge.

   The concept performance has two meanings: a) Performing ability (activities, processes) and b) its achievements (results). Business performance may be generally described by using different terms: 

• Grade – Fulfilling general standard requirements
• Quality – Degree of fulfilling the needs and expectations of all stakeholders
• Service level – Agreed requirements by contracting parties
• Excellence – Quality excelling challenging references

   Organizations should be interested in their overall business performance that is a broad concept including four categories of performance:

• Customer-focused performance: Organization’s performance seen by the customers
• Operational performance: Organization-internal performance including hard business processes performance (e.g. cost efficiency, throughput or lead time) and soft performance (e.g. workforce skills)
• Product performance: Characteristics of the products (goods and services)
• Financial and market performance (e.g. competitiveness and market share)

    There are many different purposes for business related measurements. These purposes include the following:

• Research activities for getting new knowledge
• Acquisition of information for planning business or operations
• Controlling operations and processes

•  Performance monitoring
• Determination measurements (assessment)
• Verification measurements, requirement / acceptance testing (e.g. acceptable quality level, AQL, or limiting quality, LQ, approaches)
•  Validation testing

• Measurements for problem solving and performance improvement: Kaizen, breakthrough and learning
• Measurements for quality assurance  

   In business performance management and measurements, one should follow the basic principles of the metrology. That means that quantitative indication of business performance requires the following steps:

• Understanding the phenomena to be considered of a business interest and their characters
• Defining measures or indicators, characteristics, and quantity (metrics, measurement unit)
• Selecting numerical values and value range of the quantity

   After this, one is possible to determine logically practicalities needed for performance management and measurements that include the consideration of the following:

• Target / required values, observed values
• Meter, gauge, means to measure / observe the quantity
• Measurement, assessment technique and process
• Uncertainty of the measurements
• Conclusions, decisions and actions based on measurements
• Approach and practices for performance improvement

   Significant business phenomena typically involve both operational and strategic dimension.

   Measurements are important in all kinds of business activities but their purpose is not always clear or their emphasis may even be wrong. You don’t necessarily get what you measure, and what you get through measurements is not necessarily good for the organization or people being managed. Too often organizations measure easy things or follow guidelines of general standards or consultants instead of developing measurements according to the particular business situation.

   There are also many problems in the strict fact-based business management. The problems may be caused by the following extreme phenomena:

•  Data hell – Chaotic data disturbance
•  Data bulimia or data anorexia
•  Data mechanization – overly emphasized information technology and impoverished people knowledge and skills 
•  Information and knowledge splintering into isolated information pieces
•  Data pedantry or data naiveté
• Information popularization – Complex realities overly simplified and information contents impoverished
•  Information insecurity
•  Information capitalism or information proletariat – misuse of information power
•  Information terror and data crime

   According to Deming information is not knowledge. Knowledge comes from theory. Without theory, there is no way to use information that comes to us on the instant. He also emphasized that there is no true value of any characteristic, state, or observation. Therefore he called for creating profound knowledge for leadership.

   Deming also warned about rewarding people by quantitative measurements. This is his well-known deadly disease #3 of leadership. According to his arguments the “evaluation of performance, merit rating, or annual review nourish short-term performance, annihilates long-term planning, builds fear, demolishes teamwork, nourishes rivalry and politics. It leaves people bitter, crushed, bruised, battered, desolate, despondent, dejected, feeling inferior, some even depressed, unfit for work for weeks after receipt of rating, unable to comprehend why they are inferior. It is unfair, as it ascribes to the people in a group differences that may be caused totally by the system that they work in.”

   More important than to create scrupulous and even harmful business measurement  and information systems, is to get tacit knowledge of business leaders and employees who effectively interact with each other for creating extensive profound knowledge within an organization. After that also measurements-based explicit knowledge gives new views for managing an organization to sustained success development.

(A reference)