What is the difference between Data, Information, Knowledge and Wisdom?

What is the difference between Data, Information, Knowledge and Wisdom?

Need help with your KM Strategy? Click here for the Serious Insights KM Services

The debate over the relationship between data, information, knowledge and wisdom continues to evolve as new forms of representation emerge. My personal perspective runs contrary to IT and philosophical definitions. I find for knowledge management, and many IT applications, my definitions provide a clarity that doesn’t exist in conflated definitions of data and information, knowledge and wisdom. This discussion will restrict itself to technology though extensions to include biology will become increasingly important.

Data: Passive and inert

Outside the presence of knowledge, everything is data. Raw data, or summaries of data, including charts and graphs, commonly called information, isn’t information if not actively informing anything.

Data in the absence of analysis or another form of engagement proves meaningless. Information requires transformation and awareness.

Data is passive. Bits, data stored in database rows, e-mail, spreadsheets, word processing documents and images all create a collection that promises the potential for “information.” No matter how organized these collections, data not being viewed becomes inert.

Inert data does not imply a lack of value. Personal interpretation or market forces ultimately determine the absolute value of any data. If the data, when used creates value, then it holds that value regardless of if it is being active or not. Even data considered highly valuable, however, only manifest that value while it is being used.

Data completeness

Most data is incomplete. Transaction data from an online store much may be gleaned about user preferences and buying trends, but transactions continue to occur. No inference will ever be informed by the very latest information. As soon as an extract occurs, new records create more data.

The only data that may be considered complete would be a model for which some arbitrary definition of completeness exists, such as a computer emulator. When emulator runs, the data being interpreted represents a complete model of the target system. No additional information would make the data better, so the data can be considered complete.

Most data is historical. Beyond data that can be considered complete, all data is historical. There is no such thing as realtime data. Even data delivered in a continuous stream arrives at a process aged by pico-, nano- or microseconds. New data sits behind it in the stream that is more recent than the data currently being interpreted.

Copies, procedures and trust

Data may be copied and exist in two states at the same time. If data is being used to offer navigation guidance for example, and that data sits simultaneously on a server not being used, the instances of that data are inert and active at the same time. This remains true for datasets pulled into a computer’s working memory as the input to an application or algorithm, or if a person actively reasoning over the data. With current computing architectures, many copies of the same data may exist simultaneously in inert and actively used states. Rarely is the primary source data employed for anything beyond being the active repository for additional accumulation or individual record changes, which renders it perpetually inert.

Stored procedures act as a knowledge when they execute, otherwise, they are also data.

Data should not be trusted without context. Data, for instance, that modeled the toy market ahead of the demise of Toys’R Us can no longer predict the activity of the toy market because no data exists about consumer behavior, supply chains to other related activities or processes in a post-Toys’R Us world. New models will need to be built as data arrives to inform them. Data, therefore, requires context. Data stewards should clearly mark toy store market data needs to as pre and post-Toys’R Us.

There is no data from the future. All predictions are guesses.

Is Big Data different?

Big Data modifies none of the above attributes of data. Big Data simply applies a label to large collections of data. Websites, social media platforms, and transaction systems constantly collect data, and they do so in a structured way that makes that data easy to use for queries, reports, and visualizations. These massive collections have become known as Big Data.

Nothing in Big Data implies any structures or attributes that distinguish it from other data. Machine learning (ML) benefits from Big Data because the ML algorithms require substantial training sets to reach reasonable accuracy in their recognition of patterns. Big Data remains relatively narrow in each instance of machine learning and therefore does not contribute to any synthesis of cognitive abilities by computers. More data about a topic does not increase the likelihood of a computer making inferences about another topic.

Information: Ephemeral and Transformational

Data informs knowledge which creates information.

No information exists without knowledge interpreting it.

Information is ephemeral. Information only exists at the moment an active agent, be it a mind or an algorithm, interprets meaning from data or takes action based on that data.

Action-based information is simple: the data suggests that a pump number 12 is overheating. The algorithm sends a command to shut down pump number 12.

Transformational information is much more complicated. Transformational information may change the knowledge-base of the human or algorithm which interprets it. This is how machine learning learns. As new cases arrive, the machine learning algorithm refines its ability to create positive results based on its goal.

Transformation information may also suggest net new knowledge, such as offering the ability to identify new physical objects or discovering a connection between two data points. To be clear, knowledge must be present that seeks new connections. The data will not manifest such connections on their own. New connections require the model of the problem, or perhaps the interpretation of the world, to change.

Action, analysis, and interpretation can take place in the human brain, or inside the working memory of a computer. The analysis must be active for information to exist. Once attention shifts to another task that no longer requires the data, information ceases to exist and the data reverts to an inert state. In the human mind, however, data may remain active in unconscious subprocesses even if the person interpreting the data shifts attention to something else. Regardless of any subprocesses or background processes in human minds or in computers, information lasts only as long as someone or something pays attention to it.

Infographics as data

People talk about “rich information graphics,” or “infographics” as if they contain information independent of activity. They do not. An infographic printed and placed in the middle of a stack of papers is just data waiting for someone to use it. Infographics simplify data, or through the application of knowledge, organize it in such a way that the brain takes less time to infer meaning than it would if presented with the raw data.

The infographic, however, remains inert data when not being interpreted. Data in this form, though, may be said to have a higher information potential for people than the raw data (an algorithm, by contrast, would need likely need common sense reasoning to interpret an infographic, but would have no problem creating a chart to include in the infographic from the underlying data).

Attention shifts return information to data

When attention shifts from data, information usually ceases to exit. In some cases, information transforms into knowledge. As the information integrates and assimilates with existing knowledge it becomes part of the knowledge corpus, losing its individual characteristics as it gains new context.

Information exists in punctuated, non-continuous time. Redirecting attention from data does not mean the information ceases to exist for all time, just for that moment. If attention returns, so too does the information derived from that data.

For humans, mulling over information, reprioritizing attention, even procrastination—pushing off data to linger in subprocesses—may iteratively increase the value of the data as the subprocesses enhance the ability to discover relationships and insights that may lead to actions or knowledge not initially apparent.

Knowledge as Data

Knowledge is data.Unless it is being used, it is inert. Knowledge comes in complex forms such as rules, algorithms, patterns (and synaptic connections) that process simpler data when active in a computer or a human brain.

An analytics application, while not being used, exists on disk only as data. It is inert. Once the analytics program executes within in its interpreter (the operating system) and reads data, it becomes active at transforming data into information.

Knowledge, especially digitally based knowledge, can be interpreted in more than one manner. It can be interpreted by a compiler which executes it, but it can also be interpreted by an editor which reveals its “knowledge” logic and structure. Unlike data, knowledge has intent and purpose that can be derived.

A feedback loop that exists between information and knowledge. Insights gained from the interpretation of data lead to new knowledge, to a new actionable rule that can be applied to other data. We commonly call this feedback loop learning.

Consider that after running several experiments you realize you an entire set of procedures add nothing to the results. In the future, your algorithm, your knowledge, will reduce the data set and apply only to the meaningful processes. Updated knowledge tends to become more efficient.

Knowledge also has the special characteristic of being able to, in its most sophisticated forms, deal with incomplete data. Humans often face incomplete data in situations that still require action. Heuristics guess, often testing multiple data possibilities in quick succession to find the historical data most likely to fit the current situation (This is another reason why the absolute value of data can’t be determined, because in a Serendipity Economy one can never accurately anticipate when something currently inert and of low information value may be become both informationally and potentially economically valuable.)

Science often seeks knowledge through simplification of data. Physicists reduce fluid dynamics to equations, or algorithms, that take in data and model the behavior of a fluid. But there is also a version of the same problem, in which a model represents the fluid and interactions at the molecular level. Such a model would likely take more bits than stars in the known universe to be complete, therefore the simplification and approximations of equations and their algorithmic implementations must often suffice.

Common Sense Reasoning

Common sense reasoning uses the world as data to inform knowledge. The enormous amounts of data that humans and other beings use to interpret the world creates issues for creating sentient computers. Any device not fashioned in humanoid form will likely never approximate human consciousness due to architectural differences in the way a box and a human interact with the world.

If a robotics arrives at a point where machines include all human inputs at similar fidelity, it is conceivable that some meta-cognitive set of algorithms could approximate human intelligence. It is far more likely that the differences of architecture would create a being with a unique set of cognitive abilities, especially when the sensory inputs to such a machine would probably far outstrip those available to ordinary humans. Common sense reasoning represents a form of meta-cognition for which humans continue to seek a reliable model.

Short of religious perspectives, data existed before knowledge. Knowledge seems an evolutionary adaptation that assists plants and animals in navigating and surviving their environment. As our environments increased in complexity, so too did the human need to interpret the data, which lead first to larger brains, then to computers as a form of a mental prosthesis or auxiliary processing unit.

Wisdom as Constraint and Context

Wisdom is specialized knowledge that places constraints on knowledge. Not that it limits how knowledge performs but selects which knowledge applies to a situation or set of data. Wisdom is a selection process. Wisdom is the most tacit of knowledge, a black box of personally encrypted data.

Humans find it most difficult to describe what they know about what they know, and also how they know what they know about what they know. Wisdom proves unfathomable even to its owner. This is the primary reason successful people have a difficult time transferring success. They try to understand what it is they did to become successful, reverse engineering their wisdom into executable knowledge which disregards much of the subtlety for which they may not even have the language.

At the highest levels of intellect, wisdom remains primarily a human form of data, but lower-level wisdom exists, for instance, in data center operations where automation must decide what routines to balance an unexpected processor load. Wisdom also acts as meta-controller, suggesting when inaction may be more valuable than action. And that remains a very human act, one that even humans struggle to master.

As drone and automobile makers create more autonomous devices, they will need to address the concept of wisdom. The Trolley Problem represents an instance where wisdom fails, but where the situation requires a choice. Self-driving vehicle makers also encode their machines to make this choice.

AI and learning

Companies like IBM and others focused on cognitive computing can also see wisdom as the meta-process for sense-making. Some cognitive applications apply “wisdom” when identifying useful, or missing data required to drive a model’s to a meaningful conclusion. In this sense, wisdom acts a content creator to synthesize potential options where tested knowledge fails. Scaling this up leads to higher-forms of meta-processes that mimic wisdom.

Wisdom, like all forms of data, fails over time as new data changes context in the models and in the world. Wisdom needs to evolve to remain relevant. Wisdom processes should require learning when they recognize their irrelevance.

As organizations apply analytics to education to “create better outcomes” we need to question the dependency of outcomes on universal or regional conditions and temporal context. Given there is no data about the future, how can we know that a particular outcome today will be an outcome relevant to some future context? That is the point of worker retraining programs. Wisdom, like all data, ages, becomes irrelevant as context shifts—but wisdom, unlike basic data, employs feedback loops.

Consider the wisdom leveraged in an authoritarian state focused on military growth suddenly displaced by democracy. Would the learning outcomes not change? Would the skills required to survive and thrive not prove radically different? The authoritarian state would seek entrepreneurs and disruptors to suppress and weed them out. The new government might find it difficult to repurpose data sources and the models once so focused on military aptitudes, followership, nationalism and internal consistency to discover entrepreneurs, inventors, and disrupters capable of growing the economy. The outcome inverts. While data labels remain consistent, definitions would not.

For most compute-based systems, a shift this dramatic would make them permanently irrelevant. Only the richness of human wisdom, the vast stores of unrelated data and the ability to learn along multiple vectors protects humanity, if not all humans, from significant discontinuities.

Data, Information, Knowledge and Wisdom

Data, information, knowledge and wisdom act as data in their most basic form. When invoked by mind or algorithm, knowledge and wisdom use data to create information. This offers the ability for knowledge and wisdom to use that information to evolve with more precise knowledge, with more effective wisdom.

By thinking about these four elements that make up the cognitive portions of our technological and biological existence, we can be more aware of not only how we learn, but when to actively leverage data as feedback for our own growth—and where current technology meets limitations to artificial cognition and the obstacles that remain before any artificial intelligence can reach beyond its data.

(For more cautions on analytics see my expert blog post at Fast Company: Why Big Data Won’t Make You Smart, Rich, Or Pretty )

This is an updated position that replaces a similar post from April 11, 2012.