Expert Systems

Populating an organization with expert systems may seem like nirvana for many gurus of knowledge management: The ability to capture an expert’s knowledge in a software system for other members to use seems to answer many of the problems raised when trying to get an organization to exploit knowledge as a valuable resource. Unsurprisingly, expert systems fall quite short of this – but they are still worth investigating. Here we will look at some of the strengths and weaknesses of them. (See Box 1 for a quick answer to what is an expert systems.)

Expert systems were launched in the early Eighties to a fanfare of hype proclaiming that they would soon prove cheaper and more efficient than human experts. Wilder pundits even predicted that expert systems would do for white-collar workers what factory automation did for many blue collar workers: make them redunadant.

The vacuity of these claims were quickly realized and by the late Eighties, expert systems were at a watershed. A lot of commercial experience had been gained from the deployment of prototype expert systems, and whilst the wilder claims were evaporating, there was a belief that by being "realistic", expert systems could be useful. Organizations across the commercial spectrum were going on record as realizing benefits from expert systems trials. Amongst them were Ford, BP, Unilever, Shell, IBM, Prudential and TSB. However, even with this more modest view on the role of expert systems led to some predictions being wide of the mark. (See Box 2 on Ovum gets it wrong!)

Ten years on, it is much clearer what is happening with expert systems. Essentially, modesty seems to be the key: Small PC-based applications which provide computer-based guides on regulations and procedures are ideal for expert system technology. The information that goes into manuals can often be used straightforwardly as the knowledge for an expert system. Such a system increases efficiency and can be useful for integrating amendments.

This kind of application by-passes the knowledge engineering bottleneck: The problem of getting the right sort of information with which to engineer a knowledgebase. An experienced professional may know how to do a particular task but may be unable to articulate the knowledge for use in an expert system, since it may depend on a "feel-for-the-job" or "intuition". This problem seems to occur in interviewing experts in any profession and is one of the most significant obstacles in developing expert systems. Indeed, the knowledge engineering bottleneck is key to understanding the failure of expert systems to realize many proponents expectations.

Many companies providing expert systems consultancy services have distilled their experience into a list of criteria for the viability of expert systems projects (See Box 3 as a example listig). This captures some of the issues involved in deciding whether or not it is possible to build an expert system straightforwardly. However, the benefits of a potential expert system should be considered in addition to the cost. Those that may be quantifiable include more uniform decision making, and the reduction both in errors and in the workload of experienced high-cost staff.

Expert systems may also be adopted because of a scarcity of knowledgeable staff. For example, they have been deployed for use by technical support staff who provide 24-hour helpdesk services for banking software suppliers. Some of the major helpdesk software suppliers such as Siebel provide rule-based systems embedded within their software packages.

Relatively few companies market software tools for building expert systems. This is partly because many of the organizations that do use expert systems technology develop rule-based reasoning within existing software using standard programming languages such as C, C++ and Java. And a number of software systems have embedded rule-based systems that are used without the user even being aware they are accessing an expert system.

Attar Software is now one of the more prominent suppliers of expert systems packages in the UK with its XpertRule toolset. An example of its use is as the basis of the Blending Requirements Expert System for United Distilleries. The system is used to advise on the types of whisky required for their whisky blending. The company has barrels of whisky distributed around 49 warehouses in Scotland, and each week 20,000 casks are moved between them for blending. The expert system is used to optimize the management of this process by choosing efficient combintations of movements.

Another user of XpertRule is Norwich Union who have developed an Expert Underwriting System. This has been running for a few years and has processed almost 100,000 life proposals with 100% conformance with the specified underwriting rules. The Expert Underwriting System has met its original objective of achieving 50% automated underwriting and referring only 50% to head office.

One of the most recent customers of Attar is The New Zealand Social Welfare Department, which used XpertRule to build a welfare advisor for use by income support staff who deal with questions of eligibility, allowances and benefit amounts. While the legislators strive to make the benefits, allowances and taxes as simple to calculate as possible, the net result, in a typical case, is an extremely complex calculation. Typically, an officer of the department will have a folder with 50 pages of notes and rates to guide them through the calculation.

This complexity denied the department the ability to give quick consistent advice, either over the phone or at the counter. Undoubtedly, clients were frustrated by the delays and errors in these manual calculations. The XpertRule calculator gives the officer a simple interface to enter the clients circumstances and calculate the eligible benefits, allowances and

amounts, including printing an advice note. The speed of this calculator also allows advice to be given by telephone operators, now a major part of the Department’s new strategy. The calculator also provides an explanation of the calculation. Furthermore, the robust rule-management techniques built into XpertRule make the system easily maintainable by an experienced programmer.

Whilst a small number of niche suppliers such as Attar Software are making a living out of expert systems, it does raise the question of what happened to the wilder predictions of expert systems uptake. Well the answer is that a number of new technologies have been developed to address the shortcomings of expert systems (See Box 4). These new technologies to been to address the problem from a number of angles: (1) Acquiring knowledge using automated acquisition techniques; (2) Use more sophisticated models of the uncertainty that is inherent in most kinds of knowledge; and (3) Shifting from developing systems that give expert conclusions to systems that enter into a dialogue with the user to advise rather than dictate. So whilst expert systems may be limited they have spurred the development of range of more sophisticated AI technologies for knowledge management.

A common definition of an expert system is a software system that emulates the problem solving behaviour of an expert in a restricted domain. In practice, it is a software system that codifies some of the knowledge of an expert in rules of the following form: "IF X1 & … & Xn THEN Y" where X1,…,Xn are conditions, and Y is some evaluation or action that can be inferred if X1,…,Xn are true. A simplistic example is "IF there are a lot of grey clouds in the sky, THEN take an umbrella". An expert system may include many such rules.

Developments of this idea include the facility to reason with sequences of these rules and to associate numerical uncertainty values with rules. In some focussed application, where uncertainty is limited, this representation can be useful. But in general we need more sophisticated technology to represent and reason with knowledge.

1. Resolution of the problem must be important for the business.
2. Problem arises from scarcity of expertise within the business.
3. Problem is recurrent.
4. Problem is too difficult for non-experts yet manageable for experts.
5. Specific expertise, and not just common sense is needed to solve the problem.
6. Resolution of the problem demands reasoning and not just calculation.
7. The problem area has clear boundaries and is well understood by experts.
8. Knowledge of the problem does not change constantly.
9. Human experts are available to impart and codify their expertise.
10. The problem size is modest.

Ovum is an independent UK-based IT research and consultancy company. In 1988, they stuck their neck out and predicted that the expert systems market would be $4.3 billion by 1992. Well you don’t need to pay £500 to buy another industry report to know this was extremely wide of the mark. In their research, Ovum contacted a number of orgnaizations who had experimented with expert systems, and believed that on the basis of these pioneers, there would be wide spread uptake. This failed to materialize, and as a result most of the key suppliers they analysed went bust in the following couple of years.

Whilst experts sytems based on rules have limitations, there has been considerable progress in artificial intelligence technologies that take us a few steps closer to the goal of representation and reasoning with sophisticated expert knowledge. Some of the technologies to watch are:

Bayesian networks: These are probabilistic models based on directed graphs capturing causal relationships between a number of variables being modelled. These can provide very accurate tools for predication and diagnosis. Microsoft is a big supporter basing various diagnostic tools on the technology.

Argumentation system: These are models based on capturing arguments for and against inferences. For professionals needing decision-support, seeing the inter-relationships between evidence and viewpoints can be much more valuable than just being given a computer-generated decision.

Machine learning: These are techniques for deriving useful knowledge from data or examples. Data mining is a key approach that looks for using useful patterns in databases. By using machine leanting, some of the problems of the knowledge engineering bottleneck can be obviated.