Explainable AI is the hype! Or maybe not so much. But depending on the use case the AI has to be explainable. Imagine if your loan broker rejected you without proper reason and you would have to move out of your house, or if the insurance premium were to be set by a black box with no real way to know what effects the resulting premium. Hence the need for explainable AI systems.

What is a system that provides explainable AI? It is a system that supports their decisions with compelling arguments. In essence, the system should be able to spit out a report that presents the proofs. These proofs should be verifiable without access to the AI system. But how do we create such a system?

One way to create explainable AI systems is to create strong isolation between different types of knowledge. Some knowledge is of factual nature, some knowledge is objects recognized in image material / sound etc. Some knowledge is based on trustworthy sources while others are highly circumstantial. A way to create this division is by using multi-layer networks. A layer is merely a network graph constructed in a way such that it is possible to consolidate individual layers and traverse the multilayered graph.

Each layer has a single knowledge responsibility, such a responsibility could be information about who owns which companies, or what landmarks are present in pictures. Furthermore, each layer includes information about provenance, confidence, verification steps, etc.

For the example about a layer that holds responsibility about the company ownership, we construct the lay by reading the relevant company register. We note that we have high confidence in the correctness of the data represented (this might be different for different jurisdictions). The verification step would be to log in to the registry and very the stated ownership.

For the example about landmarks present in pictures, we can construct the layer by using image recognition techniques. Ie. we would contain the use of black-box techniques (eg. neural networks) to that layer only. Provenance would include information about the source image, confidence would be low to medium depending on the evaluation metrics of the algorithm in use, and verification steps would include visual inspection of the source image.

When combining these layers we can do complex reasoning. We can make chains like Picture P features Landmark L, Landmark L has address A, picture P was taken at time T, Picture P was shared by Person G to support the conclusion that a person was present at a specific place and time. This syllogism is verified manually such that:

  1. It is visually inspected that the picture indeed features the proposed landmark
  2. The landmark is checked to have the address
  3. It is checked that the proposed time of the picture being taken works out
  4. It is checked that the picture was indeed shared by Person G, and that it is possible that he was present when sharing.

The next step would be to add compound layers which is a product of existing layers. Besides enabling more complex reasoning for the system, they would also inherit provenance, confidence, and verification from previous layers.