Steve Scott

I’ve been interested in the way brains work all through my life and I’ve written elsewhere about how our eternal vanity has led us to underestimate the brains of other animals. Watching our inevitable rush to put AI everywhere and recently driving a car with elements of steering control has brought back to mind some conclusions I reached many years ago, when researching elements of AI.

This briefing is intended to bring some clarity to this area in the vain hope that it might slow the inevitable rush to implement without understanding the implications. There is a general overview of AI in another briefing.

As far as we can tell there are two groups of thinking processes in our brains. One group is what Daniel Kahneman characterises as the fast thinking group and is almost certainly the earlier group to have developed to enable us to survive. It works amazingly quickly and accurately in most situations but has known failings, especially in the areas of probability and risk assessment. As far as I can tell we still have little knowledge of how it actually works.

The other group of thinking processes is related to logic and reasoning. This is much slower and more tiring. People generally overestimate the amount of this sort of thinking they do and social systems have a bias in that direction too. The big advantage of this set of thinking systems is that they often allow us to keep an eye on the the other set. The two working together are probably what has allowed us to become such a dominant species. This sort of thinking is also what has allowed us to develop things like maths, logic and scientific analysis to allow us to develop things like automation and computerisation.

At some point we realised that some problems we were trying to solve were so complex that plain logic was not enough and we had to adopt more complex processes involving estimation and statistical prediction, as used in Quantum Physics. AI does some stunning things combining such processes with computing’s astonishing speeds.

Unfortunately, as AI has got better at mimicking our fast instinctive brains, it has often been done with insufficient attention to the checking and auditing side of our brain processes. People have always produced computer systems with insufficient attention to detail and practicality, but when there is a particular place on a narrow bridge where your car tries to steer you into the side of the bridge because of a white line nearer than it deems sensible, then things have got very silly. Currently AI is full of such bad pieces of design.