Early in my life I heard the story that the ancient Chinese worked out how to make a jug spout that didn’t dribble down the side. That may or may not be true but I’ve been amazed how many jugs still get that wrong. I’ve used it as a guiding principle throughout my life to make things that actually work. I also try to help others get the knack too.
Unfortunately we humans keep changing things for change’s sake and often produce ineffective products. A couple of years ago I bought a nearly new electric car. Like all modern cars it has ‘safety’ features that well meaning public bodies impose. I soon realised that the safety features were inadequate. For instance the car might attempt to pull you towards a parked car because it had spotted a white line approaching on the other side of the car. Even if it didn’t actually attempt to steer it would peep at you, unless you indicated your minor manoeuvre. While I am a driver who respects the needs of other road users, there is no need to indicate when there are clearly no other users in the vicinity. Also in some situations indicating is actually more confusing for other users, for instance when you are also coming up to a turn that you are not going to take. The car simply does not have the ability to judge the situation as a whole.
That lack of ability to judge led to other worrying decisions by the car. Both drivers had situations where the car suddenly did an emergency stop, much to the consternation of the driver and following drivers. The car quite simply misjudged the space available round the car.
The car also gets very confused by speed limits and what type of road it is. Again this has led to at least one occasion where the car violently swerved towards a car being overtaken because it thought the driver was was moving onto the opposite carriageway, rather than into an overtaking lane.
The well-meaning public bodies have also decreed that you can’t permanently switch the safety features off. That means that, on a drive with several stops, you have to keep switching them off every time. That would be fine if the switching off worked properly, but on this model of car there are inevitable bugs that mean it either hasn’t actually switched them off or, like ours, it switches one of them back on again when you put your foot on the brake to prepare to move.
So much of this inadequate design now resides in software. and one of the troubles with software is that it is easy to change but less easy to fully test. We have also moved into the age where information is provided to users on the go and in multiple ways. This is used to sell products. Unfortunately, in a car, every moment you are trying to work out what the car is telling you is a moment when you are not concentrating on the job of driving.
In the 1970’s I stumbled into making complex software and also the means of assessing how effective it was. The software I made would look at the roughly formatted information it was given and decide how to analyse it, using other hints from within the information and other guideline information that it had available. It would hand over control to another piece of software when it had satisfied a control limit measuring how well it was doing. If it was not able to solve the problem, it would hand control back to humans, if possible giving them information that would help them sort it out.
When testing the software I would set an arbitrary threshold on how effectively the software would perform and test it against that. If it failed to reach the threshold, I’d look to see if I could make it better but I also did that if it met or exceeded the threshold. Of course I had to set a limit on how long I worked at that before I took it to the other humans to see if that was going to work overall.
This model of software development is just as relevant today as it was back then. The software that controls safety features in a modern car is no different, just more complex and dependent on feedback from more sensors to make decisions. It is particularly important to take this into account when when you are thinking about AI but I’ll get to that another time.