Probably should take into account people with learning disabilities and processing disorders

As an option, definitely. As a default though I too would prefer the standard spoken form if the time is going to be spoken rather than displayed. It's a bit like how simplified wikipedia is a good idea but I prefer regular English to be the default version.