Being a (soft) Sapir-Whorfist, I have long thought that it was at least partly due to how easy and reasonable-sounding it is to say to a kid “O-rusu-ban o-negai shimasu” (something like “Please look after the house while I’m out”) as you leave, as it sounds a lot less neglectful than the English “I’ll be back in a couple of minutes” or “Be good while I’m gone”. This nugget of information from East Asia: A Cultural, Social, and Political History that I’ve just come across might also have some relevance though if the habits have long outlasted their roots (as is common in any culture):
“For reasons of etiquette and security, old-style houses had required that someone always be home to greet guests and guard the premises. Western-style front doors made it easy for the housewife to lock the door and go shopping, visit friends or see a movie” (page 480)
Then again, some Japanese parents also seem happy to let their kindergarten-aged kids take the train home without any adult supervision, so maybe it is something more general.