This novel is a time regression story, not really an isekai. The author is ham-fisted in how he depicts the old man; he walks with a cane and refers to younger folk as “lad” or “lass”, and generally speaks in a stiff manner. This doesn’t change when he’s spirited away into his younger body. My guess is that the author can’t figure out any other way to convey the character’s age. This is a continual problem in entertainment media where the writers want to depict someone ancient and wise: since the writer aren’t ancient and wise, they have no idea how to write such characters. Anyway, this makes the character hard to relate to or like.
The problem is compounded by the narrator, who uses his old man voice for the younger version of himself, a monotone, gravelly voice with a bad British accent. This was a poor choice by the voice actor, who should have tried to humanize the character.