By default the voiceovers in Persona 5 will be in English. However, with some DLC you can change that.
To enable the Japanese voiceovers, simply head to the PlayStation Store and look for the Persona 5 page. From there, press down until you get to the Add-Ons area, and locate "Japanese Audio Track". Upon downloading and installing this head into the game.
You can enable the Japanese Audio Track DLC from the New Game menu.
Note that this DLC is free and accessible to all owners of Persona 5. However, it is a 2.9GB download.
Check Out More Persona 5 Coverage:

Embark on an inspiring mission of human connection beyond the UCA. Sam — with companions by his side — sets out on a new journey to save humanity from extinction. Join them as they traverse a world beset by otherworldly enemies, obstacles and a haunting question: should we have connected?