By default the voiceovers in Persona 5 will be in English. However, with some DLC you can change that.
To enable the Japanese voiceovers, simply head to the PlayStation Store and look for the Persona 5 page. From there, press down until you get to the Add-Ons area, and locate "Japanese Audio Track". Upon downloading and installing this head into the game.
You can enable the Japanese Audio Track DLC from the New Game menu.
Note that this DLC is free and accessible to all owners of Persona 5. However, it is a 2.9GB download.
Check Out More Persona 5 Coverage: