Iwata has explained it a little bit. The experience is the same whether you buy your game digitally or physically, and that experience is what you are paying for. Whether you buy a disk or download the game, you are paying to get access to the experience. Like a concert or going to a movie. The faster, easiest way for you to get a game is digitally, physical games only still exist for people whose isn't up to snuff, or prefer clutter for whatever reason. If you need a physical game bigger game companies will absorb all the increased costs of physical (trucks, employees, printing disks, etc..) and not charge you an extra penny for it! Lucky you. Smaller companies just cant afford to absorb that cost, so indies largely just release games digitally.
Also it isn't free to make/maintain something like PSN. There is a cost associated with digital. A lot of you pretend digital games just appear on your PS4 from heaven.
Digital games being cheaper would also devalue IP, which is a major problem in the industry. The app store and humble bundles have devalued games to the point where people complain whenever something costs anything. This devaluation is also a huge part of what is turning gamers into entitled, whiny, butt hurt children.
Lastly it doesn't piss off retailers.