I'm not sure backwards compatibility is a thing of the past. It is true that the Xbox one and the PS4 are not, but (I know, most people here don't care) the Wii U and the 3DS are. The case of the 3DS is very interesting. It actually has two processors inside, one for each screen, in a way. The processor for the lower screen is fundamentally a DS processor, while the other one is much newer and powerful. The DS processor is used to run the lower screen, normally with much less information, and usually static, as well as sound, internet connectivity, etc. The other processor is used to render the two images needed for 3D in the top screen. The 3DS can play all DS games, but loses some of the 3DS operative system capabilities. There is, in fact, a DS inside the 3DS.
What I'm saying is that the architecture is a huge factor in whether a console is backwards compatible or no. The first PS3s were backwards compatible, and probably that feature was removed with the slim models because of some redesign in the internal architecture (I haven't been able to foun the information).
Why would things change in the future? Well, for starters, the PS4 is x86, so it can't run PS3 software... but if Sony keeps using x86 for the "PS5" or whatever, it's much easier for them to maintain the compatibility.
It is a huge factor in buying a new console for anyone with a large game collection from the previous generation. It also encourages brand loyalty, as you are more and more invested in Sony hardware line. Therefore, I think, at least on the beggining, they'd rather have the console be backwards compatible, but in this case it was impossible without huge size and price distortion