I would have to say both for this, these two aspects are the core when it comes to consoles. You have the PS4 and the Xbox One that has improved their specs by a lot. Xbox One went even further, and overclocked their GPU by I believe 53 mHZ which is a pretty decent upgrade. 512MB to 8GB RAM is a huge improvement for the Xbox One, and 256MB to 8GB RAM is big for the PS4 as well. Both GPU's and CPU's has increased significantly, and this guarantees that the consoles will run games at decent, and playable speeds. Without decent specs, consoles would be non-functional, and wouldn't be able to run Minesweeper.
On the other hand, if there are no games, then what's the point of buying the console? Some people decide whether or not to buy a console just because of their library. The better games there are available for the specific console, they will most likely buy it. Right now, I'm sort of leaning towards the Xbox One because of their wide selection of games and exclusives. Their library of games is a bit better when it comes to me, and their specs are fairly similar to the PS4.