I did some research and here are some interesting stats on Federer. First compare his last three years:
2012: 71-12 (86%)
2013: 45-17 (73%)
2014: 44-9 (83%)
Now how did he fare against different ranked players? This is where it gets interesting.
Vs. Top 10 Players
2012: 16-9 (64%)
2013: 4-10 (29%)
2014: 9-4 (69%)
Vs. #11-100 Players
2012: 49-3 (94%)
2013: 33-5 (87%)
2014: 28-5 (85%)
Vs. #101+ Players
2012: 6-0 (100%)
2013: 8-2 (80%)
2014: 7-0 (100%)
So here we can see the big differences in the three seasons. In 2013 Federer was terrible against top 10 opponents, winning only 29% of his matches, while 2012 and 2014 are very similar. We can also see that in 2013 Roger did something he didn't do in either of the other two years, lose not one but two matches to players outside of the top 100.
Why was 2012 better than 2014 so far? It is less significant than we might think. He has had similar results so far against top 10 players and the same against players outside of the top 100, but the main difference is the players ranked #11-100 - Roger was solidly more effective against them in 2012 than he has been in 2014. In 2012 he lost to only three players in that range: #17 John Isner, #34 Andy Roddick, and #87 Tommy Haas. In 2014 he's lost to five players in that range: #15 Jo-Wilfried Tsonga, #17 Ernests Gulbis, #21 Kei Nishikori, #47 Jeremy Chardy, and #60 Lleyton Hewitt.
One could argue that Tsonga is really a top 10 player, so the numbers would change a bit.
Anyhow, the main take-away for me is that his biggest problem in 2013 is that he was more prone to really bad upsets (Stakhovsky and Delbonis in particular), and not nearly as competitive against top 10 players. 2014 is almost as good as 2012, but he is still prone to be occasionally upset, just be superior players. The lowest ranked player Roger has lost to in 2014 is #60 Lleyton Hewitt. After that it is #47 Jeremy Chardy, then his other seven losses are all #21 or better.
His record in 2012 was even more remarkably consistent: He lost only twice to players ranked lower than #17, that being #34 Andy Roddick and #87 Tommy Haas - two players whose highest level is much greater than their rank at the time.
It is also worth noting that in 2012, Roger was 6-6 against Djokovic, Nadal and Murray--2-3 against Novak, 1-1 against Rafa, and 3-2 against Andy. In 2013 he was 0-7 against them (although only faced Novak and Rafa), while in 2014 he is 3-3 against them (2-2 against Novak, 0-1 against Rafa, 1-0 against Andy).
The other big difference, of course, is his record in Finals. In 2012 he was 6-4, in 2013 1-2, and in 2014 he is 2-5.
So despite my initial post, Roger is holding his own against the other elite players. In terms of overall record, he (83%) isn't quite on the level of Novak (88%) but he's just a hair below Rafa (85%)--and was tied with him until his loss to Tsonga today--and quite a better better than Andy (71%).