I will weigh in on this one--and I grew up playing baseball and football before I ever picked up a basket ball, golf club or basketball. Baseball was second place for me because I was a kid in the Sixties and early Seventies without a baseball team (I live in South Florida near Miami)--although Spring training season was up the coast or in Tampa and lots of folks went to see the Yankees and others come down here for the Spring session. On the other hand, WE had the MIAMI DOLPHINS which, back then, was the first professional dynasty after Lombardi retired from the Green Bay Packers. Football was king down here because we had legends--and no baseball, hockey or basketball (for that matter).
Now, I loved baseball and the stories and the statistics and all of that. I devoured that stuff and can still tell you more about Mantle, Mays, Aaron, Seaver, Koufax, Feller, Ruth and whatnot than very much about the last ten years of baseball. This is sad, but it is largely true. Free agency played a big role in my view, just as it has in other sports. Baseball was full of glory players who defined their team and, by extension, the town. I mean, Stan "the Man" Musiel is Mr. St. Louis (along with Bob Gibson) and that kind of thing is not common anymore. Derek Jeter has a bit of that with all his years as the Yankees Captain, but baseball has lost a lot of that with free agency and there can be no denying that the PED scandal just gutted the baseball fan base. Yet, I don't think that is the main reason why baseball has been passed up by football and, in my view, basketball now.
To me, the biggest culprits are the automobile and live television. Baseball became our national pastime from playing fields throughout the nation and games heard on RADIO. You could imagine the skillful interplay with the color commentary as the pitcher worked the count, tried to pick off a runner on first base, and whatnot. It was passed on from generation to generation when folks tended to live near their parents and even grandparents in the same town; it was a family affair. With Eisenhower funding and building our incredible interstate highway system in the 1950s and the rise of affordable automobiles, we started going mobile. That family connection of going to the ballpark became part of nostalgia. The sport was left to defend itself based on what it offered compared to other diversions, and that is where live television came in.
Football was fast, men were strong and huge, and the play on the field was usually fairly certain and the season was not that long. It was like watching gladiators for the early generation I sometimes surmise. Baseball players were SKILLED, but these football players were MONSTERS to most folks. When you compared the two sports on the small screen, one looked comparatively slow, meticulous and for those with a lot of time to spend analyzing the "quiet" part of the game. I think the dye was probably cast when the great Johnny Unitas led the Colts to victory of the Giants in the 1958 Championship game--along with the Dodgers leaving Brooklyn for Los Angeles (folks still can't even talk about that without getting angry), followed by the Giants going to San Francisco. Things had changed and baseball has been on defense ever since.