I was watching Mad Men recently, and I don't think I'd like it nearly as much as I do if it were set in modern times.
I also like the movies Jet Li did back before he came to Hollywood, the crazy colonial China martial arts epics. I suppose a big part of the fun is discovering things about the past ("The past is a foreign country: they do things differently there.") Period pieces seem richer in meaning somehow, because everything has a meaning. I always like discovering new things, and there is always ample opportunity. For instance, Mad Men had a throwaway line about a Unitarian Minster who was killed in Selma. That was enough information to find James Reeb, a figure in American history of whom I would have otherwise been ignorant.
It also enables tricks you can't play with contemporary stories, like in Shada, when the Doctor brings a cassette of Bonnie Tyler's greatest hits back to 1974,
In general, period pieces tend to be crafted with fussy, meticulous care, which is something I like in a story, and which is why I give them an edge of their non-period counterparts.