Already Comcast has put new download caps on user systems where I live. I wonder if the cap will include NBC and Comcast's demand services. Microsoft has failed in a TV connection. Set top box makers are also unhappy. You can add to the list consumers with already sagging broadband lines. Imagine three teenagers trying to do Google TV while they do homework. Or the sag in the already stanched bandwidth at airport WiFi access points. Would you sit through commercials-- and video buffering episodes?
Some of these problems would have an effect on any TV-over-broadband providers. That's one part of Apple's cleverness, being captive to a carrier that they can essentially control, as bad as they might be (yes, you, AT&T). Google would have an international appeal, however. It's a big market out there across the planet. Google already knows about the problems via YouTube in terms of localizing and media latency issues. If this becomes popular, latency issues will be exacerbated. Does your GoogleVoice call go thru, or does the video? Do we get to pick?
Obstacle 5: Google itself.
Google also handily competes against itself. It's said that YouTube doesn't make much money, and is likely a money pit. If people (remember 2i's/24hrs?) start to watch ad-supported video, then the drain of resources undertaken with YouTube go down. Google also gets to capture one more piece of your turf.
And Google has turf. Yes they have market capitalization and yes they have smart people. Will they be able to make this reliable and not embarrassing as other Google products have occasionally been? Can they sustain the effort it takes to take on iTunes and RealNetworks, not to mention Amazon, Hulu, and others? Will the product be first rate, or will we have to wait for generations and betas before we really like Google TV? It's unknown. Check back in a year.