HTTP referer spoofing, cookies, User Agent strings
There was a post on one of the groups recently by a developer who was making a number of WebRequests for various pages, claiming that one of them would strangely fail. Yet, this individual stated that if he would paste the respective URL into his browser, that page would come up just fine. There are several things that could come into play here with various sites: 1) Many sites will reject a request that doesn't match a particular one or more User Agent strings ( Here are some samples , if your memory is rusty). So you can add the UserAgent header to the WebRequest. I've even seen some wise-asses who detect Internet Exploder and give you a nasty message about how immoral you are and you should go download Firefox to become a real person and how dare you try to view my site with ..etc... (Listen, Pal: I already figured out you are a Webtard, so I'm not going to bother to fire up my copy of Firefox to see your page, which I already know is worthless. Besides, your little shen...