Screen-Scraping Stock Market Data with HtmlAgilityPack
I’ve been involved with the stock market since the early 80’s when I was a rookie broker with Merrill Lynch in Orlando. In fact probably the main reason I got out of stock trading and into programming was because I spent more time looking at the technical indicators than selling stocks and mutual funds! Back in the 1970’s legendary market guru Norman Fosback developed a “Daily Market Forecast” index that had an uncanny accuracy with predicting the direction of the next day’s market. Most of the items I have in this article were included in that index. There is a page on Yahoo Finance, http://finance.yahoo.com/advances which provides most of these indicators daily. Another page on the WSJ, http://online.wsj.com/mdc/public/page/2_3021-tradingdiary2.html provides the daily TICK and TRIN (Arms index) that are also components of Fosback’s indicator. I put together a console application that can be run daily via Task Scheduler to scrape these values daily and store them in...