keenlearner
June 21st, 2007, 03:36 AM
I am learning to build a web robot for my own interest, I saw lots of article building scalable server using completion port which can accept lots of clients efficiently. However, I am thinking of how can I use the features of IO completion method in building client application such as web robot to increase the efficiency of crawling. I want to make request to many servers at the same time, such as maybe I create lot of sockets to request web pages to many servers and then a few worker thread (thread pool) will process the IO completion request (server html response). Is it possible to do this ? Or what other things that I can benefits from using I/O completion for Client TCP ?Thank you