Bots hit and miss
Bots miss:
n XML pages, pdf files
n Dynamically created HTML pages
n Frames-based pages
n New pages or recent updated text
n Some say the Invisible Web is 500
times larger than Web