What does Octocular do?
Octocular primarily provides spidering functions to a new type of search engine which is still in development.However, it also serves a number of other functions. If you've seen Octocular's user agent in your web server's logs and its search patterns show an interest in your Javascript files, then that's probably because it's scanning your site for viruses and malware. Octocular provides the malware scanning facilities associated with the Fair Pay Northeast Jobs List. At the present time this is the most likely cause of Octocular's interest in your site.
Some of Octocular's software is also used by the free web page change notification website, Changewatching.
Octo is a complicated but flexible beast and has been (and is) used for a wide variety of private research purposes, including somewhat insecure (but entertainingly unusual) cryptography and linguistic investigation. Octocular's main database (the 'search core') borrows ideas from Markov chains, which I independently re-invented, influenced (20 years ago) by reading the source code of the Amiga NIALL software (Non-Intelligent AMOS Language Learner). The search core currently houses a body of work greater than the largest publicly available corpus of written English AI training data.
As is generally the case with anything to do with programming, linguistic Markov chains are actually much simpler than Wikipedia's convoluted explanation would have you believe.