# File System Crawler

We need to bring it to V4 stack.

## About distributed crawling.

I guess it makes sense when dealing with huge file systems.  Plus, some
of the supported file systems are in fact Internet services, like CMS,
S3, etc.

## Questions

* Do we investigate merging it with the Web Crawler?  Different concepts,
  maybe not realistic (like the concept of "directories").

