# This file specifies what directories and files a robot # should not follow. The intent is to make our web log # files more accurately represent users and not robots. # Also, this file exists so that don't get spurious # file not found messages in the error log. # Server: ??? User-agent: * Disallow: /servlets/ Disallow: /video/ Disallow: /s/ Disallow: /agencies/ Disallow: /wireless/