DominicaSearchBot

The crawler that builds this index. It reads publicly reachable pages of websites belonging to the Commonwealth of Dominica — ministries, councils, companies, schools, churches, media, hotels — so that people can find them in one place.

How it identifies itself

DominicaSearchBot/1.0 (+https://dominica1.com/search/bot)

What it respects

• robots.txt, including Crawl-delay and rules written for DominicaSearchBot specifically;
• <meta name="robots" content="noindex"> — such a page is fetched but never enters the search results;
• nofollow — links on such a page are not followed;
• a pause of at least 1.5 seconds between requests to the same host, longer if your robots.txt asks for it.

Pictures

Images are indexed together with their captions — alt, title, <figcaption> and the photo credit — because on a national archive the caption is often the only place a person, a village or an event is named. The original file stays on your server: we keep a small thumbnail and always link back to the page it came from.

To keep the crawler out

User-agent: DominicaSearchBot
Disallow: /

The rule takes effect on the next visit — robots.txt is re-read regularly. To have a site removed from the index straight away, write to contact@dominica1.com.