Who operates it
HostingpaperBot is operated by Hostingpaper Media B.V. for Hostingpaper, an independent trade publication covering domains, hosting, cloud, security, policy, and business.
The bot does not accept third-party jobs, crawl on demand, or index the open web for search. It runs only on our own editorial infrastructure to monitor feeds we have configured for news intake.
User-Agent string
Feed polling requests identify themselves explicitly:
HostingpaperBot/1.0 (+https://hostingpaper.com/about-bot)
If you block or rate-limit automated clients, please allow this User-Agent for the feed URLs you publish. This page is the contact URL referenced in the token.
What it fetches
HostingpaperBot is used for RSS and Atom feed polling only — the scheduled engine:fetch step that checks configured industry feeds for new items.
Typical request profile:
- Method:
GET - Accept: RSS, Atom, and XML media types
- Conditional requests: sends stored
If-None-Match/If-Modified-Sincevalidators when available, so unchanged feeds return304 Not Modified - Timeout: 15 seconds per attempt
- Retries: up to 2 retries on transient failures, with short exponential backoff
- Redirects: follows up to 5 redirects
We do not use HostingpaperBot to spider your site, scrape arbitrary pages, or hammer endpoints outside the feed URLs we monitor.
Full-text article fetching
When a feed item is selected for reporting, a separate HTTP client may retrieve the linked article page for full-text extraction during writing and verification. That step does not use the HostingpaperBot User-Agent; it uses browser-like request headers so publishers are less likely to receive truncated or bot-specific responses.
Before any full-text fetch, we check the site's robots.txt for the article URL. If the path is disallowed for our fetch User-Agent (or for *), we fall back to the feed summary instead of retrieving the page.
For how those sources are used in published reporting, see Editorial standards and How we work.
Frequency and load
Feeds are polled on a schedule driven by our editorial engine, not continuously. Polling frequency depends on how the feed is configured and whether the previous fetch succeeded. Conditional HTTP means most polls are inexpensive when nothing has changed.
If you operate a feed we monitor and need a lower cadence or a dedicated endpoint, contact us with the feed URL and your preference.
Blocking or reporting a problem
If HostingpaperBot is causing a problem on your infrastructure — unexpected traffic, a misconfigured feed URL, or a robots concern — email us with:
- The affected URL or feed address
- Your server or CDN logs showing the request (timestamp and User-Agent help)
- What you would like us to change (stop polling, reduce frequency, fix a redirect loop, etc.)
We review operator reports promptly. You can also use Contact for general editorial enquiries unrelated to bot traffic.
Further reading
- How we work — editorial pipeline from feed intake to publication
- Editorial standards — sourcing and verification rules
- About — the publication and what we cover