ConvertlyftBot

ConvertlyftBot is the crawler behind Convertlyft’s site audits. If you found it in your logs, this page says what it is doing and how to control it.

It identifies itself with this user agent:

ConvertlyftBot/1.0 (+https://convertlyft.com/bot)

Before this name it identified itself as SEO-Spider-Pro/1.0. Rules you wrote for that name are still honoured until 9 January 2027.

Why it visits

It reads a site when someone asks Convertlyft to audit it: a Convertlyft customer auditing their own site, or a visitor running the free SEO scan on convertlyft.com (16 pages at most). It reads pages the way a search engine does, to report problems such as broken links, missing titles or slow responses back to the person who asked.

How it behaves

• It reads robots.txt first and follows it, and reads it again at least once a day.
• It honours Crawl-delay.
• By default it keeps at most 4 requests open to one host and waits 200 ms between requests to that host.
• When your server answers 429 or 503, it slows down: the wait between requests to your host doubles, up to 10 seconds. It honours Retry-After (up to 30 seconds).
• Pages it loads in a browser to see the rendered result carry the same user agent.
• To check whether AI assistants can read a site, it asks a few of the site’s pages while identifying as those assistants’ crawlers (for example GPTBot). It does not send these requests as an agent your robots.txt disallows.
• When it checks whether a link on your site to another site still works and that site refuses it, it may ask that other site once more as an ordinary browser. It never does this to the site being audited.

How to block or limit it

Add a group for it to your robots.txt:

User-agent: ConvertlyftBot
Disallow: /

Or allow it but slow it down:

User-agent: ConvertlyftBot
Crawl-delay: 2

We do not publish a list of IP addresses. The crawler runs on cloud infrastructure whose addresses are shared and change, so an IP rule is not a reliable way to identify it. Use the user agent and robots.txt.

Contact

If ConvertlyftBot is causing a problem on your site, write to support@convertlyft.com with your domain and roughly when you saw it.

Convertlyft LLC, Dubai, United Arab Emirates