IndexNow, Yandex and Bing: The Search Traffic Adult Sites Overlook

Most adult SEO advice is about Google. But on our own network, Google crawls our weaker sites a few hundred times a month, while Yandex and Bing crawl them thousands of times. In one baseline week Yandex sent our network around 550 search clicks, more than Google sent most of those sites. If you only optimise for Google, you’re ignoring the engines that actually visit you.

What IndexNow is

IndexNow is a simple protocol: when you publish or change a page, you tell the search engines its URL directly instead of waiting for them to find it. Bing, Yandex, Seznam and Naver accept it. Google does not. You host a key file on your site to prove ownership, then send URLs in batches of up to 10,000.

POST https://yandex.com/indexnow
{
  "host": "yoursite.com",
  "key": "your-key",
  "keyLocation": "https://yoursite.com/your-key.txt",
  "urlList": ["https://yoursite.com/video/123/title", "..."]
}

A 200 or 202 answer means the batch was accepted. It does not mean the pages will be indexed, only that the engine knows about them.

Send to each engine directly

The protocol says that submitting to one engine shares the URLs with the others. In practice we found this unreliable, and worse: the shared endpoint and Bing’s own endpoint share one rate limit. When it closes, everything sent through it is lost. We now submit to every engine’s own endpoint separately:

• https://www.bing.com/indexnow
• https://yandex.com/indexnow
• https://search.seznam.cz/indexnow
• https://searchadvisor.naver.com/indexnow

On a typical day all four answer 200 or 202. When one of them rate-limits you, the others still get the URLs.

The rate limit that silently ate our submissions

If you run several sites from one server, this is the part that matters. All our sites used the same IndexNow key from the same IP address. So the rate limit was network-wide, not per site.

We proved it one afternoon: one site submitted two batches of 10,000 URLs and got HTTP 429. Half an hour later a different site submitted a single URL and also got 429, and was still blocked forty minutes after that. Each site’s job only knew its own URLs. The only symptom was a “rate limited, will retry” line in a log nobody read.

Simply capping each site didn’t fix it either. The allocation became first-come: on one day the two sites that ran earliest each used their full allowance, and two later sites that wanted 1,400 URLs each got zero.

A setup that keeps working

• One daily budget for the whole server, split between your sites, rather than each site sending as much as it wants.
• Send only what is new or changed. Keep a record of URLs already announced. Re-announcing your whole catalogue every day burns the budget and teaches the engine to ignore you.
• Treat a 429 as a signal. Stop sending to that engine for the rest of the day, don’t retry in a loop.
• Log the answer per engine (for example bing=200 yandex=202 seznam=200 naver=200) and alert when an engine has failed for more than a day.

Yandex Webmaster is worth the setup

Verify every site in Yandex Webmaster and submit the sitemap there. It shows you pages in search, the queries you rank for, and crawl problems Google never mentions. Two things we learned:

• Old checks linger. Yandex flagged several of our sites as slow long after the problem was fixed, because its last check was weeks old. Don’t panic over one flag. Check its date.
• Its API can be unreachable for days. The Webmaster API (used for recrawl requests and statistics) was down for more than a week while IndexNow to Yandex kept working normally. If you automate recrawl requests, record when they last worked, and keep IndexNow as your main way of announcing new pages.

Bing Webmaster Tools

Bing’s own tools accept your sitemap and show its own backlink data, which is a useful free second opinion next to paid SEO tools. Bing’s index also feeds several AI assistants, so being in Bing increasingly means being visible there too.

Checklist

• Host an IndexNow key file and verify it loads.
• Submit new and changed URLs daily to each engine’s own endpoint.
• Share one budget across all sites on the same server and key.
• Log every answer per engine and alert on repeated failures.
• Verify every site in Yandex Webmaster and Bing Webmaster Tools and submit sitemaps there.