Do you know that generally these bot safety providers will serve Google and different serps noindex directives? Google’s John Mueller mentioned this on X, saying, “Typically there’s bot-protection (or a login, interstitial, and so forth) triggering that has a noindex on it.”
He additionally added that it could be higher for them to serve 503 server standing codes. “Higher can be to make use of 503 for server-side blocks like bot-protection,” he wrote.
Ultimately, the individual he replied to was having a special difficulty inflicting the noindex, it was not bot safety. He mentioned it was “some shenanigans with NextJS doing shopper aspect rendering as a substitute of server rendering.” “The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering, which is why I couldn’t discover it and thought it was a GSC bug,” he added.
This was the error he was seeing:
Listed here are these posts:
wtf is happening right here? why is Google not indexing this one?
They are saying there is a “noindex” tag, however there is not one. robots.txt appears to be like good too.
Anybody? #search engine marketing assist pic.twitter.com/DYzJsLQ4Yk
— Guilherme (@goenning) December 26, 2023
Thanks John, ultimately discovered it due to @HamilcDev
Some shenanigans with NextJS doing shopper aspect rendering as a substitute of server rendering
The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering,… https://t.co/NffU7suVLM
— Guilherme (@goenning) December 26, 2023
Discussion board dialogue at X.
