Artifacts Gallery Guilds Search Wiki Login Register

Welcome, Guest. Please login or register. - Thinking of joining?
May 24, 2026 - @308.54 (what is this?)
Activity rating: Four Stars Posts & Arts: 66/1k.beats ~ Boop! The forum will close in 692.beats! Random | Recent Posts | Guild Recents
News: :wizard: You can make anything on the web! :wizard: Guild Events: Spring Themed Projects

+  MelonLand Forum
|-+  Life & The Web
| |-+  ✁ ∙ Web Crafting
| | |-+  ☔︎ ∙ I need Help!
| | | |-+  Stop my blog from Google search result?


« previous next »
Pages: [1] Print Embed
Author Topic: Stop my blog from Google search result?  (Read 61 times)
chilblands
Casual Poster ⚓︎
*
View Profile WWW


make it happen or regret
⛺︎ My Room

Guild Memberships:
Artifacts:
Joined 2026!
« on: May 23, 2026 @295.63 » Embed

One of my blog on BearBlog went trending, when I check my referrers, I can see Google and Facebook. I am guessing my posts ended up on the Google search result. Is there a way like robot.txt to hide me from Google and other big search engines like Bing and Yahoo?

edit: this is my robot.txt
Code
User-agent: *
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Omgilibot
Disallow: /
User-Agent: FacebookBot
Disallow: /
User-Agent: Applebot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: ImagesiftBot
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Omgili
Disallow: /
User-agent: YouBot
Disallow: /

edit:edit: I just added this <meta name="robots" content="noindex, nofollow">
via: this, I hope no more Google jumpscare!!
« Last Edit: May 23, 2026 @304.47 by chilblands » Logged

IDK WHAT I'M DOIN HEHE XD

Artifact Swap: I met Dan Q on Melonland!Devil Heart
shouldseethis
Newbie ⚓︎
*
View Profile WWW

⛺︎ My Room

Artifacts:
Joined 2026!
« Reply #1 on: May 23, 2026 @370.60 » Embed

how else would we find you?  :melon:
Logged
mrparker
Casual Poster ⚓︎
*
View Profile WWW

⛺︎ My Room

Guild Memberships:
Artifacts:
Joined 2026!
« Reply #2 on: May 23, 2026 @395.63 » Embed

You'll want to add the bots listed here if you want to block every google bot:

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

The only entry you have (Google-Extended) relates to Google's AI training crawler. To block all usage of AI from Google - you'll have to add some of these (or all):

https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers

However, the most reliable way to block these types of scrapers is to block IP ranges - depending on your host, you'll have to add firewall rules to block those.
Logged
Pages: [1] Print Embed 
« previous next »
 

Melonking.Net © Always and ever was! SMF 2.0.19 | SMF © 2021 | Privacy Notice | Send Feedback | Supporters ♥ Forum Guide | Rules | RSS | WAP | Mobile


MelonLand Badges and Other Melon Sites!

MelonLand Project! Visit the MelonLand Forum! Support the Forum
Visit Melonking.Net! Visit the Gif Gallery! Pixel Sea TamaNOTchi
MelonLand @000

Want to Login or Join ?

Minecraft: Online
Join: craft.melonking.net