Artifacts Gallery Guilds Search Wiki Login Register

Welcome, Guest. Please login or register. - Thinking of joining?
Autumn, 20 Oct 2026 - @124.90
Forum Activity: Two Stars Random | Recent Posts | Guild Recents
News: :4u: ~~~~~~~~~~~ Guild Events: Weekly zine jam 17: Fashion
Super News: ~~MelonLand's Halloween Profile Design Contest 2026~~
+  MelonLand Forum
|-+  Da Web Revival
| |-+  ✁ ∙ Web Crafting
| | |-+  ☔︎ ∙ I need Help!
| | | |-+  Stop my blog from Google search result?


« previous next »
Pages: [1] Print Embed
Author Topic: Stop my blog from Google search result?  (Read 636 times)
chilblands
Jr. Member ⚓︎
**
View Profile WWW


make it happen or regret
⛺︎ My Room

Guild Memberships:
Artifacts:
Visited on Melon's 10th Anniversary!I got robbed by Dan Q on Melonland!Joined 2026!
« on: Spring, 23 May 2026 @296 » Toggle Font Embed

One of my blog on BearBlog went trending, when I check my referrers, I can see Google and Facebook. I am guessing my posts ended up on the Google search result. Is there a way like robot.txt to hide me from Google and other big search engines like Bing and Yahoo?

edit: this is my robot.txt
Code
User-agent: *
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: Omgilibot
Disallow: /
User-Agent: FacebookBot
Disallow: /
User-Agent: Applebot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: ImagesiftBot
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Omgili
Disallow: /
User-agent: YouBot
Disallow: /

edit:edit: I just added this <meta name="robots" content="noindex, nofollow">
via: this, I hope no more Google jumpscare!!

« Last Edit: Spring, 23 May 2026 @304 by chilblands » Logged

https://chilblands.neocities.org/files/horseicon.gif it's gettin cold in here...

Artifact Swap: Garden snailCursor!funguyShocked SharaDevil HeartI met Dan Q on Melonland!PurbymargiesimpSunny the Puppydancibg monkeWhite pawnJIJIbluejiggle~
shouldseethis
Newbie ⚓︎
*
View Profile

⛺︎ My Room

Artifacts:
Joined 2026!
« Reply #1 on: Spring, 23 May 2026 @371 » Embed

how else would we find you?  :melon:

Logged
mrparker
Jr. Member ⚓︎
**
View Profile WWW

⛺︎ My Room

Guild Memberships:
Artifacts:
Visited on Melon's 10th Anniversary!lighthouse keeper artemisUserJoined 2026!
« Reply #2 on: Spring, 23 May 2026 @396 » Embed

You'll want to add the bots listed here if you want to block every google bot:

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

The only entry you have (Google-Extended) relates to Google's AI training crawler. To block all usage of AI from Google - you'll have to add some of these (or all):

https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers

However, the most reliable way to block these types of scrapers is to block IP ranges - depending on your host, you'll have to add firewall rules to block those.

Logged
chilblands
Jr. Member ⚓︎
**
View Profile WWW


make it happen or regret
⛺︎ My Room

Guild Memberships:
Artifacts:
Visited on Melon's 10th Anniversary!I got robbed by Dan Q on Melonland!Joined 2026!
« Reply #3 on: Summer, 07 Jun 2026 @339 » Toggle Font Embed

how else would we find you?  :melon:

hope and dreams :sleep:

You'll want to add the bots listed here if you want to block every google bot:

https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers

The only entry you have (Google-Extended) relates to Google's AI training crawler. To block all usage of AI from Google - you'll have to add some of these (or all):

https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers

However, the most reliable way to block these types of scrapers is to block IP ranges - depending on your host, you'll have to add firewall rules to block those.
I did not know this! I don't think my host allow firewall rules but the Google bots doc is very useful. Thank you so much for sharing  :transport:


Logged

https://chilblands.neocities.org/files/horseicon.gif it's gettin cold in here...

Artifact Swap: Garden snailCursor!funguyShocked SharaDevil HeartI met Dan Q on Melonland!PurbymargiesimpSunny the Puppydancibg monkeWhite pawnJIJIbluejiggle~
Pages: [1] Print Embed 
« previous next »
 

Melonking.Net © Always and ever was! SMF 2.0.19 | SMF © 2021 | Privacy Notice | Send Feedback | Supporters ♥ Forum Guide | Rules | RSS | WAP | Mobile


MelonLand Badges and Other Melon Sites!

MelonLand Project! Visit the MelonLand Forum! Support the Forum
Visit Melonking.Net! Visit the Gif Gallery! Pixel Sea TamaNOTchi
@000 Melon
Land
Forum Art Hub Chat Webring Explore Random Recent Tenement Arcade Want to Login or Join ? Web Craft Guide Graphic Catalogue Wiki Help I'm Lost! Link Land Newsletters Image Stream Minecraft: Online
Zap!
Passport Sites tamaNOTchi Asatte Add your site?
Aero Archive Onio Cafe 32 Bit Cafe Status Cafe