Discussion
Loading...

Discussion

  • About
  • Code of conduct
  • Privacy
  • About Bonfire
Jürgen Hubert
@juergen_hubert@mementomori.social  ·  activity timestamp 2 days ago

1/ I have a problem, which is: My websites (a #Wordpress site and a #MediaWiki installation) are slow as hell.

So I need to identify the cause. The problem is that I don't know nearly as much about website administration as I ought to be.

I contacted the support people at my website provider, who looked at my (Apache) logs and suggested that my Wordpress site might suffer from a "pingback xmlrpc attack". I did the proposed remedy, which made things a little better. But I don't know enough about reading website logs to identify such problems myself, which I ought to.

So what I am trying to say is: Is there some kind of beginners guide for reading website logs, identifying malicious traffic, and what to do about it?

  • Copy link
  • Flag this post
  • Block
Jürgen Hubert
@juergen_hubert@mementomori.social replied  ·  activity timestamp 2 days ago

2/ Okay, I think I might already have some ideas.

My latest #Apache log has 26,694 lines.

In these 26.694 lines, I have:

- 10,724 access requests from "https://developers.facebook.com/docs/sharing/webmasters/crawler"
- 4.562 access requests from "https://developer.amazon.com/support/amazonbot"
- 3.316 access requests from "https://openai.com/gptbot"

So yeah, I suspect these are the #LLM crawling bots from #Facebook , #Amazon , and #OpenAI who jointly make up for more than half the traffic - and they are hogging the more resource intensive functions, like "Recent Changes" on my wiki.

Fuck those fuckers for causing outages on my websites.

And any suggestions on how to block them (no snark, please - I _am_ new at this.)

OpenAI Platform

Explore developer resources, tutorials, API docs, and dynamic examples to get the most out of OpenAI's platform.
Developer Portal Master

About AmazonBot

Customer facing page of Amazonbot crawler which all web content publishers can refer to.

Meta Web Crawlers

This page lists the User Agent (UA) strings that identify Meta’s most common web crawlers and what each of those crawlers are used for.
  • Copy link
  • Flag this comment
  • Block
Log in

Bonfire Dinteg Labs

This is a bonfire demo instance for testing purposes. This is not a production site. There are no backups for now. Data, including profiles may be wiped without notice. No service or other guarantees expressed or implied.

Bonfire Dinteg Labs: About · Code of conduct · Privacy ·
Bonfire social · 1.0.0-rc.3.15 no JS en
Automatic federation enabled
  • Explore
  • About
  • Code of Conduct
Home
Login