Reddit says that it doesn’t want companies scraping the site for AI. Microsoft says it’s not doing that.
  • Moonrise2473
    10·
    1 month ago

    for common people they respect and even warn a webmaster if they submit a sitemap that has paths included in robots.txt