One thing Reddit dominates on is search results. I’m looking things up and seeing so many links to reddit, which I guess is going to help keep that place relevant (unless those subreddits stay dark).

I wondered how Lemmy and this fed thingy stuff all works for that? With more posts can we expect to see people arriving through search results?

  • fizzym4d@lemmy.fmhy.mlEnglish
    0·
    2 years ago

    Your “off-topic” sounded pretty cool to me! I love that that is something anyone can do when hosting a lemmy instance. You get to choose if it’s searchable on the web! Obviously there are search engines which ignore the no scraping/indexing header, but the rest of what you did should counteract that, noice.

    • wpuckering@lm.williampuckering.comEnglish
      3·
      2 years ago

      Yeah, if you’re running something yourself, you can do pretty much whatever you want in order to protect it. Especially if it’s behind a reverse proxy. Firewalls are great for protecting ports, but reverse proxies can be their own form of protection, and I don’t think a lot of people associate them with “protection” so much. Why expose paths (unauthenticated) that don’t need to be? For instance, in my case with my Lemmy instance, all any other instance needs is access to the /api path which I leave open. And all the other paths are behind basic authentication which I can access, so I can still use the Lemmy web interface on my own instance if I want to. But if I don’t want others browsing to my instance to see what communities have been added, or I don’t want to give someone an easy glance into what comments or posts my profile has made across all instances (for a little more privacy), then I can simply hide that behind the curtain without losing any functionality.

      It’s easy to think of these things when you have relevant experience with things such as web development, debugging web applications, full stack development, and subject matter knowledge in those and related areas.

      • Arinshot@lemmy.worldEnglish
        1·
        2 years ago

        I’d be interested in how you did this, this seems like one of the best ways I’ve seen for securing a lemmy instance.

        • maynarkh@feddit.nlEnglish
          1·
          2 years ago

          One easy way to do that is to set up something like Nginx as a reverse proxy in front and forward /api clean, but forward everything else with basic auth.

          The steps broadly would be:

          And you’re done.

        • wpuckering@lm.williampuckering.comEnglish
          1·
          2 years ago

          I have a single Nginx container that handles reverse proxying of all my selfhosted services, and I break every service out into its own configuration file, and use include directives to share common configuration across them. For anyone out there with Nginx experience, my Lemmy configuration file should make it fairly clear in terms of how I handle what I described above:

          server {
  include ssl_common.conf;
  server_name lm.williampuckering.com;
  set $backend_client lemmy-ui:1234;
  set $backend_server lemmy-server:8536;
  
  location / {
    set $authentication "Authentication Required";
    include /etc/nginx/proxy_nocache_backend.conf;
    
    if ($http_accept = "application/activity+json") {
      set $authentication off;
      set $backend_client $backend_server;
    }
    if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
      set $authentication off;
      set $backend_client $backend_server;
    }
    if ($request_method = POST) {
      set $authentication off;
      set $backend_client $backend_server;
    }
    
    auth_basic $authentication;
    auth_basic_user_file htpasswd;
    proxy_pass http://$backend_client;
  }
  
  location ~* ^/(api|feeds|nodeinfo|.well-known) {
    include /etc/nginx/proxy_nocache_backend.conf;
    proxy_pass http://$backend_server;
  }
  
  location ~* ^/pictrs {
    proxy_cache lemmy_cache;
    include /etc/nginx/proxy_cache_backend.conf;
    proxy_pass http://$backend_server;
  }
  
  location ~* ^/static {
    proxy_cache lemmy_cache;
    include /etc/nginx/proxy_cache_backend.conf;
    proxy_pass http://$backend_client;
  }
  
  location ~* ^/css {
    proxy_cache lemmy_cache;
    include /etc/nginx/proxy_cache_backend.conf;
    proxy_pass http://$backend_client;
  }
}

          It’s definitely in need of some clean-up (for instance, there’s no need for multiple location blocks that do the same thing for caching, a single expression can handle all of the ones with identical configuration to reduce the number of lines required), but I’ve been a bit lazy to clean things up. However it should serve as a good example and communicate the general idea of what I’m doing.