I just spun up Lemmy on my Kubernetes cluster with nginx-unprivileged and ingress-nginx. All is well so far! I’m thinking about posting the Kustomization manifests and continuing to maintain and publish OCI’s per version release of Lemmy.

  • gabe565@lemmy.cook.gg
    link
    fedilink
    English
    arrow-up
    2
    ·
    edit-2
    1 year ago

    Yep I’m still working on a helm chart. Currently, each service is deployed with the bjw-s app-template helm chart, but I’d like to combine it all into a single chart.

    The hardest part was getting ingress-nginx to pass ActivityPub requests to the backend, but we settled on a hack that seems to work well. We had to add the following configuration snippet to the frontend’s ingress annotations:

    nginx.ingress.kubernetes.io/configuration-snippet: |
      if ($http_accept = "application/activity+json") {
        set $proxy_upstream_name "lemmy-lemmy-8536";
      }
      if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
        set $proxy_upstream_name "lemmy-lemmy-8536";
      }
      if ($request_method = POST) {
        set $proxy_upstream_name "lemmy-lemmy-8536";
      }
    

    The value of the variable is $NAMESPACE-$SERVICE-$PORT.
    I tested this pretty thoroughly and haven’t been able to break it so far, but please let me know if anybody has a better solution!

    • anthr76@lemmy.kutara.ioOP
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      1 year ago

      Firstly, awesome to hear you’re using bjw-s app-template helm chart. He’s my good friend and former coworker :)

      I’m also doing what @seang96@exploding-heads.com is doing.

      While I don’t consider this completed yet I have posted how I’m doing things so far here

      • gabe565@lemmy.cook.gg
        link
        fedilink
        English
        arrow-up
        3
        ·
        edit-2
        1 year ago

        That’s awesome! I love his Helm chart. It’s the most impressive Helm library I’ve ever seen. I maintain a bunch of charts and I exclusively use his library chart :)

        I just mentioned in a response to @seang96@exploding-heads.com, but I feel like deploying a separate nginx is probably cleaner, I just didn’t want another SPOF that I could break at some point in the future.

    • seang96@exploding-heads.com
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      Doing this made / path with Lemmy ui break, but posts and comments were actually updating according to logs. i set it up with a nginx container behind my ingress now and it appears everything works besides my comments and posts not being federated even though I can curl the links for troubleshooting federation without issue. Got any ideas?

      • gabe565@lemmy.cook.gg
        link
        fedilink
        English
        arrow-up
        2
        ·
        1 year ago

        Hmm I’m not sure! That code snippet should only affect routing conditionally. When you added the configuration snippet, did your ingress logs show the requests to / going to the frontend or backend?

        An nginx container behind ingress seems cleaner, I just didn’t want to add another point that I could possibly break lol

        • seang96@exploding-heads.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          I just found out my posts are finally going through without any changes using the nginx proxy container to nginx ingress method! If you do have a way to do it all directly through nginx I’d love to see how it’s all done, maybe I was missing something outside of the snippet you posted.

          Since it’s currently working I’ll look into spinning a test instance up when I get a chance and play with that ingress annotation.

          • gabe565@lemmy.cook.gg
            link
            fedilink
            English
            arrow-up
            2
            ·
            1 year ago

            Awesome! A separate nginx container is fine, so if it’s working I’d probably leave it. I’ll look through and see if there’s anything I missed in my comment though for brevity.