Hi Lemmy!
I’m studying webdevelopment, i love the concept of the fediverse and activitypub, but i still see it as really new and needing to get better of course.
I will just throw out some first impressions, these are just from a first sight so i’d be happy if you make me notice i’m saying something false or inaccurate.
As i said in the title i would really love to collaborate, so i’m just exposing stuff i would love to work to, just to see what others think about it and if they could be theoretically possible.
The Fediverse claims to be highly interconnected, yet i cannot use the account i have on a mastond instance on Lemmy, and neither the opposite. I don’t see why there has to be many instances of the same thing and so many accounts needed to access, for example if i want to browse Kbin with my Lemmy account, i won’t be able to, but i can create a new account and find some content of Lemmy on it or browse privately.
The system of mentions is quite confusing as well and i don’t see these differentiation of istances a pro for the network, if an istance goes down the content on that istance goes down as well.
More that having to reference each server istance, it would maybe be an idea to have one single domain holding all the istances, much like IPFS and blockchain nodes. More servers hosting the content for the same platform, backed up by a pinning service such as Pinata for IPFS, not different servers holding different content loads.
I want to be able to subscribe to one single Fediverse account, and be able to use it in every different federated platforms.
I’ll take the user login as an example: If i login in a federated platform, IPFS would perform a node lookup and find my account details on one of the federated servers, instead of having to specify which server i want the account info to be coming from.
All of this would still need to be backed up: 1- because node lookup is quite slow 2- because if one or more istances go down, there has to be a reliable and fast backup available to keep the platform running, even if the content is already duplicated in more IPFS servers.
Of course with this option there HAS to be a central server with all the data on it in order to make them readily available and make the user experience acceptable.
Another possibility would be using really light languages (such as Rust that doesn’t have a runtime for example) to build the platforms and see what the performances are WITHOUT a pinning service and so without the need of a central server.
Please don’t be harsh on me, i’m just jotting down some ideas!
Quick comments:
i cannot use the account i have on a mastond instance on Lemmy, and neither the opposite
I’ve seen people comment here using Mastodon accounts. Not sure how they had done it, but it’s definitely doable. Not sure about using a Lemmy account on Mastodon, but it really should be doable.
if i want to browse Kbin with my Lemmy account, i won’t be able to, but i can create a new account and find some content of Lemmy on it or browse privately.
That is a temporary issue caused by the main kbin instance deploying defensive measures against the onslaught of new users - they’ve hidden themselves behind Cloudflare CDN. Once they figure out the performance issues and expose themselves publicly the federation will come back.
The system of mentions is quite confusing as well and i don’t see these differentiation of istances a pro for the network, if an istance goes down the content on that istance goes down as well.
Can’t argue the mention bit, as I’m probably too old to care about having a server bit added to the username - that’s just normal to me. The instance going down is no different that a subreddit going private. It can happen on a whim of the mod/owner. And just like with a subreddit content being cached externally, other Lemmy nodes will keep cached copies of existing posts for some time. The only real issue here would be user accounts that would become inaccessible, but even that may be sorted by people more intelligent than me.
have one single domain holding all the istances
No. Just no. If you prefer having single points of failure - either stick to Reddit or spin up a private unfederated Lemmy instance.
I want to be able to subscribe to one single Fediverse account, and be able to use it in every different federated platforms.
I believe we’re very close to that. You can already access many federated things with a single account.
If i login in a federated platform, IPFS would perform a node lookup and find my account details on one of the federated servers, instead of having to specify which server i want the account info to be coming from.
This is the cost of owning your data yourself. Fully. I prefer hosting my own stuff.
I don’t know IPFS stuff all that well, can’t comment.
Of course with this option there HAS to be a central server with all the data on it in order to make them readily available and make the user experience acceptable.
No. Just no.
No. Just no. If you prefer having single points of failure - either stick to Reddit or spin up a private unfederated Lemmy instance.
Sorry i didn’t explain myself very well.
By central server i meant a pinning service, but that could even be hosted by the already existent fediverse istances.
I’ll paste one answer i added below:
I meant sharding. that’s what IPFS is about, the files on it are sharded between different computers accross all over the world that request the specific resource. When you ask through ipfs for a resource, it would perform a lookup and see if that’s available and in which node, then serve it to the client and cache it on its node. Content can be pinned to specific nodes in order not to be deleted through garbage collection, and that was what i was meaning by “central server”, there are services (like Pinata) doing just that: pinning content on fast servers and making it readily available. But i guess those nodes can also be the current fediverse istances, i had a central server in mind more due to that and because of the fact that smaller servers (available to the little associations that host fediverse istances) couldn’t probably keep up with something like sharding right now, but i don’t really know.
I think something very similar is done already. If I look a thing up on another instance - my instance saves it. If the origin goes away - I can still see it.
Not the kind of sharding you’re talking about, but the federated nature of stuff does prevent something popular from just dropping off the surface of earth.
I tried to follow a community from Mastodon, an the connection worked without issue. Every answers was it’s own post tho, so its a bit messy (but there might as well be a solution to that)
I’ll try and get through as many of your questions as I can:
The Kbin - Lemmy connection should be fixed as soon as kbin.social sorts out their cloudflare issues (it’s blocking federation messages).
Mastodon accounts can follow communities and post here. We can respond to mastodon accounts that post here. Until/unless we have a “Subscribe to user” feature, following mastodon accounts doesn’t make a huge amount of sense.
If an instance goes down temporarily, all other instances with subscribers can still interact with the cached content, which will get synced back once that instance comes back up. If it goes down permanently, yeah. Nothing we can do about that.
If a single domain holds all instances, that’s a single point of failure and the admin of that single domain de-facto controls the network. That’s not what we’re trying to do here. If you mean sharding an instance so that a single instance can be run on multiple servers, that’s a huge amount of coding work. Maybe if things keep growing, sometime in the future that’ll be necessary, but for the time being that’s a stupidly gigantic amount of work for not much gain.
Single account for all instances would trample on their autonomy. Each instance should have control over their own users, servers, communities, etc. You’re still thinking about this as Reddit 2.0 instead of an interconnected network.
I agree instances should back up their content. I think we should make sure our respective instance admins do their due diligence. I don’t agree in this centralized mandatory backup authority you’re talking about.
The whole backend of Lemmy is already written in Rust, for exactly the performance reasons you mentioned.
Thank you very much for the quick and detailed answer!
The Kbin - Lemmy connection should be fixed as soon as kbin.social sorts out their cloudflare issues (it’s blocking federation messages).
This is actually really good, i didn’t know they were working on that!
If a single domain holds all instances, that’s a single point of failure and the admin of that single domain de-facto controls the network. That’s not what we’re trying to do here. If you mean sharding an instance so that a single instance can be run on multiple servers, that’s a huge amount of coding work. Maybe if things keep growing, sometime in the future that’ll be necessary, but for the time being that’s a stupidly gigantic amount of work for not much gain.
Yeah i meant sharding. that’s what IPFS is about, the files on it are sharded between different computers accross all over the world that request the specific resource. When you ask through ipfs for a resource, it would perform a lookup and see if that’s available and in which node, then serve it to the client and cache it on its node. Content can be pinned to specific nodes in order not to be deleted through garbage collection, and that was what i was meaning by “central server”, there are services (like Pinata) doing just that: pinning content on fast servers and making it readily available. But i guess those nodes can also be the current fediverse istances, i had a central server in mind more due to that and because of the fact that smaller servers (available to the little associations that host fediverse istances) couldn’t probably keep up with something like sharding.
Single account for all instances would trample on their autonomy. Each instance should have control over their own users, servers, communities, etc. You’re still thinking about this as Reddit 2.0 instead of an interconnected network.
I understand that this would trample the autonomy of the single istances. In the hypotesis of sharding through IPFS however each istance could pin its users accounts and by logging in through different istances, users would possibly be able to be cached in the other servers istances (which would handle the request to the original istance at first) and still be able to access even if the original istance went down, how does this work in the fediverse now? With this hypotesis users wouldn’t need to login in a specific istance, but just ask for a login that would be managed by the network itself. The request would be handled to the nodes and the nearest/latest-updated content would be served. If the user doesn’t access for a while, the content can still be pinned and available on the original istance. This way each istance can keep its autonomy, but the user can access to it even from other istances, doesn’t need to specify a path and can bear the original istance going down for a while, even if it goes down forever another istance can take charge of that and decide to pin the content. Of course this would require more space for each istance to keep cached content and to make sure there’s at least a couple of copies available between original istance and other cached content bearing istances. There’s also a process of garbage collection, for which if the content isn’t requested for a while, it would be deleted from nodes that haven’t pinned it.
Sorry if i’m talking so much about ipfs but that’s just the example i have in mind for distributed files along different networks.
I agree instances should back up their content. I think we should make sure our respective instance admins do their due diligence. I don’t agree in this centralized mandatory backup authority you’re talking about.
The whole backend of Lemmy is already written in Rust, for exactly the performance reasons you mentioned.
This is lovely! The more i look at Rust the more it seems interesting, where can i find some documentation on how the fediverse as a whole works?
About the whole IPFS thing, the answer is that it’s not a priority because instances aren’t big enough yet for that to matter. If you feel that strongly about it, you can code it yourself and make a pull request.
For documentation about Lemmy specifically, the above link is to their github and their docs page is here. If you want more information about the ActivityPub protocol that underpins the fediverse, the documentation is here.
