• zero_iq@lemm.ee
    link
    fedilink
    arrow-up
    7
    ·
    edit-2
    2 years ago

    Lots of traffic, lots of posts, lots of comments, … a torrent of incoming data. That’s going to need more storage, more bandwidth, more CPU power, higher running costs. The original instance hosting the community bears a higher load than the instances that duplicate it.

    Ideally, there would be a way to more evenly distribute this load across instances according to their resources, but from my (currently limited) knowledge, I don’t think Lemmy/ActivityPub is really geared for that kind of distributed computing, and currently I don’t believe that there’s a way to move subs between instances to offload them (although I believe some people may be working on that).

    Perhaps the Lemmy back-end could use a distributed architecture for serving requests and storage, such that anyone could run a backend server to donate resources without necessarily hosting an instance.

    For example, I currently have access to a fairly powerful spare server. I’m reluctant to host a Lemmy instance on it as I can’t guarantee its availability in the long term (so any communities/user accounts would be lost when it goes down), but while it’s available I’d happily donate CPU/storage/bandwidth to a Lemmy cloud, if such a thing existed.

    There are pros and cons to this approach, but it might be worth considering as Lemmy grows in popularity.

    • SQL_InjectMe@partizle.com
      link
      fedilink
      arrow-up
      0
      ·
      2 years ago

      I don’t think it’s a problem. If you weren’t using activity pub and just something like reddit then if you were reddit (the sysadmin) you’d also deal with having to scale if your community gets really popular

      Stuff that gets linked to also has the same problem

      https://www.jwz.org/blog/2022/11/mastodon-stampede/

      (Btw I don’t like jwz but he mentions it here)

      • zero_iq@lemm.ee
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        2 years ago

        Funny how you say it’s not a problem, then go on to describe the problem that needs to be dealt with. Dealing with scaling is a problem, and it’s a problem that costs money.

        Posts like this: https://lemm.ee/post/58472 suggest it is a problem. The rise in traffic seen by Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope. The recent growth in user registrations represents only about 0.007% of reddit’s active user base. (~60K new Lemmy users vs 861,000,000 active monthly reddit users). A site like reddit costs millions to run.

        There are 190+ Lemmy instances last time I checked, yet almost all the brunt of this load has been borne by a handful of servers, which see an inordinate amount of traffic while 100+ other servers sit around idle. Why should a handful of “lucky” servers have to pay all the hosting costs? What if a volunteer-run instance explodes to reddit-like levels of popularity? It will simply fold, unless the volunteer has serious money to throw at the problem.

        • SQL_InjectMe@partizle.com
          link
          fedilink
          arrow-up
          1
          ·
          2 years ago

          Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope.

          While this is true, 5 days ago lemmy.ml, the biggest instance, was on a 67 EUR server which is very small. https://news.ycombinator.com/item?id=36270094

          Posts like this: https://lemm.ee/post/58472 suggest it is a problem

          This is a scaling problem (having more users means you need more mods) but I disagree with how they handled it and it isn’t a money related thing. My thoughts on this are in an older post when this was first announced https://partizle.com/comment/64178

          Why should a handful of “lucky” servers have to pay all the hosting costs?

          My initial idea is to use the something awful model of paying a one time fee to register an acount. The problem is that people would just sign up on another instance that doesn’t charge a fee but still add load to the lucky instance. Another approach could be to participate in communities on one of those lucky servers then you need to pay a one time fee to that server (comments would need to be removed by a bot if they’re not made by an approved user). I’m not saying that’s perfect, but it’s an idea. Adsense is another idea.