Just some Internet guy

He/him/them 🏳️‍🌈

  • 1 Post
  • 20 Comments
Joined 3 years ago
cake
Cake day: June 25th, 2023

help-circle
  • They’re just examples of things you could pipe curl into, but no not really. If the download fails you end up with an incomplete file in your tmpfs anyway, and have to retry. Another use I have is curl | mysql to restore a database backup.

    If the server supports resuming, I guess that can be better than the pipe, but that still needs temporary disk space, and downloads rarely fail. You can’t corrupt downloads over HTTPS either as the encryption layer would notice it and kill the connection, so it’s safe to assume if it downloaded in full, it’s correct.

    With downloads being IO bound these days, it’s nice to not have to read it all back and write the extracted files to disk afterwards. Only writes the final files once.

    That’s far from the weirdest thing I’ve done with pipes though, I’ve installed Windows 11 on a friend’s PC across the ocean with a curl | zstd | pv | dd, and it worked. We tried like 5 different USBs and different ISOs and I gave up, I just installed it in a VM and shipped the image.


  • I’ve had to use that flag.

    --silent is useful when you don’t want the progress bar or you’re piping curl into something else. I like to do curl | tar -zxv to download and decompress at the same time, I’ve even tar -zc | curl to upload a backup taking no disk space to do so.

    The problem however is it’s really silent: if it fails, it exits with a non-zero code and that’s it. Great when you don’t want debug info to interfere, annoying when you need to debug it.

    So you can opt-in to print some errors when in silent mode, but otherwise be silent.


  • It’ll tolerate a few hours no problem, mine’s been down for a bit over 24h and caught up fine.

    I think it marks instances as down after 2-3 days, but I’m not sure if it’ll resume once it comes back up at this point. I think if your instance reaches out it might start pushing events again but it could also result in dropping the previous days.



  • For all its flaws and mess, NFS is still pretty good and used in production.

    I still use NFS to file share to my VMs because it still significantly outperforms virtiofs, and obviously network is a local bridge so latency is non-existent.

    The thing with rsync is that it’s designed to quickly compute the least amount of data transfer to sync over a remote (possibly high latency) link. So when it comes to backups, it’s literally designed to do that easily.

    The only cool new alternative I can think of is, use btrfs or ZFS and btrfs/zfs send | ssh backup btrfs/zfs recv which is the most efficient and reliable way to backup, because the filesystem is aware of exactly what changed and can send exactly that set of changes. And obviously all special attributes are carried over, hardlinks, ACLs, SELinux contexts, etc.

    The problem with backups over any kind of network share is that if you’re gonna use rsync anyway, the latency will be horrible and take forever.

    Of course you can also mix multiple things: rsync laptop to server periodically, then mount the server’s backup directory locally so you can easily browse and access older stuff.


  • Technically it wasn’t really designed with megainstances in mind that swallows the entire fediverse.

    My instance has no problem whatsoever keeping up and storage is well under control. But we’re few here subscribed to a subset of available communities so my instance isn’t 90% filled with content I don’t care about and will never look at. Also reduces the moderation burden because it’s slow enough I can actually mostly see everything that comes through.

    Lemmy itself is also pretty inefficient in that regard, you can very much make software that pulls instead and backfill local cache as needed.

    Even my Reddit subscriptions would be pretty easy on my instance.


  • Technically it wasn’t really designed with megainstances in mind that swallows the entire fediverse.

    My instance has no problem whatsoever keeping up and storage is well under control. But we’re few here subscribed to a subset of available communities so my instance isn’t 90% filled with content I don’t care about and will never look at. Also reduces the moderation burden because it’s slow enough I can actually mostly see everything that comes through.

    Lemmy itself is also pretty inefficient in that regard, you can very much make software that pulls instead and backfill local cache as needed.


  • One thing to keep in mind is ActivityPub isn’t exactly made for social media in the sense most people use it nowadays. It’s intended to be more like RSS feeds: you’re support to subscribe to stuff like news sites and be able to bring it all into a content aggregator. Seen that way, its design makes a lot of sense.

    It kinda works well for public microblogging as well. It’s when you start involving moderation, voting, sharing, boosting that things get kinda weird.

    I’ll add some of my comments to that discussion.



  • The main issue is when your instance starts federating, accounts are created with a key pair that you will lose when changing software, and generally a whole bunch of URLs will no longer be valid. The actor ID of your user is https://feddit.org/u/buedi, not just buedi. Mastodon might make it https://feddit.org/@buedi instead. As per the spec, that is the canonical URL for the user/actor.

    Other instances will still try to push content to your instance assuming the software it was registered with. So you may continue to receive data for Lemmy communities which Mastodon has no clue what that is or what to do with it.

    You can host the API/frontend on a different domain no problem, but the actual ActivityPub service should be on a dedicated subdomain to avoid the issues.

    That said, I believe after a couple days/weeks, it should eventually sort itself out as your instance keeps erroring out and gets dropped and reregisters with the new software.

    https://seb.jambor.dev/posts/understanding-activitypub/



  • Max-P@lemmy.max-p.metoLinux@lemmy.ml•aaa
    link
    fedilink
    arrow-up
    2
    ·
    1 year ago

    Aside from the other answers, no you can’t offload computations to memory. Memory stores data, it doesn’t compute.

    The only way having more memory can possibly improve performance, is by having a cached copy of files so they don’t have to be fetched from disk, and applications potentially caching the results of heavy but reusable computations. (Unless you run out of memory and starts spilling over to disk, then more memory will make it fast again by avoiding swapping).

    I mean I guess technically yes you could transcode into H264 into a tmpfs mount, and then play the H264, but you’re still not doing it faster and certainly not fast enough to watch in real time, you’re just decoding the AV1 well in advance before actually watching it.



  • As commenters on the LWN thread said, I doubt that many firmwares even bother to check anyway. My motherboard happens to have had a bug where you can corrupt the RTC and end up in 2031 if you overclock it wrong. I didn’t use secure boot then though so I don’t know if it would have still booted Windows. But I imagine it would.

    That said, I’ve always just enrolled my own keys. I know some other distros that make you enroll their keys as well like Bazzite. At least that way you don’t depend on Microsoft’s keys and shim or anything, clean proper secure boot straight into UKI.


  • The main issue you’ll run into is nicher proprietary software being hard to install, but that’s what containers are for. The main one I see is if you need to install some proprietary VPN client it gets annoying, but since you’ll be running a VM anyway you can do some network trickery. My work’s antivirus only works on Ubuntu and RHEL, proprietary kernel modules so it’s got to be at least one of those kernels.

    Linux is Linux, nothing’s impossible to solve even with Bazzite’s immutability. Worst comes to worst you make your own images and it’s not that hard, you basically just fork it on GitHub and let the CI do its thing.

    But do you have time to fiddle to make it work and take the risk, or do you want to play it safe? How confident are you with Bazzite’s more advanced topics?




  • If a community is hosted on instance A and a user that’s on instance B creates a post in this community, is the post hosted on A, B? Or are individual comments hosted on the instance of their respective authors?

    Both, but the instance the community is on is the one that forwards it to all the other instances will get it from.

    what happens when your account gets deleted? Are all my messages deleted? Does this happen to all instances (it syncs the deletion?)

    It does delete everything, but it’s a bit buggt and cannot be guaranteed (because I can just restore from backup and undelete everything if I want).

    do instances cache posts and comments posted on other instances? If so, RAM or disk?

    Everything is pushed to all interested instances and host their own copy.

    will having too many instances increase the load of all instances? (If they all have to sync?)

    Only the ones that you subscribe to. But yes generally it would increase load on the big ones like lemmy.ml and lemmy.world since they’re popular.

    But also in a way, it’s no different than one user viewing the post, and your instance have a copy of it and can serve many more users. And the remote instance gets to push it to you when it’s convenient. So not really a problem.

    if I want to check the comments of a post, does my client ask this to my instance of to the instance of the author or to the community’s instance?

    Your instance already have a copy of it all. You always go through your instance (except media, depending on your instance’s settings if the media cache/proxy is enabled).


    Roughly, how ActivityPub works is that instance A subscribes to B (by sending it a subscription request to a given community), and then B just sends A everything that happens from that point on. If you post, then A goes to B to inform it of the post, and then B broadcasts it it to everyone else. A owns the user, B owns the community.

    Most questions can then be answered by thinking of what would happen. What would happen if B bans a user from A? Well A doesn’t care, neither does C, B will just ignore everything from that user. What happens if A bans the user? Well, that user can’t post at all so indirectly also banned from B and C. What happens if A bans a user from C but C posts to B? A will ignore it, while B and C sees it. And so on.

    Each instance is independent and makes its own decisions, so the view is slightly different from instance to instance.

    And yes the fact everyone basically have a fully copy of everything does have some considerable privacy implications. A rogue instance can just ignore deletes and keep full edit histories. Every post, every comment and every vote is public information. It’s entirely an honor system when it comes to deleting stuff.


  • The BIOS does a lot less than you’d expect, it doesn’t really have an impact on gaming performance. For what it’s worth, I’ve been gaming in a VM for years, and it uses the TianoCore/OVMF/EDK2 firmware, and no issues. Once Linux is booted, it doesn’t really matter all that much. You’re not even allowed to use firmware services after the OS is booted, it’s only meant for bootloaders or simple applications. As long as all the hardware is initialized and configured properly it shouldn’t matter.