Back in the day I used Nagios to get an overview of large systems, and it made it very obvious if something wasn’t working and where. But that was 20 years ago, I’m sure there are more modern approaches.

Come to think of it, at work we have grafana running, but I’m not sure exactly what scope it’s operating under.

Barring any hardware issues or external factors, will it run for 10000 years? Any logs not properly rotated? And other outputs accumulating and eventually filling up a filesystem?

Sounds more like what you need is a combination of a VPN and RDP. Have your machines connect to somewhere via whichever VPN protocol you prefer, and then you can access them via whichever protocol you prefer.

I’m old and crusty, so I mostly use openvpn, but wireguard will probably do as well.

Debian on homeservers, centos on work servers, and mint on desktops

In this day and age, shouldn’t Huntarr be replaced by Gatherarr? You know, sustainability and all…

A jumpbox. Set up a VPS somewhere, have some remote hands at home set up a VPN client to connect to the VPS, and then you connect to the VPS as well.

Alternatively, is it possible that your ISP can remote config your router and set up the port forwarding again for you?

How do we know it wasn’t the other way around?

You know what’s never happened? Someone coming home way too late after too much to drink, stumbling into the kitchen going “I could really go for an apple right now!”

STORY TIME!

Once upon a time, I worked an offshore rotation. So while I was home, I didn’t have much better to do than to hang out with my friend and his coworkers. They all worked for the local branch of a huge international corporation that shall remain anonymous, so I will only refer to the corporation by their initials: IBM.

This local branch dealt with banking systems, handling large clients in Europe, ensuring that their systems ran the way they should. And to make sure said banks could have their stuff sorted when a problem arose, there was always someone on call.

Well, it sucks being the guy on call when the one who’s the perfect guy to fix it is off, and in the spirit of solidarity, they did the only thing reasonable: Went to a local pub, and placed the on-call phone on the table, so if it rang, the expertise to get it sorted quickly was present.

I usually joined them, and more than once did I go for a piss, passing someone with their phone on their shoulder with a laptop in a bathroom sink, trying to sort out banking issues after having had waaay too many drinks.

When I come home from a humid bar crawl, I always crave greasy food and a messy kernel upgrade

Running arbitrary text from the internet through an interpreter… what could possibly go wrong.

I need to set up a website with

fork while 1

…Just so I can (try to) convince people to

curl | perl

it

…rhyme intended.

The gooner gender

If you’re going for software RAID, I recommend taking it a step further and go for ZFS: If set up correctly you get all the advantages of raid6, while remaining very flexible.

Used/refurb SAS drives aren’t that expensive. Can someone with better memory than I please link to that site for second hand server components?

The reason why SAS drives are usually more expensive isn’t because the tech itself is more expensive (It’s largelt just a different kind of interface), but rather that “enterprise grade” hardware have a few additional Q&A steps, such as running a break-in cycle at the factory to weed out defective units.

While a server such as the one you described is slightly power hungry, it’s not that bad. Plus, if you wanna get into servers long term, it could serve as a useful way to get used to the hardware involved.
Server hardware is at its core not that different from consumer hardware, but it does often come with some nice and useful additions, such as:

• Botswana drive bays (I tried to write “hotswap”, but autocorrect is probably correct.
• IPMI/iDRAC or equivalent for headless management
• Dual PSUs
• Rack mount capability
• Easy maintenance access to most hardware
• A ridiculous amount of sensors with automated warnings.

RAID is entirely optional. I seem to be the only one in here who actually like hardware RAID, as software RAID is more popular in the self hosting community. Using it is entirely optional and depends on your use case, though. If you wanna live without, use JBOD mode, and access each drive normally. Alternatively, pool as many disks as you want into RAID6 and you have one large storage device with built-in redundancy. RAIDs can either be managed from the BIOS, or from the OS using tools such as storcli.

Some VLAN-related nuggets that you may find useful for your post/blog:

• 99% of the time when people refer to VLAN, they’re talking about 802.1Q (tag-based VLANs). There are others (Such as port based), so it’s up whether you want to cover those as well.
• The word “Trunk” can mean different things, depending on vendor. In the Cisco world, it means a line/port carrying multiple VLANs. With many other vendors, such as Aruba/HPE, it refers to link aggregation which isn’t necessarily relevant to VLANs
• A lot of hardware still use VLANs even if none have been configured. For example, defaulting all switch ports to have an Access tag of 1 makes it behave like a dumb switch. This can cause issues later if you’re configuring VLANs elsewhere
• Anything non-vlany connected to a VLAN-enabled switch will have to be connected to a port with a default VLAN tag. This is usually referred to as an “Access port” or an “Untagged port”
• “How do I configure the switch to allow units on VLAN 123 to talk to VLAN 321?”. You don’t. Connect both VLANs to a router which will route between them. Either connect the router to both VLANs individually and skip the tagging on the router, or you can run a single trunk between the switch and the router which carries both VLANs. The latter requires you to configure VLANs on your router accordingly.
• It might make sense in many cases to have the VLAN tag the same as the last octet in the IPv4 subnet. Makes it easier to keep track of.
• A PC can implement VLANs on its network port, allowing you to connect to a trunk port and access several VLANs with one cable.

Source: VLANs have been an integral part of my career for 20ish years.

If it works on mint, it’ll most likely work on debian, with the caveat that debian is a lot more CLI and a lot less handholding. Depending on your setup, debian might be a better choice for you, as Mint is desktop oriented.

But don’t fix something that already works. If there’s no issues with your Mint setup, I’d say keep it. Next time you set up a server, you can go for debian instead.

Source: I use both extensively. Mint on desktop, debian on headless stuff.

Noted. I’m mostly an X11 kind of guy. A few X12s.

Are yoy able to switxh to HTML5 instead of Java? I never managed to get that Java applet to run properly without issues, and it sucks that older supermicro machines default to it. But many (most? All?) Have an HTML5 option you can use instead.

Also, the BMC croaks sometimes - pull bios battery and any other backup batteries during a power cycle.

I use beegfs at work for the redundancy and clustering aspect. 1.8PB of storage with 100% redundancy.

While it supports a lot and CAN be quite involved, a very basic setup is in fact pretty simple:

A filesystem on a machine is a storage target.
A machine with storage targets is a storage node. (beegfs-storage)
A management server (beegfs-mgmtd) connects these together into a filesystem.
Any machine runs beegfs-client to mount this filesystem.
One machine needs to run beegfs_meta for the Metadata. It doesn’t require a lot.

Both dhcpd and bind support failover.

If you want to have failover storage you might want to look into beegfs, as storage targets can be mirrored across hosts.

Source: Using all of the above at work. I’ve had motherboards die on me without causing downtime.