I disagree with most here.

You need to reduce swappiness even more, and tweak some other variables (like the memory “threshold” Linux starts to swap, and disk cache aggressiveness) to stop Linux from swapping so opportunistically under such a light load.

IMO, Linux is configured for “old” systems by default: slow HDDs, and constrained RAM pools, where disk IO really, really needs caching, and where idle background processes take a large fraction of RAM.

You have superfluous RAM for your workload. And very fast disk IO that isn’t such a hindrance to apps anyway. And a disk you don’t want to wear. This is the opposite scenario: you don’t want Linux to swap unless it absolutely has to.

For reference, this is part of my config. It’s rather niche and you probably shouldn’t use it, but you should consider looking up the variables:

# Keep min reserve reasonable for 8GB usable space
vm.min_free_kbytes = 262144        # Lower absolute minimum to 256MB

vm.watermark_scale_factor = 10     # Lower to 0.1% (which is ~128MB on 128GB)

# Disable watermark boosting completely
vm.watermark_boost_factor = 0

# Normal-ish metadata pressure so desktop doesn't stutter on disk reads
vm.vfs_cache_pressure = 120

# Allow reasonable swapping of inactive anonymous desktop pages, could be lower
vm.swappiness = 10

# Dirty bytes limits to limit caching
vm.dirty_background_bytes = 67108864
vm.dirty_bytes = 268435456

# Disable compaction & proactive scans to stop freezing with large portions of RAM mlocked
vm.compaction_proactiveness = 0
vm.compact_unevictable_allowed = 0

vm.page-cluster = 0 # 4kb pages for SSD

My system still uses RAM as disk cache with this config, it just won’t go out of its way to swap just to keep that cache, especially I lower swappiness to 1-3.

I also have a 1GB zram pool, prioritized over ssd swap. But you should make yours even larger (maybe 4GB?). This will intercept anything that does swap first.

I also start some applications with systemd-run and specify memory caps and swap limits (often forbidding them from swapping entirely).

It makes a night-and-day difference for some workloads on my system, that would otherwise swap pointlessly, just tank performance and even de-stabilize the system.