Have you tried leaving any giant rooms you may be in? Like Riot/Matrix HQ? I use my personal homeserver for my friends/some IRC channels and my load is nonexistent. May not be the answer you're looking for, but those huge (10,000+ users) rooms are a huge strain on homeservers right now. Look for performance updates in the next version of synapse.
Yup -- that's exactly what the end solution was. Unfortunately reaching out to those 20 users and saying "hey, could you be judicious about which federated rooms you join? otherwise you're gonna hose us all" kinda sucks.
