It rings true for me as well, but there's still a floor on it. I like a quieter environment than most extroverts, but there is no such thing as an absolutely silent environment (not unless you spend a lot of money to make one of those soundless rooms).

There's also the matter of acute vs chronic noise. Any steady, constant noise, like a fan or music, I can tune out. Aggressive, sudden noises (like somebody shouting on the street) trigger something primal in me. It attracts my attention and if it's loud enough will even trigger my fight or flight reflex.

It makes me wonder if the reason the sensory processing dial on introverts is a little jacked up is because that trait results in a more sensitive fight or flight reflex.