Don't forget, that if using SSB you will not be running anything like 100 watts output....only using a constant carrier mode with achieve that. SSB power out will be in the region of 50 - 6o watts at the most.
Those who give up liberties for the sake of security deserve neither.
If you are well out of the noise at the other end, no one would notice a change from 100 watts to 400 watts or more.
If you are just a few dB out of the noise, even a change from 100 watts to 200 watts can be huge.
Since signal to noise is what really matters, and since S units are largely meaningless, it's a little silly to talk about S units.
S units are not always (or often) 6dB. We PRETEND they are all six dB, but they have historically been all over the place. There was an unsuccessful push around WW II to make them six dB, no one paid any attention. Drake and most others used 5 dB, Collins around 3 dB, and none of it was linear across the scale. Recently the is another push to make calibration six dB per S unit, but few receivers actually are that way and almost none from before the last several years are. S units are for the most part meaningless.
The bottom line is this. People that are barely hearing you will appreciate a change from 100 watts to even 200 watts, because that is quite noticeable if you are near the noise. If you are below the noise, every extra watt can count. If you are well out of the noise, power does not matter.
The less efficient your antennas, the more often even a little more power will help. Even a change from 100 watts to 200 watts can be dramatic at the other end if your signal is in or near noise (or QRM) floor. Anyone telling you the generalization that a change like that never makes a big difference is misleading you.
For several years, Ameritron sold a 400 watt output sweep tube amp. That amp sold as well as 600 watt amps, and much better than 1500 watt amps.