Not sure what brand or model number you are working with, but some of the supplies rely on the 6 or 8 1200uf caps that are on a Motherboard to be on the 5V red connections.
I have seen power supplies fail to start when the Motherboard caps go bad. The 1200uf 6.3V that normally fail.
5 volts is normally always on for wake on lan and the front push button.
You may try to add some caps outboard to the power supply 5 volt line and see if that may help.
Many motherboards have a total of 7200uf and that power supply may rely on them to work properly.
There are some on the 12V also, but I don't think that would keep the power supply from at least trying to start up, it would try to start then shut down after a second of so.
How do you have "Power Good" connected ?
"Books tell how it should be, Experience tells how it really is,
Theory only works perfect in a vacuum." KA9JLM Don