Other than the Second Wave option Itchy Tenticle (which allows aliens a chance to fire instead of scattering) I do not think there is much you can do to chance their reaction to spotting you. You can however set the distance by alien type that your soldiers can be spotted. Play around with the sightradius under charactertype to find something you are happy/comfortable with. You may also want to look at the Weapons irange (Both in the DefaultGameCore.ini). Just be careful not to make yourself too OP. Again this will not change how they react once they spot you, instead allows you more of a chance to maneuver for potential ambushes.