I have observed quite a large number of damaged ESCs which have caused not only Loss of control but also fires both in the air and on the ground. An ESC comes with voltage and current limits as part of its specification but one thing not specified is the ESC's cooling requirements.
You should always take the manufacturer specifications as the ABSOLUTE MAXIMUM that the ESC can handle safely. When selecting the propeller for the installation you should not allow the maximum current to be exceeded. So, when changing the battery fitted you should swap out the propeller as well. When going from 3S to 4S you should reduce the Propeller diameter, the propeller pitch, or both!
In addition you need to be aware that as the battery voltage increases you should pay more careful attention to the cooling of both the ESC and the battery. Where an ESC is cheap and has an onboard SBEC the BEC will actually generate far more heat with higher battery voltage, and due to over temperature protection it could even shut down the power rail to the Rx. This can and does result in a total loss of control.
Where pushing the ESC to its limits regularly I would recommend that you consider a separate UBEC for the electronics/control. The output drivers on the ESC that control the current supplied to the motor can become very hot and this in turn can also trigger the over temperature protection feature of the BEC and again can/will result in a total loss of control.
If you use an ESC that can supply more than enough current safely to the motor you run into another issue and that is the motor's ability to handle the current and the heat it generates. If you fail to adjust the motor load, (the propeller diameter/pitch), you can actually cause the motor to overheat. This causes the insulation on the motor windings to be heated greatly, (it becomes darker in colour over time), until the insulation fails totally, (it becomes charcoal). At this point the windings are no longer properly insulated and short circuits can easily develop.
A shorted winding can cause the current supplied by the ESC to spike way beyond its normal supply amperage and the ESC can still fail! The motor must not be overheated if at all possible. Ensure that the motor also has adequate cooling, (as a minimum).
In summary the propeller is the key item in not only the plane's performance but also in the life of the motor and ESC especially when increasing the cell count of the flight battery. Ensure that the ESC and the motor get sufficient cooling. Finally the throttle use is another factor in the temperature and current used by the motor and passing through the ESC. Allowing time to cool the motor and ESC between high speed runs can really help in averting a failure when the motor and ESC are operating at their absolute limits.
When under propped a motor and ESC will last for years but over prop them and their life can be as little as seconds!
Just my thoughts!
Have fun!