A faulty servo can be the cause but not always!
Recently I had a rudder servo start to do the same and I discovered that the potentiometer that is connected to the rudder stick in the transmitter was faulty or somehow full of dirt Etc. I simply replaced the entire gimbal with a replacement unit from Banggood and now all is working fine again!
Have fun!