Rudder gives you yaw... like sitting on an office chair and spinning round.
Rudder helps you counteract something horrible called adverse yaw. If you can believe the rule where you don't get something for nothing then you will understand that when you use ailerons to roll, the wing going up is doing this because the aileron is making the wing generate more lift. Due to the above rule, more lift means more drag. So, if you roll left, whilst your ailerons aren't neutral, your aircraft will want to yaw right. To compensate, use a bit of left rudder whilst you have the aileron left of neutral. right aileron needs right rudder... this is called coordinated controls.
Most powered aircraft can live without people coordinating their controls as the motor effectively pulls the aircraft round the way you want it to go. However, better pilots will use some to make the whole flying that bit smoother.