Technically, yes, every time you fly, if you are within 5 mi. of an airport, you are supposed to notify the air traffic control tower.
THAT SAID!!!!
There are regions which may have waivers or "fly under X number of feet and it's ok" regions. My father's RC aircraft field in East County San Diego is within 5 mi. of a small airport, one Gillespie Field. When they applied with the AMA for club charter, and the city granted them the park land for their runway and flying area, they also gave them a waiver to fly there. It's generally not a problem because their club flies at the mouth of a box canyon area; if you keep your planes below the tops of the hills surrounding the runway, you have nothing to worry about. Full scale aircraft aren't going to come in and fly through the little canyon area; it's somewhat narrow and the guys in the Cessnas and Beechrafts are more trying to get OVER those hills, so the full sized vs the RC aircraft are never really in competition for airspace.
Are you supposed to notify every time? Like I said, technically, yes, but it may be more hassle than it's worth. I'm going to tell you this - use common sense. If you are not flying in the landing path or takeoff path of an active runway, or directly in an area where planes are lining up for final approach, and if you DO see planes/helicopters coming and you get out of their way or land, you shouldn't have an issue. Oh, and if you are flying lower than treetops or adjacent buildings/structures? You probably won't have to worry about it - pilots are going to be more worried about hitting the 20 story office building or a giant tree, than a midair collision with your aircraft.

Don't give them a reason to worry, and you should be fine.