I don't have time to read all the applicable rules, particularly since the law invokes AMA, or other nationwide group, and its safety regulations. However, it does appear you can fly over over 400' in appropriate areas with appropriate line of site. It's a complicated matter because the NAS is a complex system designed to keep everyone safe.
I've taken a DJI Phantom to 400' before. That's pretty darn high for a model aircraft. I'm quite certain I've never flown a plane anywhere near that high.
One easy example of when it is safe: suppose you are in a valley and within 400' of a mountain. You could tow a glider to over 400' from the takeoff point so long as you stay within 400' of the mountain.
At least that's a quick interpretation I think is correct based on quick reading. It's really best to learn yourself, and only follow posts with appropriate links to specifics. Note that this is not such a post and I am most certainly not giving advice you should actually follow.
The only advice I am giving is read for yourself and only follow advice that includes links to official regulations and rules.
But, flying below 400' won't always save you. Aircraft are allowed to fly below 500' under specific guidelines and you are required to yield to such aircraft. For instance, I often fly over a wide river, and in the winter time, pilots will often "buzz" the river. They really shouldn't in the summer because of boat traffic. So, when flying over the middle of the river, I need to be prepared to drop to less than 40-50' or leave the area if possible. There is also a designated seaplane landing area to the North of me that I need to keep in mind.
Mike