As an avid wing flyer, I agree with @Kananga, wings, when property setup, are no more difficult to fly than any other plane.
They are very sensitive to your setup, keep your CG forward (@ 25% of wing area) and keep your throws low. About half of throws used on a plane with a tail. Like all planes, wings will fly better when you get the plane trimmed. When trimmed, slowly turn the throws up to suit your flying stile, mild or wild.
It's really hard to judge without looking at some video. BUT something I would suggest is dialing down your rates even more. Maybe a 3 position switch for 3 different rates. Perhaps make your low rates even lower. Could also bump up the expo a little bit. Just something super easy while you work out the kinks. Do some glide tests too and see if it glides ok, battery attached, but no power.
Minis can be a whole different experience...I don't have a ton of experience with them either. But one common thing I've heard is weight. Keep your glue usage with them to as little as possible. Use real light servos, lightweight motors, light batteries, etc.
I agree. Video would have helped people help me easier with some early issues if I'd had it sooner, but I'm learning to record and upload footage for the first time as well as flying planes, so by the time I had the means and technique to record videos reliably and upload them, which was quite recently, I was already proficient with a trainer type plane(Hobby Zone Champ and had two similar but slightly different prototypes of my own designs flying reliably. I'm still learning, and when I start to branch out in completely different directions with my builds and designs, I'm sure I'll have some times where something doesn't work the way I want it too right on the first try, and then I'll be glad I have video footage to share.
You seem to not do well with anything that has elevons. I don't do well with elevons yet either, but part of that is the precision they require in building. I'll be working more on that soon. Other than that, planes that need a lot of elevator authority to climb and very little aileron input to roll, usually have sizeable elevons which further this bias a bit. For these reasons, I have some low aspect delta type planes that I designed to have a RET control scheme, to lead me in sort of gently. They are still quite sensitive and would likely still make your sketchy list. Simulator flight helped me out a lot. Perhaps get a free sim such as Multiflight and fly a bunch of stuff you have trouble with, like flying wings, elevon jets, low-wing warbirds, etc. I have heard the FT flyer is very docile for a plane with elevons. It is very responsive, but can fly slow. I have not built or flown one myself, but I'm sure those who have would agree.