Ardupilot or pixhawk are going to be (by far) the easier path towards full autonomous capability. Unless you have really specific reasons not to do that, I would point you in that direction. The people I hang out with lean more towards px4/qgroundcontrol, but it's super cool to consider running ardupilot on the really inexpensive flight controller hardware now.
I'm a little crazy (but not as crazy as the guy that shows up in a nearby lake and swims every morning at 6am) so I designed my own fully autonomous system based on a teensy-3.2 (or teensy-4.0 now) and a beaglebone which is similar to a raspberry pi (runs linux, etc.) I've been wrenching on this system since 2005 or so, we run a high quality EKF developed at the university of Minnesota that mathematically blends gps and imu to accurately estimate roll, pitch, true heading (and location, velocity.)
My core teensy code has all the sensor reading, EKF, control allocation, simple rate damping. The beaglebone runs the higher level stuff, speed/altitude/heading hold, route management, logging, communication. Much of the beaglebone code is written in python as an attempt to be open and accessible to more people. The teensy side is written all in arduino style C++. My thinking is I have 2 simpler pieces that plug together versus one really complicated massive do-everything thing like px4. The ardupilot and px4 software stacks have become so incredibly complex over the years. My system is probably complicated too, but I've tried focus on simplicity. I see this kind of like keeping an airplane design light weight ... you really have to think about it at every step of the design and build to actually come out light in the end. Same with software development ... complexity creeps up on you and compounds if you aren't super careful.
Anyway, I don't want to side track your thread with my stuff, but happy to talk more about it if you are interested.