The vibrations I'm getting are definitely only at the upper end of the throttle range. They don't appear to be P-related oscillations, but I guess it could be possible.
I'm running under the hypothesis that the issue is with prop stall. I was running HQ 5x4 props on that video. 5x4 is super-aggressive and not at all something that makes sense for hovering, but for some reason when you get to mini quads the convention is to put a prop on that only a few people in the world can actually use efficiently.
3D planes and larger multirotors always use low-pitched props, usually around 2:1 or larger diameter to pitch ratio. 8x4, 10x5, 11x5, 9x4, etc. You would never see a 10x8 prop on a 500-class multi, why are we putting 5x4 on our 250-class multis? We aren't all charpu, no matter how hard we try.
So I've been doing a little experimenting. Currently I've swapped out the 5x4 to some 6x3. These motors and ESCs can handle those props, and they have plenty of thrust for me. This frame can outfly my skill level, so I am going to stay at 3-inch pitch until i really feel like it's slowing me down.
Results: The shakiness is largely eliminated at full throttle. Much less throttle is needed to hover. The overall acceleration is hugely improved, and that includes arresting a descent as well as maneuvering. For example, when I drop from on high and swoop underneath something, a small increase in throttle is all that's needed to keep me off the ground. With the 5x4 props, I really had to blip the throttle almost to max as I went under. Carving corners feels a lot different as well. It's hard to describe actually. It's almost like with the 6-inch props theres more traction on the air. To be honest it reminds me of piloting a fixed wing. The craft seems to just want to glide in the direction I put it.
Going from a hover to full climb, I imagine the 6x3 props, when compared to the 5x4, have such a good bite of air to them that they are actually climbing up into still air with every rotation, where the 5x4 are just beating up the turbulent air around them into submission, trying to move it downwards.
I would have loved to get closer to the plane, but I didn't fully know what I was doing, plus I was not coordinating with the other pilots. That is to say, they were aware of what I was doing, but they were just flying around doing their own thing, not trying to make it easy for me.