Most pf these products like a VTX and camera are made as matched sets by a company. When used as sets those connectors will mate up properly and more or less be plug and play. When mixing gear you will in most cases have to splice wires and route things differently. Most builders custom fit their wires to make them more of a clean build so things do not look like a birds nest or wires hang out with the potential to get caught on things.
An example is most cameras do not have a microphone so they have a 3 wire connector. +5v gnd and the signal wire. CMOS cameras will not have a port to do programming of settings in the cameras. CCD cameras will tend to have a separate two pin connector for the switch to be plugged in so settings can be changed. Some cameras put all those ports into one connector.
Some VTX do not transmit sound. Some do so they will have +(voltage range) and ground to power the unit. Then a filtered +5v and gnd out to power a camera, video signal, (maybe) Audio signal, as well as extra grounds for both sound and video to help control electrical noise.
I know it can be confusing but a little reading and a little match game before the final assembly can go a long way. I usually make a wiring diagram for everything I do and follow that step by step and check things multiple times before applying power. Some complicated builds I will even have friends check my work before powering up. You can sometimes simplify things by cutting and pasting the pictures of each component from the PDF and pasting it in paint. Then run color coded connector lines to and from each one to where it goes to the other.
You can use higher power but you take a chance of getting into trouble if it interferes with other things in the same band and you are not licenced to do so. That is a choice we all have to decide for ourselves. As far as the added channels above the 40 that are authorized for use in the USA again that is a choice but the likelihood of you interfering with someone else on those is higher. If you bother the wrong person like a HAM radio group they like to triangulate signals to find people illegally using the air waves and file detailed reports to the F.C.C. who is then obligated to come find that signal and shut it down.
Now do not be afraid someone will come kick your door down. It is highly unlikely someone will bother you unless you are in areas where RC stuff is not supposed to be in the first place. The test for the technician level licence is relatively easy to take and pass with minimal study. There are whole threads in here devoted to that stuff so you can start there or like I have been doing log into HAMSTUDY.org and do the flashcards they have posted for the various tests and read up on the requirements to use FPV gear properly and legally.
here is the link to get started so you better understand what you are getting into.
https://hamstudy.org/tech2014
Feel free to ask any questions in here as people more informed then I will have better answers.