Quick run down:
Hardware: If you don't want your recording to impact gameplay performance (especially on high end games) then you need a separate piece of hardware called a capture card. Which simply uses a split of your signal going from your GPU to your monitor and stores it on a hard drive.
If that's a bit too techy techy for you then there are also software solutions for you. Both AMD and Nvidia have their own recorders which is good since the GPU does the recording with that much more effectively.
That's it for AMD:
http://www.amd.com/e...es-gaming/raptr
Editing: Again there's various software you can utilize to do that. For most of them you would have to pay (well you don't have to if you know what I mean but that's not the point) the best and most commonly used application for that would be Adobe Premiere. It's easy to grasp you have enough control and if you desire to have more effects for example you could also other products of Adobe and can easily port clips over with the use of dynamic linking. For example to after effects without actually having to export the file before final render.
If you want to stick to free ones go here: http://filmora.wonde...re-windows.html
As for the other questions they are kinda tricky to answer with one "here you go" answer. If you record in one go then you will have one massive file. (well most programs make a cut at 4 GB and then continue writing a new file and so on) Your software (or hardware) will usually only record in one specific codec and will then continue to put it in a certain container file.
Usually those program will save to mpeg4 (some may have less compressed codec) which is also called h264 and put it in a container for many it's AVI. (in case you don't know AVI is a container not a codec)
As for your resolution. You want to record in your monitors resolution so you get all the detail and all the information you can have (helps a lot for editing, CC etc) so don't cheap out on that.
Now assuming you have your edit ready to go and now you think how do I compress this crap. If you're exporting using a TIFF sequence you end up with many gigabytes if not terabytes of data so you don't want that. That's where your codecs come into play again. Most codecs are lossy compression codecs. Meaning if you compress them you actually lose information. I could explain to you how compression actually works but I think that's not in the scope of this. Anyway losing that in most cases doesn't matter if it's your final delivery. The most efficient and the most used codec is again h264 for sub 4k resolutions and h265 for 4k resolutions.
Most likely you will compress your file right in your editor when you're exporting. The process is actually quite simple. In your editor go to export settings and choose codecs then you'll select h264 (install quicktime for that) and set the bit rate. You can either use a fixed bit rate and be done with it but for smaller and more efficient sizes you pick VBR (variable bit rate) in which each frame gets bits according to how much information is currently present (a black screen doesn't have much so it gets less). FOr youtube select 35 bits per channel for your maximum and 5 bits for your minimum. If you even want to make it more efficient use a two pass render so your software looks at the video file at the first pass and determines how much information is in each frame and distribute your bits accordingly. And the second pass is the actual rendering.
And that's basically all you need to do then you just hit okay and press render in your editor. There are specific compression software solutions but they're really not necessary.