MSI Afterburner uses Predator for capture.
"Predator software supports H.264 codec and multiple file formats, (wmv,avi etc..) size and quality"
I've used it in game and the videos are excellent quality the audio capture is clear and clean. You can get MSI here:
Here's a tutorial on it's use
I use Cyberlink Power Director for editing and formatting and there are no issues importing the vids created by this software.
here's and example of online capture from ESO
I recommend watching in HD to see the quality.