What is this guide about?

This guide aims to explain, to people just starting out with H.264 encoding, what the various options in x264 mean and how they affect the encoded video. For each option, a description is provided, as well as a list of possible options (and their descriptions), the command line version of the option and the recommend setting is shown in underline whenever possible.

If you are reading this guide, then it must mean that you already know all about H.264 and H.264 playback. If you don't, head on to our H.264 Playback Guide to get acquainted with this new video compression format.

Future versions of this guide will feature more information regarding compatibility (eg. with QuickTime, iPod, PSP, Xbox).

What is x264?



x264 is a free H.264 encoder. It is mainly aimed at intermediate users, so it has lots of options that will be confusing to the average user. If you don't know your trellis from your pyramid, then this guide will hopefully help you to understand things a bit better (or just enough so that you don't have to select options at random).

The x264 options discussed in this article are ones that can be inputted using the command line version of the tool or through a Windows GUI software called StaxRip. A separate StaxRip H.264 Encoding Guide is also available, so don't fret if you can't figure out how to use this software (or to even install it).

Why use StaxRip?

Or rather, why not use MeGUI? MeGUI is probably the best tool around with dealing with x264, but it can be daunting for new users because of the numerous options that are included. StaxRip is aimed more at beginners, and so has (as the author states) only the important x264 options available to configure. Plus, StaxRip assumes no knowledge of Avisynth scripting.

You can also read our MeGUI H.264 Conversion Guide.

Software Requirements:

• x264
  
• StaxRip
  

Basic x264 usage:

At it's most basic, x264 is a command line tool, where you can specify the input, output and encoding options all through a glorious black-background/white-text command line interface.

The command line syntax is as follows



A simple example of a x264 command line entry:



A more complicated example of a x264 command line entry:



There are many graphical user interfaces (GUIs) which allow you to set these options through a normal Windows program, and StaxRip is one of them.

Throughout this guide, whenever an x264 option is explained, both the command line equivalent and the StaxRip/GUI version of the option will be shown.

Frames Basics:

Modern compressed video are made up of 3 different types of frames. Intra (I) frames (also known as key frames), Predictive (P) and Bidirectional (B) frames. Certain types of frames can be referenced, which means a frame sometimes only need to stores the difference between it-self and the referenced frame. The problem with referencing is that any error that occurs when frame Y references frame X will be carried over and propagated if frame Y is it-self referenced later on. This is why there are also frames which can't be referenced.

The I frame stores the entire image and so are the least compressible. Video seeking (when you skip to a part of a video) requires I-frames, as otherwise a symptom called seeking lag will occur when the video is blank until the next I-frame is encountered.

P-frames store the difference (called "referencing") between itself and an I-frame or other P-frames. This means P-frames can be referenced, so it needs to store enough information for this to happen, but not as much I-frames because of it's encoding technique (motion compensation).

B frames also store the differences between itself and other frames as well (uses both previous and future frames for reference), but it can't be referenced ("B-frame pyramid" excepted), so they are the most compressible (holds least amount of information). 264 supports "B-frame pyramids", which allow B-frames to be referenced for other consecutive B-frames, but otherwise B-frames can't be referenced by any other frame, so less information is stored.

So basically, a compressed video will be a combination of I, P and B frames, some referencing others. The successful combination of these frames and the referencing determines how well the video is compressed.

x264 Profiles:

Profiles are the easiest way to configure x264. Profiles are basically pre-configured x264 settings that have been saved and can be loaded time and time again to encode files. They save you time by not requiring you to re-set all the options whenever you use x264, and most importantly, they give new users a chance to experience x264 without understanding what the options are.

Sharktooth from the Doom9 forums has produced several profiles for everyday usage, and has made it available for download. You should read the official forum thread for details of what each profile is intended for, although the profiles "PD-PSP", "PD-Xbox_Hires", "PD-Xbox_LoRes" and "PD-iPod" should come in handy for encoding a PSP/Xbox/iPod compatible video. There are even profiles specifically designed for Anime/cartoons

The other piece of good news is that software like MeGUI allow you to import profiles, and StaxRip has these profiles built into the software it self (and also allow you to edit and create your own profiles).

But what if you are curious and you want to find out just what options are being selected in a profile? It's actually very easy to view profiles, because they are actually just XML files. Clicking on the XML file should open them in Internet Explorer, and you should see a screen similar to the one below:



What the above shows is that for this particular profile, NbBframes to 3, KeyframeInterval to 250, trellis to false and so on. On the x264 command line, the three options I've mentioned would translates to the following:

---

Description: Selects the encoding mode to be used by the encoder. Single pass encoding can give unexpected output file sizes (Quantizer and Quality modes) and/or poorer quality compared to multi pass encoding.
Available Options:
Single Pass - Bitrate: Encodes the video once (single pass) with a set constant bitrate for each frame
Single Pass - Quantizer: Single pass encoding with a set quantizer (higher quantizer => lower quality) for each frame
Single Pass - Quality: Single pass encoding with a set quality rating for each frame
Two Pass: Encodes the video twice (once to determine it's properties, another to ensure the selected output file size is reached with maximum efficiency). This is the most common setting.
Three Pass: Same as Two Pass except for an extra encoding pass to ensure even better quality/accurate file size
Command Line: --bitrate, --qp, --crf, --pass 1/2/3

---

Description: Only available with "Single Pass - Quantizer" mode is selected. Set the quantizer value for the encoding. A high quantizer will equal a low quality (and smaller file size). A quantizer value between 20 (high quality) and 30 (low quality) should be used. Anime/cartoons or scenes with large patches of color or less details benefit from having a higher quantizer, typical movies require a lower quantizer.
Command Line: --qp n
(where 'n' is the quantizer value)

---

Description: Only available with "Single Pass - Quality" mode is selected. Set the quality value for the encoding. A high quality value equals a higher quality encoding (and larger file size).
Command Line: --crf n
(where 'n' is the quality value)

---

Description: Selects the number of consecutive B-frames x264 should use. It is also the maximum number of consecutive B-frames x264 can use if the "Adaptive" B-frames option (see next section) is selected. B-frames are frames that are small in size, but when placed correctly, offer no loss of quality. This can help improve compression. With the "Adaptive" option on (x264 will decide how many B-frames to use), having a setting of 3 is recommended as this gives x264 enough room to work with.
Command Line: --bframes n
(where 'n' is the number of B-frames)

---

Description: Selects the maximum number of reference frames that can be used. Referenced frames are frames that refer to other frames (eg. if both frames are similar). Having a high referenced frame will improve quality but slow up encoding. For typical content, a reference frame of 3 to 5 is recommended. For content with a lot of repetition (eg. animation), a reference frame of 8 to 10 can be used. If a large number of reference frames is selected, then the "Mixed References" options should also be selected to allow x264 greater control (see "Analysis Options" section).
Command Line: --ref n
(where 'n' is the number of reference frames)

---

Description: Also known as "de-blocking" filter. Connected with the "Alpha" and "Beta" setting next to it. One of the fundamental differences between H.264 and previous codecs. Should always be enabled or excessive video artifacts may appear. Controversial because blocking (when the video looks like a series of different colored squares in low bitrate video) and artifacts are treated as details by the human visual system, and removing them makes people think details is lost, when it's actually the other way around (more "actual" details shown due to less artifacts). This is an interesting topic that is better discussed in DeathTheSheep's x264 guide.
Command Line: --nf
(disables loop filter)

---

Description: This controls the "Loop Filter" (see above). Also known as Strength (Alpha) and Threshold (Beta). Alpha determines the strength of the deblocking action. Beta determines when x264 decides something is a block and when something isn't - a high value means x264 sees more video as blocks than a lower value. For example, a too high Beta value and a too high Alpha means x264 will identify too many blocks (even when they aren't artifacts) and apply too much filtering to remove them, making the picture look "washed out" and lacking detail. Both values can be positive or negative. 0/0 (Alpha/Beta) is the default and recommended setting, unless you find the video quality unacceptable, then you should try combinations. Do not go under -2 or above 3. This is an interesting topic that is better discussed in DeathTheSheep's x264 guide.
Command Line: --filter

---

Description: This option when selected can improve compression efficiency. Disabled by default, Auto seems to be the most common value, with Spatial being used occasionally.
Available Options:
None
Spatial
Temporal: least used (at least in Sharktooth's profiles)
Auto: most common
Command Line: --direct x
(where 'x' is 'none', 'spatial', 'temporal' or 'auto')

---

Description: Sets how much bias x264 should give the usage of B-frames (higher means more use of B-frames). Setting this to 100 is the equivalent of not selecting the "Adaptive" option (see below). A default value of 0 works well. Option only available when at least 1 B-frame has been set.
Command Line: --b-bias n
(where 'n' is the bias value)

---

Description: Turns on adaptive B-frames, which allows x264 to determine the number of B-frames to use (within the set B-frames limit in the previous section). A default value of On works well. Option only available when at least 1 B-frame has been set.
Command Line: --no-b-adapt
(disables adaptive B-frames)

---

Description: Turns on B-frame Pyramid option, which allows B-frames to be used as references for other B-frames in prediction. A default value of On (when available) works well. Option only available when at least 2 B-frame has been set.
Command Line: --b-pyramid
(enables adaptive B-frames)

---

Description: Turns on/off RDO for B-Frames. Turning it On will improve quality at the expense of encoding speed, but it is almost always worth it. Must have at least 1 B-frame and must have a "Subpixel Motion Estimation" (see "Analysis options" section) of 6 or 7.
Command Line: --b-rdo
(enables RD based mode decision for B-frames)

---

Description: Turns on weighted prediction for B-frames, which results in improved accuracy and therefore a more efficient encoding. Set it to On in most cases. Option only available when at least 1 B-frame has been set.
Command Line: --weightb
(enables weighted prediction)

---

Description: Turns on bidirectional M.E, which allows predictions based on motion both before and after the B-frames. Set it to On in most cases.
Command Line: --bime
(enables bidirectional M.E.)

---

Description: This option sets how x264 determines when a scene change has occurred and hence when a key frame is needed. A higher value will allow x264 to be more sensitive to scene changes. A video that has very few camera movements may require a higher scene cut value, for example. The default value of 40 should work well in most cases, plus or minus 5 for low/high action videos.
Command Line: --scenecut n
(where 'n' is the scene cut value)

---

Description: Stands for Context Adaptive Binary Arithmetic Coding. Improves encoding efficiency at the expense of playback/decoding efficiency. The default option of leaving this setting On is recommended, unless the encoded video is to be played back on devices with limited decoding power (eg. portable devices such as the iPod or high resolution clip playback on the Xbox).
Command Line: --no-cabac
(disables CABAC)

---

Description: This option sets the minimum and maximum number of frames before a key frame has to be inserted by x264. For example, a minimum setting that is the same as the framerate of the video (eg. 24/25/30) will prevent the encoded video from having two subsequent key frames within a second of each other, which only aims to increase file size without any noticeable quality improvements.

Similarly, a maximum setting ensures that a key frame is inserted at least every X number of frames. A video without key frames will have lower quality and there will be problem with seeking (if you try to skip to a part of the video without a key frame, there won't be any video until the next key frame is reached, also known as "seeking lag"). A recommend setting, as with DivX/XviD encoding, is to set this as 10 times the framerate, which equates to 10 seconds of video between key frames (ensures "seeking lag" is at most 10 seconds).
Command Line: --keyint m --min-keyint n
(where 'm' and 'n' are the max and min GOP size values)

---

Description: This option allows x264 to have greater control over "Reference Frames" (see "Common options" section). Especially needed if the "Reference Frames" is set to a high value.
Command Line: --mixed-refs
(enables mixed references)

---

Description: Fast P Skip can improve encoding speed at the expense of a very slight quality drop. Leave this option set to Off (to allow for Fast P Skips) is recommended unless encoding for maximum quality with no regard to encoding time.
Command Line: --no-fast-pskip
(disables fast P skip)

---

Description: This option select the way motion is detected by x264. Motion is what compression codecs are all about, tracking differences between scenes to allocate the various frame types and bitrates.
Available Options (listed from fastest to slowest, encoding speed wise):
Diamond: Only use when maximum encoding speed is required.
Hexagon: A good option for when quality is not the most important factor, but still important.
Multi Hex: Also known as "Uneven Multi-Hexagon". This is the most common setting, as it is a good trade off between speed and quality.
Exhaustive: Slowest and totally unnecessary (it isn't estimation anymore, rather, it's Motion Calculation, pixel by pixel).
Command Line: --me x
(where 'x' is 'dia', 'hex', 'umh' or 'esa')

---

Description: Also known as "Partition Decision". A very important option that determines how x264 makes decisions about motion estimation. The options are available from 1 to 7, with 1 being the fastest (lowest quality) and 7 being the slowest (best quality). A setting of 6 is the most common, with 1 or 7 being used in extreme cases (fastest encoding or best quality). A setting of 6 or 7 enables the B-frames RDO option (see "B-Frames options" section).
Command Line: --subme n
(where 'n' is the estimation value)

---

Description: This option when turned on is supposed to improve quality, but at a severe slow down of encoding. The quality improvements are also subjective. Leaving it Off is recommended unless you have a fast computer and using multi-pass encoding. Never turn it on in single pass quantizer mode, as it would lead to unexpected results.
Available Options:
Disabled: Use this for 1-pass encoding
Final MB: Use this for typical 2-pass encoding
Always: Use this for maximum quality (slowest speed)
Command Line: --trellis n
(where 'n' is 0 {disabled}, 1 {Final MB} or 2 {Always})

---

Description: These options determine the partition search types. Generally, you want to enable all of these options whenever possible, as they will improve encoding quality. The "P4x4" option requires the "P8x8" option.
Command Line: --analyse x
(where 'x' is 'p8x8', 'p4x4' {requires 'p8x8'}, 'b8x8', 'i8x8', 'i4x4' , 'none' or 'all')

---

Description: This option has been separated from the above because it has a different command line flag. This option is required if you want to select the "I8x8" option above.
Command Line: --8x8dct
(enables 8x8 DCT)

---

Description: This option when enabled shows you how many frames have been encoded and how many frames are left. Best to leave it On if you don't want to sit there guessing how much is still left to do.
Command Line: --progress
(enables progress indicator)

---

Description: This option disables noise reduction calculations. It is recommended that you leave this option On (to diable PSNR).
Command Line: --no-psnr
(disables PSNR computation)

---

Description: This sets the number of CPU threads to use in encoding. An Intel Pentium processor with HyperThreading counts as 2 threads. Dual Core processors are also 2 threads. Dual core processor with HyperThreading (eg. Intel Extreme Edition) will have 4 threads.
Command Line: --threads n
(where 'n' is the number of threads)

---

Description: Selectable within the x264 command line by the output filename. A file with extension .264 will equal the "RAW" setting.
Command Line: --output x
(where 'x' is the output filename with the extension being .264, .mp4 or .mkv to indicate output type)