Difference between revisions of "Bluray"

From Jon's Wiki
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
State of play as of March 2009. Hopefully things will improve...
+
State of play as of December 2012.
  
 
== Rip ==
 
== Rip ==
Currently the only way to rip Bluray discs on a PC is with AnyDVD HD in Windows. In Linux, one can run Windows XP with AnyDVD HD installed in a VM, then share the Bluray drive and mount it with samba in the host. In theory, the mounted share should be the decrypted Bluray disc, as AnyDVD HD should decrypt on the fly. I tried this once and it didn't work.
+
Until recently the only way to rip Bluray discs on a PC was with AnyDVD HD in Windows, which copies the contents of the disc into a folder after removing the encryption and DRM. You then had to use something else to select streams from the M2TS files, and remux them into a sensible container (Matroska).
 +
 
 +
=== Use MakeMKV ===
 +
Now you can skip several steps by using [http://www.makemkv.com/ MakeMKV] instead. This will produce a Matroska file straight from the disc, with the desired streams you select, leaving out all the various subtitles and foreign language dub tracks you don't need.
  
 
== Transcode ==
 
== Transcode ==
  
=== Modified MPEG-TS Container ===
+
=== Transcode to H.264 ===
Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called [http://www.avchd-info.org/ AVCHD]. So in theory, a decrypted M2TS file should be sufficient for playback, but most players choke probably due to the modified MPEG-TS format. Currently only mplayer works reliably.
+
 
 +
;NOTE: ffmpeg developers fed up with Michael Niedermayer forked it in March 2011 and formed the ''libav'' project, with its ''avconv'' tool. Most of the key ffmpeg developers now contribute to this project instead. Update 2020: looks like it's fizzled out.
 +
 
 +
In this example, I am processing my [http://www.blu-ray.com/movies/Baraka-Blu-ray/1529/#Review Bluray copy] of [http://www.imdb.com/title/tt0103767 Baraka].
 +
 
 +
Having obtained a Matroska file containing the main title of the disc, we can use avconv to squash the video track from 28 GB down to about 10 GB using the constant rate factor quality setting:
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 Baraka.mkv
  
=== VC-1 Support ===
+
=== Crop ===
Players based on mplayer and ffmpeg 0.5 can decode VC-1. It is more CPU intensive than H264, probably due to code maturity.
+
We can determine the crop using the ''cropdetect'' video filter, which will output some numbers we can then use in the encode:
 +
avconv -i Baraka-rip.mkv -vf cropdetect /tmp/x.mkv
  
=== Transcode to H264 ===
+
This will splurge output for an optimum crop per frame, which should settle out on:
In this example I'm transcoding my copy of Baraka. If ffmpeg could deal with the M2TS container, I could go
+
[cropdetect @ 0xa3c540]  ''[...]''  crop=1920:876:0:102
ffmpeg -i 00017.M2TS -acodec copy -vcodec libx264 -crf 15 Baraka.mkv
 
  
However since it can't, let's use tsmuxer<ref>http://www.smlabs.net/tsmuxer_en.html</ref> to fish out the video, audio and subtitle tracks. It can also extract the AC3 and DTS cores from TruHD and HD Master Audio tracks respectively as well:
+
Which we now add to our transcode command:
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
 +
    -vf crop=1920:876:0:102 Baraka.mkv
  
[[Image:Tsmuxer.png]]
+
=== User defined H.264 preset ===
 +
For best quality, we can improve the H.264 options by creating our own H.264 preset.<ref>https://libav.org/avconv.html#Preset-files</ref> To do this, we stick a bunch of good quality x264 options in ''~/.avconv/libx264-gnarly.avpreset''. Then we simply go
  
Now we can use ffmpeg:
+
  avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
  ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 Baraka.mkv
+
    -pre gnarly \
 +
    -vf crop=1920:876:0:102 Baraka.mkv
  
Since Baraka has 102 pixels of letterbox top and bottom, the ffmpeg command is:
+
Since quality is paramount, we should use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel search methods, increase the motion estimation range (especially for HD resolutions), turn on the 8x8 DCT, enable weighted B frame prediction, bump up the B frame options, use mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas. This produces a great quality, highly efficient encode (small file size), at the expense of much longer encoding time and higher CPU demand when decoding. Here's my preset file:
ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
 
    -croptop 102 -cropbottom 102 -aspect 2.19 Baraka.mkv
 
  
=== H264 Options ===
+
# THIS PRESET FILE IS NOW OBSOLETE
For best quality, we can improve the H264 options on the ffmpeg command line.<ref>http://sites.google.com/site/linuxencoding/x264-ffmpeg-mapping</ref>
+
# It was written for ffmpeg and needs an avconv rewrite.
 +
# For now just use the slower libx264 preset, with -pre slower
 +
g=250
 +
keyint_min=25
 +
sc_threshold=40
 +
i_qfactor=0.71
 +
qcomp=0.6
 +
qmin=10
 +
qmax=51
 +
qdiff=4
 +
me_range=24
 +
me_method=umh
 +
subq=7
 +
cmp=+chroma
 +
partitions=+parti8x8+parti4x4+partp8x8+partb8x8
 +
refs=5
 +
bf=16
 +
wpredp=2
 +
b_strategy=2
 +
directpred=3
 +
coder=1
 +
trellis=2
 +
flags=+loop
 +
flags2=+bpyramid+wpred+mixed_refs+dct8x8-fastpskip
  
Single pass constant rate factor (CRF) is best, since two-pass is only relevant if we care about exact file size. Since quality is paramount, I should also use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel searching methods, and use multiple threads to speed it up:
+
Single pass constant rate factor (CRF) is best, since two-pass is only relevant if we care about exact file size. A good CRF range is 18-23, with 0 being lossless and 51 being maximum compression and probably awful. We can also use multiple threads to speed it up:
ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
 
    -croptop 102 -cropbottom 102 -aspect 2.19 \
 
    -bf 16 -refs 6 -coder 1 -trellis 2 -me_method umh -subq 9 -threads 4 Baraka.mkv
 
  
Or for a crazy HQ encode that takes ages, increase the motion estimation range (good for HD), turn on the 8x8 DCT, enable weighted B frame prediction and mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas:
+
  avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
  ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
+
     -pre gnarly \
     -croptop 102 -cropbottom 102 -aspect 2.19 \
+
     -threads auto \
     -bf 16 -refs 6 -coder 1 -trellis 2 -me_method umh -subq 9 -threads 4
+
     -vf crop=1920:876:0:102 Baraka.mkv
     -me_range 32 -flags2 +dct8x8+wpred+bpyramid+mixed_refs-fastpskip Baraka.mkv
 
  
 
=== Cartoons ===
 
=== Cartoons ===
For anime and cartoons, bump up the deblocking and noise reduction filters:
+
For anime and cartoons, bump up the deblocking and noise reduction filters.
ffmpeg -deblockalpha 2 -deblockbeta 2 -nr 150 ...
+
 
 +
== Subtitles ==
 +
Bluray subtitle streams can now mux into Matroska files fine. You may want to convert them to plain text SRT using some of the subtitle OCR tools out there if you really have nothing better to do, or just pull one off one of the dozens of dodgy websites dedicated to that sort of thing.
 +
 
 +
== M2TS files ==
 +
 
 +
Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called [http://www.avchd-info.org/ AVCHD]. As of 2012 most players can now handle playing decrypted M2TS files.
 +
 
 +
=== TS Muxer ===
 +
We can also use tsmuxer<ref>http://www.smlabs.net/tsmuxer_en.html</ref> to deal with M2TS files, and in particular extract the AC3 and DTS cores from Dolby TrueHD and DTS Master Audio tracks, respectively.
 +
 
 +
[[Image:Tsmuxer.png]]
  
== MIME Type ==
+
=== Adding a MIME type ===
  
Edit /usr/share/mime/packages/bluray.xml and copy in this:
+
Ubuntu still doesn't see these files as video files - edit /usr/share/mime/packages/bluray.xml and copy in this:
  
 
  <?xml version="1.0" encoding="UTF-8"?>
 
  <?xml version="1.0" encoding="UTF-8"?>
Line 70: Line 111:
 
You should then be able to open M2TS files with an mplayer-based media player.
 
You should then be able to open M2TS files with an mplayer-based media player.
  
== Subtitles ==
+
== Playback ==
Bluray subtitle streams currently need converting to SUB/IDX format before they will reliably go in Matroska files. This conversion can easily be done with BDSup2Sub<ref>http://www.videohelp.com/tools/BDSup2Sub</ref>. Choose Lanczos3 for best quality if changing resolution.
+
 
 +
NVidia has VDPAU. Dear ATI, where's that fancy video acceleration you promised? In the meantime, we can speed up playback using multithreaded mplayer<ref>http://www.mplayerhq.hu/design7/news.html</ref>:
 +
git clone git://repo.or.cz/mplayer
 +
cd mplayer
 +
git checkout origin/mt
 +
git submodule init
 +
git submodule update
 +
sudo apt-get build-dep mplayer
 +
./configure --prefix=/usr
 +
make
 +
sudo checkinstall --fstrans=no --install=no --pkgname=mplayer-mt --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default
 +
 
 +
mplayer -lavdopts threads=4 Baraka.mkv
 +
And if it still sucks,
 +
 
 +
mplayer -lavdopts skipframe=nonref:skiploopfilter=all:fast=1:threads=4 Baraka.mkv
  
 
== References ==
 
== References ==
  
 
<references />
 
<references />

Latest revision as of 21:58, 14 December 2020

State of play as of December 2012.

Rip

Until recently the only way to rip Bluray discs on a PC was with AnyDVD HD in Windows, which copies the contents of the disc into a folder after removing the encryption and DRM. You then had to use something else to select streams from the M2TS files, and remux them into a sensible container (Matroska).

Use MakeMKV

Now you can skip several steps by using MakeMKV instead. This will produce a Matroska file straight from the disc, with the desired streams you select, leaving out all the various subtitles and foreign language dub tracks you don't need.

Transcode

Transcode to H.264

NOTE
ffmpeg developers fed up with Michael Niedermayer forked it in March 2011 and formed the libav project, with its avconv tool. Most of the key ffmpeg developers now contribute to this project instead. Update 2020: looks like it's fizzled out.

In this example, I am processing my Bluray copy of Baraka.

Having obtained a Matroska file containing the main title of the disc, we can use avconv to squash the video track from 28 GB down to about 10 GB using the constant rate factor quality setting:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 Baraka.mkv

Crop

We can determine the crop using the cropdetect video filter, which will output some numbers we can then use in the encode:

avconv -i Baraka-rip.mkv -vf cropdetect /tmp/x.mkv

This will splurge output for an optimum crop per frame, which should settle out on:

[cropdetect @ 0xa3c540]  [...]  crop=1920:876:0:102

Which we now add to our transcode command:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -vf crop=1920:876:0:102 Baraka.mkv

User defined H.264 preset

For best quality, we can improve the H.264 options by creating our own H.264 preset.[1] To do this, we stick a bunch of good quality x264 options in ~/.avconv/libx264-gnarly.avpreset. Then we simply go

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -pre gnarly \
    -vf crop=1920:876:0:102 Baraka.mkv

Since quality is paramount, we should use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel search methods, increase the motion estimation range (especially for HD resolutions), turn on the 8x8 DCT, enable weighted B frame prediction, bump up the B frame options, use mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas. This produces a great quality, highly efficient encode (small file size), at the expense of much longer encoding time and higher CPU demand when decoding. Here's my preset file:

# THIS PRESET FILE IS NOW OBSOLETE
# It was written for ffmpeg and needs an avconv rewrite.
# For now just use the slower libx264 preset, with -pre slower
g=250
keyint_min=25
sc_threshold=40
i_qfactor=0.71
qcomp=0.6
qmin=10
qmax=51
qdiff=4
me_range=24
me_method=umh
subq=7
cmp=+chroma
partitions=+parti8x8+parti4x4+partp8x8+partb8x8
refs=5
bf=16
wpredp=2
b_strategy=2
directpred=3
coder=1
trellis=2
flags=+loop
flags2=+bpyramid+wpred+mixed_refs+dct8x8-fastpskip

Single pass constant rate factor (CRF) is best, since two-pass is only relevant if we care about exact file size. A good CRF range is 18-23, with 0 being lossless and 51 being maximum compression and probably awful. We can also use multiple threads to speed it up:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -pre gnarly \
    -threads auto \
    -vf crop=1920:876:0:102 Baraka.mkv

Cartoons

For anime and cartoons, bump up the deblocking and noise reduction filters.

Subtitles

Bluray subtitle streams can now mux into Matroska files fine. You may want to convert them to plain text SRT using some of the subtitle OCR tools out there if you really have nothing better to do, or just pull one off one of the dozens of dodgy websites dedicated to that sort of thing.

M2TS files

Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called AVCHD. As of 2012 most players can now handle playing decrypted M2TS files.

TS Muxer

We can also use tsmuxer[2] to deal with M2TS files, and in particular extract the AC3 and DTS cores from Dolby TrueHD and DTS Master Audio tracks, respectively.

Tsmuxer.png

Adding a MIME type

Ubuntu still doesn't see these files as video files - edit /usr/share/mime/packages/bluray.xml and copy in this:

<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
  <mime-type type="video/bluray">
    <alias type="application/bluray"/>
    <alias type="video/avchd"/>
    <sub-class-of type="video/mpeg"/>
    <comment>Blu-ray/AVCHD Video</comment>
    <magic priority="50">
      <match value="\x47\x40\x00\x10\x00\xb0\x00\x11" type="string" offset="4"/>
    </magic>
    <generic-icon name="video-x-generic"/>
    <glob pattern="*.m2ts"/>
    <glob pattern="*.mts"/>
  </mime-type>
</mime-info>

Then update your MIME database:

sudo update-mime-database

You should then be able to open M2TS files with an mplayer-based media player.

Playback

NVidia has VDPAU. Dear ATI, where's that fancy video acceleration you promised? In the meantime, we can speed up playback using multithreaded mplayer[3]:

git clone git://repo.or.cz/mplayer
cd mplayer
git checkout origin/mt
git submodule init
git submodule update
sudo apt-get build-dep mplayer
./configure --prefix=/usr
make
sudo checkinstall --fstrans=no --install=no --pkgname=mplayer-mt --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default
mplayer -lavdopts threads=4 Baraka.mkv

And if it still sucks,

mplayer -lavdopts skipframe=nonref:skiploopfilter=all:fast=1:threads=4 Baraka.mkv

References