Difference between revisions of "Bluray"

From Jon's Wiki
 
(33 intermediate revisions by the same user not shown)
Line 1: Line 1:
State of play as of March 2009. Hopefully things will improve...
+
State of play as of December 2012.
  
 
== Rip ==
 
== Rip ==
Currently the only way to rip Bluray discs on a PC is with AnyDVD HD in Windows. In Linux, one can run Windows XP with AnyDVD HD installed in a VM, then share the Bluray drive and mount it with samba in the host. In theory, the mounted share should be the decrypted Bluray disc, as AnyDVD HD should decrypt on the fly. I tried this once and it didn't work.
+
Until recently the only way to rip Bluray discs on a PC was with AnyDVD HD in Windows, which copies the contents of the disc into a folder after removing the encryption and DRM. You then had to use something else to select streams from the M2TS files, and remux them into a sensible container (Matroska).
 +
 
 +
=== Use MakeMKV ===
 +
Now you can skip several steps by using [http://www.makemkv.com/ MakeMKV] instead. This will produce a Matroska file straight from the disc, with the desired streams you select, leaving out all the various subtitles and foreign language dub tracks you don't need.
  
 
== Transcode ==
 
== Transcode ==
  
=== EASY! ===
+
=== Transcode to H.264 ===
 +
 
 +
;NOTE: ffmpeg developers fed up with Michael Niedermayer forked it in March 2011 and formed the ''libav'' project, with its ''avconv'' tool. Most of the key ffmpeg developers now contribute to this project instead. Update 2020: looks like it's fizzled out.
 +
 
 +
In this example, I am processing my [http://www.blu-ray.com/movies/Baraka-Blu-ray/1529/#Review Bluray copy] of [http://www.imdb.com/title/tt0103767 Baraka].
 +
 
 +
Having obtained a Matroska file containing the main title of the disc, we can use avconv to squash the video track from 28 GB down to about 10 GB using the constant rate factor quality setting:
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 Baraka.mkv
 +
 
 +
=== Crop ===
 +
We can determine the crop using the ''cropdetect'' video filter, which will output some numbers we can then use in the encode:
 +
avconv -i Baraka-rip.mkv -vf cropdetect /tmp/x.mkv
 +
 
 +
This will splurge output for an optimum crop per frame, which should settle out on:
 +
[cropdetect @ 0xa3c540]  ''[...]''  crop=1920:876:0:102
 +
 
 +
Which we now add to our transcode command:
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
 +
    -vf crop=1920:876:0:102 Baraka.mkv
 +
 
 +
=== User defined H.264 preset ===
 +
For best quality, we can improve the H.264 options by creating our own H.264 preset.<ref>https://libav.org/avconv.html#Preset-files</ref> To do this, we stick a bunch of good quality x264 options in ''~/.avconv/libx264-gnarly.avpreset''. Then we simply go
 +
 
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
 +
    -pre gnarly \
 +
    -vf crop=1920:876:0:102 Baraka.mkv
 +
 
 +
Since quality is paramount, we should use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel search methods, increase the motion estimation range (especially for HD resolutions), turn on the 8x8 DCT, enable weighted B frame prediction, bump up the B frame options, use mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas. This produces a great quality, highly efficient encode (small file size), at the expense of much longer encoding time and higher CPU demand when decoding. Here's my preset file:
 +
 
 +
# THIS PRESET FILE IS NOW OBSOLETE
 +
# It was written for ffmpeg and needs an avconv rewrite.
 +
# For now just use the slower libx264 preset, with -pre slower
 +
g=250
 +
keyint_min=25
 +
sc_threshold=40
 +
i_qfactor=0.71
 +
qcomp=0.6
 +
qmin=10
 +
qmax=51
 +
qdiff=4
 +
me_range=24
 +
me_method=umh
 +
subq=7
 +
cmp=+chroma
 +
partitions=+parti8x8+parti4x4+partp8x8+partb8x8
 +
refs=5
 +
bf=16
 +
wpredp=2
 +
b_strategy=2
 +
directpred=3
 +
coder=1
 +
trellis=2
 +
flags=+loop
 +
flags2=+bpyramid+wpred+mixed_refs+dct8x8-fastpskip
 +
 
 +
Single pass constant rate factor (CRF) is best, since two-pass is only relevant if we care about exact file size. A good CRF range is 18-23, with 0 being lossless and 51 being maximum compression and probably awful. We can also use multiple threads to speed it up:
 +
 
 +
avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
 +
    -pre gnarly \
 +
    -threads auto \
 +
    -vf crop=1920:876:0:102 Baraka.mkv
 +
 
 +
=== Cartoons ===
 +
For anime and cartoons, bump up the deblocking and noise reduction filters.
 +
 
 +
== Subtitles ==
 +
Bluray subtitle streams can now mux into Matroska files fine. You may want to convert them to plain text SRT using some of the subtitle OCR tools out there if you really have nothing better to do, or just pull one off one of the dozens of dodgy websites dedicated to that sort of thing.
 +
 
 +
== M2TS files ==
 +
 
 +
Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called [http://www.avchd-info.org/ AVCHD]. As of 2012 most players can now handle playing decrypted M2TS files.
 +
 
 +
=== TS Muxer ===
 +
We can also use tsmuxer<ref>http://www.smlabs.net/tsmuxer_en.html</ref> to deal with M2TS files, and in particular extract the AC3 and DTS cores from Dolby TrueHD and DTS Master Audio tracks, respectively.
 +
 
 +
[[Image:Tsmuxer.png]]
 +
 
 +
=== Adding a MIME type ===
  
Just use tsmuxer - see http://www.smlabs.net/tsmuxer_en.html
+
Ubuntu still doesn't see these files as video files - edit /usr/share/mime/packages/bluray.xml and copy in this:
  
=== Modified MPEG-TS Container ===
+
<?xml version="1.0" encoding="UTF-8"?>
Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called [http://www.avchd-info.org/ AVCHD]. So in theory, a decrypted M2TS file should be sufficient for playback, but most players choke probably due to the modified MPEG-TS format. Currently only mplayer works reliably.
+
<nowiki><mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'></nowiki>
 +
  <mime-type type="video/bluray">
 +
    <alias type="application/bluray"/>
 +
    <alias type="video/avchd"/>
 +
    <sub-class-of type="video/mpeg"/>
 +
    <comment>Blu-ray/AVCHD Video</comment>
 +
    <magic priority="50">
 +
      <match value="\x47\x40\x00\x10\x00\xb0\x00\x11" type="string" offset="4"/>
 +
    </magic>
 +
    <generic-icon name="video-x-generic"/>
 +
    <glob pattern="*.m2ts"/>
 +
    <glob pattern="*.mts"/>
 +
  </mime-type>
 +
</mime-info>
  
=== VC-1 Support ===
+
Then update your MIME database:
Players based on mplayer and ffmpeg 0.5 can decode VC-1. It is more CPU intensive than H264, probably due to code maturity.
+
sudo update-mime-database
  
=== Transcode to H264 ===
+
You should then be able to open M2TS files with an mplayer-based media player.
In this example I'm transcoding my copy of Baraka. If ffmpeg could deal with the M2TS container, I could go
 
ffmpeg -i 00017.M2TS -acodec copy -vcodec libx264 -crf 15 Baraka.mkv
 
  
However since it can't, mplayer comes to the rescue by extracting everything first to eliminate the container from the equation:
+
== Playback ==
mplayer 00017.M2TS -dumpvideo -dumpfile 00017.vc1
 
mplayer 00017.M2TS -dumpaudio -dumpfile 00017.ac3
 
ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 Baraka.mkv
 
  
;NOTE: With other films, you may need to fiddle about with the mplayer -aid option to get it to fish out the right audio track, otherwise you'll end up with Spanish or something. I suppose that's fine if it's a Spanish film.
+
NVidia has VDPAU. Dear ATI, where's that fancy video acceleration you promised? In the meantime, we can speed up playback using multithreaded mplayer<ref>http://www.mplayerhq.hu/design7/news.html</ref>:
 +
git clone git://repo.or.cz/mplayer
 +
cd mplayer
 +
git checkout origin/mt
 +
git submodule init
 +
git submodule update
 +
sudo apt-get build-dep mplayer
 +
./configure --prefix=/usr
 +
make
 +
sudo checkinstall --fstrans=no --install=no --pkgname=mplayer-mt --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default
  
Since Baraka has 102 pixels of letterbox top and bottom, the ffmpeg command is:
+
  mplayer -lavdopts threads=4 Baraka.mkv
  ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
+
And if it still sucks,
    -croptop 102 -cropbottom 102 -aspect 2.19 Baraka.mkv
 
  
=== H264 Options ===
+
mplayer -lavdopts skipframe=nonref:skiploopfilter=all:fast=1:threads=4 Baraka.mkv
See http://ffmpeg.x264.googlepages.com/mapping
 
  
Single pass constant rate factor (CRF) is sufficient. Since quality is paramount, I should also use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel searching methods, and use multiple threads to speed it up:
+
== References ==
ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
 
    -croptop 102 -cropbottom 102 -aspect 2.19 \
 
    -bf 16 -refs 6 -coder 1 -trellis 2 -me_method umh -subq 9 -threads 4 Baraka.mkv
 
  
Or for a crazy HQ encode that takes ages, increase the motion estimation range (good for HD), turn on the 8x8 DCT, enable weighted B frame prediction and mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas:
+
<references />
ffmpeg -i 00017.ac3 -acodec copy -i 00017.vc1 -vcodec libx264 -crf 20 \
 
    -croptop 102 -cropbottom 102 -aspect 2.19 \
 
    -bf 16 -refs 6 -coder 1 -trellis 2 -me_method umh -subq 9 -threads 4
 
    -me_range 32 -flags2 +dct8x8+wpred+mixed_refs-fastpskip Baraka.mkv
 

Latest revision as of 21:58, 14 December 2020

State of play as of December 2012.

Rip

Until recently the only way to rip Bluray discs on a PC was with AnyDVD HD in Windows, which copies the contents of the disc into a folder after removing the encryption and DRM. You then had to use something else to select streams from the M2TS files, and remux them into a sensible container (Matroska).

Use MakeMKV

Now you can skip several steps by using MakeMKV instead. This will produce a Matroska file straight from the disc, with the desired streams you select, leaving out all the various subtitles and foreign language dub tracks you don't need.

Transcode

Transcode to H.264

NOTE
ffmpeg developers fed up with Michael Niedermayer forked it in March 2011 and formed the libav project, with its avconv tool. Most of the key ffmpeg developers now contribute to this project instead. Update 2020: looks like it's fizzled out.

In this example, I am processing my Bluray copy of Baraka.

Having obtained a Matroska file containing the main title of the disc, we can use avconv to squash the video track from 28 GB down to about 10 GB using the constant rate factor quality setting:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 Baraka.mkv

Crop

We can determine the crop using the cropdetect video filter, which will output some numbers we can then use in the encode:

avconv -i Baraka-rip.mkv -vf cropdetect /tmp/x.mkv

This will splurge output for an optimum crop per frame, which should settle out on:

[cropdetect @ 0xa3c540]  [...]  crop=1920:876:0:102

Which we now add to our transcode command:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -vf crop=1920:876:0:102 Baraka.mkv

User defined H.264 preset

For best quality, we can improve the H.264 options by creating our own H.264 preset.[1] To do this, we stick a bunch of good quality x264 options in ~/.avconv/libx264-gnarly.avpreset. Then we simply go

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -pre gnarly \
    -vf crop=1920:876:0:102 Baraka.mkv

Since quality is paramount, we should use B frames as references, turn CABAC and trellis encoding on, bump up the motion estimation and subpixel search methods, increase the motion estimation range (especially for HD resolutions), turn on the 8x8 DCT, enable weighted B frame prediction, bump up the B frame options, use mixed reference frames (especially if using multiple B frame references), and disable the fast P frame skip for smoother skies and plain areas. This produces a great quality, highly efficient encode (small file size), at the expense of much longer encoding time and higher CPU demand when decoding. Here's my preset file:

# THIS PRESET FILE IS NOW OBSOLETE
# It was written for ffmpeg and needs an avconv rewrite.
# For now just use the slower libx264 preset, with -pre slower
g=250
keyint_min=25
sc_threshold=40
i_qfactor=0.71
qcomp=0.6
qmin=10
qmax=51
qdiff=4
me_range=24
me_method=umh
subq=7
cmp=+chroma
partitions=+parti8x8+parti4x4+partp8x8+partb8x8
refs=5
bf=16
wpredp=2
b_strategy=2
directpred=3
coder=1
trellis=2
flags=+loop
flags2=+bpyramid+wpred+mixed_refs+dct8x8-fastpskip

Single pass constant rate factor (CRF) is best, since two-pass is only relevant if we care about exact file size. A good CRF range is 18-23, with 0 being lossless and 51 being maximum compression and probably awful. We can also use multiple threads to speed it up:

avconv -i Baraka-rip.mkv -c:a copy -c:v libx264 -crf 21 \
    -pre gnarly \
    -threads auto \
    -vf crop=1920:876:0:102 Baraka.mkv

Cartoons

For anime and cartoons, bump up the deblocking and noise reduction filters.

Subtitles

Bluray subtitle streams can now mux into Matroska files fine. You may want to convert them to plain text SRT using some of the subtitle OCR tools out there if you really have nothing better to do, or just pull one off one of the dozens of dodgy websites dedicated to that sort of thing.

M2TS files

Bluray video is stored in a modified version of the MPEG-TS container format in the BDMV/STREAM folder. The modification is an extra 4 byte time code on each MPEG-TS packet, increasing the packet size from 188 bytes to 192 bytes, apparently in order to better support buffering and out-of-order writing to random-access media. It is unclear whether this is an MPEG Committee sanctioned modification, but there appears to be some standardisation through something called AVCHD. As of 2012 most players can now handle playing decrypted M2TS files.

TS Muxer

We can also use tsmuxer[2] to deal with M2TS files, and in particular extract the AC3 and DTS cores from Dolby TrueHD and DTS Master Audio tracks, respectively.

Tsmuxer.png

Adding a MIME type

Ubuntu still doesn't see these files as video files - edit /usr/share/mime/packages/bluray.xml and copy in this:

<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
  <mime-type type="video/bluray">
    <alias type="application/bluray"/>
    <alias type="video/avchd"/>
    <sub-class-of type="video/mpeg"/>
    <comment>Blu-ray/AVCHD Video</comment>
    <magic priority="50">
      <match value="\x47\x40\x00\x10\x00\xb0\x00\x11" type="string" offset="4"/>
    </magic>
    <generic-icon name="video-x-generic"/>
    <glob pattern="*.m2ts"/>
    <glob pattern="*.mts"/>
  </mime-type>
</mime-info>

Then update your MIME database:

sudo update-mime-database

You should then be able to open M2TS files with an mplayer-based media player.

Playback

NVidia has VDPAU. Dear ATI, where's that fancy video acceleration you promised? In the meantime, we can speed up playback using multithreaded mplayer[3]:

git clone git://repo.or.cz/mplayer
cd mplayer
git checkout origin/mt
git submodule init
git submodule update
sudo apt-get build-dep mplayer
./configure --prefix=/usr
make
sudo checkinstall --fstrans=no --install=no --pkgname=mplayer-mt --pkgversion "1:0.svn`date +%Y%m%d`-0.0ubuntu1" --default
mplayer -lavdopts threads=4 Baraka.mkv

And if it still sucks,

mplayer -lavdopts skipframe=nonref:skiploopfilter=all:fast=1:threads=4 Baraka.mkv

References