Python AsyncIO & FFmpeg:视频时光倒流术

前言

本文将通过编程的方式倒序视频的每一帧,实现视频的倒放。

开发环境

效果预览


实现流程

1、获取视频基本信息

ffprobe -i v1.mp4 -v error -print_format json -show_streams
{
    "streams": [
        {
            "index": 0,
            "codec_name": "h264",
            "codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
            "profile": "High",
            "codec_type": "video",
            "codec_tag_string": "avc1",
            "codec_tag": "0x31637661",
            "width": 1280,
            "height": 720,
            "coded_width": 1280,
            "coded_height": 720,
            "closed_captions": 0,
            "has_b_frames": 2,
            "sample_aspect_ratio": "1:1",
            "display_aspect_ratio": "16:9",
            "pix_fmt": "yuv420p",
            "level": 31,
            "color_range": "tv",
            "color_space": "bt709",
            "color_transfer": "bt709",
            "color_primaries": "bt709",
            "chroma_location": "left",
            "refs": 1,
            "is_avc": "true",
            "nal_length_size": "4",
            "r_frame_rate": "30/1",
            "avg_frame_rate": "30/1",
            "time_base": "1/15360",
            "start_pts": 0,
            "start_time": "0.000000",
            "duration_ts": 108032,
            "duration": "7.033333",
            "bit_rate": "1756853",
            "bits_per_raw_sample": "8",
            "nb_frames": "211",
            "disposition": {
                "default": 1,
                "dub": 0,
                "original": 0,
                "comment": 0,
                "lyrics": 0,
                "karaoke": 0,
                "forced": 0,
                "hearing_impaired": 0,
                "visual_impaired": 0,
                "clean_effects": 0,
                "attached_pic": 0,
                "timed_thumbnails": 0
            },
            "tags": {
                "language": "und",
                "handler_name": "VideoHandler",
                "vendor_id": "[0][0][0][0]"
            }
        }
        # 此处省略音频流信息...
    ]
}
  • width/height:视频尺寸
  • r_frame_rate:视频帧率
  • nb_frames:总帧数
  • pix_fmt:视频像素格式

2、视频编解码

Tips:FFmepg 命令中的参数 pipe: 可以简写为 -

首先,需要将原视频做为输入,将原视频 RGB 数据输出到 PIPE 管道。

ffmpeg -v error -i v1.mp4 -f rawvideo -pix_fmt rgb24 -
  • -v:日志级别,仅显示错误信息
  • -i:输入源
  • -f:编码格式,这里使用 rawvideo,仅包含原始视频数据,无任何其他的附加信息
  • -pix_fmt:像素格式,这里使用 RGB 格式

然后,从 PIPE 管道读取所有数据,数据长度为 nb_frames * width * height * 3,每一帧长度为 width * height * 3,拆分数据为长度为 nb_frames 的数组,倒序该数组形成新的字节序列。

最后,将重排序好的帧数据,顺序写入 PIPE 管道,编码为新的视频。

ffmpeg -y -v error -f rawvideo -pix_fmt rgb24 -r 30 -s 1280x720 -i - -pix_fmt yuv420p t1.mp4

Python 源码

from asyncio.subprocess import PIPE
import asyncio
import json


async def probe(file_path: str):
    cmd = f'ffprobe -i {file_path} -v quiet -print_format json -show_streams'
    args = cmd.split(' ')
    process = await asyncio.create_subprocess_exec(*args, stdout=PIPE)
    stdout = await process.stdout.read()
    return json.loads(stdout)['streams']


async def main():
    source = '/Users/admin/Downloads/videos/v1.mp4'
    target = '/Users/admin/Downloads/videos/t1.mp4'

    # get video stream info
    v_info = await probe(source)
    v_stream_info = next((stream for stream in v_info if stream['codec_type'] == 'video'), None)
    width, height = v_stream_info['width'], v_stream_info['height']
    nb_frames = int(v_stream_info['nb_frames'])
    rate = v_stream_info['r_frame_rate']
    bit_rate = v_stream_info['bit_rate']
    pix_fmt = v_stream_info['pix_fmt']

    # create read process
    cmd = f'ffmpeg -v error -i {source} -f rawvideo -pix_fmt rgb24 -'
    args = cmd.split(' ')
    read_process = await asyncio.create_subprocess_exec(*args, stdout=PIPE)

    # read frames
    data = await read_process.stdout.read()

    # create write process
    cmd = f'ffmpeg -y -v error -f rawvideo -pix_fmt rgb24 -r {rate} -s {width}x{height} -i - -pix_fmt {pix_fmt} -b:v {bit_rate} {target}'
    args = cmd.split(' ')
    write_process = await asyncio.create_subprocess_exec(*args, stdin=PIPE)

    # write reversed frames
    frame_size = width * height * 3
    for i in range(nb_frames):
        write_process.stdin.write(data[(nb_frames - i - 1) * frame_size:(nb_frames - i) * frame_size])
    await write_process.stdin.drain()
    write_process.stdin.close()

    await write_process.wait()
    print(f'End...')


if __name__ == '__main__':
    asyncio.run(main())


相关问题

1、文章开头的效果图是如何生成的?

首先,将两段视频分别添加上文字【Normal】和【Reversed】;然后垂直拼接并缩放尺寸为原来的一半;最后导出为 GIF 图。命令如下:

ffmpeg -y -i v1.mp4 -i t1.mp4 \
-filter_complex "[0:v]drawtext=text='Normal':fontcolor=white:fontsize=24:x=40:y=20[v1];\
[1:v]drawtext=text='Reversed':fontcolor=white:fontsize=24:x=40:y=20[v2];\
[v1][v2]vstack,scale=w=iw/2:h=ih/2[v]" \
-map "[v]" -an -r 8 output.gif

注意,使用 -r 8 设置 FPS,不然生成的 GIF 占用的存储空间会很大,放到浏览器加载就会非常耗时!

2、千万不要使用 await read_process.stdout.read(width * height * 3) 从 PIPE 读取一帧数据

以下是 python 官方函数的说明,read 只会尝试读取 n bytes,但是返回的结果一定 <= n

    async def read(self, n=-1):
        """Read up to `n` bytes from the stream.

        If n is not provided, or set to -1, read until EOF and return all read
        bytes. If the EOF was received and the internal buffer is empty, return
        an empty bytes object.

        If n is zero, return empty bytes object immediately.

        If n is positive, this function try to read `n` bytes, and may return
        less or equal bytes than requested, but at least one byte. If EOF was
        received before any byte is read, this function returns empty byte
        object.

        Returned value is not limited with limit, configured at stream
        creation.

        If stream was paused, this function will automatically resume it if
        needed.
        """

经多次测试,最多读取 65536 bytes,最大值可能与操作系统某些配置有关。

但是,官方提供了另外一个函数 readexactly(n),可以使用 await read_process.stdout.readexactly(width * height * 3) 读取完整的一帧数据。

3、尽量不是使用 await read_process.stdout.read() 从 PIPE 中一次性读取所有数据

假设视频尺寸为 1280 x 720,时长 60s,FPS 30,那么一次性读取所有数据会占用内存 1280 * 720 * 3 * 60 * 30 ≈ 4.63 GB


参考文档

推荐阅读更多精彩内容