Python – 3.x – uses IO to pass bytes to ffmpeg in python

Sorry, new to stackoverflow

Just wondering if it is possible to pass byte data from io.
I’m trying to extract frames from a gif using ffmpeg and then resize it using Pillow.
I know you can use Pillow to extract frames from GIFs, but sometimes it breaks some GIFs. So I used ffmpeg as a fix.
As for why I want to read the gif from memory because I’m changing this setting, the gif from the url will be wrapped in Bytesio instead of saved.
As for why I have extra Pillow code, I did succeed in getting it to work by passing the actual filename to the ffmpeg command.

original_pil = Image.open("1.gif")

bytes_io = open("1.gif", "rb")
bytes_io.seek(0)

ffmpeg = 'ffmpeg'

cmd = [ffmpeg,
       '-i', '-',
       '-vsync', '0',
       '-f', 'image2pipe',
       '-pix_fmt', 'rgba',
       '-vcodec', 'png',
       '-report',
       '-']

depth = 4
width, height = original_pil.size
buf_size = depth * width * height + 100
nbytes = width * height * 4

proc = SP. Popen(cmd, stdout=SP. PIPE, stdin=SP. PIPE, stderr=SP. PIPE, bufsize=buf_size, shell=False)
out, err = proc.communicate(input=bytes_io.read(), timeout=None)

FFMPEG Report:

ffmpeg started on 2021-06-07 at 18:58:14
Report written to "ffmpeg-20210607-185814.log"
Command line:
ffmpeg -i - -vsync 0 -f image2pipe -pix_fmt rgba -vcodec png -report -
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 -- enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass -- enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi -- enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse -- enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame -- enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi -- enable-lv2 --ena  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 -- enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass -- enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi -- enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse -- enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame -- enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi -- enab  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Splitting the commandline.
Reading option '-i' ... matched as input url with argument '-'.
Reading option '-vsync' ... matched as option 'vsync' (video sync method) with argument '0'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'image2pipe'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'rgba'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'png'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option vsync (video sync method) with argument 0.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url -.
Successfully parsed a group of options.
Opening an input file: -.
[NULL @ 0x55b59c38f7c0] Opening 'pipe:' for reading
[pipe @ 0x55b59c390240] Setting default whitelist 'crypto'
[gif @ 0x55b59c38f7c0] Format gif probed with size=2048 and score=100
[AVIOContext @ 0x55b59c398680] Statistics: 4614093 bytes read, 0 seeks
pipe:: Input/output error

For a single image, your code works fine.
Looks like you’re missing proc.wait() at the end and that’s it.

Multiple images can be seen in my post here .
You can simplify your code to work with images.

I

made some changes to your code to make it more elegant (I think) :

• You do not need the '-vsync', '0' parameter.

I

• replaced ‘-‘ with ‘pipe:' (I think that’s clearer).
• You don’t need to set bufsize unless you know that the default value is too small.
• I removed stderr=SP. PIPE, because I don’t want to see FFmpeg logs in the console.
• I added proc.wait() after proc.communicate.

The code example first builds a composite GIF image file for testing.

Here is the code example:

import subprocess as sp
import shlex
from PIL import Image
from io import BytesIO

# Build synthetic image tmp.gif for testing
sp.run(shlex.split(f'ffmpeg -y -f lavfi -i testsrc=size=128x128:rate=1:duration=1 tmp.gif'))

original_pil = Image.open('tmp.gif')

bytes_io = open('tmp.gif', "rb")
bytes_io.seek(0)

ffmpeg = 'ffmpeg'

cmd = [ffmpeg,
       '-i', 'pipe:',
       #'-vsync', '0',
       '-f', 'image2pipe',
       '-pix_fmt', 'rgba',
       '-vcodec', 'png',
       '-report',
       'pipe:']

proc = sp. Popen(cmd, stdout=sp. PIPE, stdin=sp. PIPE)
out = proc.communicate(input=bytes_io.read())[0]

proc.wait()

bytes_io_png = BytesIO(out)
img = Image.open(bytes_io_png)
img.show()

Output:

Pass multiple images:

If you have multiple images, you can use proc.communicate only if all images are in RAM.
Instead of grabbing all the images to RAM and passing the images to FFmpeg, use a writer thread and a for loop.

I’ve tried uploading PNG images, but it’s too messy.
I changed the code to pass an image in RAW format.
The advantage of RAW images is that the byte size of all images is known in advance.

Here is a code example (without BytesIO):

import numpy as np
import subprocess as sp
import shlex
from PIL import Image
import threading

# Write gif images to stdin pipe.
def writer(stdin_pipe):
    # Write 30 images to stdin pipe (for example)
    for i in range(1, 31):
        in_file_name = 'tmp' + str(i).zfill(2) + '.gif'

with open(in_file_name, "rb") as f:  
            proc.stdin.write(f.read())  # Write bytes to stdin pipe

stdin_pipe.close()

# Build 30 synthetic images tmp01.gif, tmp02.gif, ..., tmp31.gif for testing
sp.run(shlex.split(f'ffmpeg -y -f lavfi -i testsrc=size=128x128:rate=1:duration=30 -f image2 tmp%02d.gif'))

original_pil = Image.open("tmp01.gif")
depth = 4
width, height = original_pil.size
nbytes = width * height * 4

ffmpeg = 'ffmpeg'

cmd = [ffmpeg,
       '-i', 'pipe:',
       '-f', 'image2pipe',
       '-pix_fmt', 'rgba',
       '-vcodec', 'rawvideo',  # Select rawvideo codec
       '-report',
       'pipe:']

proc = sp. Popen(cmd, stdout=sp. PIPE, stdin=sp. PIPE)

thread = threading. Thread(target=writer, args=(proc.stdin,))
thread.start()  # Strat writer thread

while True:
    in_bytes = proc.stdout.read(nbytes)  # Read raw image bytes from stdout pipe.
    raw_imag = np.frombuffer(in_bytes, np.uint8).reshape([height, width, 4])

img = Image.fromarray(raw_imag)
    img.show()

# Break the loop when number of bytes read is less then expected size.
    if len(in_bytes) < nbytes:
        break

proc.wait()
thread.join()