Scaling up Unix Commands Quickly

The other day, it annoyed me I could not get a one line program to run on every single file in a directory. I was close and now I want to have a reference to use for all situations. (Tethering really uses up a mobile phone’s data!)

I will share two techniques to scale up a command I use to make a gif to work on any matching file in a directory without additional human intervention. These techniques will work with any other Unix command you need to scale up.

This article will take you less than four minutes to read.

source and more information

Introduction

I have been making gifs for my last two articles on org-mode, lists and checklists.

Whenever possible, I want to show and tell, gifs of the screen capture allow that.

After making gifs for two articles in a row where I had to make at least three gifs each, I noticed a bottleneck: creating gifs through the command line gets annoying.

The command I use is:

ffmpeg -i source.mov -vf "scale=iw/2:ih/2" -pix_fmt rgb24 -r 15 -f gif - | gifsicle --delay=3 > source.gif

source

It is a sweet one-liner and it’s great if I have to make one gif because the only changes to the command is source.mov and source.gif, the input and output names.

Upon closer inspection, the only difference between the changes is: changing the file ending from mov to gif. I name my screen captures appropriately and want the gif output name to match the input name.

Can I make gif from mov files in a better way? Especially when I have multiple ones all queued up in a folder?

Yes, there is a better way to do this than converting each one by one, each time renaming the input and output file ending:

• Write a Bash Script
• Beef up the one-liner

Bash Script

The following would be the bash script to take all the files that end with .mov and makes a gif of them. The directory will have both source.mov and source.gif

for f in *.mov; do
  filename=${f%.mov}
  ffmpeg -i $filename.mov -vf "scale=iw/2:ih/2" -pix_fmt rgb24 -r 15 -f gif - | gifsicle --delay=3 > $filename.gif
done

inspiration

• for f in *.mov; do - this is the start of a for loop, this grabs all the files that have an ending of .mov in a directory.
• filename=${f%.mov} - this removes the .mov from the file name using bash parameter expansion.
• To use the original command, change source value to $filename.

Codifing the operation in a bash script provides:

• simplicity - if you know bash loops and how to get everything in a directory.
• extendable - if you need to add another step in between, say deleting the source.mov.
• a record - allowing you to easily refer back to it in the future.

Maybe the only drawback would be that this is yet another file to save somewhere (or manage in your script repository - which you have, right?!)

Either way, this script makes quick work of converting every .mov file into a gif. One, two or even a million files. Better than what I was starting with!

Beef up the one-liner

Another option, inspired by my parsing SSH logs article is to make a beefed up one-liner. No script, just do everything in place.

This took work, but this is the current result:

ls *.mov | sed -E "s/\.[a-z][a-z][a-z]//g" | xargs -I % sh -c "ffmpeg -i %.mov -vf "scale=iw/2:ih/2" -pix_fmt rgb24 -r 15 -f gif - | gifsicle --delay=3 > %.gif"

Description of the key moving parts:

• ls *.mov - use ls has the filtering mechanism. I know there are issues with this, it feels the most natural to me.
• sed -E "s/\.[a-z][a-z][a-z]//g" - this uses sed to remove any file ending, only preserving the filename.
• xargs -I % - use xargs to process each line and run a particular command. The -I % parameter specifies xargs to replace % in its argument with the output from the previous command.
• sh -c - as the command to make a gif uses two commands, ffmpeg and gifsicle, proper command execution requires sh -c to encapsulate the command.

I find this solution useful whenever:

• I cannot save a file - especially when I’m working on a remote server instance.
• I do not have the original bash script handy.
• I figured out the one line I need to get things done, and just want to scale up immediately.

This method is trickier as I do have to memorize the structure of the command. It was not easy deriving it on the spot. I believe I will start remembering this more.

Conclusion

From here on, whenever I need to scale up a Unix command to process different files, I have two methods at my disposal:

• write a bash script in the form: for f in *; do...
• after figuring out the one-liner for what I need done, beef it up using sed and xargs

Each have their place in my toolbox, most of the time it comes to access and urgency.

One thing nice about both of these methods: they scale and avoid human interactions in that scaling process.