MacOS

Slow Motion A.I. Tutorial

Artificial Intelligence has come a long way. Now days it can be used for all kinds of things related to Videography and Photography. From automatically removing unwanted things from pictures in Photoshop to putting your face right into a movie clip via the Deep Fake A.I., it’s uses are only going to continue to grow

One such use intrigued me after I saw it in action. A group of programmers developed an A.I. called Super-SloMo using the PyTorch framework which allows you to take normal footage and believable turn it into slow motion. https://github.com/avinashpaliwal/Super-SloMo

This is challenging because the A.I. has to predict what should fill in the missing frames. If you convert a video at 30 FPS to 240 FPS, the A.I. has to auto generate 7 extra frames between every real frame, which is no small task.

My desire to try it out was limited until I was editing some footage from my recent mountain climbing trip. I had done a motion time-lapse video that involved taking a photo every 2 seconds for 10 minutes as it panned across a valley. The shot, below, is cool enough, but it got me wondering if I could use this A.I. to smooth out the motion.

Time-lapse that I wanted to smooth out

So I ran this video through the A.I. and was so amazed by the results that I had to share the process with the world so that other’s could use it. The first video below is a comparison between slowing the video down as shot and what the A.I. was able to do.

See the video below to learn how to do this yourself.

Here are the typed instructions:

  1. Make a folder to keep all of the Slow Motion files together. Call it slomo

  2. Make an Input and Output folder in your slomo folder.

  3. Install a Python Distribution. I’m using Anaconda Distribution. Available for Windows, Mac and Linux - Download 3.7 64bit https://www.anaconda.com/distribution/

  4. Download the Super-SloMo project from GitHub Repo - Click Clone or Download, Download ZIP https://github.com/avinashpaliwal/Super-SloMo

  5. Extract Super-SlowMo Project into a folder. Rename superslomo

  6. Train the model (if you want). Or use a pre-trained model here - https://drive.google.com/file/d/1IvobLDbRiBgZr3ryCRrWL8xDbMZ-KnpF/view

  7. Place that model in the superslomo directory

  8. Download FFmpeg Video Converter - Download the stable version, the correct architecture and choose the Static Linking https://ffmpeg.zeranoe.com/builds/

  9. Unzip that in your slomo folder. Rename it to ffmpeg

  10. Open Terminal (Mac) or Anaconda Prompt (Windows)

    1. On a Mac, this requires using bash, not the zsh default that comes with Catalina. You can check what version you are using by typing: echo $SHELL. It will either say /bin/bash or /bin/zsh. If it says /bin/zsh you must change your shell. To switch to bash type: chsh -s /bin/bash. Then you must restart terminal. To change back to zsh after you're done with this project type: chsh -s /bin/zsh and restart terminal.

  11. Get the correct command for your setup - Choose Stable, Pick your OS, Choose Conda, Choose whatever version of Python you downloaded, Choose None for CUDA unless you have an External NVIDIA GPU https://pytorch.org/get-started/locally/

  12. Copy the command in the box labeled “Run this Command:” and paste it into your Python Terminal. This will download and install all the files and and environments you need

  13. cd into the slomo/superslomo directory

  14. Adjust the following script to match your computer setup

    • python video_to_slomo.py --ffmpeg /Users/user/slomo/ffmpeg/bin/ --video /Users/user/slomo/input/mountain_timelapse.mp4 --sf 4 --checkpoint /Users/user/slomo/superslomo/SuperSloMo.ckpt --fps 119.880 --output /Users/user/slomo/output/mountain120.mp4 --batch_size 1

    • Definitions of terms in script

      • sf = multiple of the frame rate

      • fps = target frame rate

      • batch _size = how fast it will run, limited by your computer’s hardware. 1 is default

  15. Take the finished footage into a video editor and slow it down by whatever multiple your sf was to see the magic.

OpenCL or Metal Sept. 2018 Update!

In January, 2016, I had just completed my 2nd video for Flarecorp Media. I had decided to use my MacBook Pro at the time to do the editing instead of my Desktop Windows computer. When exporting, I noticed I had several options for the Renderer in Adobe Media Encoder. I could use OpenCL GPU Acceleration, Metal GPU Acceleration or software only.

Not wanting to waste time in the future on which setting I should be using, I searched all over the internet and couldn’t find an answer anywhere. I decided it was up to me to figure out the answer and let the world know. The results at the time were to use OpenCL and not Metal.

Apparently I wasn’t the only person curious about which setting to use because over the next year, the video climbed week by week on the number of views. It’s success allowed me to become a YouTube preferred partner and start getting monetization (before they changed the rules in 2018). In a way, it was great to have a “hit” video early on in my YouTube career because it gave me a taste for success and showed me that any video could be popular. You never know what people are going to be interested in so all that maters is to just keep making them.

However, over the past 20 months, people have been asking for an update. Is my advice still true with modern hardware, the newer operating systems and new versions of Adobe Products. Well, I finally got access to some newer hardware and that allowed me to FINALLY make an updated video.

So here it is. Is OpenCL still the go to choice or has Metal finally caught up. Find out in the video below.