Auto-Generate Subtitles for Foreign Content

Download it from potplayer.tv, the x64 one. The old potplayer.daum.net site doesn’t load anymore.

potplayer.tv

Install with the defaults. It goes to C:\Program Files\DAUM\PotPlayer. Codecs it offers are fine, untick anything else you do not recognize, pretty straightforward I guess and obviously if you use ARM computer, download that version but I don’t have one and idk how well it works over there.

Right-click menu

Right-click on a video and go to Subtitles. There are two Whisper options in there. ”Create subtitles from audio” which creates them based on the whole file, and that’s the one I use because Whisper is much better when it has full context. The real-time one works while the video plays but their use-case is only for streamed content I think

Everything else is inside “Create subtitles from audio⸺

Subtitles menu

Create subtitles from audio, I have set it to be automatic for kind of content I usually consume these days but most people are better off with option 4, I am also mostly watching tutorials these days, so I don’t need subtitles saved either, turn them on.

Choosing the Engine

I use (and recommend) Faster-Whisper-XXL. It’s Purfview’s build of faster-whisper, which is a lot faster than OpenAI’s version and uses less VRAM. It uses your NVIDIA card if you have one and the CPU if you don’t. The whisper.cpp ones are backups, CUDA for NVIDIA, Vulkan for AMD or Intel. I never tried them, and I don’t have an AMD chip either, so just adapt the tutorial to your needs if you have a different setup.

Whisper AI engine

Faster-Whisper-XXL on GitHub

Choosing the Model

I only use two, large-v3-turbo and tiny, and I switch between them as needed. turbo is much more accurate. tiny is under 100 MB and barely touches the VRAM, so I use it for English stuff if I ever need it or when I’m multitasking. You can see comparison below.

Whisper AI models, tiny is selected because I was testing it

PotPlayer is supposed to download the model when you pick it, but usually it just doesn’t. Downloading it yourself and pasting it into the model folder is easy though, just download it and click open folder to see the model folder. Same goes for the engine, for a full tutorial see below.

PotPlayer’s own download fails oftentimes and did fail for me. Either way, I would recommend you download it yourself and paste it into the right folder.

PotPlayer keeps both in AppData, not Program Files:

%APPDATA%\PotPlayerMini64\Model\
%APPDATA%\PotPlayerMini64\Engine\

You can paste those into the Explorer address bar, or use “Open model folder” and “Open engine folder” from the engine and model menus above in PotPlayer ui.

How to download Engine Manually?

Faster-Whisper-XXL is on Purfview’s GitHub.

1. Close PotPlayer.
2. Download the latest Faster-Whisper-XXL_..._windows.7z from the releases page. It’s big, mine takes up over 4.5 GB.
3. Extract it, so you end up with Engine\Faster-Whisper-XXL\faster-whisper-xxl.exe
4. Open PotPlayer and pick Faster-Whisper-XXL, it shouldn’t ask to download anymore.

It should look like this⸺

%APPDATA%\PotPlayerMini64\Engine\
└── Faster-Whisper-XXL\
    ├── _xxl_data\
    ├── faster-whisper-xxl.exe
    └── ffmpeg.exe

How to download Models manually?

The models are on Hugging Face (that’s where faster-whisper’s models are hosted, not GitHub). Each model has its own folder called faster-whisper-<model>:

inside %APPDATA%\PotPlayerMini64\Model

These are CTranslate2 models, a folder with model.bin and some small config files. They’re not the .pt files from OpenAI’s GitHub and not the ggml-*.bin files whisper.cpp uses, so don’t mix them up, just follow the link.

faster-whisper-large-v3-turbo on Hugging Face, it used to be under mobiuslabsgmbh and that link still redirects here

faster-whisper-tiny on Hugging Face

The easiest way is to just download the whole repo into the Model folder, (you can ignore the README and other extra files don’t matter if you download them or not). either click the download arrow and paste them appropriately or open a terminal in %APPDATA%\PotPlayerMini64\Model\ and run one of these:

git lfs install
git clone https://huggingface.co/dropbox-dash/faster-whisper-large-v3-turbo faster-whisper-large-v3-turbo
git clone https://huggingface.co/Systran/faster-whisper-tiny faster-whisper-tiny

or with the Hugging Face CLI (pip install -U huggingface_hub first):

hf download dropbox-dash/faster-whisper-large-v3-turbo --local-dir faster-whisper-large-v3-turbo
hf download Systran/faster-whisper-tiny --local-dir faster-whisper-tiny

Then open PotPlayer and pick the model, it shouldn’t ask to download anymore. It should look something like this in folders (maybe plus the README and whatever else came with the repo)⸺

%APPDATA%\PotPlayerMini64\Model\
├── faster-whisper-large-v3-turbo\
│   ├── config.json
│   ├── model.bin                  (1.62 GB)
│   ├── preprocessor_config.json
│   ├── tokenizer.json
│   └── vocabulary.json
└── faster-whisper-tiny\
    ├── config.json
    ├── model.bin                  (75.5 MB)
    ├── tokenizer.json
    └── vocabulary.txt

If you use a whisper.cpp engine instead, it wants a single ggml-<model>.bin file loose in the same folder. PotPlayer’s server has those at https://twg.kakaocdn.net/potplayer/PotPlayer/whisper.model/ggml-<model>.bin.

Whisper writes subtitles in whatever language the video is in, and PotPlayer can translate them line by line as they come up.

1. Right-click, Subtitles, Subtitle Translation (it’s in the Subtitles menu screenshot above).
2. Change it from “Do not use” to “Always use”.
3. Pick an engine in the same menu.
4. Set the target language in the translation settings at the bottom of that menu. Source can stay on auto.

Whisper also has its own translate-to-English mode but turbo wasn’t trained for it and just returns the original language, so use PotPlayer’s translation for this.

Dual subtitles for language learning

This is how I actually use it most of the time. Instead of only showing the translation, PotPlayer can show the original line and the translation together, so you read the French or Japanese and have your own language right there when you get stuck.

1. Set Whisper AI language to the language of the video, not auto. You want Whisper writing the original, not guessing.
2. Turn on translation like above and set the target to your language.
3. In the same Subtitle Translation menu there are display options, chose your preffered mode.

OpenAI's model table, speeds are on an A100 and not a laptop

On VRAM, OpenAI’s numbers are about 1 GB for tiny and 6 GB for turbo, but that’s for their original code. Faster-Whisper uses a lot less and turbo has to hold its 1.6 GB of weights plus some working memory, so expect a few GBs. I haven’t measured either.

I’ve only tested tiny and large-v3-turbo, with Faster-Whisper-XXL.

Accuracy

I picked English, French, German, Chinese and Japanese because that’s content I usually watch, so check the links below if you need another language. In English tiny is usable if you just want the gist, but in everything else it is pretty lame but despite it’s errors I didn’t mind using it much for learning French given I hardly care for grammar but just new words but it might be bad for absolute beginners. Chinese and Japanese are counted per character since they don’t use spaces, I found tiny unusable for these two and most non European languages.

These are official numbers from OpenAI, just from two places. tiny is from table 13 of the Whisper paper (appendix D.2.4, there’s a readable version here), and turbo is from the chart in their turbo release post. Both are on the FLEURS dataset. So they’re comparable. FLEURS is also people reading sentences clearly, and real videos with music and people talking over each other will be worse for both.

Speed

I timed both on my laptop (RTX 3050, 4GB VRAM varient, Faster-Whisper-XXL r245.4 running on CUDA) with an 18 min 49 s English video essay as a test. Same engine PotPlayer uses, just run directly so I could time it properly, though i was browsing and doing other tasks so expect pretty relatable conditions.

So turbo takes a bit over twice as long as tiny on my laptop, but both are way faster than the video itself, tiny is about 40× real time and turbo about 19×. Most of the wait for the first subtitle is just the model loading, turbo is a 1.6 GB file so it takes longer to get going. After that it catches up fast.

In practice, if you press play straight away, the first line of speech in video I used for testing it was at 0:14. tiny had it ready at 8 s, so it was there in time. turbo needed 17 s, so its first few lines would get skipped in real time. If someone starts talking in the first few seconds, you’ll miss the first line or two with turbo, so I either wait a few seconds before pressing play or just rewind a bit, you can alternatively just generate subtitles first and then watch the video, but it works really well with real time use case too. Movies and Animes tend to start well after an introduction so it’s only required for lectures and video essays or similar content.

The official speeds are from an A100, so they only tell you that tiny is a bit faster. On my laptop the gap was bigger, 2.2× instead of the 1.25× you’d expect from OpenAI’s numbers, since turbo is the heavier model to run on a small GPU, specially when multitasking.

The difference in quality showed up straight away too. In the first 30 seconds tiny made two spelling mistakes. Small stuff, but for a language you’re learning that can be an issue, personally I don’t think so. (I think Tiny was reliable enough for learning French and mistakes enhanced my experience by increasing resistance helping me recognize similar words precisely because they were mistaken haha) (❁^◡^❁)

So I switch between them.

Ask me on discord if you’ve other issues, or if you suggest a common FAQ I’ll add them too⸺