Desktop app for automatically translating comics - BDs, Manga, Manhwa, Fumetti and more in a variety of formats (Image, Pdf, Epub, cbr, cbz, etc) and in multiple languages.
https://github.com/ogkalu2/comic-translate/assets/115248977/b57360d3-eaad-4a93-bc46-94c01d38927c
Many Automatic Manga Translators exist. Very few properly support comics of other kinds in other languages. This project was created to utilize the ability of GPT-4 and translate comics from all over the world. Currently, it supports translating to and from English, Korean, Japanese, French, Simplified Chinese, Traditional Chinese, Russian, German, Dutch, Spanish and Italian.
For a couple dozen languages, the best Machine Translator is not Google Translate, Papago or even DeepL, but GPT-4, and by far. This is very apparent for distant language pairs (Korean<->English, Japanese<->English etc) where other translators still often devolve into gibberish. Excerpt from "The Walking Practice"(보행 연습) by Dolki Min(돌기민)
GPT-4-Vision as Translator. Note: Some of these also have Official English Translations
Install Python (<=3.10). Tick "Add python.exe to PATH" during the setup.
https://www.python.org/downloads/
Currently, this doesn't work fully on python 3.11 or higher because of issues with PaddleOCR. If you have no intention of translating from Chinese with the Default Option(Paddle), you can use this with 3.11 by replacing
paddleocr==2.7.0.3
paddlepaddle==2.5.2
with
PyMuPDF==1.23.8
in the requirements.txt file.
Clone the repo (or download the folder), navigate to the folder
git clone https://github.com/ogkalu2/comic-translate
cd comic-translate
and install the requirements
pip install -r requirements.txt
If you have an NVIDIA GPU, then it is recommended to run
pip uninstall torch torchvision
pip install torch==2.1.0+cu121 -f https://download.pytorch.org/whl/torch_stable.html
pip install torchvision==0.16.0+cu121 -f https://download.pytorch.org/whl/torch_stable.html
Note: The 121 in +cu121 represents the CUDA version - 12.1. Replace 121 with your CUDA version. E.g 118 if you are running CUDA 11.8
In the comic-translate directory, run
python comic.py
This will launch the GUI
raise RarCannotExec("Cannot find working tool")
In that case, Instructions for Windows, Linux, Mac
To following selections will require access to closed resources and subsequently, API Keys:
speech-bubble-detector, text-segmenter. Two yolov8m models trained on 8k and 3k images of comics (Manga, Webtoons, Western) respectively.
By Default:
Optional:
These can be used for any of the supported languages. An API Key is required.
A Manga/Anime finetuned lama checkpoint to remove text detected by the segmenter. Implementation courtsey of lama-cleaner
Currently, this supports using GPT-4-Vision, GPT-4, GPT-3.5, DeepL and Google Translate. All GPT models are fed the context of the entire page text to aid translations. GPT-4-Vision specifically is also provided the image of the page, the page with the original text for languages it is competent at recognizing (French, Russian, German, Dutch, Spanish, Italian) and the Inpainted Image for the rest.
PIL for rendering wrapped text in bounding boxes obtained from bubbles and text.