A High-Quality Real Time Upscaler for Anime Video
Installation Instructions for GLSL/MPV (v4.0.1)
What's new:
Restore_Soft
shaders (5 variants) to address different types of artifacts. Soft variants significantly reduces ringing, aliasing and other artifacts at the expense of less sharpness.Major Changes:
Restore
shaders are replaced by Soft variants.Minor Changes:
Restore_Moderate
shaders are renamed to simply Restore
.AutoDownscalePre_x2
shader for better compatibility with 1440p/1080p monitors.Installation Instructions for GLSL/MPV (v4.0)
What's new:
Major Changes:
RA
, Upscale+Deblur CNN
and Deblur CNN
shaders, as they are superseded by v4.x Restore
shaders.Heavy
variant for line reconstruction. The Moderate
variant has been tweaked for better reconstruction of low quality images.Minor Changes:
Anime4K-[Type-Subtype]-[Strength]-[Quality/Speed]
.Restore
.Installation Instructions for GLSL/MPV (v3.0)
What's new:
Minor changes:
What's new:
Major Changes:
RA
, Upscale+Deblur CNN
and Deblur CNN
shaders, as they are superseded by v4.x Restore
shaders.Minor Changes:
Anime4K-[Type-Subtype]-[Strength]-[Quality/Speed]
.Upscale
and Deblur
shaders are now deprecated. They might be removed in future versions.DarkLines
and ThinLines
shaders to a shorter name: Darken
and Thin
.Huge quality improvement on ITW (In the wild) anime. Previous super resolution methods were suited for artificially downscaled anime, while anime found in the wild exhibit a large amounts of variance. We propose and develop a method to correct this distribution shift for already existing CNN upscalers and that can be applied to non-Anime4K upscalers (The upscaling shaders bundled in this version are the exact same as v3.1). The line reconstruction algorithm will be explained in a future informal paper.
Some small issues such as occasional ghosting or ringing will need to be fixed before official release.
The three versions of line reconstruction shaders can be used as follows in mpv's input.conf
:
CTRL+2 works on almost all 1080p anime, if blurriness and aliasing is noticeable, use the heavier version (CTRL+3), if ghosting and ringing is visible, use the lighter version (CTRL+1).
For downscaled 720/480p anime, start with CTRL+1, for native standard definition anime, start with CTRL+2.
input.conf
for 4K screens (screen size, not anime size)
CTRL+1 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Light_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Heavy_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+2 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Medium_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Heavy_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+3 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Heavy_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Medium_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+0 change-list glsl-shaders clr ""
input.conf
for 1080p screens (screen size, not anime size)
CTRL+1 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Light_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Medium_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+2 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Medium_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Light_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+3 change-list glsl-shaders set "~~/shaders/Anime4K_Line_Reconstruction_Heavy_L.glsl;~~/shaders/Anime4K_Upscale_CNN_L_x2_Denoise.glsl;~~/shaders/Anime4K_Auto_Downscale_Pre_x4.glsl;~~/shaders/Anime4K_Line_Reconstruction_Light_L.glsl;~~/shaders/Anime4K_Upscale_CNN_M_x2_Denoise.glsl;~~/shaders/Anime4K_DeRing.glsl;"
CTRL+0 change-list glsl-shaders clr ""
Note: This version is only compatible with v3.1 and older shaders. Mixing with newer shaders might cause issues.
Installation Instructions for GLSL/MPV (v3.1)
What's new:
Minor changes:
Bug fixes:
RA
, Upscale_CNN
, and Upscale_CNN_Deblur
shaders that caused faulty processing when stacked one after the other.RA
shaders now correctly ignore chroma channels. They used to wrongly process chroma channels.Installation Instructions for GLSL/MPV (v3.0)
What's new:
Minor changes:
Installation Instructions for GLSL/MPV Version
Hotfix: Algorithm now performs downscaling before applying the x4 prescaler (rather than after). 720p scaled to x4 is bigger than 4K, which is unnecessary and slows the algorithm down.
Added a L version that contains a larger prescaler. It offers a significant improvement to image quality, especially to low resolution anime. (eg. 480p) It has 2741 parameters, which is still smaller than FSRCNNX 8. If your GPU can run FSRCNNX, it should be fine. However, since the prescaler was trained using perceptual loss, some banding might occur on very old/blurry anime. It is also slower. (+1ms compared to non-L version for 1080p -> 4K upscaling) Use at your own discretion.
Installation Instructions for GLSL/MPV Version
Integrated a pre-scaler in Anime4K. It is a tiny ConvNet (916 parameters, 3 hidden layers of 4 feature maps) trained with the differentiable Anime4K algorithm and other techniques that will be published soon. For comparison, FSRCNNX has more than 3000 parameters (depending on the version) and waifu2x has more than 1 million.
This pre-scaler gives a major quality improvement to lower resolution anime, especially for 720p and 480p. 480p -> 1080p, 480p -> 4K and 720p -> 4K are now officially supported. Texture loss and aliasing issues are again reduced.
The pre-scaler uses 1.5ms for 1080p -> 4K upscaling, bringing the total to 4ms on a Vega 64 GPU.
Installation Instructions for GLSL/MPV Version
Replaced the Bilateral Mean Filter with a "Regularized" Bilateral Mode Filter. The mode is more robust than the mean or median, and introduces the least amount of blur.
The mode filter needs to be "regularized" since the mode of a set is not defined when the number of data points is finite and contains unique data (all data points are different). This undefined case happens extremely often in images when using a small kernel (eg. 3x3), since there are 256 possible luma values, but only 9 samples. Often all 9 pixels will have different values, and rigging up a continuous fallback to median/mean that does not introduce discontinuities is not trivial.
However, we can use Kernel Density Estimation (KDE) as a continuous approximation of the discrete probability color histogram.