waifu2x-ncnn-vulkan
waifu2x converter ncnn version, runs fast on intel / amd / nvidia / apple-silicon GPU with vulkan
Top Related Projects
Image Super-Resolution for Anime-Style Art
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, Real-ESRGAN, Real-CUGAN, RTX Video Super Resolution VSR, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.
waifu2xのCaffe版
Quick Overview
waifu2x-ncnn-vulkan is an image upscaling and denoising tool that uses convolutional neural networks. It's based on the original waifu2x project but implemented using the ncnn framework and Vulkan API for improved performance. This tool is particularly useful for enhancing anime-style artwork and photos.
Pros
- High-quality image upscaling and noise reduction
- Utilizes GPU acceleration for faster processing
- Cross-platform support (Windows, Linux, macOS, Android)
- Command-line interface for easy integration into scripts and workflows
Cons
- Requires a Vulkan-capable GPU for optimal performance
- May produce artifacts or undesired results on non-anime style images
- Limited customization options compared to some other image processing tools
- Larger file size compared to the original waifu2x implementation
Getting Started
- Download the latest release from the GitHub repository.
- Extract the files to a directory of your choice.
- Open a terminal or command prompt and navigate to the extracted directory.
- Run the tool using the following command structure:
./waifu2x-ncnn-vulkan -i input_image.jpg -o output_image.png -n noise_level -s scale_factor
Replace input_image.jpg
with your input image file, output_image.png
with your desired output filename, noise_level
with a value from 0 to 3 (0 = no denoising), and scale_factor
with the desired upscaling factor (1, 2, 4, 8, 16, or 32).
Example:
./waifu2x-ncnn-vulkan -i my_image.jpg -o my_image_upscaled.png -n 1 -s 2
This command will upscale my_image.jpg
by a factor of 2 and apply a light noise reduction (level 1).
Competitor Comparisons
Image Super-Resolution for Anime-Style Art
Pros of waifu2x
- Original implementation with a focus on image quality
- Supports both CPU and CUDA processing
- Includes a web-based demo for easy testing
Cons of waifu2x
- Slower processing speed, especially on CPU
- Requires more dependencies and setup
- Limited platform support (mainly Linux and macOS)
Code Comparison
waifu2x (Lua):
function Waifu2x()
local model = torch.load(model_path)
local input = image.load(input_path)
local output = model:forward(input)
image.save(output_path, output)
end
waifu2x-ncnn-vulkan (C++):
int Waifu2x::process(const ncnn::Mat& inimage, ncnn::Mat& outimage)
{
ncnn::VulkanCompute cmd(vkdev);
cmd.record_upload(inimage);
cmd.record_pipeline(pipeline);
cmd.record_download(outimage);
cmd.submit_and_wait();
return 0;
}
The waifu2x code is simpler and more straightforward, while waifu2x-ncnn-vulkan utilizes Vulkan for GPU acceleration, resulting in more complex but faster processing.
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, Real-ESRGAN, Real-CUGAN, RTX Video Super Resolution VSR, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.
Pros of Waifu2x-Extension-GUI
- User-friendly graphical interface for easier operation
- Supports multiple AI models, including waifu2x, Real-ESRGAN, and Anime4K
- Offers batch processing and video upscaling capabilities
Cons of Waifu2x-Extension-GUI
- Larger file size and more complex installation process
- May have higher system requirements due to additional features
- Potentially slower processing speed compared to the lightweight waifu2x-ncnn-vulkan
Code Comparison
waifu2x-ncnn-vulkan:
int waifu2x(const cv::Mat& inimage, cv::Mat& outimage, int noise_level, int scale, int tilesize_x, int tilesize_y, int prepadding, int gpu_id)
{
ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device(gpu_id);
// ... (implementation details)
}
Waifu2x-Extension-GUI:
def process_file(self, input_file, output_file):
command = self.generate_command(input_file, output_file)
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
# ... (implementation details)
The code comparison shows that waifu2x-ncnn-vulkan is implemented in C++ and directly uses the ncnn framework, while Waifu2x-Extension-GUI is written in Python and acts as a wrapper for various AI upscaling tools, executing them as subprocesses.
waifu2xのCaffe版
Pros of waifu2x-caffe
- Supports CPU processing, making it more versatile for systems without dedicated GPUs
- Offers a graphical user interface (GUI) for easier use by non-technical users
- Provides more customization options for advanced users
Cons of waifu2x-caffe
- Generally slower processing speed compared to waifu2x-ncnn-vulkan
- Requires more setup and dependencies, potentially making installation more complex
- Less actively maintained, with fewer recent updates
Code Comparison
waifu2x-caffe:
cv::Mat image = cv::imread(inputFileName, cv::IMREAD_COLOR);
boost::shared_ptr<caffe::Net<float> > net(new caffe::Net<float>(modelFileName, caffe::TEST));
net->CopyTrainedLayersFrom(modelFileName);
waifu2x-ncnn-vulkan:
ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device();
ncnn::VkAllocator* blob_vkallocator = vkdev->acquire_blob_allocator();
ncnn::VkAllocator* staging_vkallocator = vkdev->acquire_staging_allocator();
The code snippets highlight the different approaches:
- waifu2x-caffe uses OpenCV and Caffe for image processing and neural network operations
- waifu2x-ncnn-vulkan utilizes the ncnn framework and Vulkan API for GPU acceleration
Both projects aim to upscale and denoise anime-style artwork, but they differ in their implementation and target use cases. waifu2x-caffe offers more flexibility and a user-friendly interface, while waifu2x-ncnn-vulkan focuses on performance and GPU acceleration.
Pros of ailab
- Offers a wider range of AI-powered image processing tools beyond upscaling
- Provides more advanced features for video enhancement and processing
- Includes pre-trained models for various tasks, such as face detection and style transfer
Cons of ailab
- May have a steeper learning curve due to its broader scope and more complex features
- Potentially higher resource requirements for running the full suite of tools
- Less focused on a single task, which might impact performance for specific use cases like image upscaling
Code Comparison
waifu2x-ncnn-vulkan:
int waifu2x(const cv::Mat& inimage, cv::Mat& outimage, int noise_level, int scale, int tilesize_x, int tilesize_y, int prepadding, int gpu_id)
{
ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device(gpu_id);
// ... (implementation details)
}
ailab:
def enhance_image(image_path, model_name, output_path):
model = load_model(model_name)
image = load_image(image_path)
enhanced_image = model.process(image)
save_image(enhanced_image, output_path)
Note: The code snippets are simplified representations and may not reflect the exact implementation in the repositories.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
waifu2x ncnn Vulkan
ncnn implementation of waifu2x converter. Runs fast on Intel / AMD / Nvidia / Apple-Silicon with Vulkan API.
waifu2x-ncnn-vulkan uses ncnn project as the universal neural network inference framework.
Download
Download Windows/Linux/MacOS Executable for Intel/AMD/Nvidia GPU
https://github.com/nihui/waifu2x-ncnn-vulkan/releases
This package includes all the binaries and models required. It is portable, so no CUDA or Caffe runtime environment is needed :)
Usages
Example Command
waifu2x-ncnn-vulkan.exe -i input.jpg -o output.png -n 2 -s 2
Full Usages
Usage: waifu2x-ncnn-vulkan -i infile -o outfile [options]...
-h show this help
-v verbose output
-i input-path input image path (jpg/png/webp) or directory
-o output-path output image path (jpg/png/webp) or directory
-n noise-level denoise level (-1/0/1/2/3, default=0)
-s scale upscale ratio (1/2/4/8/16/32, default=2)
-t tile-size tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
-m model-path waifu2x model path (default=models-cunet)
-g gpu-id gpu device to use (-1=cpu, default=auto) can be 0,1,2 for multi-gpu
-j load:proc:save thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
-x enable tta mode
-f format output image format (jpg/png/webp, default=ext/png)
input-path
andoutput-path
accept either file path or directory pathnoise-level
= noise level, large value means strong denoise effect, -1 = no effectscale
= scale level, 1 = no scaling, 2 = upscale 2xtile-size
= tile size, use smaller value to reduce GPU memory usage, default selects automaticallyload:proc:save
= thread count for the three stages (image decoding + waifu2x upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.format
= the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded
If you encounter a crash or error, try upgrading your GPU driver:
- Intel: https://downloadcenter.intel.com/product/80939/Graphics-Drivers
- AMD: https://www.amd.com/en/support
- NVIDIA: https://www.nvidia.com/Download/index.aspx
Build from Source
- Download and setup the Vulkan SDK from https://vulkan.lunarg.com/
- For Linux distributions, you can either get the essential build requirements from package manager
dnf install vulkan-headers vulkan-loader-devel
apt-get install libvulkan-dev
pacman -S vulkan-headers vulkan-icd-loader
- Clone this project with all submodules
git clone https://github.com/nihui/waifu2x-ncnn-vulkan.git
cd waifu2x-ncnn-vulkan
git submodule update --init --recursive
- Build with CMake
- You can pass -DUSE_STATIC_MOLTENVK=ON option to avoid linking the vulkan loader library on MacOS
mkdir build
cd build
cmake ../src
cmake --build . -j 4
Speed Comparison with waifu2x-caffe-cui
Environment
- Windows 10 1809
- AMD R7-1700
- Nvidia GTX-1070
- Nvidia driver 419.67
- CUDA 10.1.105
- cuDNN 10.1
Measure-Command { waifu2x-ncnn-vulkan.exe -i input.png -o output.png -n 2 -s 2 -t [block size] -m [model dir] }
Measure-Command { waifu2x-caffe-cui.exe -t 0 --gpu 0 -b 1 -c [block size] -p cudnn --model_dir [model dir] -s 2 -n 2 -m noise_scale -i input.png -o output.png }
cunet
Image Size | Target Size | Block Size | Total Time(s) | GPU Memory(MB) | |
---|---|---|---|---|---|
waifu2x-ncnn-vulkan | 200x200 | 400x400 | 400/200/100 | 0.86/0.86/0.82 | 638/638/197 |
waifu2x-caffe-cui | 200x200 | 400x400 | 400/200/100 | 2.54/2.39/2.36 | 3017/936/843 |
waifu2x-ncnn-vulkan | 400x400 | 800x800 | 400/200/100 | 1.17/1.04/1.02 | 2430/638/197 |
waifu2x-caffe-cui | 400x400 | 800x800 | 400/200/100 | 2.91/2.43/2.7 | 3202/1389/1178 |
waifu2x-ncnn-vulkan | 1000x1000 | 2000x2000 | 400/200/100 | 2.35/2.26/2.46 | 2430/638/197 |
waifu2x-caffe-cui | 1000x1000 | 2000x2000 | 400/200/100 | 4.04/3.79/4.35 | 3258/1582/1175 |
waifu2x-ncnn-vulkan | 2000x2000 | 4000x4000 | 400/200/100 | 6.46/6.59/7.49 | 2430/686/213 |
waifu2x-caffe-cui | 2000x2000 | 4000x4000 | 400/200/100 | 7.01/7.54/10.11 | 3258/1499/1200 |
waifu2x-ncnn-vulkan | 4000x4000 | 8000x8000 | 400/200/100 | 22.78/23.78/27.61 | 2448/654/213 |
waifu2x-caffe-cui | 4000x4000 | 8000x8000 | 400/200/100 | 18.45/21.85/31.82 | 3325/1652/1236 |
upconv_7_anime_style_art_rgb
Image Size | Target Size | Block Size | Total Time(s) | GPU Memory(MB) | |
---|---|---|---|---|---|
waifu2x-ncnn-vulkan | 200x200 | 400x400 | 400/200/100 | 0.74/0.75/0.72 | 482/482/142 |
waifu2x-caffe-cui | 200x200 | 400x400 | 400/200/100 | 2.04/1.99/1.99 | 995/546/459 |
waifu2x-ncnn-vulkan | 400x400 | 800x800 | 400/200/100 | 0.95/0.83/0.81 | 1762/482/142 |
waifu2x-caffe-cui | 400x400 | 800x800 | 400/200/100 | 2.08/2.12/2.11 | 995/546/459 |
waifu2x-ncnn-vulkan | 1000x1000 | 2000x2000 | 400/200/100 | 1.52/1.41/1.44 | 1778/482/142 |
waifu2x-caffe-cui | 1000x1000 | 2000x2000 | 400/200/100 | 2.72/2.60/2.68 | 1015/570/459 |
waifu2x-ncnn-vulkan | 2000x2000 | 4000x4000 | 400/200/100 | 3.45/3.42/3.63 | 1778/482/142 |
waifu2x-caffe-cui | 2000x2000 | 4000x4000 | 400/200/100 | 3.90/4.01/4.35 | 1015/521/462 |
waifu2x-ncnn-vulkan | 4000x4000 | 8000x8000 | 400/200/100 | 11.16/11.29/12.07 | 1796/498/158 |
waifu2x-caffe-cui | 4000x4000 | 8000x8000 | 400/200/100 | 9.24/9.81/11.16 | 995/546/436 |
Sample Images
Original Image
Upscale 2x with ImageMagick
convert origin.jpg -resize 200% output.png
Upscale 2x with ImageMagick Lanczo4 Filter
convert origin.jpg -filter Lanczos -resize 200% output.png
Upscale 2x with waifu2x noise=2 scale=2
waifu2x-ncnn-vulkan.exe -i origin.jpg -o output.png -n 2 -s 2
Original waifu2x Project
Other Open-Source Code Used
- https://github.com/Tencent/ncnn for fast neural network inference on ALL PLATFORMS
- https://github.com/webmproject/libwebp for encoding and decoding Webp images on ALL PLATFORMS
- https://github.com/nothings/stb for decoding and encoding image on Linux / MacOS
- https://github.com/tronkko/dirent for listing files in directory on Windows
Top Related Projects
Image Super-Resolution for Anime-Style Art
Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, Real-ESRGAN, Real-CUGAN, RTX Video Super Resolution VSR, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.
waifu2xのCaffe版
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot