Convert Figma logo to code with AI

nihui logowaifu2x-ncnn-vulkan

waifu2x converter ncnn version, runs fast on intel / amd / nvidia / apple-silicon GPU with vulkan

2,976
205
2,976
80

Top Related Projects

27,441

Image Super-Resolution for Anime-Style Art

Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, Real-ESRGAN, Real-CUGAN, RTX Video Super Resolution VSR, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.

waifu2xのCaffe版

5,589

Quick Overview

waifu2x-ncnn-vulkan is an image upscaling and denoising tool that uses convolutional neural networks. It's based on the original waifu2x project but implemented using the ncnn framework and Vulkan API for improved performance. This tool is particularly useful for enhancing anime-style artwork and photos.

Pros

  • High-quality image upscaling and noise reduction
  • Utilizes GPU acceleration for faster processing
  • Cross-platform support (Windows, Linux, macOS, Android)
  • Command-line interface for easy integration into scripts and workflows

Cons

  • Requires a Vulkan-capable GPU for optimal performance
  • May produce artifacts or undesired results on non-anime style images
  • Limited customization options compared to some other image processing tools
  • Larger file size compared to the original waifu2x implementation

Getting Started

  1. Download the latest release from the GitHub repository.
  2. Extract the files to a directory of your choice.
  3. Open a terminal or command prompt and navigate to the extracted directory.
  4. Run the tool using the following command structure:
./waifu2x-ncnn-vulkan -i input_image.jpg -o output_image.png -n noise_level -s scale_factor

Replace input_image.jpg with your input image file, output_image.png with your desired output filename, noise_level with a value from 0 to 3 (0 = no denoising), and scale_factor with the desired upscaling factor (1, 2, 4, 8, 16, or 32).

Example:

./waifu2x-ncnn-vulkan -i my_image.jpg -o my_image_upscaled.png -n 1 -s 2

This command will upscale my_image.jpg by a factor of 2 and apply a light noise reduction (level 1).

Competitor Comparisons

27,441

Image Super-Resolution for Anime-Style Art

Pros of waifu2x

  • Original implementation with a focus on image quality
  • Supports both CPU and CUDA processing
  • Includes a web-based demo for easy testing

Cons of waifu2x

  • Slower processing speed, especially on CPU
  • Requires more dependencies and setup
  • Limited platform support (mainly Linux and macOS)

Code Comparison

waifu2x (Lua):

function Waifu2x()
  local model = torch.load(model_path)
  local input = image.load(input_path)
  local output = model:forward(input)
  image.save(output_path, output)
end

waifu2x-ncnn-vulkan (C++):

int Waifu2x::process(const ncnn::Mat& inimage, ncnn::Mat& outimage)
{
    ncnn::VulkanCompute cmd(vkdev);
    cmd.record_upload(inimage);
    cmd.record_pipeline(pipeline);
    cmd.record_download(outimage);
    cmd.submit_and_wait();
    return 0;
}

The waifu2x code is simpler and more straightforward, while waifu2x-ncnn-vulkan utilizes Vulkan for GPU acceleration, resulting in more complex but faster processing.

Video, Image and GIF upscale/enlarge(Super-Resolution) and Video frame interpolation. Achieved with Waifu2x, Real-ESRGAN, Real-CUGAN, RTX Video Super Resolution VSR, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.

Pros of Waifu2x-Extension-GUI

  • User-friendly graphical interface for easier operation
  • Supports multiple AI models, including waifu2x, Real-ESRGAN, and Anime4K
  • Offers batch processing and video upscaling capabilities

Cons of Waifu2x-Extension-GUI

  • Larger file size and more complex installation process
  • May have higher system requirements due to additional features
  • Potentially slower processing speed compared to the lightweight waifu2x-ncnn-vulkan

Code Comparison

waifu2x-ncnn-vulkan:

int waifu2x(const cv::Mat& inimage, cv::Mat& outimage, int noise_level, int scale, int tilesize_x, int tilesize_y, int prepadding, int gpu_id)
{
    ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device(gpu_id);
    // ... (implementation details)
}

Waifu2x-Extension-GUI:

def process_file(self, input_file, output_file):
    command = self.generate_command(input_file, output_file)
    process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    # ... (implementation details)

The code comparison shows that waifu2x-ncnn-vulkan is implemented in C++ and directly uses the ncnn framework, while Waifu2x-Extension-GUI is written in Python and acts as a wrapper for various AI upscaling tools, executing them as subprocesses.

waifu2xのCaffe版

Pros of waifu2x-caffe

  • Supports CPU processing, making it more versatile for systems without dedicated GPUs
  • Offers a graphical user interface (GUI) for easier use by non-technical users
  • Provides more customization options for advanced users

Cons of waifu2x-caffe

  • Generally slower processing speed compared to waifu2x-ncnn-vulkan
  • Requires more setup and dependencies, potentially making installation more complex
  • Less actively maintained, with fewer recent updates

Code Comparison

waifu2x-caffe:

cv::Mat image = cv::imread(inputFileName, cv::IMREAD_COLOR);
boost::shared_ptr<caffe::Net<float> > net(new caffe::Net<float>(modelFileName, caffe::TEST));
net->CopyTrainedLayersFrom(modelFileName);

waifu2x-ncnn-vulkan:

ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device();
ncnn::VkAllocator* blob_vkallocator = vkdev->acquire_blob_allocator();
ncnn::VkAllocator* staging_vkallocator = vkdev->acquire_staging_allocator();

The code snippets highlight the different approaches:

  • waifu2x-caffe uses OpenCV and Caffe for image processing and neural network operations
  • waifu2x-ncnn-vulkan utilizes the ncnn framework and Vulkan API for GPU acceleration

Both projects aim to upscale and denoise anime-style artwork, but they differ in their implementation and target use cases. waifu2x-caffe offers more flexibility and a user-friendly interface, while waifu2x-ncnn-vulkan focuses on performance and GPU acceleration.

5,589

Pros of ailab

  • Offers a wider range of AI-powered image processing tools beyond upscaling
  • Provides more advanced features for video enhancement and processing
  • Includes pre-trained models for various tasks, such as face detection and style transfer

Cons of ailab

  • May have a steeper learning curve due to its broader scope and more complex features
  • Potentially higher resource requirements for running the full suite of tools
  • Less focused on a single task, which might impact performance for specific use cases like image upscaling

Code Comparison

waifu2x-ncnn-vulkan:

int waifu2x(const cv::Mat& inimage, cv::Mat& outimage, int noise_level, int scale, int tilesize_x, int tilesize_y, int prepadding, int gpu_id)
{
    ncnn::VulkanDevice* vkdev = ncnn::get_gpu_device(gpu_id);
    // ... (implementation details)
}

ailab:

def enhance_image(image_path, model_name, output_path):
    model = load_model(model_name)
    image = load_image(image_path)
    enhanced_image = model.process(image)
    save_image(enhanced_image, output_path)

Note: The code snippets are simplified representations and may not reflect the exact implementation in the repositories.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

waifu2x ncnn Vulkan

CI download

ncnn implementation of waifu2x converter. Runs fast on Intel / AMD / Nvidia / Apple-Silicon with Vulkan API.

waifu2x-ncnn-vulkan uses ncnn project as the universal neural network inference framework.

Download

Download Windows/Linux/MacOS Executable for Intel/AMD/Nvidia GPU

https://github.com/nihui/waifu2x-ncnn-vulkan/releases

This package includes all the binaries and models required. It is portable, so no CUDA or Caffe runtime environment is needed :)

Usages

Example Command

waifu2x-ncnn-vulkan.exe -i input.jpg -o output.png -n 2 -s 2

Full Usages

Usage: waifu2x-ncnn-vulkan -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -n noise-level       denoise level (-1/0/1/2/3, default=0)
  -s scale             upscale ratio (1/2/4/8/16/32, default=2)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        waifu2x model path (default=models-cunet)
  -g gpu-id            gpu device to use (-1=cpu, default=auto) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • noise-level = noise level, large value means strong denoise effect, -1 = no effect
  • scale = scale level, 1 = no scaling, 2 = upscale 2x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + waifu2x upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter a crash or error, try upgrading your GPU driver:

Build from Source

  1. Download and setup the Vulkan SDK from https://vulkan.lunarg.com/
  • For Linux distributions, you can either get the essential build requirements from package manager
dnf install vulkan-headers vulkan-loader-devel
apt-get install libvulkan-dev
pacman -S vulkan-headers vulkan-icd-loader
  1. Clone this project with all submodules
git clone https://github.com/nihui/waifu2x-ncnn-vulkan.git
cd waifu2x-ncnn-vulkan
git submodule update --init --recursive
  1. Build with CMake
  • You can pass -DUSE_STATIC_MOLTENVK=ON option to avoid linking the vulkan loader library on MacOS
mkdir build
cd build
cmake ../src
cmake --build . -j 4

Speed Comparison with waifu2x-caffe-cui

Environment

  • Windows 10 1809
  • AMD R7-1700
  • Nvidia GTX-1070
  • Nvidia driver 419.67
  • CUDA 10.1.105
  • cuDNN 10.1
Measure-Command { waifu2x-ncnn-vulkan.exe -i input.png -o output.png -n 2 -s 2 -t [block size] -m [model dir] }
Measure-Command { waifu2x-caffe-cui.exe -t 0 --gpu 0 -b 1 -c [block size] -p cudnn --model_dir [model dir] -s 2 -n 2 -m noise_scale -i input.png -o output.png }

cunet

Image SizeTarget SizeBlock SizeTotal Time(s)GPU Memory(MB)
waifu2x-ncnn-vulkan200x200400x400400/200/1000.86/0.86/0.82638/638/197
waifu2x-caffe-cui200x200400x400400/200/1002.54/2.39/2.363017/936/843
waifu2x-ncnn-vulkan400x400800x800400/200/1001.17/1.04/1.022430/638/197
waifu2x-caffe-cui400x400800x800400/200/1002.91/2.43/2.73202/1389/1178
waifu2x-ncnn-vulkan1000x10002000x2000400/200/1002.35/2.26/2.462430/638/197
waifu2x-caffe-cui1000x10002000x2000400/200/1004.04/3.79/4.353258/1582/1175
waifu2x-ncnn-vulkan2000x20004000x4000400/200/1006.46/6.59/7.492430/686/213
waifu2x-caffe-cui2000x20004000x4000400/200/1007.01/7.54/10.113258/1499/1200
waifu2x-ncnn-vulkan4000x40008000x8000400/200/10022.78/23.78/27.612448/654/213
waifu2x-caffe-cui4000x40008000x8000400/200/10018.45/21.85/31.823325/1652/1236

upconv_7_anime_style_art_rgb

Image SizeTarget SizeBlock SizeTotal Time(s)GPU Memory(MB)
waifu2x-ncnn-vulkan200x200400x400400/200/1000.74/0.75/0.72482/482/142
waifu2x-caffe-cui200x200400x400400/200/1002.04/1.99/1.99995/546/459
waifu2x-ncnn-vulkan400x400800x800400/200/1000.95/0.83/0.811762/482/142
waifu2x-caffe-cui400x400800x800400/200/1002.08/2.12/2.11995/546/459
waifu2x-ncnn-vulkan1000x10002000x2000400/200/1001.52/1.41/1.441778/482/142
waifu2x-caffe-cui1000x10002000x2000400/200/1002.72/2.60/2.681015/570/459
waifu2x-ncnn-vulkan2000x20004000x4000400/200/1003.45/3.42/3.631778/482/142
waifu2x-caffe-cui2000x20004000x4000400/200/1003.90/4.01/4.351015/521/462
waifu2x-ncnn-vulkan4000x40008000x8000400/200/10011.16/11.29/12.071796/498/158
waifu2x-caffe-cui4000x40008000x8000400/200/1009.24/9.81/11.16995/546/436

Sample Images

Original Image

origin

Upscale 2x with ImageMagick

convert origin.jpg -resize 200% output.png

browser

Upscale 2x with ImageMagick Lanczo4 Filter

convert origin.jpg -filter Lanczos -resize 200% output.png

browser

Upscale 2x with waifu2x noise=2 scale=2

waifu2x-ncnn-vulkan.exe -i origin.jpg -o output.png -n 2 -s 2

waifu2x

Original waifu2x Project

Other Open-Source Code Used