Convert Figma logo to code with AI

anishathalye logoneural-style

Neural style in TensorFlow! 🎨

5,537
1,513
5,537
3

Top Related Projects

Torch implementation of neural style algorithm

TensorFlow CNN for fast style transfer ⚡🖥🎨🖼

Code and data for paper "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511

Style transfer, deep learning, feature transform

TensorFlow (Python API) implementation of Neural Style

Quick Overview

Neural-style is an implementation of the neural style transfer algorithm, which combines the content of one image with the style of another image using convolutional neural networks. This project, created by Anish Athalye, allows users to generate artistic images by applying the style of one image to the content of another.

Pros

  • High-quality results with fine-tuned control over style transfer
  • Supports multiple style images and content images
  • Includes pre-trained VGG19 model for easy setup
  • Well-documented with clear instructions and examples

Cons

  • Computationally intensive, requiring a powerful GPU for reasonable processing times
  • Limited to static images, not suitable for real-time video processing
  • Requires some understanding of neural networks and image processing concepts
  • Dependency on older versions of TensorFlow and NumPy

Code Examples

  1. Basic style transfer:
import neural_style

content_image = 'path/to/content/image.jpg'
style_image = 'path/to/style/image.jpg'
output_image = 'path/to/output/image.jpg'

neural_style.stylize(content_image, style_image, output_image)
  1. Adjusting style weight:
neural_style.stylize(
    content_image,
    style_image,
    output_image,
    style_weight=1e4  # Increase style influence
)
  1. Using multiple style images:
style_images = ['style1.jpg', 'style2.jpg', 'style3.jpg']
style_blend_weights = [0.3, 0.3, 0.4]

neural_style.stylize(
    content_image,
    style_images,
    output_image,
    style_blend_weights=style_blend_weights
)

Getting Started

  1. Clone the repository:

    git clone https://github.com/anishathalye/neural-style.git
    cd neural-style
    
  2. Install dependencies:

    pip install -r requirements.txt
    
  3. Download the pre-trained VGG19 model:

    ./models/download_models.sh
    
  4. Run the style transfer:

    python neural_style.py --content <content_image> --styles <style_image> --output <output_image>
    

Replace <content_image>, <style_image>, and <output_image> with your desired file paths.

Competitor Comparisons

Torch implementation of neural style algorithm

Pros of neural-style (jcjohnson)

  • More extensive documentation and usage instructions
  • Supports multiple GPUs for faster processing
  • Includes pre-trained VGG models for convenience

Cons of neural-style (jcjohnson)

  • Less actively maintained (last update in 2018)
  • Requires older versions of Torch and CUDA
  • Limited to specific image sizes due to VGG network constraints

Code Comparison

neural-style (jcjohnson):

local cmd = torch.CmdLine()
cmd:option('-style_image', 'examples/inputs/seated-nude.jpg', 'Style target image')
cmd:option('-content_image', 'examples/inputs/tubingen.jpg', 'Content target image')
cmd:option('-output_image', 'out.png', 'Output image')
cmd:option('-image_size', 512, 'Maximum height / width of generated image')

neural-style (anishathalye):

parser.add_argument('--content', type=str,
                    dest='content', help='content image',
                    metavar='CONTENT', required=True)
parser.add_argument('--styles', type=str,
                    dest='styles', nargs='+', help='style images',
                    metavar='STYLE', required=True)
parser.add_argument('--output', type=str,
                    dest='output', help='output path',
                    metavar='OUTPUT', required=True)

Both repositories implement neural style transfer, but they differ in programming language and command-line argument structure. The jcjohnson version uses Lua and Torch, while the anishathalye version is implemented in Python, making it more accessible to a wider range of developers.

TensorFlow CNN for fast style transfer ⚡🖥🎨🖼

Pros of fast-style-transfer

  • Significantly faster processing time for style transfer
  • Real-time style transfer capability for video streams
  • Pre-trained models available for immediate use

Cons of fast-style-transfer

  • Limited flexibility in adjusting style parameters
  • Potentially lower quality output compared to neural-style
  • Requires training a separate model for each style

Code Comparison

neural-style:

def stylize(content_image, style_image, content_weight, style_weight):
    # Optimization-based approach
    # Iteratively updates the output image
    # ...

fast-style-transfer:

def stylize(content_image, model):
    # Feed-forward network approach
    # Single pass through a pre-trained model
    # ...

The key difference is that neural-style uses an optimization-based approach, iteratively updating the output image, while fast-style-transfer uses a feed-forward network that can generate stylized images in a single pass through a pre-trained model.

neural-style offers more flexibility and potentially higher quality results but at the cost of longer processing times. fast-style-transfer sacrifices some quality and flexibility for significantly faster processing, making it suitable for real-time applications.

Code and data for paper "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511

Pros of deep-photo-styletransfer

  • Focuses on photorealistic style transfer, maintaining the structure and details of the original photo
  • Implements a photorealistic smoothing step to reduce artifacts and improve output quality
  • Utilizes semantic segmentation to better preserve content boundaries

Cons of deep-photo-styletransfer

  • More complex implementation and setup compared to neural-style
  • Requires additional inputs like segmentation masks for optimal results
  • May have longer processing times due to the additional smoothing step

Code Comparison

neural-style:

def stylize(content_image, style_image, content_weight, style_weight):
    # Simple implementation using VGG19 for feature extraction
    model = vgg19.VGG19(weights='imagenet', include_top=False)
    # ... (content and style loss calculation)
    # Optimization step

deep-photo-styletransfer:

function [stylized_im] = deep_photo_style_transfer(content_im, style_im, content_seg, style_seg)
    % Photorealistic style transfer with segmentation and smoothing
    [feat_content, feat_style] = vgg19_forward(content_im, style_im);
    % ... (content and style loss calculation with segmentation)
    % Photorealistic smoothing step
    stylized_im = wls_smoothing(raw_output, content_im);
end

The code snippets highlight the differences in approach, with deep-photo-styletransfer incorporating segmentation and smoothing for more photorealistic results.

Style transfer, deep learning, feature transform

Pros of FastPhotoStyle

  • Faster processing time for style transfer
  • Better preservation of content structure and details
  • Supports both photo-realistic and artistic style transfer

Cons of FastPhotoStyle

  • Requires more computational resources (GPU)
  • Less flexibility in style manipulation compared to Neural Style
  • More complex setup and dependencies

Code Comparison

Neural Style:

content_image = imread(args.content_image)
style_image = imread(args.style_image)
network = build_vgg19(content_image)
content_features = get_content_features(network, content_image)
style_features = get_style_features(network, style_image)
output = optimize_image(content_features, style_features)

FastPhotoStyle:

content = load_image(args.content)
style = load_image(args.style)
content_seg = load_seg(args.content_seg)
style_seg = load_seg(args.style_seg)
output = stylize(content, style, content_seg, style_seg)

Both repositories focus on neural style transfer, but FastPhotoStyle aims for more photo-realistic results and faster processing. Neural Style offers more artistic flexibility but may be slower. FastPhotoStyle requires segmentation maps for optimal results, while Neural Style operates directly on images. The code structure reflects these differences, with FastPhotoStyle incorporating segmentation and Neural Style focusing on feature extraction and optimization.

TensorFlow (Python API) implementation of Neural Style

Pros of neural-style-tf

  • Implemented in TensorFlow, offering potential performance benefits and GPU acceleration
  • Supports multiple style images for blended styles
  • Includes additional features like color preservation and masked styling

Cons of neural-style-tf

  • May require more setup due to TensorFlow dependencies
  • Potentially more complex codebase for users unfamiliar with TensorFlow

Code Comparison

neural-style (Lua/Torch):

local img = image.load(params.content_image, 3)
local img = image.scale(img, params.image_size, 'bilinear')
local content_image = preprocess(img):float()

neural-style-tf (Python/TensorFlow):

content_image = scipy.misc.imread(content_image)
content_image = scipy.misc.imresize(content_image, (image_height, image_width))
content_image = content_image.astype(np.float32)

Both repositories implement the neural style transfer algorithm, but neural-style-tf offers a more modern TensorFlow implementation with additional features. The choice between them may depend on your familiarity with Lua/Torch vs. Python/TensorFlow and specific requirements for style transfer tasks.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

neural-style Build Status

An implementation of neural style in TensorFlow.

This implementation is a lot simpler than a lot of the other ones out there, thanks to TensorFlow's really nice API and automatic differentiation.

TensorFlow doesn't support L-BFGS (which is what the original authors used), so we use Adam. This may require a little bit more hyperparameter tuning to get nice results.

Running

python neural_style.py --content <content file> --styles <style file> --output <output file>

Run python neural_style.py --help to see a list of all options.

Use --checkpoint-output and --checkpoint-iterations to save checkpoint images.

Use --iterations to change the number of iterations (default 1000). For a 512×512 pixel content file, 1000 iterations take 60 seconds on a GTX 1080 Ti, 90 seconds on a Maxwell Titan X, or 60 minutes on an Intel Core i7-5930K. Using a GPU is highly recommended due to the huge speedup.

Example 1

Running it for 500-2000 iterations seems to produce nice results. With certain images or output sizes, you might need some hyperparameter tuning (especially --content-weight, --style-weight, and --learning-rate).

The following example was run for 1000 iterations to produce the result (with default parameters):

output

These were the input images used (me sleeping at a hackathon and Starry Night):

input-content

input-style

Example 2

The following example demonstrates style blending, and was run for 1000 iterations to produce the result (with style blend weight parameters 0.8 and 0.2):

output

The content input image was a picture of the Stata Center at MIT:

input-content

The style input images were Picasso's "Dora Maar" and Starry Night, with the Picasso image having a style blend weight of 0.8 and Starry Night having a style blend weight of 0.2:

input-style input-style

Tweaking

--style-layer-weight-exp command line argument could be used to tweak how "abstract" the style transfer should be. Lower values mean that style transfer of a finer features will be favored over style transfer of a more coarse features, and vice versa. Default value is 1.0 - all layers treated equally. Somewhat extreme examples of what you can achieve:

--style-layer-weight-exp 0.2 --style-layer-weight-exp 2.0

(left: 0.2 - finer features style transfer; right: 2.0 - coarser features style transfer)

--content-weight-blend specifies the coefficient of content transfer layers. Default value - 1.0, style transfer tries to preserve finer grain content details. The value should be in range [0.0; 1.0].

--content-weight-blend 1.0 --content-weight-blend 0.1

(left: 1.0 - default value; right: 0.1 - more abstract picture)

--pooling allows to select which pooling layers to use (specify either max or avg). Original VGG topology uses max pooling, but the style transfer paper suggests replacing it with average pooling. The outputs are perceptually different, max pool in general tends to have finer detail style transfer, but could have troubles at lower-freqency detail level:

--pooling max --pooling avg

(left: max pooling; right: average pooling)

--preserve-colors boolean command line argument adds post-processing step, which combines colors from the original image and luma from the stylized image (YCbCr color space), thus producing color-preserving style transfer:

--pooling max --pooling max

(left: original stylized image; right: color-preserving style transfer)

Requirements

Data Files

  • Pre-trained VGG network (MD5 106118b7cf60435e6d8e04f6a6dc3657) - put it in the top level of this repository, or specify its location using the --network option.

Dependencies

You can install Python dependencies using pip install -r requirements.txt, and it should just work. If you want to install the packages manually, here's a list:

Related Projects

See here for an implementation of fast (feed-forward) neural style in TensorFlow.

Try neural style client-side in your web browser without installing any software (using TensorFire).

Citation

If you use this implementation in your work, please cite the following:

@misc{athalye2015neuralstyle,
  author = {Anish Athalye},
  title = {Neural Style},
  year = {2015},
  howpublished = {\url{https://github.com/anishathalye/neural-style}},
}

License

Copyright (c) Anish Athalye. Released under GPLv3. See LICENSE.txt for details.