Top Related Projects
Torch implementation of neural style algorithm
TensorFlow CNN for fast style transfer ⚡🖥🎨🖼
Code and data for paper "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511
Style transfer, deep learning, feature transform
TensorFlow (Python API) implementation of Neural Style
Quick Overview
Neural-style is an implementation of the neural style transfer algorithm, which combines the content of one image with the style of another image using convolutional neural networks. This project, created by Anish Athalye, allows users to generate artistic images by applying the style of one image to the content of another.
Pros
- High-quality results with fine-tuned control over style transfer
- Supports multiple style images and content images
- Includes pre-trained VGG19 model for easy setup
- Well-documented with clear instructions and examples
Cons
- Computationally intensive, requiring a powerful GPU for reasonable processing times
- Limited to static images, not suitable for real-time video processing
- Requires some understanding of neural networks and image processing concepts
- Dependency on older versions of TensorFlow and NumPy
Code Examples
- Basic style transfer:
import neural_style
content_image = 'path/to/content/image.jpg'
style_image = 'path/to/style/image.jpg'
output_image = 'path/to/output/image.jpg'
neural_style.stylize(content_image, style_image, output_image)
- Adjusting style weight:
neural_style.stylize(
content_image,
style_image,
output_image,
style_weight=1e4 # Increase style influence
)
- Using multiple style images:
style_images = ['style1.jpg', 'style2.jpg', 'style3.jpg']
style_blend_weights = [0.3, 0.3, 0.4]
neural_style.stylize(
content_image,
style_images,
output_image,
style_blend_weights=style_blend_weights
)
Getting Started
-
Clone the repository:
git clone https://github.com/anishathalye/neural-style.git cd neural-style
-
Install dependencies:
pip install -r requirements.txt
-
Download the pre-trained VGG19 model:
./models/download_models.sh
-
Run the style transfer:
python neural_style.py --content <content_image> --styles <style_image> --output <output_image>
Replace <content_image>
, <style_image>
, and <output_image>
with your desired file paths.
Competitor Comparisons
Torch implementation of neural style algorithm
Pros of neural-style (jcjohnson)
- More extensive documentation and usage instructions
- Supports multiple GPUs for faster processing
- Includes pre-trained VGG models for convenience
Cons of neural-style (jcjohnson)
- Less actively maintained (last update in 2018)
- Requires older versions of Torch and CUDA
- Limited to specific image sizes due to VGG network constraints
Code Comparison
neural-style (jcjohnson):
local cmd = torch.CmdLine()
cmd:option('-style_image', 'examples/inputs/seated-nude.jpg', 'Style target image')
cmd:option('-content_image', 'examples/inputs/tubingen.jpg', 'Content target image')
cmd:option('-output_image', 'out.png', 'Output image')
cmd:option('-image_size', 512, 'Maximum height / width of generated image')
neural-style (anishathalye):
parser.add_argument('--content', type=str,
dest='content', help='content image',
metavar='CONTENT', required=True)
parser.add_argument('--styles', type=str,
dest='styles', nargs='+', help='style images',
metavar='STYLE', required=True)
parser.add_argument('--output', type=str,
dest='output', help='output path',
metavar='OUTPUT', required=True)
Both repositories implement neural style transfer, but they differ in programming language and command-line argument structure. The jcjohnson version uses Lua and Torch, while the anishathalye version is implemented in Python, making it more accessible to a wider range of developers.
TensorFlow CNN for fast style transfer ⚡🖥🎨🖼
Pros of fast-style-transfer
- Significantly faster processing time for style transfer
- Real-time style transfer capability for video streams
- Pre-trained models available for immediate use
Cons of fast-style-transfer
- Limited flexibility in adjusting style parameters
- Potentially lower quality output compared to neural-style
- Requires training a separate model for each style
Code Comparison
neural-style:
def stylize(content_image, style_image, content_weight, style_weight):
# Optimization-based approach
# Iteratively updates the output image
# ...
fast-style-transfer:
def stylize(content_image, model):
# Feed-forward network approach
# Single pass through a pre-trained model
# ...
The key difference is that neural-style uses an optimization-based approach, iteratively updating the output image, while fast-style-transfer uses a feed-forward network that can generate stylized images in a single pass through a pre-trained model.
neural-style offers more flexibility and potentially higher quality results but at the cost of longer processing times. fast-style-transfer sacrifices some quality and flexibility for significantly faster processing, making it suitable for real-time applications.
Code and data for paper "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511
Pros of deep-photo-styletransfer
- Focuses on photorealistic style transfer, maintaining the structure and details of the original photo
- Implements a photorealistic smoothing step to reduce artifacts and improve output quality
- Utilizes semantic segmentation to better preserve content boundaries
Cons of deep-photo-styletransfer
- More complex implementation and setup compared to neural-style
- Requires additional inputs like segmentation masks for optimal results
- May have longer processing times due to the additional smoothing step
Code Comparison
neural-style:
def stylize(content_image, style_image, content_weight, style_weight):
# Simple implementation using VGG19 for feature extraction
model = vgg19.VGG19(weights='imagenet', include_top=False)
# ... (content and style loss calculation)
# Optimization step
deep-photo-styletransfer:
function [stylized_im] = deep_photo_style_transfer(content_im, style_im, content_seg, style_seg)
% Photorealistic style transfer with segmentation and smoothing
[feat_content, feat_style] = vgg19_forward(content_im, style_im);
% ... (content and style loss calculation with segmentation)
% Photorealistic smoothing step
stylized_im = wls_smoothing(raw_output, content_im);
end
The code snippets highlight the differences in approach, with deep-photo-styletransfer incorporating segmentation and smoothing for more photorealistic results.
Style transfer, deep learning, feature transform
Pros of FastPhotoStyle
- Faster processing time for style transfer
- Better preservation of content structure and details
- Supports both photo-realistic and artistic style transfer
Cons of FastPhotoStyle
- Requires more computational resources (GPU)
- Less flexibility in style manipulation compared to Neural Style
- More complex setup and dependencies
Code Comparison
Neural Style:
content_image = imread(args.content_image)
style_image = imread(args.style_image)
network = build_vgg19(content_image)
content_features = get_content_features(network, content_image)
style_features = get_style_features(network, style_image)
output = optimize_image(content_features, style_features)
FastPhotoStyle:
content = load_image(args.content)
style = load_image(args.style)
content_seg = load_seg(args.content_seg)
style_seg = load_seg(args.style_seg)
output = stylize(content, style, content_seg, style_seg)
Both repositories focus on neural style transfer, but FastPhotoStyle aims for more photo-realistic results and faster processing. Neural Style offers more artistic flexibility but may be slower. FastPhotoStyle requires segmentation maps for optimal results, while Neural Style operates directly on images. The code structure reflects these differences, with FastPhotoStyle incorporating segmentation and Neural Style focusing on feature extraction and optimization.
TensorFlow (Python API) implementation of Neural Style
Pros of neural-style-tf
- Implemented in TensorFlow, offering potential performance benefits and GPU acceleration
- Supports multiple style images for blended styles
- Includes additional features like color preservation and masked styling
Cons of neural-style-tf
- May require more setup due to TensorFlow dependencies
- Potentially more complex codebase for users unfamiliar with TensorFlow
Code Comparison
neural-style (Lua/Torch):
local img = image.load(params.content_image, 3)
local img = image.scale(img, params.image_size, 'bilinear')
local content_image = preprocess(img):float()
neural-style-tf (Python/TensorFlow):
content_image = scipy.misc.imread(content_image)
content_image = scipy.misc.imresize(content_image, (image_height, image_width))
content_image = content_image.astype(np.float32)
Both repositories implement the neural style transfer algorithm, but neural-style-tf offers a more modern TensorFlow implementation with additional features. The choice between them may depend on your familiarity with Lua/Torch vs. Python/TensorFlow and specific requirements for style transfer tasks.
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
neural-style
An implementation of neural style in TensorFlow.
This implementation is a lot simpler than a lot of the other ones out there, thanks to TensorFlow's really nice API and automatic differentiation.
TensorFlow doesn't support L-BFGS (which is what the original authors used), so we use Adam. This may require a little bit more hyperparameter tuning to get nice results.
Running
python neural_style.py --content <content file> --styles <style file> --output <output file>
Run python neural_style.py --help
to see a list of all options.
Use --checkpoint-output
and --checkpoint-iterations
to save checkpoint images.
Use --iterations
to change the number of iterations (default 1000). For a 512Ã512 pixel content file, 1000 iterations take 60 seconds on a GTX 1080 Ti, 90 seconds on a Maxwell Titan X, or 60 minutes on an Intel Core i7-5930K. Using a GPU is highly recommended due to the huge speedup.
Example 1
Running it for 500-2000 iterations seems to produce nice results. With certain
images or output sizes, you might need some hyperparameter tuning (especially
--content-weight
, --style-weight
, and --learning-rate
).
The following example was run for 1000 iterations to produce the result (with default parameters):
These were the input images used (me sleeping at a hackathon and Starry Night):
Example 2
The following example demonstrates style blending, and was run for 1000 iterations to produce the result (with style blend weight parameters 0.8 and 0.2):
The content input image was a picture of the Stata Center at MIT:
The style input images were Picasso's "Dora Maar" and Starry Night, with the Picasso image having a style blend weight of 0.8 and Starry Night having a style blend weight of 0.2:
Tweaking
--style-layer-weight-exp
command line argument could be used to tweak how "abstract"
the style transfer should be. Lower values mean that style transfer of a finer features
will be favored over style transfer of a more coarse features, and vice versa. Default
value is 1.0 - all layers treated equally. Somewhat extreme examples of what you can achieve:
(left: 0.2 - finer features style transfer; right: 2.0 - coarser features style transfer)
--content-weight-blend
specifies the coefficient of content transfer layers. Default value -
1.0, style transfer tries to preserve finer grain content details. The value should be
in range [0.0; 1.0].
(left: 1.0 - default value; right: 0.1 - more abstract picture)
--pooling
allows to select which pooling layers to use (specify either max
or avg
).
Original VGG topology uses max pooling, but the style transfer paper suggests
replacing it with average pooling. The outputs are perceptually different, max pool in
general tends to have finer detail style transfer, but could have troubles at
lower-freqency detail level:
(left: max pooling; right: average pooling)
--preserve-colors
boolean command line argument adds post-processing step, which
combines colors from the original image and luma from the stylized image (YCbCr color
space), thus producing color-preserving style transfer:
(left: original stylized image; right: color-preserving style transfer)
Requirements
Data Files
- Pre-trained VGG network (MD5
106118b7cf60435e6d8e04f6a6dc3657
) - put it in the top level of this repository, or specify its location using the--network
option.
Dependencies
You can install Python dependencies using pip install -r requirements.txt
,
and it should just work. If you want to install the packages manually, here's a
list:
Related Projects
See here for an implementation of fast (feed-forward) neural style in TensorFlow.
Try neural style client-side in your web browser without installing any software (using TensorFire).
Citation
If you use this implementation in your work, please cite the following:
@misc{athalye2015neuralstyle,
author = {Anish Athalye},
title = {Neural Style},
year = {2015},
howpublished = {\url{https://github.com/anishathalye/neural-style}},
}
License
Copyright (c) Anish Athalye. Released under GPLv3. See LICENSE.txt for details.
Top Related Projects
Torch implementation of neural style algorithm
TensorFlow CNN for fast style transfer ⚡🖥🎨🖼
Code and data for paper "Deep Photo Style Transfer": https://arxiv.org/abs/1703.07511
Style transfer, deep learning, feature transform
TensorFlow (Python API) implementation of Neural Style
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot