Convert Figma logo to code with AI

emilwallner logoScreenshot-to-code

A neural network that transforms a design mock-up into a static website.

16,367
1,554
16,367
21

Top Related Projects

Keras model to generate HTML code from hand-drawn website mockups. Implements an image captioning architecture to drawn source images.

11,940

pix2code: Generating Code from a Graphical User Interface Screenshot

7,528

A tool for defining design systems and using them to generate cross-platform UI code, Sketch files, and other artifacts.

164,677

Flutter makes it easy and fast to build beautiful apps for mobile and beyond

A framework for building native applications using React

Quick Overview

Screenshot-to-code is an innovative project that uses machine learning to convert screenshots of web designs into functional HTML and CSS code. It aims to bridge the gap between design and development by automating the process of translating visual designs into code, potentially saving time and reducing errors in the frontend development process.

Pros

  • Automates the process of converting designs to code, potentially saving development time
  • Helps maintain consistency between design and implementation
  • Can be a useful learning tool for aspiring web developers
  • Demonstrates the potential of AI in web development workflows

Cons

  • May not produce production-ready code, requiring manual refinement
  • Limited to simpler layouts and designs; complex or unique designs may not be accurately translated
  • Depends on the quality and diversity of the training data
  • May not keep up with the latest web design trends or technologies

Code Examples

# Load and preprocess the screenshot
screenshot = load_image('website_screenshot.png')
preprocessed_image = preprocess_image(screenshot)

# Generate HTML and CSS code
generated_code = model.predict(preprocessed_image)

# Output the generated code
print(generated_code)
# Fine-tune the model with custom data
custom_dataset = load_custom_dataset('my_designs/')
model.fine_tune(custom_dataset, epochs=10)
# Evaluate the model's performance
test_images = load_test_images('test_screenshots/')
accuracy = model.evaluate(test_images)
print(f"Model accuracy: {accuracy}")

Getting Started

  1. Clone the repository:

    git clone https://github.com/emilwallner/Screenshot-to-code.git
    cd Screenshot-to-code
    
  2. Install dependencies:

    pip install -r requirements.txt
    
  3. Run the model:

    from screenshot_to_code import ScreenshotToCode
    
    model = ScreenshotToCode()
    code = model.generate('path/to/screenshot.png')
    print(code)
    

Note: Detailed setup instructions and model training processes can be found in the repository's README and documentation.

Competitor Comparisons

Keras model to generate HTML code from hand-drawn website mockups. Implements an image captioning architecture to drawn source images.

Pros of sketch-code

  • Focuses specifically on converting hand-drawn wireframes to HTML/CSS
  • Includes a more comprehensive dataset of wireframe images
  • Offers a web-based demo for easy testing

Cons of sketch-code

  • Less versatile, as it's limited to wireframe-to-code conversion
  • Hasn't been updated recently, potentially outdated
  • Lacks some of the advanced features found in Screenshot-to-code

Code Comparison

Screenshot-to-code:

def generate_html(model, screenshot, output_file):
    predicted_code = model.predict(screenshot)
    with open(output_file, 'w') as f:
        f.write(predicted_code)

sketch-code:

def convert_sketch(model, sketch_image):
    preprocessed = preprocess_image(sketch_image)
    html_output = model.predict(preprocessed)
    return html_output

Both projects aim to convert visual designs into code, but they differ in their specific focus and implementation. Screenshot-to-code offers a more general approach, converting various types of screenshots to HTML/CSS, while sketch-code specializes in transforming hand-drawn wireframes. The code snippets demonstrate the core functionality of each project, with Screenshot-to-code directly generating and saving HTML, while sketch-code focuses on the conversion process itself.

11,940

pix2code: Generating Code from a Graphical User Interface Screenshot

Pros of pix2code

  • More comprehensive research paper with detailed methodology
  • Supports multiple platforms (web, Android, iOS)
  • Includes a custom domain-specific language (DSL) for UI representation

Cons of pix2code

  • Less active development and community engagement
  • Limited to simpler UI layouts and components
  • Requires more setup and dependencies

Code Comparison

pix2code DSL example:

header {
  btn-active, btn-inactive, btn-inactive
}
row {
  double {
    label, text
  }
}

Screenshot-to-code HTML output:

<div class="container">
  <h1>Welcome</h1>
  <p>This is a sample page</p>
  <button class="btn btn-primary">Click me</button>
</div>

Both projects aim to convert UI designs to code, but they differ in approach and output. pix2code uses a custom DSL as an intermediate step, while Screenshot-to-code generates HTML/CSS directly. Screenshot-to-code has seen more recent updates and community contributions, making it potentially more adaptable to current web development practices. However, pix2code's multi-platform support and detailed research backing may be advantageous for certain use cases.

7,528

A tool for defining design systems and using them to generate cross-platform UI code, Sketch files, and other artifacts.

Pros of Lona

  • Provides a comprehensive design system tool for cross-platform development
  • Offers a visual editor for creating and managing design components
  • Supports code generation for multiple platforms (iOS, Android, React)

Cons of Lona

  • Steeper learning curve due to its more complex feature set
  • Requires more setup and configuration compared to Screenshot-to-code
  • Limited to specific platforms and languages supported by the tool

Code Comparison

Screenshot-to-code (HTML output):

<div class="container">
  <h1>Welcome</h1>
  <p>This is a sample page</p>
</div>

Lona (Swift output):

let container = UIView()
let welcomeLabel = UILabel()
welcomeLabel.text = "Welcome"
let descriptionLabel = UILabel()
descriptionLabel.text = "This is a sample page"

Screenshot-to-code focuses on generating HTML/CSS from images, while Lona provides a more comprehensive design system approach with code generation for multiple platforms. Screenshot-to-code is simpler to use but limited in scope, whereas Lona offers more features and flexibility at the cost of increased complexity.

164,677

Flutter makes it easy and fast to build beautiful apps for mobile and beyond

Pros of Flutter

  • Mature, production-ready framework for cross-platform app development
  • Extensive documentation and large community support
  • Rich set of pre-built widgets for creating responsive UIs

Cons of Flutter

  • Larger app size compared to native applications
  • Steeper learning curve for developers new to Dart language
  • Limited access to platform-specific features without plugins

Code Comparison

Screenshot-to-code (Python):

def generate_html(screenshot):
    model = load_model('model.h5')
    prediction = model.predict(screenshot)
    return convert_to_html(prediction)

Flutter (Dart):

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(title: Text('Flutter App')),
        body: Center(child: Text('Hello, World!')),
      ),
    );
  }
}

Summary

Screenshot-to-code is an experimental project that uses machine learning to convert screenshots into code, while Flutter is a comprehensive framework for building cross-platform applications. Flutter offers a more robust solution for app development with a wide range of features and community support, but it may require more resources and has a steeper learning curve. Screenshot-to-code, on the other hand, focuses on a specific use case and may be more suitable for rapid prototyping or design-to-code conversion tasks.

A framework for building native applications using React

Pros of React Native

  • Mature, widely-adopted framework with extensive community support and resources
  • Cross-platform development for iOS and Android from a single codebase
  • Backed by Facebook, ensuring long-term maintenance and updates

Cons of React Native

  • Steeper learning curve, requiring knowledge of React and mobile development concepts
  • Performance may not match native development for complex, graphics-intensive apps
  • Occasional challenges with third-party library compatibility and native module integration

Code Comparison

React Native:

import React from 'react';
import { View, Text, StyleSheet } from 'react-native';

const App = () => (
  <View style={styles.container}>
    <Text>Hello, React Native!</Text>
  </View>
);

Screenshot-to-code:

from model import *
from data import *

model = load_model('bin/model_weights.h5')
input_path = 'examples/input.png'
output_path = 'examples/output.html'

generate_html(model, input_path, output_path)

Key Differences

  • React Native focuses on building mobile apps, while Screenshot-to-code aims to convert images to HTML/CSS
  • React Native requires manual coding, whereas Screenshot-to-code uses AI to generate code from visual input
  • React Native offers more control and flexibility, while Screenshot-to-code prioritizes rapid prototyping

Use Cases

  • React Native: Developing full-featured, cross-platform mobile applications
  • Screenshot-to-code: Quickly converting design mockups or screenshots into initial HTML/CSS code

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README


A detailed tutorial covering the code in this repository: Turning design mockups into code with deep learning.

Plug: 👉 Check out my 60-page guide, No ML Degree, on how to land a machine learning job without a degree.

The neural network is built in three iterations. Starting with a Hello World version, followed by the main neural network layers, and ending by training it to generalize.

The models are based on Tony Beltramelli's pix2code, and inspired by Airbnb's sketching interfaces, and Harvard's im2markup.

Note: only the Bootstrap version can generalize on new design mock-ups. It uses 16 domain-specific tokens which are translated into HTML/CSS. It has a 97% accuracy. The best model uses a GRU instead of an LSTM. This version can be trained on a few GPUs. The raw HTML version has potential to generalize, but is still unproven and requires a significant amount of GPUs to train. The current model is also trained on a homogeneous and small dataset, thus it's hard to tell how well it behaves on more complex layouts.

Dataset: https://github.com/tonybeltramelli/pix2code/tree/master/datasets

A quick overview of the process:

1) Give a design image to the trained neural network

Insert image

2) The neural network converts the image into HTML markup

3) Rendered output

Screenshot

Installation

FloydHub

Run on FloydHub

Click this button to open a Workspace on FloydHub where you will find the same environment and dataset used for the Bootstrap version. You can also find the trained models for testing.

Local

pip install keras tensorflow pillow h5py jupyter
git clone https://github.com/emilwallner/Screenshot-to-code.git
cd Screenshot-to-code/
jupyter notebook

Go do the desired notebook, files that end with '.ipynb'. To run the model, go to the menu then click on Cell > Run all

The final version, the Bootstrap version, is prepared with a small set to test run the model. If you want to try it with all the data, you need to download the data here: https://www.floydhub.com/emilwallner/datasets/imagetocode, and specify the correct dir_name.

Folder structure

  |  |-Bootstrap                           #The Bootstrap version
  |  |  |-compiler                         #A compiler to turn the tokens to HTML/CSS (by pix2code)
  |  |  |-resources											
  |  |  |  |-eval_light                    #10 test images and markup
  |  |-Hello_world                         #The Hello World version
  |  |-HTML                                #The HTML version
  |  |  |-Resources_for_index_file         #CSS,images and scripts to test index.html file
  |  |  |-html                             #HTML files to train it on
  |  |  |-images                           #Screenshots for training
  |-readme_images                          #Images for the readme page

Hello World

HTML

Bootstrap

Model weights

  • Bootstrap (The pre-trained model uses GRUs instead of LSTMs)
  • HTML

Acknowledgments

  • Thanks to IBM for donating computing power through their PowerAI platform
  • The code is largely influenced by Tony Beltramelli's pix2code paper. Code Paper
  • The structure and some of the functions are from Jason Brownlee's excellent tutorial