Pros of TTS

• More comprehensive and actively maintained project
• Supports multiple TTS architectures and models
• Includes pre-trained models and easy-to-use inference scripts

Cons of TTS

• More complex setup and usage due to broader feature set
• Larger codebase, potentially harder to understand for beginners
• Requires more computational resources for training and inference

Code Comparison

TTS:

from TTS.utils.synthesizer import Synthesizer

synthesizer = Synthesizer(
    tts_checkpoint="path/to/model.pth",
    tts_config_path="path/to/config.json",
    vocoder_checkpoint="path/to/vocoder.pth",
    vocoder_config="path/to/vocoder_config.json"
)
wavs = synthesizer.tts("Hello world!")

Tacotron:

from synthesizer import Synthesizer

synthesizer = Synthesizer()
wav = synthesizer.synthesize("Hello world!")

The TTS code snippet demonstrates its more flexible architecture, allowing for separate TTS and vocoder models. Tacotron's implementation is simpler but less customizable. TTS offers more control over the synthesis process, while Tacotron provides a more straightforward API for basic usage.

Pros of tacotron2

• Improved audio quality with WaveNet vocoder integration
• Better performance and faster training due to NVIDIA optimizations
• More extensive documentation and examples

Cons of tacotron2

• Higher computational requirements for training and inference
• More complex architecture, potentially harder to modify or customize

Code Comparison

tacotron:

def prenet(inputs, is_training, scope="prenet"):
    with tf.variable_scope(scope):
        outputs = tf.layers.dense(inputs, units=256, activation=tf.nn.relu)
        outputs = tf.layers.dropout(outputs, rate=0.5, training=is_training)
        outputs = tf.layers.dense(outputs, units=128, activation=tf.nn.relu)
        outputs = tf.layers.dropout(outputs, rate=0.5, training=is_training)
    return outputs

tacotron2:

class Prenet(nn.Module):
    def __init__(self, in_dim, sizes):
        super(Prenet, self).__init__()
        in_sizes = [in_dim] + sizes[:-1]
        self.layers = nn.ModuleList(
            [LinearNorm(in_size, out_size, bias=False)
             for (in_size, out_size) in zip(in_sizes, sizes)])

    def forward(self, x):
        for linear in self.layers:
            x = F.dropout(F.relu(linear(x)), p=0.5, training=True)
        return x

The tacotron2 implementation uses PyTorch, while the original tacotron uses TensorFlow. The tacotron2 version offers more flexibility with customizable layer sizes and uses a more object-oriented approach.

Pros of Tacotron (keithito)

• More active development and maintenance
• Better documentation and code organization
• Includes pre-trained models for quicker start

Cons of Tacotron (keithito)

• Slightly more complex implementation
• May require more computational resources

Code Comparison

Tacotron (keithito):

def create_hparams(hparams_string=None, verbose=False):
    hparams = tf.contrib.training.HParams(
        # Comma-separated list of cleaners to run on text prior to training and eval:
        cleaners='english_cleaners',
        # Audio:
        num_mels=80,
        num_freq=1025,
        sample_rate=20000,
        frame_length_ms=50,
        frame_shift_ms=12.5,
        preemphasis=0.97,
        min_level_db=-100,
        ref_level_db=20,
    )

Tacotron (Kyubyong):

def get_spectrograms(sound_file):
    # Loading sound file
    y, sr = librosa.load(sound_file, sr=None)

    # Trimming
    y, _ = librosa.effects.trim(y)

    # Preemphasis
    y = np.append(y[0], y[1:] - hp.preemphasis * y[:-1])

    # stft
    linear = librosa.stft(y=y,
                          n_fft=hp.n_fft,
                          hop_length=hp.hop_length,
                          win_length=hp.win_length)

The code snippets show different approaches to audio processing and hyperparameter management between the two implementations.

Pros of Real-Time-Voice-Cloning

• Offers real-time voice cloning capabilities
• Includes a user-friendly interface for easy interaction
• Supports multi-speaker voice cloning

Cons of Real-Time-Voice-Cloning

• More complex setup and dependencies
• Requires more computational resources
• May have longer processing times for voice synthesis

Code Comparison

Tacotron

def griffin_lim(spectrogram):
    X_best = copy.deepcopy(spectrogram)
    for i in range(n_iter):
        X_t = invert_spectrogram(X_best)
        est = librosa.stft(X_t, n_fft, hop_length, win_length=win_length)
        phase = est / np.maximum(1e-8, np.abs(est))
        X_best = spectrogram * phase
    X_t = invert_spectrogram(X_best)
    y = np.real(X_t)
    return y

Real-Time-Voice-Cloning

def load_model(model_dir: Path):
    json_config = model_dir.joinpath("config.json")
    with json_config.open() as f:
        config = json.load(f)
    model = SV2TTS(config)
    checkpoint = torch.load(model_dir.joinpath("model.pt"))
    model.load_state_dict(checkpoint["model_state"])
    return model

The code snippets showcase different aspects of each project. Tacotron focuses on audio processing with the Griffin-Lim algorithm, while Real-Time-Voice-Cloning emphasizes model loading and configuration for voice synthesis.

Pros of WaveRNN

• Faster inference time due to efficient WaveRNN architecture
• Supports real-time audio synthesis
• More recent implementation with active development

Cons of WaveRNN

• Requires more computational resources for training
• Less extensive documentation compared to Tacotron
• Narrower focus on vocoder implementation

Code Comparison

Tacotron (preprocessing):

def preprocess(text):
    text = text.lower()
    seq = [char2idx[c] for c in text]
    seq += [char2idx[EOS]]  # End of Sequence
    return np.array(seq, dtype=np.int32)

WaveRNN (audio processing):

def load_wav(path):
    return librosa.load(path, sr=hp.sample_rate)[0]

def save_wav(x, path):
    scipy.io.wavfile.write(path, hp.sample_rate, x.astype(np.float32))

The code snippets highlight different focuses: Tacotron emphasizes text preprocessing for sequence-to-sequence models, while WaveRNN concentrates on audio processing tasks, reflecting their respective roles in the text-to-speech pipeline.