Zero-Shot Voice Cloning with OmniVoice on Kaggle

Voice cloning has transitioned from a niche research topic to an accessible, powerful tool for developers. But most high-quality models require massive computational resources or complex local setups. What if you could run state-of-the-art, zero-shot voice cloning entirely in the cloud, for free?

In this article, I’ll walk you through how I successfully deployed OmniVoice by k2-fsa inside a Kaggle notebook to synthesize natural-sounding speech from a short reference audio clip.

What is OmniVoice?

OmniVoice is an advanced text-to-speech (TTS) and voice cloning model available on Hugging Face. Developed by the k2-fsa team (known for their groundbreaking work on Kaldi and next-generation speech recognition), OmniVoice excels at zero-shot voice cloning. This means it can replicate the tone, timbre, and prosody of a speaker using just a brief reference audio sample, without requiring any fine-tuning, retraining, or massive datasets.

Step 1: Setting Up the Environment

Kaggle provides a fantastic environment with free GPU access (like the T4 or P100). The first step is to install the necessary dependencies. We need the omnivoice package, along with audio processing libraries like soundfile and torchaudio.

!pip install omnivoice
!pip install -q huggingface_hub IPython soundfile torchaudio

Note: The omnivoice package automatically handles underlying dependencies like transformers, torch, and librosa, ensuring a smooth and conflict-free setup.

Step 2: Downloading the Model Checkpoints

Instead of loading the model directly over the network during inference (which can be slow or prone to timeouts on free tiers), we use huggingface_hub to download the model weights directly into the notebook’s working directory.

import os
from huggingface_hub import snapshot_download

model_dir = "./checkpoints"
os.makedirs(model_dir, exist_ok=True)

# Download model from k2-fsa/OmniVoice
snapshot_download(
    repo_id="k2-fsa/OmniVoice",
    local_dir=model_dir
)
print("Model downloaded successfully!")

This caches the model files locally, making subsequent loads lightning fast and immune to transient network drops.

Step 3: Initializing the Model on GPU

With the weights downloaded, we can initialize the model. Kaggle’s GPU acceleration (CUDA) makes the inference process highly efficient.

import torch
from omnivoice import OmniVoice

# Check CUDA availability
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Using device: {device}")

# Initialize model
model = OmniVoice.from_pretrained("./checkpoints").to(device)

Step 4: Zero-Shot Voice Cloning in Action

The magic happens here. We provide a reference audio file (in my case, a .wav file of my own voice from a previous test) and the text we want the model to speak.

# Reference audio for zero-shot voice cloning
ref_audio_path = "/kaggle/input/datasets/zohirmalek/my-voice/Testing the model Gemma-4-E4B-it on Kaggle notebook.wav"

# Text to synthesize
text = "Hello everyone and welcome to my website! This is OmniVoice running inside a Kaggle notebook."

# Generate audio
output_audio = model.generate(
    text=text,
    ref_audio=ref_audio_path,
    device=device
)

Step 5: Saving and Listening to the Output

The generated output is a numerical array representing the audio waveform. We can easily save this as a standard .wav file using the soundfile library.

import soundfile as sf
import numpy as np

output_path = "output.wav"
audio = np.array(output_audio).squeeze()

sf.write(
    output_path,
    audio,
    samplerate=24000,
    format="WAV",
    subtype="PCM_16"
)

print(f"Audio saved to {output_path}")

You can then use IPython.display.Audio to listen to the result directly in the notebook. The fidelity and resemblance to the reference voice are remarkably high, capturing natural pauses and inflection.

Why This Workflow Matters

1. Zero Cost, High Power: By leveraging Kaggle’s free GPU tier, you can experiment with cutting-edge TTS models without investing in expensive local hardware.
2. Reproducibility: The entire pipeline—from dependency installation to audio generation—is encapsulated in a single, shareable notebook. Anyone can fork it and run it with their own voice samples.
3. Democratization of AI: Tools like OmniVoice, combined with accessible cloud platforms, lower the barrier to entry for developers wanting to build voice-enabled applications, accessibility tools, dynamic NPC dialogue, or interactive media.

Conclusion

Running advanced zero-shot voice cloning models no longer requires a PhD in machine learning or a massive compute budget. With k2-fsa’s OmniVoice and Kaggle, you can build, test, and deploy high-quality voice synthesis pipelines in minutes.

If you’re building a project that needs a personalized voice interface, I highly recommend giving this workflow a try.

---

Useful Links

• OmniVoice Model on Hugging Face
• k2-fsa GitHub Organization
• Kaggle Notebooks Documentation
• Hugging Face huggingface_hub Library

Have you experimented with voice cloning or TTS models? Let me know your experiences or share your own Kaggle notebooks in the comments below!