Official implementation of SawSing (ISMIR'22)
Authors: Da-Yi Wu*, Wen-Yi Hsiao*, Fu-Rong Yang*, Oscar Friedman, Warren Jackson, Scott Bruzenak, Yi-Wen Liu, Yi-Hsuan Yang
*equal contribution
Official PyTorch Implementation of ISMIR2022 paper "DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluation".
In this repository:
pip install -r requirements.txt
Please refer to dataset.md for more details.
Train vocoders from scratch.
..config/<model_name>.yaml
# SawSing as an example
python main.py --config ./configs/sawsinsub.yaml \
--stage training \
--model SawSinSub
--model
argument to try different vocoders. Currently, we have 5 models: SawSinSub
(Sawsing), Sins
(DDSP-Add), DWS
(DWTS), Full
, SawSub
. For more details, please refer to our documentation - DDSP Vocoders.Our training resources: single Nvidia RTX 3090 Ti GPU
Run validation: compute loss and real-time factor (RTF).
..config/<model_name>.yaml
# SawSing as an example
python main.py --config ./configs/sawsinsub.yaml \
--stage validation \
--model SawSinSub \
--model_ckpt ./exp/f1-full/sawsinsub-256/ckpts/vocoder_27740_70.0_params.pt \
--output_dir ./test_gen
Synthesize audio file from existed mel-spectrograms. The code and specfication for extracting mel-spectrograms can be found in preprocess.py
.
# SawSing as an example
python main.py --config ./configs/sawsinsub.yaml \
--stage inference \
--model SawSinSub \
--model_ckpt ./exp/f1-full/sawsinsub-256/ckpts/vocoder_27740_70.0_params.pt \
--input_dir ./path/to/mel
--output_dir ./test_gen
In Sawsing, we found there are buzzing artifacts in the harmonic part singals, so we develop a post-processing codes to remove them. The method is simple yet effective --- applying a voiced/unvoiced mask. For more details, please refer to here.
./exp/f1-full/sins/ckpts/
./exp/f1-full/sawsinsub-256/ckpts/
exp
folder.@article{sawsing,
title={DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluation},
author={Da-Yi Wu, Wen-Yi Hsiao, Fu-Rong Yang, Oscar Friedman, Warren Jackson, Scott Bruzenak, Yi-Wen Liu, Yi-Hsuan Yang},
journal = {Proc. International Society for Music Information Retrieval},
year = {2022},
}