projects

Work in progress...

We propose a novel vector quantization method, GQ-VAE, which incorporates a dual latent space, regularized to a Gaussian distribution prior to quantization. This design proved highly effective, simultaneously mitigating both posterior and codebook collapse while offering the versatility to support downstream models operating on either continuous or discrete representations.

We introduce a novel framework that bridges the modality gap between continuous diffusion models and the generation of inherently discrete symbolic music. Our approach consists of two core components: a Mel Encoder that processes log-mel spectrogram inputs into a latent representation, and a Bar Tokenizer that maps piano rolls into a continuous latent space. This continuous embedding makes the discrete musical data amenable to any standard diffusion model for sampling and generation. A key innovation of our method is its focus on bar-level segmentation, a departure from conventional time-based splitting that allows for a more musically coherent representation.

I use a custom-built neural network, which is an ensemble of 10 modified ResNet18 models, to classify voice types in song snippets. The system categorizes audio into four classes: no voice, a male-like voice, a female-like voice, or multiple voices, using Mel spectrograms as input features.

Code Report