nuewa pytorch online with Winfy

We have hosted the application nuewa pytorch in order to run this application in our online workstations with Wine or directly.

Quick description about nuewa pytorch:

Implementation of N�WA, state of the art attention network for text-to-Video synthesis, in Pytorch. It also contains an extension into Video and audio generation, using a dual decoder approach. It seems as though a diffusion-based method has taken the new throne for SOTA. However, I will continue on with NUWA, extending it to use multi-headed codes + hierarchical causal transformer. I think that direction is untapped for improving on this line of work. In the paper, they also present a way to condition the Video generation based on segmentation mask(s). You can easily do this as well, given you train a VQGanVAE on the sketches beforehand. Then, you will use NUWASketch instead of NUWA, which can accept the sketch VAE as a reference. This repository will also offer a variant of NUWA that can produce both Video and audio. For now, the audio will need to be encoded manually.

Features:

• Train the VAE
• Conditioning on Sketches
• Text to Video and audio
• Te audio will need to be encoded manually
• This library will offer some utilities to make training easier
• This library depends on this vector quantization library

Programming Language: Python.
Categories: