The website offers an AI-powered multimodal project focused on music, generating chords, beats, lyrics, melody, and tabs for any song. Using various transformer models like U-Net and Segment-Net, the platform can detect music chords, beats, tempo, pitch, structure, and lyrics, supporting multiple languages. Users can generate playable sheet music with editing capabilities. Another feature is audio source separation and speed adjustment. The model isolates the vocal track, extracts pitch features, and combines different aspects of the audio using TimbreBlock before generating the final output. The site also includes a demo of training on Donald Trump’s speech.
https://github.com/JoinMusic/fish