How to Create Vocal Songs with NEUTRINO Diffusion and MuseScore: A Beginner’s Guide to Japanese Vocal Synthesis

What is MuseScore?

MuseScore is a free software that allows you to easily create and edit sheet music. It is designed for everyone, from music beginners to professional composers. With this software, you can not only input notes and edit existing sheet music data easily, but also print the sheet music beautifully and play back the music to check the sound.

Background of Development

MuseScore is developed in Brussels, Belgium, as an open-source project and is continuously improved by developers from around the world. As a result, it is available for free and regularly receives new features and enhancements to existing ones.

Main Features of MuseScore

Note Input and Editing
With MuseScore, you can easily input notes and create sheet music. The intuitive operation allows you to add or delete notes with just a click.

Lyric Input
You can also add lyrics to the sheet music you create. For example, you can assign multiple characters to a single note, accurately representing the melody line of the lyrics.

MIDI Data Import and Export
You can import MIDI data into MuseScore and convert it into sheet music. Conversely, you can save the sheet music you create as MIDI data and use it in other music production software.

For Those Who Want to Learn More

The official MuseScore website offers software downloads and detailed tutorials. If you are interested, please visit the link below.

Official MuseScore Website

Additionally, for a detailed explanation on how to display MIDI data in MuseScore, please watch the video linked below.

View and Playback Sheet Music from MIDI Data in MuseScore

Downloading and Using NEUTRINO

Next, you will download NEUTRINO, a powerful voice synthesis software capable of generating realistic singing voices from text. Recently, this software has been rebranded as “NEUTRINO Diffusion,” offering even greater capabilities, but the basic usage remains the same as the original NEUTRINO.

What is NEUTRINO?

NEUTRINO is software that can use sheet music with lyrics created in MuseScore to produce singing voices using a specified voice library. This allows you to, for example, add vocals to your original compositions or recreate existing sheet music with different voices.

Introduction to Voice Libraries

NEUTRINO comes with various voice libraries featuring different character voices. For example, you can use the voices of popular characters like Shikoku Metan or Zundamon to make your sheet music sing. This enables you to easily create high-quality vocals that sound like professional singers.

Steps to Download NEUTRINO

Here’s how to download the Windows version of NEUTRINO. First, download the software from the official website below.

Official NEUTRINO Website

After downloading, install the software and follow the same steps as the original NEUTRINO for usage. There are many guides and support resources available on the official website and within the community, making it easy even for beginners to get started. Be sure to check them out.

Steps to Download and Extract NEUTRINO

First, let’s download NEUTRINO by following the instructions on the download screen, using the provided screenshots as a guide. The downloaded file is typically compressed in a ZIP format, so you will need to extract it. Once extracted, unless you change the default settings, a folder named “NEUTRINO” will be created, and the files will be unpacked into this folder.

Checking Important Files

Next, pay attention to the run.bat batch file located in the extracted “NEUTRINO” folder. This file is crucial for running NEUTRINO, and it is recommended to check its contents before using the software.

Reviewing the Batch File Contents

Right-click the run.bat file and open it with “Notepad” or your preferred text editor. The first important item to verify is the line that reads set BASENAME. Here, you will need to specify the name of the file that was created and exported in MuseScore. This file name must match the name of the sheet music with lyrics that NEUTRINO will read, so be sure to set it correctly.

Preparing NEUTRINO and Using MuseScore

Once you have prepared NEUTRINO, the next step is to create sheet music using MuseScore and export it in a format that can be used with NEUTRINO. Follow the steps below to proceed.

Checking the Settings in run.bat

First, review the contents of the run.bat file. This file specifies which file NEUTRINO will read. Refer to the provided screenshots to check the settings, and change the file name if necessary.

Preparing MuseScore

Next, launch MuseScore. To ensure that NEUTRINO synthesizes the singing voice correctly, you need not only notes but also corresponding lyrics attached to those notes. If there are notes without lyrics, an error may occur when processing the file in NEUTRINO. Additionally, if there are empty measures without notes, the exported file may not function as expected, so it’s best to delete those measures beforehand.

Tips for Efficient Sheet Music Creation

In MuseScore, it’s effective to create the sheet music gradually rather than all at once. This approach is particularly useful for large or complex compositions, where creating the file can take considerable time. Therefore, start by creating parts that you are satisfied with, and later combine them. This method allows you to divide the work and proceed more efficiently.

Exporting the Sheet Music

Once your sheet music is complete, it’s time to move on to the export process. From the MuseScore menu, select “File” and then “Export.” In the export settings, change the format from “PDF” to “MusicXML,” and make sure to set the file type to “Uncompressed.” This will create a file with the .musicxml extension. This file will be used as the lyrics-embedded sheet music data in NEUTRINO.

Now, go ahead and export the file from the “File” menu in MuseScore. In the export settings, switch the format from PDF to MusicXML, and ensure that the file type is set to Uncompressed. The file extension will be .musicxml.

Specifying the Export Location

Next, specify the location where the exported file will be saved. If you don’t save it in the correct location, NEUTRINO might not be able to read the sheet music correctly, so be sure to double-check this step.

Choosing the Save Location

Set the export destination to the following location within the NEUTRINO folder:

NEUTRINO\score\musicxml

Once you select this folder, pay attention to the following points:

  • File Format: Choose the uncompressed format. Make sure the file extension is set to .musicxml.
  • File Name: It is recommended to use a clear and understandable file name. For example, you might name it test.musicxml.

Confirming the Export

Before finalizing the export, be sure to verify that the export destination, file name, and file format are correct. Refer to the provided screenshots to check that the settings are accurate. This will ensure that NEUTRINO can correctly read the file.

Editing the Batch File (run.bat)

Next, you’ll need to enter the name of the MuseScore file you just created into the batch file. This step is necessary for NEUTRINO to correctly read the sheet music file, so proceed carefully.

How to Enter the File Name

First, open the run.bat file again in Notepad or another text editor. Once opened, look for the line that says set BASENAME=. Here, you will enter the name of the file you just exported, “test”. There’s no need to include the file extension, so just input the file name.

Confirming the Changes

In the provided screenshots, the correctly edited section is highlighted with a red line. Specifically, the part that reads set BASENAME=test is what you should focus on. Once you’ve confirmed that the changes are correct, save the file and close the editor.

Running the Batch File Using Command Prompt

Next, you’ll use the Command Prompt to run NEUTRINO. The Command Prompt is a tool that allows you to input commands directly to your computer to instruct it on what to do. While it may seem a bit challenging for beginners, just follow the steps, and you’ll find it easy to operate.

Launching Command Prompt and Navigating Directories

First, launch the Command Prompt. You can do this by typing “cmd” into the Start menu search and then clicking on Command Prompt.

Next, you need to navigate to the location where the NEUTRINO batch file is located. This is known as “changing directories.” To change directories, you use the cd command, which stands for “change directory.”

For example, you would enter the following command to navigate to the folder where the batch file is located:

cd C:\Users\minok\Downloads\NEUTRINO

An Easy Way to Copy the Directory Path

If you find it tedious or are unsure about manually entering the directory path, you can easily copy it using File Explorer. Navigate to the desired folder in Explorer, click on the address bar, and the entire path will be highlighted. Right-click and select “Copy,” then right-click in the Command Prompt and choose “Paste” to accurately input the path.

Running the Batch File

Once you’ve correctly navigated to the directory, you can run the batch file by entering the following command:

run.bat

After typing this command and pressing Enter, NEUTRINO will start processing the specified sheet music file and begin synthesizing the singing voice.

Checking Screenshots

Refer to the provided screenshots and output examples to check the command input and execution results. Comparing these with your actual operation should help you proceed smoothly.

Content Displayed in Command Prompt

C:\Users\minok\Downloads\NEUTRINO>run.bat
2023/04/09 11:41:14.75 : start MusicXMLtoLabel
    Convert MusicXML to label -> score\musicxml\test.musicxml
        warning: The score does not start with a rest. Added a rest to the first measure.
        warning: The score does not start with a rest, so a rest was added to the first measure.
        warning: The end of the score is not a rest. Added a rest to the last measure.
        warning: The end of the score is not a rest, so a rest was added to the last measure.
    output full label -> score\label\full\test.lab
    output mono label -> score\label\mono\test.lab

2023/04/09 11:41:14.90 : start NEUTRINO
    NEUTRINO - NEURAL SINGING SYNTHESIZER (Electron v1.3.0-Stable)
               Copyright (c) 2020-2022 STUDIO NEUTRINO All rights reserved.

    Setup linguistic feature (timing)   : 0 [msec]
    Load timing model                   : 4 [msec]
        -> Load completed.
        -> Merrow - NEUTRINO Singer Standard Library (v1.3.0-Stable-Timing model)
    Predict timing feature              : 181 [msec]
    Linguistic feature (acoustic)       : 228 [msec]
    Setup GPU                           : 231 [msec]
        -> NVIDIA GeForce GTX 1060 (GPU 0)
    Load acoustic model                 : 276 [msec]
        -> Load completed.
        -> Merrow - NEUTRINO Singer Standard Library (v1.3.0-Stable-Acoustic model)
    Predict acoustic features           : 1648 [msec]
    Finish                              : 4049 [msec]
    Generation rate                     : 1.18548 [gen/sec]

    -- File and Parameter information --
    label length                        : 16 [line]
    wav length                          : 4.8 [sec]
    frame period                        : 5 [frame]
    full label file                     : score\label\full\test.lab
    timing label file                   : score\label\timing\test.lab
    output f0 file                      : output\test.f0
    output mgc file                     : output\test.mgc
    output bap file                     : output\test.bap
    model directory                     : model\MERROW\
    timing model                        : model\MERROW\model_timing.bin
    timing model (stats)                : model\MERROW\stats_timing.bin
    acoustic model                      : model\MERROW\model_acoustic.bin
    acoustic model (stats)              : model\MERROW\stats_acoustic.bin
    skip timing prediction              : 0
    skip acoustic prediction            : 0
    single phrase prediction            : 0
    single phrase number                : -1 [phrase number]
    number of threads (CPU)             : 1 [thread]
    style shift                         : 0 [key]
    use GPU                             : 1
    GPU ID                              : 0
    trace phrase information            : 0
    trace information                   : 1
    ------------

2023/04/09 11:41:19.78 : start WORLD
    WORLD - NEUTRINO Edition (v1.2.0-Stable)

    Load Acoustic features   : 4 [msec]
    Decode Acoustic features : 7 [msec]
    Synthesis                : 51 [msec]
    Finish                   : 364 [msec]
    Generation rate          : 13.1868  [gen/sec]
    -- File and Parameter information --
    wav Length               : 4.8 [sec]
    sampling rate            : 48000 [Hz]
    sampling bit             : 16 [bit]
    pitch shift              : 1
    formant shift            : 1
    number of parallel       : 4 [thread]
    hi-speed synthesis       : 0
    realtime synthesis       : 0
    smooth pitch             : 0
    smooth formant           : 0
    enhance breathiness      : 0
    -------------------

2023/04/09 11:41:20.25 : start NSF
    NSF - Neural Source Filter (v1.3.0-Stable)
          Copyright (c) 2020-2022 STUDIO NEUTRINO All rights reserved.

    Load acoustic features              : 2 [msec]
    Load phrase information             : 8 [msec]
    Setup GPU                           : 9 [msec]
        -> NVIDIA GeForce GTX 1060 (GPU 0)
    Predict waveform (NSF)              : 48 [msec]
        -> Progress = 100 % (4800 / 4800 msec)
    output waveform                     : 5839 [msec]
    Finish                              : 5854 [msec]
    Generation rate                     : 0.819952 [gen/sec]

    -- File and Parameter information --
    wav length                  : 4.8 [sec]
    number of frames            : 960 [frame]
    input f0 file               : output\test.f0
    input mgc file              : output\test.mgc
    input bap file              : output\test.bap
    output wav file             : output\test_nsf.wav
    multi phrase prediction     : 1
    phrase info file            : score\label\timing\test.lab
    num threads (CPU)           : 1 [thread]
    num threads (session)       : 1 [thread]
    use GPU                     : 1
    GPU ID                      : 0
    trace information           : 1
    ------------

2023/04/09 11:41:26.60 : end

C:\Users\minok\Downloads\NEUTRINO>

Generating Audio Files with NEUTRINO

When you run the run.bat file in the Command Prompt, NEUTRINO will load the specified music score file and synthesize the singing voice. This process may take some time, so please be patient and wait. During execution, the progress will be displayed on the Command Prompt screen.

Checking the Execution Results

After some time, a message indicating that the process has completed will be displayed in the Command Prompt. A screenshot has been provided for reference to confirm this status.

Verifying the Generated Audio Files

If NEUTRINO is functioning correctly, audio files will be generated in the specified folder. Specifically, these files should be created within the output folder. Use File Explorer to open this folder and verify that the audio files have been generated properly.

Location of the Audio Files

The audio files are saved in the following location: C:\Users\minok\Downloads\NEUTRINO\output. Open File Explorer and navigate to this path to check the folder. The provided screenshot illustrates that audio files have been created within this output folder.

Checking the Actually Generated Audio Files

The audio files created by NEUTRINO are saved in the output folder. These files contain the synthesized singing voice data generated by NEUTRINO based on the music score. Here, you can check the actual output audio files.

Downloading and Playing the Audio Files

Next, let’s listen to the actual music. Below, we’ve provided the WAV file output by NEUTRINO. You can download this file and play it on your computer or smartphone. Alternatively, you can use the provided playback bar to listen to the music directly here by clicking on it.

Steps to Verify the Audio Files

Downloading: Click the link to download the audio file. Once downloaded, you can play it using a standard media player.

Using the Playback Bar: Simply click on the playback bar to listen to the music in your browser. This option is convenient for a quick check.

Contents of the Audio Files

This audio file includes the synthesized singing voice generated by NEUTRINO, reflecting the melody and lyrics input into the score. Check the sound quality and pronunciation to experience how NEUTRINO synthesizes the singing voice.

Changing the Singing Voice in a Batch File

In NEUTRINO, you can generate music with different character voices by changing the singing voice library. This time, let’s use the “Zundamon” singing voice library.

Downloading the Singing Voice Library

First, download the “Zundamon” singing voice library from the official NEUTRINO website. Once the download is complete, the file is typically compressed in ZIP format, so you will need to extract it.

Checking and Moving the Folder

After extracting, you’ll find a folder with an English name beneath a folder with a Japanese name. This English-named folder contains the library files that NEUTRINO will actually use. You need to move this entire folder to a specified location.

Destination Folder

The destination is the “model” folder within the NEUTRINO installation directory. Drag and drop the extracted folder into this “model” folder.

Using File Explorer

You can refer to the provided screenshot to see how to move the folder using File Explorer. By moving the folder correctly, NEUTRINO will recognize the new singing voice library, and the next time you run it, the music will be synthesized using Zundamon’s voice.

Setting the Singing Voice Library in the Batch File

Next, to properly use the downloaded and moved singing voice library in NEUTRINO, you’ll need to add a setting to the batch file. Specifically, you’ll modify the “set ModelDir” line to include the folder name of the new singing voice library.

Editing the Batch File

First, open the batch file (run.bat) again using Notepad or any text editor. Once opened, look for the line that says “set ModelDir.” On this line, write the name of the English-named folder that you moved earlier using File Explorer. This folder name must match the name of the downloaded singing voice library.

Refer to the Screenshot

In the provided screenshot, you can see how the English-named folder is specified in the “set ModelDir” line. Use this as a reference to make the same configuration.

Save and Execute

Once you’ve completed the setting, save the batch file with an overwrite. After that, run the batch file again using Command Prompt, following the same steps as before. This will allow NEUTRINO to synthesize music using the new singing voice library.

Detailed Explanation in the Video

While I’ve explained the steps in writing, I’ve also prepared a video tutorial for those who prefer to follow along with the actual operations. In the video, I provide a detailed walkthrough of editing the batch file and setting up the singing voice library, allowing you to understand the process while watching the screen.

Video Link

You can watch the video on YouTube by clicking the link below. Following the steps in the video will further deepen your understanding as you proceed step-by-step.

Demonstration in a Masterfully Created Video

In this video, I provide an in-depth demonstration of how to create a song with vocals using NEUTRINO Diffusion. The video showcases the process of inputting all the notes and lyrics in MuseScore and then using NEUTRINO Diffusion to generate the singer’s voice based on that input.

Video Content

Inputting Notes and Lyrics: I used MuseScore to input all the notes and lyrics. This allows NEUTRINO Diffusion to accurately generate a singing voice that matches the pronunciation of the lyrics.

Adding the Melody: Additionally, I created a melody in MuseScore and overlaid it with the singing voice generated by NEUTRINO Diffusion. This method allows you to create what is commonly referred to as a “song with vocals.”

Timing Adjustment: The video also introduces the process of synchronizing the vocal file and melody to play together in perfect timing, resulting in natural-sounding music.

This video not only allows you to visually confirm the operation methods but also provides a firsthand experience of what kind of music can be created using NEUTRINO Diffusion.

Please share if you like it!
TOC