Ultrastar (singstar)

[DeusExMachina]

Ultrastar support would be a nice added feature, both for the occasional person with large Ultrastar collections, as well as for allowing karaoke "competitions."
Also, the source code for both Ultrastar and Ultrastar Deluxe are open source.

[dave]

sadly it's a windoze program which pushes it back pretty far.

after 1.0 mac will maybe come 1.0 windows (1.1 likely)
then after THAT maybe for 2.0 some ultrastar features

[DeusExMachina]

Actually, there is a port for OSX, albeit for intel machines only.

Since what is at issue initially is just file compatibility, I should think that would offer a very good head start, since all it consists of is an audio file and a separate text file with vocals and timing cues.
As for actually having competitions, USNG (for Linux) offers better pitch detection using fast fourier transforms rather than autocorrelation, but that is not that big a deal.
Ultimately what would be nice would be a pre processor that sampled the original song vocals, and then stripped them for playback. The original vocals would then be used for the compmparison. Blah blah blah

But right now I was just thinking of file compatibility as a nice feature, mostly for the home use market.

[dave]

there is a port for OSX

where?? all i could find was an .exe

Since what is at issue initially is just file compatibility

actually what is at issue is a huge code base, written using it's own framework, for it's specific purpose, and in it's own style, and somehow grafting parts of that onto an existing separate code base, which uses a completely different frame work, and written for an entirely different purpose and in it's own style. it's a very large task even for a team of programmers and i'm just one guy. file compatibility is probably the easiest and smallest part to tackle.

a pre processor that sampled the original song vocals, and then stripped them for playback

very very smart people have been working on that problem for many many years and still have not come up with a satisfactory solution. it's an extremely difficult problem. i rather doubt that i could come up with something better. if there exists an open source library to do that then maybe. but there is hope:
http://dvice.com/archives/2008/04/miraculous_dire.php

file compatibility as a nice feature

file compatibility means i can open and play the song. it has nothing to do with analyzing the singer's vocal performance and comparing to a reference.

[DeusExMachina]

That is actually all I meant by file compatibility, the ability to play the files in the player. As I said, actually making the game work is a serperate matter. Many people have ultrastar collections in the home market, and might want to play them in your player.

That said, here is a link to the OSX version:
http://sourceforge.net/projects/ultrastar-mac/

[dave]

well i tried to run the OS X version. when i go to "play" a song, nothing happens, so i assume you have to "create" a song first. so i go to create one and the application quits.

oh well. do they make new releases frequently? this seems a rather glaring bug so i'd expect it fixed pretty quickly.

or, do you have some pre-made songs you could send me?

-dave

[DeusExMachina]

I have a bunch of Ultrastar stuff. I can e-mail it, or I can give you a torrent link I just tracked down, if you are so inclined, although the torrent'll get you a few hundred songs.

I really don't know too much about the compatibility of the OSX version, since I don't have an intel machine, but they work fin on other platforms, AFAIK.

[dave]

please put it here:

http://www.kjams.com/wiki/FTP

un: kjams
pw: kjams

[DeusExMachina]

Done.
That's a neat little trick, having a link to a server alias.

[dave]

well, the source code is written in Pascal, not C++, and the native language of the author was like czech, not english. so it's rather indecipherable. not to mention there is no documentation for the file format.

if you can scrounge up some documentation for the file format, i'll add it.

[DeusExMachina]

So I dissected the .txt file, and did some research. Here is what I was able to make out:
Header

The Header at the beginning of a TXT file MUST contain the following fields:

#TITLE:
#ARTIST:
#MP3:
#BPM:
#GAP:

Here is the header from the example file that comes with Ultrastar:

#TITLE:Superstar (demo)
#ARTIST:Jamelia
#MP3:Jamelia - Superstar (Demo).mp3
#BPM:110
#GAP:50

Additionally, it MAY contain the following fields:

#RELATIVE:yes
#VIDEO:
#VIDEOGAP:
#RESOLUTION:
#START:

The example file looks like this:

#TITLE:Superstar (demo)
#ARTIST:Jamelia
#MP3:Jamelia - Superstar (Demo).mp3
#BPM:110
#GAP:50
#RELATIVE:yes
#VIDEO:Jamelia - Superstar (Demo).mpg
#VIDEOGAP:10
#RESOLUTION:80
Meaning of the fields

#TITLE:
The song title. It will be shown in the song list in the Ultrastar interface.

#ARTIST:
The interpreter of the song.

#MP3:
The name of the mp3 file that is used by the player.

#BPM:
Speed of the song in beats per minute. It usually means the number of quarter notes or stressed beats per minute.

#GAP:
Gap between the start of the music and the start of the text display in milliseconds.

You can calculate when the text actually begins with the following formula:
start time = first time stamp / BPM / 4 * 60 seconds + GAP.

If the time stamps are relative, the formula simply is:
start time = GAP

#RELATIVE:
This shows whether the time stamps are set back to 0 after each line or not. "Yes" means they ARE set back to 0. If this line is missing in the Header, the time stamps are automatically interpreted to be absolute. Relative time stamps considerably simplify the editing of a TXT file because when you want to add a break you only need to change the time stamps of the current line (when you have absolute time stamps you have to change all time stamps in the file.) If you want to change already existing time stamps from absolute to relative, you have to change ALL time stamps which is a lot of work. But maybe there will be a tool for doing this in the near future, who knows...

#VIDEO:
Name of the video file. Only MPEG-1 files are compatible at the moment. Only the video itself is displayed, you cannot hear the sound.

#VIDEOGAP:
The same as GAP for the mp3 file. But this time for the video of course.

#RESOLUTION:
I do not know about this one yet

#START:
Offset from the beginning of the file to start playback.
With this feature you can skip a long intro.

Information on Sound and Text

Below the Header you will find the real information on sound and text. Depending on the time stamps (whether they are absolute or relative,) it looks a little different (see #RELATIVE):

Absolute Time Stamps (#RELATIVE:no):
My example is the TXT file from the UltraStar package.
: 129 2 15 Peop
: 131 1 15 le
: 132 3 15 al
: 135 1 15 ways
: 137 1 13 talk
: 139 1 15 a
: 141 2 10 bout
- 152
: 161 2 15 All
: 163 1 15 the
: 164 2 15 things
: 167 1 15 they're
The first value means the time stamp which indicates when the tone begins which is to be sung. The second value indicates the length of this tone in beats. If the following tone does not begin right after the previous one, you have breaks. In our example: there is a break lasting for 1 beat after "always", "talk", "a" and "things". The third value indicates the pitch of the tone (0 = C1.) After that you see the syllable which belongs to the tone. UltraStar determines if the following syllable still belongs to one word by blanks: if there is NO blank after the syllable, the next syllable still belongs to the word; if there is a blank after the syllable, the next syllable will be classified as the beginning of a new word.

The line
- 152
indicates that everything following it shall be shown in a new line on the screen. It can also look like this:
- 152 153
Here, the first value indicates the time when the previous line shall disappear from the screen and the second value indicates when the following line shall appear.

Now the Same Example with Relative Time Stamps (#RELATIVE:yes and #GAP:17595):
: 0 2 15 Peop
: 2 1 15 le
: 3 3 15 al
: 6 1 15 ways
: 8 1 13 talk
: 10 1 15 a
: 12 2 10 bout
- 23
: 9 2 15 All
: 11 1 15 the
: 12 2 15 things
: 15 1 15 they're
As you see, only the time stamps change.
To indicate the end of a TXT file, you need a line which only contains an "E".

So all your program would need to do that it does not do already is parse the text file for timing cues and lyric information. The rest, including audio file playback, video display, and text overlay, are already being done by kJ.
If you were being adventurous, for C++ source code, try getting the code for Ultrastar NG. It is only for Linux, but a lot of the algorithms are probably usable with only minimal change.
http://sourceforge.net/projects/ultrastar-ng/
This would only matter if you were to implement the entire feature set, not what I ws originally asking about (though it would be cool!)
BTW, as I mentioned earlier, USNG uses fft instead of ac, so it is far more accurate at note comparison. The program uses the fft or ac algorithm to sample the note in the audio file, and compare it to the length and duration of the note coming in from the mic.

The ulitmate would be if this could be modified such that it accepted two sound files, one with the lyrics, which would not be output, and a lyricless "karaoke file" that would be played. A lot of the basic funtionality is already running in kJ. Would make for a very interesting "competition feature" which is far outside the realm of this current application, but interesting nonetheless.

[dave]

wow!



<eating my words>

okay then.

here's the deal:
I currently don't have a method to display "timed text" (eg: LRC files, TunePrompter files, and now these lovely UltraStar files) However, I've got this thing called kJams Producer which can convert said files into a CD+G file (so you can burn it to disc). That's a separate program, soon to be a plugin you can purchase separately. I've already written the KTP->CDG module, and it will be trival to add LRC->CDG and now UltraStar->CDG, so that's most likely the thing that will happen first. Eventually i'll offer a built-in method to *display* "timed text" files (no conversion necessary, no separate program necessary either), but for now this is what i got. Interested?

[DeusExMachina]

Certainly!

That said, just as a side note, now that I think about it, I realize that an audio file with vocals is not necessary. I always assumed that it was comparing the notes sung (sampled) with the notes in the audio file (sampled.) Going over the file format, I realize now that it is just comparing the sung note (sampled) to the note contained in the text file. Much simpler to execute.
Hmmm…

[DeusExMachina]

So I wonder if it would be possible to write a separate application that sampled the audio stream and used the .txt file from the ultrastar file to allow kJ (or anything else) to add competition functionality without having to use kJ itself. Just run the audio in through the mac, sample the stream, run it through the fft algorithm from UltrastarNG, and add an interface to show live note accuracy bars and points scored as a windoid floating over the video window with the lyrics.
Hmm....

[dave]

you can do anything you want.

that's just not the direction that kJams is going, however.