The XNA Offline and Online Highscores Component Version 2 (distributed leaderboards, sort-of)

jwatte's picture

XNA Game Studio makes it possible to write games for the Xbox without being a developer with good publisher contacts and lots of money to pay for marketing and Xbox development kits. This is great!

However, because the XNA Indie Games system is not fully controlled by Microsoft, certain features of Xbox Live! are not available, because they would be too easily abused. These features include online Leaderboards, and unlockable Achievements.

The XNA community has developed alternatives to those functions. Many XNA games contain "Awardments" that can be unlocked, and many more XNA games use the XNA Network Highscores component to implement distributed, peer-to-peer highscore sharing. The name for this is generally "Online Highscores" rather than "Leaderboards," because the latter name is reserved for use by Microsoft-certified titles that use the real Xbox Live! functionality.

This article introduces version 2 of the XNA Online highscores component, which is free for you to use in your own game under the terms of the MIT license.

I wrote version 1 of the Highscores component mostly as a proof of concept back when the Xbox Live Indie Games marketplace was an unknown new entity. It turned out to be quite useful, and has been an important part under the hood of a lot of successful (and some not-so-successful) Xbox Live Indie games.

The general idea is simple: Host a "networked game" session. Let other gamers connect to this session. When they do, send them a selection of your highscores, and in return receive a selection of their highscores. Store the union of highscores on the local hard disk, to display as "online highscores." All of this can be done in the background without the local player having to do anything to set it up. The game will first look for highscore sharing sessions to join, and if there are none, start hosting one on its own. This means that there will always be at least one session hosted as long as anyone is playing the game at all (and has Xbox Live! Gold service), thus leading to the best possible chance of finding someone to share highscores with. Of course, the longer the game remains running on the network, the greater the chance that someone else will be playing the game at the same time, and highscores will be excanged!

Time has come to take something pretty good, and make it better!

Improvements

Over the last two years, a few feature requests have been brought up, that would make the Highscores component more useful to a wider audience of developers. These can be broken into three main categories:

  • Local-only Highscores
  • More Flexible Highscores
  • Improved Structure

Some users really didn't care much about networked highscores (they only work if you have an Xbox Live! Gold subscription, anyway), but still wanted a coherent way of dealing with saving and loading highscores. These users would try to separate out the disk management code from the network code of Highscores version 1, and would often find it hard to make it work right. In version 2, I separated the two concerns of storing highscores on disk (or flash cartridge), and sharing highscores between different Xboxes. You can choose to instantiate only the storage component, or to also add the network component.

A gamertag, a score, and a date is not enough for some kinds of games. Different games may want to record data such as "number of monsters killed" or "highest level achieved" or similar. Other games may also implement different game modes, such as "Deathmatch" versus "Co-op." To support these use cases, I added an additional value to the highscores data type (for "highest level" or similar), and a string (for "game type".) The sample application shows how to sort scores by game type, as well as how to display all the data that can be in the highscores record. It bears saying that you don't have to use these new features; if all you need is a highscore value tied to a gamertag, that usage method is still very well supported.

I toyed with the idea of letting an arbitrarily defined highscores structure be exchanged, but there are several problems with such an implementation, especially if you want to provide some level of useful baseline functionality out-of-the-box. It doesn't help that System.Reflection generates heaps of boxed data values, putting high pressure on the Xbox garbage collector, should you try to deal with arbitrary data types in that way. Instead, I hope these extra fields will solve all of the real use cases I've heard about.

The business of finding and talking to random other game consoles on the network is not entirely straightforward, so the code that dealt with highscores exchange in Highscores version 1 was somewhat obtuse. Improvement could be made. In version 2, I have attempted to structure the code using somewhat better named classes and methods, and using a common structure for similar tasks within each component. Hopefully, this leads to code that is easier to understand and modify by the community at large. To help with this, I have also added documentation not only for the high level usage of the classes, but also explaining a bit about what goes on under the hood. Crack open the code and take a look!

Sample Game

The sample game, called simply "TesterGame," is less of a game this time around. You can press A or B to generate high scores as if they came from a local player or a remote networked player. Those scores go into a highscore table, that you can view, and sort by game type (there are four sample game types). Also, those scores can be shared across the network with other instances of the TesterGame.

To run the sample game, select the four TTF files (the font Nobile by Vernon Adams, released under an open font license). Right-click and select "install" to install the font. Then open Visual Studio, and open the Highscores2.sln solution file (the current version is compatible with Visual Studio 2008 and Visual C# Express 2008, using XNA framework version 3.1). If you open Visual Studio before you install the fonts, the font compiler will not find them and will give an error.

Select "Build" to build the solution, and "Run" to run the tester game. It should run fine on both Windows and Xbox, assuming you can run XNA games from the IDE in general. Note that it will take a few seconds to connect to Live! when you start it on Windows; this is induced by the GamerServicesComponent, and while highly annoying, not something I can do anything about.

Luckily, because TesterGame doesn't really have any game screens, it fits into a single source file, making it easy to see how to integrate the Highscores components in your own game. First, add the Highscores2 project to your own game solution (it's OK to copy the entire folder into your own directory, and add the project from there). Then, here is a quick description of how the tester game is hooked up to the components:

    /* Step 1: Attach the storage to the game, and sign up for events for 
     * highscores saved/loaded. Passing true to Attach means to let the component
     * deal with the storage device itself (the sample game passes false for 
     * testing purposes)
     */
    Highscores2.Storage st = Highscores2.Storage.Attach(this, true);
    
    /* You really only need to sign up to LoadComplete and HighscoresChanged, 
     * although NewHighscoreFound may be convenient for showing some splashy 
     * overlay on the game screen, and the rest help you manage storage in more 
     * detail.
     */
    st.LoadComplete += new EventHandler(st_LoadComplete);
    st.HighscoresChanged += this.HighscoresChanged; 
    st.NewHighscoreFound += new Highscores2.HighscoreEventHandler(
        st_NewHighscoreFound);
    st.SaveComplete += new EventHandler(st_SaveComplete); 
    st.UserRefusedStorage += new EventHandler(st_UserRefusedStorage);

    /* If you don't sign up for StorageDeviceNeeded, the component can do 
     * default management of storage devices using the standard Xna storage 
     * device selection dialog when you call ReselectStorage(). If you want
     * to do your own management, use SetDevice() instead.
     */
    if (!st.StartLoading()) {
      st.ReselectStorage();
    }
  

This code snippet runs inside the constructor of the game, and configures the local highscore storage. It does not turn on automatic saving (so that you can test doing it manually from the controller), but signs up for each of the events that the component can generate. Those events will generate text strings into the log on screen.

If a GamerServicesComponent hasn't yet been created in the application and added to the component collection, the Storage component will do so, because that component is needed to show the storage device selector, which in turn is needed to figure out where to store highscores data. If you don't want the component to show that dialog by itself, there is a way to do it all yourself, and just tell the component about it -- see the code for more details.

    /* Step 2: Attach the network component to the game. This must be done 
     * after the storage component has been attached.
     */
    Highscores2.Network nw = Highscores2.Network.Attach(this);

    /* ShareSomeHighscores() is the easy way to make networked highscore 
     * sharing work. There are more advanced versions too, especally useful 
     * if you also want to support network play in your game. See the file in 
     * question for those.
     */
    nw.ShareSomeHighscores();

    /* You don't really need to sign up for any of these events if you don't
     * want to.
     */
    nw.SessionDisconnected += new EventHandler(nw_SessionDisconnected);
    nw.SessionEstablished += new Highscores2.SessionEstablishedHandler(
        nw_SessionEstablished);
    nw.TryingConnection += new EventHandler(nw_TryingConnection);
    nw.GamerJoined += new Highscores2.GamerEventHandler(nw_GamerJoined);
    nw.GamerLeft += new Highscores2.GamerEventHandler(nw_GamerLeft);
  

This code also runs in the constructor, and sets up the networked part of highscores. You don't need to add this part if all you want is to save and load highscores to the local disk. Again, this code signs up for event notifications of the different kinds of events that may be raised by the network component, and those will be printed to the log on the screen.

Inside Game.Update(), the game will call base.Update(). Because the components are real GameComponent instances, they will be automatically updated at that point. During update, the components will raise the appropriate events based on what has happend since last time Update() was called.

    Highscores2.Highscore hs = new Highscores2.Highscore(
      DateTime.Now, gamertag, gametypes[gametype],
      score, level, "note");
    Highscores2.Storage.Instance.AddHighscore(hs, false);
  

This code first creates a Highscore instance, recording information about the player's score. This includes score, level (or other arbitrary integer value), game type, gamertag, time/date, and a "note" that follows the score around but doesn't actually mean anything to the underlying score system.

Then it adds the score to the Highscores system, and in this case, tells the system that the score did not come from the network. The reason for this is that there are two sets of highscores; local scores, and network scores. The reason for this is that the local player wants to know his own scores, even if they happen to all be lower than the scores received from the network. Because not all scores ever received can be stored (or the file would be very big), a certain number of the best network highscores are stored, and a certain number of the local players' best highscores are stored.

    /* This game has four game types, 0 through 3. 
     * This array has one entry for each type to display the name of 
     * each type.
     */
    static string[] gametypes = new string[] {
        "Deathmatch", "Politics", "Group Hugs", "S & M" };
    /* Find the scores I want to display, based on filtering.
     */
    Highscores2.Highscore[] scores = Highscores2.Storage.Instance.QueryHighscores(
      null, gametypeFilter == -1 ? null : gametypes[gametypeFilter], null, null, 20);
  

This code asks for the 20 best highscores of a particular game kind. The actual values you can choose to filter on are:

  • gamertag - the gamertag that the score is for ("see my scores only")
  • gametype - the gametype recorded for the score
  • statistic - the "level achieved" or "monsters killed" value. Only values equal to or greater than the filter will be displayed.
  • score - the actual score. Only values equal to or greater than the filter will be displayed.
  • nMax - how many score entries to return (the top N)

When a value is null in the query parameters, it will match any highscore entry for that kind.

If you pass all null, then all the highscores (up to the top N) will be returned. There is also a feature where you can pass in a filter function that can look at a highscores record, and return true if it should be returned to the caller, for advanced users.

Update 2010-05-31

I found that calling DateTime.Now actually generates garbage by boxing an int32(!) I updated the timing calls in the library to not use this function during idle/hosting state, thus removing the only known source of garbage generation. The components should now not generate garbage, unless you do something that inherently generates garbage like pruning old highscores (see KeepSomeHighscores). The new version of the zip file (size 190.23 kB) fixes this.

AttachmentSize
Highscores2-Release.zip190.23 KB

Comments

Hi jwatte, Thank you for this

Hi jwatte,

Thank you for this library. We were testing it and we keep getting score duplicates. They seem to happen after data is exchanged over the network. Do you know what might be causing this?

WP7 game?

Will this code work for adding a leaderboard for a windows phone 7 game?

also I am not sure what I am

also I am not sure what I am doing wrong but old highscores are not getting loaded when I restart a game. That is quering for highscores (with all props set to null) returns an empty array. When I add the highscore in-game I can see it in the table, but if I restart it the table is empty..

Hi, thank you for your work.

Hi, thank you for your work. Is it possible to distinguish local highscores from network when retrieving highscores for display? For example my designer wants to see two hs columns "Local" and "Network".
Thanks

Maybe this will help more?!?

I made a couple of changes in my version to hopefully reduce clashing high scores between games.. I haven't tested this with XBOX though.

  1. The Assembly Name of the Game is used as the save directory.
  2. The Assembly Name of the Game is also used as the file name. i.e: [GameClassName].Highscores

I thought about using the project GUID, but could not find a way to query it from code. At least with using both, game class and assembly name, there are two points of failure instead of one. Good? Bad?

Result: SavedGames\\[AssemblyName]\\AllPlayers\\[GameClassName].Highscores

internal Context(StorageDevice dev, string name)
{
	Trace.WriteLine("Context: creating for: " + name);
	this.dev = dev;
	this.name = GameInstance.GetType().Name + "." + name;
}

StorageContainer GetContainer()
{
	if (container == null)
	{
		System.Reflection.AssemblyName an = System.Reflection.Assembly.GetEntryAssembly().GetName();
		container = dev.OpenContainer(an.Name);
		//container = dev.OpenContainer(GameInstance.GetType().Name);
	}
	return container;
}

On this line

On this line "AvailableNetworkSessionCollection availableSessions = NetworkSession.EndFind(ar);" in OnSessionsFound() in Network.cs, I get this error on the PC:

"A signed in gamer profile is required to perform this operation. A Live profile may also be required. There are no profiles currently signed in, or the profile is not signed in to Live."

I get this when I try and integrate into my project and or when running the test project.

Any suggestions on what to check so it doesn't do this? I assume others don't see it. Maybe it has to do with the fact that I logon to a Windows Live account on my PC.

entries limit?

Hi,
I get a stack overflow with 2000 entries, there's a limit you know?

jwatte's picture

I don't know of a limit, but

I don't know of a limit, but it might be possible.
Do you have a stack trace for the overflow?

found the cause

The cause is the HashSet class, having a recursive method that overflows if there are too much entry to update (around 2000 on the Xbox, much more on th Windows PC)

I rewrote that class using another HashSet Class (less optimized but do not suffer of stack overflows)

Thanks a lot for the awesome work!

Detecting Gamer Services

I would make one suggestion around your detection of gamer services in the component. Checking for GamerServicesComponent to exist isn't the best way to look for the functionality because games can call the GamerServicesDispatcher.Initialize() method directly to initialize the gamer services functionality. Thus you may actually add a component to a game and cause an exception by initializing gamer services a second time. Just one way to make this a bit more robust, though I would personally have the component simply make the dependency check and throw an exception for the game developer to handle gamer services, but that's just me. :)

jwatte's picture

I've got to stop somewhere. I

I've got to stop somewhere. I need gamer services to be pumped. The easiest way to do that is to create the component, and let the application pump it in Update(). The current component catches the 99% use case -- and if you're a proud part of the 1%, the source is available. I made it so just for you :-)

integration

so how do i integrate it into my project?
thanks

jwatte's picture

You reference the highscore

You reference the highscore component project in your own solution, and call it in the way that's outlined in the code snippets in this article.
You can also look through the sample game for specifics on how it was integrated there.

Firstly, thanks for creating

Firstly, thanks for creating such a great component.

I've modified the QueryHighscores method to return only the highest score per unique gamertag, which works great, but I have a question relating to the network component.
It looks as if it's sending all scores accross the net whether they happen to be local or not. Is this intended? Unless I'm missing something obvious, it looks as if there's no way of distinguishing whether a score was added locally allowing filtering of which ones are sent to other players.

jwatte's picture

Yes, it intentionally sends

Yes, it intentionally sends all the scores. The reason for this is that it's supposed to build up global highscore information, and when you don't have a central server, this is best done using a "gossip" protocol, which this component implements. If you only sent local scores, then whomever you sent your data to would have to be online at the same time as everyone else to get data from everyone. If you get data from everyone, then whomever you send data to only needs to be online at the same time as you. The difference is dramatic!

thanks a lot!

I'm using it on my latest XNA game THE NO BUTTON GAME, simply great. Plannig to extend it with an "achievements" module. I will share this extension as soon as it is ready.

Alfio (Running Pixel)

Thanks alot for this! Just

Thanks alot for this! Just what I was looking for my next game! Our designer keeps whining about how important online leaderboards are. Looks like there is a solution now :D

Not sure if I'm using it

Not sure if I'm using it correctly, but I noticed it on the old version also. Highscores seem to only be loaded from the network when it is first attached to the game. I tested it with a friend and scores are shared just fine but only when you first start the game. Maybe it's by design, or maybe I'm not updating it correctly, I was just wondering if this is how it's suppose to work?
Also, after the sessionCloseTime (when it disposes the session) does it disable you from sharing scores on the network or will it continue to send your scores if another gamer is found? It seemed to turn into local-only mode and would never reconnect to my friend even when he restarted to create a new session.
Since it seemed to only share the scores on first connect, I had to restart the game to update the scores list, and I'm not sure if it's by design or something I'm doing. Thanks

jwatte's picture

Highscores do load on a

Highscores do load on a schedule. However, the schedule is once every 5-10 minutes, so as not to overwhelm the network. The more players there are, the more possible matches there will be, and the more highscores will be shared.

Hi, I have same problem. We

Hi, I have same problem. We tested with my friend they seem to work very randomly. I waited for 20+ minutes for scores to update and they dont. However when I restart the game *sometimes* they will update. Sometimes my friend's will update when he restarts, but mine wont no mater how long I wait or restart... Its quite confusing. Is there a command to manually update the highscores? It seem that ShareSomeHighscores should be it, however that just throws an exception "Can't ShareSomeHighscores() when already connected.". Sorry I am abit confused about this function. What is its purpose?

It seems that after I manually cleared via ClearLoaded() func the highscores on my XBOX they stopped sharing completely. How does the ClearLoaded work? Does it delete only local highscores or does it do something else like turn off sharing? Should scores get reloaded from network after some time if cleared and if another session exists?

Also what is the use of fromNet in AddHighscore(hs, fromNet) function? I dont completely understand it. It seems that no matter what I set the highscores are stored and shared the same?

And final question... how can I distinguish between local highscores and network highscores when I display them? QueryHighscores() does not have an option for local or network?

Sorry for dumb questions, your code seems quite complex for me to understand! I will try to go through it in more detail but it will take me alot of time! If you can answer some questions above I will be very grateful!

And thanks for sharing this awesome library!

Good stuff!

I have been looking this over and testing it. I really like it and plan to use it in my upcoming XBLIG! Thanks for the hard work!

Highscores2.sln solution file missing?

I'm not seeing the Highscores2.sln solution file mentioned in the article in the download. I may have missed it. Looking forward to trying this version out!

-Tim

jwatte's picture

Thanks, I added that file

Thanks, I added that file now.

Another point: When you open the .sln, the Highscores2 project will be the start-up project, and the solution will be set to Mixed Platforms. This is because solution options don't follow the .sln file. You need to change the Target Platform to x86 (or Xbox 360) in the Configuration Manager, and right-click the TesterGame project and set as start-up project.