Network bandwidth mathematics; peer-to-peer versus client/server

jwatte's picture

The XNA Forums are now talking about the Xbox Live! networking support added to XNA 3.0. There's some discussion about how large games can be supported on top of the recommended maximum upstream bandwidth consumption of 8 KB/s. (This is a recommendation Microsoft makes for Xbox Live!, based on learnings from deployed broadband connectsion)

Note that, in more than two-player games, you need to send the state of each player, to each player, so bandwidth consumption grows as n-squared. If you trust the clients (you can, on Xbox Live, but not on PC) then you can use peer-to-peer, which distributes that load on the sending part (compared to a server), but uses more bandwidth on the receiving end (because of multiple packet headers).

On one particular case, a developer was wondering about the impact of a two-player game at 60 updates a second with 100 bytes per player, versus an 8-player game with 11 updates per second. Here's how to run the analysis on such a scenario.

In general, in a networked game, you can have the following parameters:

Number of players: P
Bytes per player per update: B
Updates per player per second: U
Overhead per packet: O (28 bytes for UDP; 40 bytes for TCP; somewhat higher for Xbox Live! which does IP-over-UDP...)




Peer-to-peer:
P2P Upstream: (P-1) * U * (B + O)
P2P Downstream: (P-1) * U * (B + O)



Client/server (assuming one gamer hosts):
Client Upstream: U *(B + O)
Client Downstream: U * ((P-1) * B + O)
Server Upstream: (P-1) * U * ((P-1) * B + O) (server send)
Server Downstream: (P-1) * U * (B + O) (server receive)





Plugging in the numbers:

P = 2, B = 100, U = 60, O = 28 -- two-player game, 60 updates per second, 100 bytes per player per update

P2P Upstream: 1 * 60 * (100 + 28) == 7680
P2P Downstream: 1 * 60 * (100 + 28) == 7680


Client Upstream: 60 * (100 + 28) == 7680
Client Downstream: 60 * (1 * 100 + 28) == 7680
Server Upstream: 1 * 60 * (1 * 100 + 28) == 7680
Server Downstream: 1 * 60 * (100 + 28) == 7680




Numbers are in bytes per second, so this would fit on 8 KB/s, assuming Xbox overhead isn't too much on top of the 28 bytes of UDP headers.

Now plug in an 8-player game at 11 updates per second:

P = 8, B = 100, U = 11, O = 28

P2P Upstream: 7 * 11 * (100+28) == 9856
P2P Downstream: 7 * 11 * (100+28) == 9856


Client Upstream: 11 * (100 + 28) == 1408
Client Downstream: 11 * (7 * 100 + 28) == 8008
Server Upstream: 7 * 11 * (1 * 100 + 28) == 9856
Server Downstream: 7 * 11 * (7 * 100 + 28) == 56056




It seems to me that if you're using P2P, and sending 100 bytes per player per update, and want to fit in 8 KB/s, you need to send at about 8 updates per second for an 8-player game. For client/server, if you can find one client that is well connected (56 KB/s) then all the other players can already fit on 8 KB/s with an update rate of 11/s.

My recommendation is to try to reduce the size of B -- 100 bytes is a ton of information about a player, to be changing every second, much less 60 times a second! You can often quantize position into 6 or 9 bytes, and velocity into 6 bytes, and look (heading/pitch) into 4 bytes. Add 20 bytes for average management of shooting and jumping and whatnot, and you only have about 40 bytes per player per update. Also, most modern first-person shooters are very playable at 20 updates per second; 60 seems like something you can cut down if you need to save some bandwidth. The difference between an update every 17 milliseconds and an update every 50 milliseconds is hard to notice for most players.