Home › Networking

Marshaling

Stuffing bits into other bits.
jwatte's picture

A simple approach to native network marshalling

January 18, 2010 - 22:30 — jwatte

I used to do serialization using all kinds of fancy templates and macros. You can create pretty elegant systems that way. However, at some point, simplicity should win out. Here's a system that might work just fine for you:

A simple packet class, which really is all you need:

class packet {
public:
  packet() : pos_(0) {}
  void append(void const *data, size_t size) {

jwatte's picture

How to structure a reusable game networking library

September 3, 2009 - 19:21 — jwatte

I recently have answered several questions about how to structure a networking library such that it can be easy to use for users of the library and/or when expanding the game you're writing. Here are some thoughts on that. (Code examples in C++)

Networking generally ends up needing to do three things:

1) Mirror state updates from one object to another.
2) Request remote services ("RPC").

jwatte's picture

marshal.cpp

July 22, 2009 - 19:42 — jwatte 

#include <assert.h>
#include <string>
#include <math.h>
#include <map>
 
#include "etwork/marshal.h"
 
namespace marshaller {
 
inline std::string operator+( std::string const & left, int right ) {
  char buf[24];
  sprintf( buf, "%d", right );
  return left + std::string( buf );
}
 
inline std::string operator+( std::string const & left, size_t right ) {
  char buf[24];

jwatte's picture

buffer.cpp

July 22, 2009 - 19:40 — jwatte 

#include "etwork/buffer.h"
 
#include <string.h>
#include <deque>
#include <assert.h>
 
using namespace etwork;
 
//! The framing protocol for etwork::Buffer is simple: each packet is 
//! preceded by a two-byte length, in network-endian order.
//! If random data is received, this may result in arbitrary packet

jwatte's picture

block.cpp

July 22, 2009 - 19:39 — jwatte 

#include "etwork/marshal.h"
 
 
Block::Block( void * base, size_t size )
{
  buf_ = (unsigned char *)base;
  size_ = size;
  pos_ = 0;
  deleteIt_ = false;
  atEof_ = false;
}
 
Block::Block( size_t size )
{
  buf_ = (unsigned char *)::operator new( size );
  size_ = size;
  pos_ = 0;
  deleteIt_ = true;
  atEof_ = false;
}
 
Block::~Block()
{
  if( deleteIt_ ) {

jwatte's picture

How to re-construct messages on the receiving side

November 15, 2008 - 13:30 — jwatte

Once you've gotten your message buffer sent to a receiving end (through a file or network interface), you can inflate and re-construct the same data structure using the MsgIn class. Note that the current marshaling code does not compensate for varying byte order across networks, but because all computers you'll be running this code on are little-endian, that probably doesn't matter to you.

jwatte's picture

How to use the generated code: MsgBase.h

November 15, 2008 - 11:23 — jwatte

The generated code makes use of the pre-defined visitors in MsgBase.h and MsgBase.cpp. It also needs you to generate a source file for your packets file, using perl -MSource Packets.pl. The generated source code is simple:

/*
 * This file was auto-generated on 2008-11-15 11:16:13
 */
 
#include "Packets.h"

jwatte's picture

The output: templates that describe your packets

November 15, 2008 - 11:12 — jwatte

If you run the perl script, passing in the "Headers" module, it will print the declaration of your data structures to the console output. The included rules file lets you do this automatically as part of a Microsoft Visual Studio project, but for now, try doing it on the command line. If you don't have perl installed, get it either as part of Cygwin, or from ActiveState.

jwatte's picture

The approach: writing a high-level description of each data unit

November 15, 2008 - 11:05 — jwatte

The idea is to describe your various data structures using a basic high-level language, and then generate the necessary code machinations to go between that description and byte streams automatically. This example uses a custom Perl script, although there are various standardized tools that can do the same thing (such as the ASN.1 standards suite).

jwatte's picture

Marshaling (serializing data) in C++ using auto-generated headers

November 15, 2008 - 10:57 — jwatte

This package provides simple generation of headers and source for marshaling binary data between C++ structs and byte streams. The nice thing about it is that you can easily extend the system to support generating property sheets, or going to/from XML, or one of a number of other things you want to do to data structures. You do so by just adding new visitor classes, without having to change the data structure code.

Syndicate content