Marshalling

warning: Creating default object from empty value in /home/hplus/enchantedage.com/modules/taxonomy/taxonomy.pages.inc on line 33.
Stuffing bits into other bits.
jwatte's picture

A simple approach to native network marshalling

I used to do serialization using all kinds of fancy templates and macros. You can create pretty elegant systems that way. However, at some point, simplicity should win out. Here's a system that might work just fine for you:

A simple packet class, which really is all you need:

class packet {
public:
  packet() : pos_(0) {}
  void append(void const *data, size_t size) {

jwatte's picture

How to structure a reusable game networking library

I recently have answered several questions about how to structure a networking library such that it can be easy to use for users of the library and/or when expanding the game you're writing. Here are some thoughts on that. (Code examples in C++)

Networking generally ends up needing to do three things:

1) Mirror state updates from one object to another.
2) Request remote services ("RPC").

jwatte's picture

marshal.cpp

#include <assert.h>
#include <string>
#include <math.h>
#include <map>
 
#include "etwork/marshal.h"
 
namespace marshaller {
 
inline std::string operator+( std::string const & left, int right ) {
  char buf[24];
  sprintf( buf, "%d", right );
  return left + std::string( buf );
}
 
inline std::string operator+( std::string const & left, size_t right ) {
  char buf[24];

jwatte's picture

buffer.cpp

#include "etwork/buffer.h"
 
#include <string.h>
#include <deque>
#include <assert.h>
 
using namespace etwork;
 
//! The framing protocol for etwork::Buffer is simple: each packet is 
//! preceded by a two-byte length, in network-endian order.
//! If random data is received, this may result in arbitrary packet 

jwatte's picture

block.cpp

#include "etwork/marshal.h"
 
 
Block::Block( void * base, size_t size )
{
  buf_ = (unsigned char *)base;
  size_ = size;
  pos_ = 0;
  deleteIt_ = false;
  atEof_ = false;
}
 
Block::Block( size_t size )
{
  buf_ = (unsigned char *)::operator new( size );
  size_ = size;
  pos_ = 0;
  deleteIt_ = true;
  atEof_ = false;
}
 
Block::~Block()
{
  if( deleteIt_ ) {

jwatte's picture

How to re-construct messages on the receiving side

Once you've gotten your message buffer sent to a receiving end (through a file or network interface), you can inflate and re-construct the same data structure using the MsgIn class. Note that the current marshalling code does not compensate for varying byte order across networks, but because all computers you'll be running this code on are little-endian, that probably doesn't matter to you.

jwatte's picture

How to use the generated code: MsgBase.h

The generated code makes use of the pre-defined visitors in MsgBase.h and MsgBase.cpp. It also needs you to generate a source file for your packets file, using perl -MSource Packets.pl. The generated source code is simple:

/*
 * This file was auto-generated on 2008-11-15 11:16:13
 */
 
#include "Packets.h"

jwatte's picture

The output: templates that describe your packets

If you run the perl script, passing in the "Headers" module, it will print the declaration of your data structures to the console output. The included rules file lets you do this automatically as part of a Microsoft Visual Studio project, but for now, try doing it on the command line. If you don't have perl installed, get it either as part of Cygwin, or from ActiveState.

jwatte's picture

The approach: writing a high-level description of each data unit

The idea is to describe your various data structures using a basic high-level language, and then generate the necessary code machinations to go between that description and byte streams automatically. This example uses a custom Perl script, although there are various standardized tools that can do the same thing (such as the ASN.1 standards suite).

jwatte's picture

Marshalling (serializing data) in C++ using auto-generated headers

This package provides simple generation of headers and source for marshalling binary data between C++ structs and byte streams. The nice thing about it is that you can easily extend the system to support generating property sheets, or going to/from XML, or one of a number of other things you want to do to data structures. You do so by just adding new visitor classes, without having to change the data structure code.

Syndicate content