XBS: A Streaming Binary Serializer for High Performance Computing

Tharaka Devadithya (Indiana University), Kenneth Chiu (SUNY, Binghamton University)

Overview

XBS is a binary serialization library. It differs from other binary serializers in that it is a streaming serializer (as opposed to message-oriented), and that it is object-oriented. XBS is written in C++, and its object-orientation facilitates its use in modern high performance applications.

Since XBS is a streaming serializer, it is suitable for situations where the data is composed of large, complex, and dynamic data structures. It does not impose any highlevel structure on the data as does XML. Any such structures would be implemented in a layer operating on top of XBS. For example, a Binary XML format [1] can be built on top of it. Also, it would be possible to build a SOAP engine that supports binary XML, in addition to the regular textual XML [2].

Basic Usage

The XBS library provides separate interfaces for serializing and deserializing messages. Since the two interfaces are almost analogous, only the serializer interface is illustrated here. Please refer the sample code provided for an illustration of the deserilaizer interface.

Creating an XBS serializer requires creating a buffer object.

StreamSerializationBuf streamBuf(fd, BUFFER_SIZE);
Serializer<> ser(&streamBuf);

The StreamSerializationBuf class derives from the SerializationBuffer class, and is responsible for the actual implementation of the fill and flush routines. The constructor argument, fd is a file descriptor, which could be a handle to a le, socket connection, etc.

The actual serialization of each data element is performed by template function pack() or one of its variations, as shown below.

// Primitives
ser.pack(1234);
ser.pack(1.111);
/* Array of doubles */
double a[2];
a[0] = 1.234;
a[1] = 2.345;
ser.packArray(a, 3);
/* Strings */
ser.packCstring("Hello!");
/* Variable size integers */
ser.packVariableSize(63); // 1 byte
ser.packVariableSize(268435455); // 4 bytes
ser.flush();

The explicit flush call is only necessary at some kind of synchronization point, such as when waiting for a reply. Otherwise, XBS will automatically call flush() internally when the buffer is full, or when the destructor of Serializer is executed.

Download

XBS can be downloaded from here.