Today we will talk about using Protocol Buffers on .Net platform. I will tell  and show you what it is and why it is needed .Net developer. A format of the post is different from the usual format and it is designed as a discussion. What I think should improve the understanding of the material by readers.

Hello, what is Protocol Buffers?

According to the definition on the official page, Protocol Buffers (protobuf) is a way of encoding structured data in an efficient and extensible format used by Google in almost all its products. For most platforms, including .Net, a process called serialization.

I use the services of Google, but as protobuf help me to develop .Net applications?

Yes you are right, Google is not engaged in writing of specialized libraries for .Net developers. However, there is protobuf-net project (one of several implementations of protobuf for the platform), that allows to use of protobuf in .Net. It is run by Marc Gravell – frequenter stackoverflow.com and participated in many other great projects. So you can always ask him, and he is happy to answer it (Thanks Mark, that answered my questions!).

Why should I use this library instead of built-in?

When it comes to serialization in .Net, all usually think of the existence of a binary formatter and xml serializer. The next thing that developers mentions is that the first one is fast and has a high degree of compression, but only works within .Net platform, and the second one represents data in human- readable format and provides a basis for SOAP, that in turn is used for cross-platform solutions. In fact, the assertion that you always have to make a choice between speed and portability, is taken for granted! But protobuf can solve both problems at once.

So you’re saying that protobuf not yield the binary serialization and also are portable?

Yes it is. Let’s consider a simple example, but at the same time learn how to use protobuf-net. Assume that we have the following entities:

public enum TaskPriority
{
	Low,
	Medium,
	High
}

[Serializable] // <-- Only for BinaryFormatter
[ProtoContract]
public class Task
{
	[ProtoMember(1)]
	public int Id { get; set; }

	[ProtoMember(2)]
	public DateTime CreatedAt { get; set; }

	[ProtoMember(3)]
	public string CreatedBy { get; set; }

	[ProtoMember(4)]
	public TaskPriority Priority { get; set; }

	[ProtoMember(5)]
	public string Content { get; set; }
}

Protobuf-net requires using of special attributes that directly follows from the main feature of the format – depending on an order of fields. Let’s write a test of performance and compression:

internal class Program
{
	private static void Main(string[] args)
	{
		var tasks = new List<Task>
						{
							new Task
								{
									Id = 1,
									CreatedBy = "Steve Jobs",
									CreatedAt = DateTime.Now,
									Priority = TaskPriority.High,
									Content = "Invent new iPhone"
								},
							new Task
								{
									Id = 2,
									CreatedBy = "Steve Ballmer",
									CreatedAt = DateTime.Now.AddDays(-7),
									Priority = TaskPriority.Low,
									Content = "Install own Skype"
								}
						};

		Console.WriteLine("The test of binary formatter:");

		const string file1 = "tasks1.bin";

		TestBinaryFormatter(tasks, file1, 1000);
		TestBinaryFormatter(tasks, file1, 2000);
		TestBinaryFormatter(tasks, file1, 3000);
		TestBinaryFormatter(tasks, file1, 4000);
		TestBinaryFormatter(tasks, file1, 5000);

		Console.WriteLine("\nThe test of protobuf-net:");

		const string file2 = "tasks2.bin";

		TestProtoBuf(tasks, file2, 1000);
		TestProtoBuf(tasks, file2, 2000);
		TestProtoBuf(tasks, file2, 3000);
		TestProtoBuf(tasks, file2, 4000);
		TestProtoBuf(tasks, file2, 5000);

		Console.WriteLine("\nThe comparision of file size:");

		Console.WriteLine("The size of {0} is {1} bytes", file1, (new FileInfo(file1)).Length);
		Console.WriteLine("The size of {0} is {1} bytes", file2, (new FileInfo(file2)).Length);

		Console.ReadKey();
	}

	private static void TestBinaryFormatter(IList<Task> tasks, string fileName, int iterationCount)
	{
		var stopwatch = new Stopwatch();
		var formatter = new BinaryFormatter();
		using (var file = File.Create(fileName))
		{
			stopwatch.Restart();

			for (var i = 0; i < iterationCount; i++)
			{
				file.Position = 0;
				formatter.Serialize(file, tasks);
				file.Position = 0;
				var restoredTasks = (List<Task>)formatter.Deserialize(file);
			}

			stopwatch.Stop();

			Console.WriteLine("{0} iterations in {1} ms", iterationCount, stopwatch.ElapsedMilliseconds);
		}
	}

	private static void TestProtoBuf(IList<Task> tasks, string fileName, int iterationCount)
	{
		var stopwatch = new Stopwatch();
		using (var file = File.Create(fileName))
		{
			stopwatch.Restart();

			for (var i = 0; i < iterationCount; i++)
			{
				file.Position = 0;
				Serializer.Serialize(file, tasks);
				file.Position = 0;
				var restoredTasks = Serializer.Deserialize<List<Task>>(file);
			}

			stopwatch.Stop();

			Console.WriteLine("{0} iterations in {1} ms", iterationCount, stopwatch.ElapsedMilliseconds);
		}
	}
}

The results:

The test of binary formatter:
1000 iterations in 423 ms
2000 iterations in 381 ms
3000 iterations in 532 ms
4000 iterations in 660 ms
5000 iterations in 814 ms

The test of protobuf-net:
1000 iterations in 1056 ms
2000 iterations in 76 ms
3000 iterations in 129 ms
4000 iterations in 152 ms
5000 iterations in 202 ms

The comparision of file size:
The size of tasks1.bin is 710 bytes
The size of tasks2.bin is 101 bytes

As you can see, we have surpassed the binary serialization, not only on speed but also on a degree of compression. The only drawback that protobuf-net took more time to “cold start”. But you can solve this problem by using the following auxiliary code:

var model = TypeModel.Create();
model.Add(typeof(Task), true);
var compiledModel = model.Compile(path);
compiledModel.Serialize(file, tasks);

Other tests and the results can be found here.

Ok. Relative to the speed and compression you have convinced me, but how is the problem of portability solved?

You know, if there is implementation for the desired platform, the issue of portability in most cases eliminated. There are protobuf’s implementation for more than 20 languages​. A full list can be seen here. I note only that in some languages ​​there are more than one implementation. So you always have a choice.

Soon we’ll continue!