Rapid Avro serializer for C# .NET
Rapid Avro serializer for C# .NET
Avro format combines readability of JSON and data compression of binary serialization.
Apache Avro format documentation
First steps with Avro in the .NET article
Benchmark and Avro API article
Introducing Avro to the projects brings three main benefits:
AvroConvert | Apache.Avro | Newtonsoft.Json | |
---|---|---|---|
Rapid serialization | ✔️ | ✔️ | ✔️ |
Easy to use | ✔️ | ❌ | ✔️ |
Built-in compression | ✔️ | ✔️ | ❌ |
Low memory allocation | ✔️ | ✔️ | ✔️ |
Support for C# data structures: Dictionary, List, DateTime... | ✔️ | ❌ | ✔️ |
Support for compression codecs | Deflate Snappy GZip Brotli |
Deflate | ❌ |
Readable schema of data structure | ✔️ | ✔️ | ✔️ |
Data encryption | ✔️ | ✔️ | ❌ |
Results of BenchmarkDotNet:
Converter | Request Time [ms] | Allocated Memory [MB] | Compressed Size [kB] |
---|---|---|---|
Json | 672.3 | 52.23 | 6044 |
Avro | 384.7 | 76.58 | 2623 |
Json_Gzip | 264.1 | 88.32 | 514 |
Avro_Gzip | 181.2 | 75.05 | 104 |
Json_Brotli | 222.5 | 86.15 | 31 |
Avro_Brotli | 193.5 | 74.75 | 31 |
Article describing Avro format specification and benchmark methodology: https://www.c-sharpcorner.com/blogs/avro-rest-api-as-the-evolution-of-json-based-communication-between-mic
Conclusion:
Using Avro for communication between your services significantly reduces data size and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.
byte[] avroObject = AvroConvert.Serialize(object yourObject);
CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);
string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)
using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
while (reader.HasNext())
{
var item = reader.ReadNext();
// process item
}
}
string resultModel = AvroConvert.GenerateModel(avroObject);
var resultJson = AvroConvert.Avro2Json(avroObject);
AvroConvertOnline - online Avro Schema to C# model converter
- Library containing functionalities, which enable communication between microservices via Http using Avro data format
- Library containing components needed for Confluent Kafka integration
The project is CC BY-NC-SA 3.0 licensed.
For commercial purposes purchase AvroConvert on website - Xabe.net
We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.