Unmanaged byte array to managed structure

Question

I have a need to turn byte arrays into various structures.

First version:

public static object ConvertBytesToStructure(object target, byte[] source, Int32 targetSize, int startIndex, int length)
 {
 if (target == null)
 return null;
 IntPtr p_objTarget = Marshal.AllocHGlobal(targetSize);
 try
 {
 Marshal.Copy(source, startIndex, p_objTarget, length);
 Marshal.PtrToStructure(p_objTarget, target);
 }
 catch (Exception e)
 {
 Console.WriteLine(e);
 }
 finally
 {
 Marshal.FreeHGlobal(p_objTarget);
 }
 return target;
 }

I have found that when i calling to the first version a lot of times in a second - i getting poor performance.

So i trying to improve that to the version 2:

 private static T ReadUsingMarshalUnsafe<T>(byte[] data, int startIndex, int length)
 {
 byte[] fixedData = new byte[length];
 unsafe
 {
 fixed (byte* pSource = data, pTarget = fixedData)
 {
 int index = 0;
 for (int i = startIndex; i < data.Length; i++)
 {
 pTarget[index] = pSource[i];
 index++;
 }
 }
 fixed (byte* p = &fixedData[0])
 {
 return (T)Marshal.PtrToStructure(new IntPtr(p), typeof(T));
 }
 }
 }

I have found that this version getting very good performance..

But i want to getting your code review - maybe i'll have any memory leak? maybe i can to do this with efficent another way?

Thanks!

• Kittoes0124: What version of .NET are you using? If on .NET Core 2.1 or above then one can rely on the native MemoryMarshal class which has a method Read<T> that I believe matches your needs. Alternatively, one can import the System.Memory NuGet package if using .NET Framework 4.5 or above; it is not as optimized but should work well.
• t3chb0t: Could also show the structs you are using?
• David Michaeli: @Kittoes0124, .NETFramework 4.0 client
• David Michaeli: @t3chb0t i'll attach an example

Answer 1

Expanding on my comment; below you'll find a very simple program that compares the method I suggested with your original example. The results on my machine show that the MemoryMarshal class is about 85x faster. You might want to experiment a bit and try running a similar test with a larger struct; maybe your method is faster for the specific problem that you're trying to solve.

Comparison Code:

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Runtime.InteropServices;
public readonly struct SomeStruct
{
 private readonly ulong m_x;
 private readonly ulong m_y;
 private readonly ulong m_z;
 public ulong X => m_x;
 public ulong Y => m_y;
 public ulong Z => m_z;
 public SomeStruct(ulong x, ulong y, ulong z) {
 m_x = x;
 m_y = y;
 m_z = z;
 }
}
public class Race
{
 private readonly byte[] m_data = new byte[] {
 0, 0, 0, 0, 0, 0, 0, 0,
 1, 0, 0, 0, 0, 0, 0, 0,
 1, 255, 0, 0, 0, 0, 0, 0,
 };
 [Benchmark(Baseline = true)]
 public SomeStruct A() => MemoryMarshal.Read<SomeStruct>(m_data);
 [Benchmark]
 public SomeStruct B() => Program.ReadUsingMarshalUnsafe<SomeStruct>(m_data, 0, m_data.Length);
}
class Program
{
 static void Main(string[] args) {
 var summary = BenchmarkRunner.Run<Race>();
 Console.ReadKey();
 }
 public static T ReadUsingMarshalUnsafe<T>(byte[] data, int startIndex, int length) {
 byte[] fixedData = new byte[length];
 unsafe {
 fixed (byte* pSource = data, pTarget = fixedData) {
 int index = 0;
 for (int i = startIndex; i < data.Length; i++) {
 pTarget[index] = pSource[i];
 index++;
 }
 }
 fixed (byte* p = &fixedData[0]) {
 return (T)Marshal.PtrToStructure(new IntPtr(p), typeof(T));
 }
 }
 }
}

BenchmarkDotNet Results:

Benchmark results.

• David Michaeli: MemoryMarshal is not exists in old framework:\

Answer 2

first of all, thanks for your great example.

I update my little program based of your answer to benchmark it.

The project should be .NETFramework 4.0 client, but for the benchmark i change it to be .NETFramework 4.0.

 [StructLayout(LayoutKind.Sequential, Pack = 4)]
public class SomeStructure
{
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 2)]
 public char[] szF1;
 public char cF2;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)]
 public char[] szF3;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 7)]
 public char[] szF4;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 7)]
 public char[] szF5;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 7)]
 public char[] szF6;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF7;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF8;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF9;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF10;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
 public char[] cF11;
 public char cF12;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 2)]
 public char[] cF13;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 7)]
 public char[] szF14;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF15;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 7)]
 public char[] szF16;
 [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
 public char[] szF17;
}
public class Race
{
 //[Benchmark(Baseline = true)]
 //public SomeStruct A() => MemoryMarshal.Read<SomeStruct>(_data);
 [Benchmark(Baseline = true)]
 public object A() => Program.ConvertBytesToStructure(new SomeStructure(), _data, Marshal.SizeOf(typeof(SomeStructure)), 0, _data.Length);
 [Benchmark]
 public SomeStructure B() => Program.ConvertBytesToStructureV2<SomeStructure>(_data, 0, _data.Length);
 private readonly byte[] _data = new byte[] {
 49, 50, 49, 49, 50, 51, 52, 53, 54, 55, 56,
 49, 50, 51, 52, 53, 54, 55, 49, 50, 51, 52,
 53, 54, 55, 49, 50, 51, 52, 53, 54, 55, 49,
 50, 51, 52, 53, 54, 55, 56, 57, 49, 50, 51,
 52, 53, 54, 55, 56, 57, 49, 50, 51, 52, 53,
 54, 55, 56, 57, 49, 50, 51, 52, 53, 54, 55,
 56, 57, 49, 97, 49, 50, 49, 50, 51, 52, 53,
 54, 55, 49, 50, 51, 52, 53, 54, 55, 56, 57,
 49, 50, 51, 52, 53, 54, 55, 49, 50, 51, 52,
 53, 54, 55, 56, 57
};
}
 public class Program
{
 static void Main(string[] args)
 {
 var summary = BenchmarkRunner.Run<Race>();
 Console.ReadKey();
 }
 public static object ConvertBytesToStructure(object target, byte[] source, Int32 targetSize, int startIndex, int length)
 {
 if (target == null)
 return null;
 IntPtr p_objTarget = Marshal.AllocHGlobal(targetSize);
 try
 {
 Marshal.Copy(source, startIndex, p_objTarget, length);
 Marshal.PtrToStructure(p_objTarget, target);
 }
 catch (Exception e)
 {
 Console.WriteLine(e);
 }
 finally
 {
 Marshal.FreeHGlobal(p_objTarget);
 }
 return target;
 }
 public static T ConvertBytesToStructureV2<T>(byte[] data, int startIndex, int length)
 {
 byte[] fixedData = new byte[length];
 unsafe
 {
 fixed (byte* pSource = data, pTarget = fixedData)
 {
 int index = 0;
 for (int i = startIndex; i < data.Length; i++)
 {
 pTarget[index] = pSource[i];
 index++;
 }
 }
 fixed (byte* p = &fixedData[0])
 {
 return (T)Marshal.PtrToStructure(new IntPtr(p), typeof(T));
 }
 }
 }
}

The results:

enter image description here

If i understand good - it seems that according to the Benchmark results,the first method is faster than the second method.

But these are not the results I see in my system.

More details: It's multi-threading system, i've a service that listen to another service and register to his event, when the event is raised with unmanaged data (byte[]) i convert that to the my cutom managed object.

So far so good, but when i stress the system by sending thousands of events (~20000) per second, The original method (ConvertBytesToStructure) getting poor performance, and the new method getting excellent performance..