Thanks for the proposal and ping @apoorvdeshmukh - it's great to see this effort starting.

I don't there's much to say from an EF perspective - it should be able to support whatever type you decide to go with; in other words, users would be able to have a SqlVectorFloat32 property on their .NET type, and EF would read and write that to the database as with any other type. I'm a bit considered about the odd requirements around NULL though: EF typically just sends DBNull.Value for nulls - that works with all other types, but it seems like vector will be a new, special exception to that. Hopefully this won't be a blocker.

Below are various thoughts. Once we're more baked around the API shape, I'd recommend we bring this to the API design panel - these are the folks that discuss all changes to BCL APIs etc. Although SqlClient isn't part of the BCL this isn't mandatory, but they can provide very valuable advice around all the issues we're discussing.

• Agree with @edwardneal that sealing SqlVectorFloat32 is a good idea.
• In fact, you may want to consider making the type a struct rather than a class. It's very small - so a good candidate for a struct - and this would save the needless extra allocation. If you do this, I'd recommend making this a readonly struct, as mutable structs are hard to work with. We can also consult with .NET API designers on this question if we want.
• I highly recommend accepting ReadOnlyMemory<float> rather than just float[] on the type; ROM<float> allows representing a slice of an array, as well as unmanaged memory, and would allow more efficient usage. Note that an implicit cast exists from T[] to ROM<T>.
• Does SqlVectorFloat32.ToArray() copy the array out, or expose access to the original array? In general, is SqlVectorFloat32 intended to be a very thin wrapper above user data (I certainly hope so) rather than copying the user's data into an internal buffer?
  • It seems quite important to just expose the data, rather than copying. If this copies, then users have no way of reading a vector from SQL Server without an additional expensive copy.
  • ToArray() in .NET generally connotates a copy rather than simple exposing. To expose, the more natural thing would likely be a simple property (a method like ToArray suggests potentially heavy processing, while a property suggests simple immediate access).
• If ISqlVector is a public interface, then it would be good to understand what it does, the motivations etc. (OTOH I'm not sure something like that makes sense). Of course, if it's internal then no problem at all (usually internal interfaces are left out from (public) API proposals).
• Naming: since the SQL Server type is called vector, I'd recommend calling the .NET type SqlVector rather than SqlVectorFloat32.
  • Unless I'm mistaken, existing Sql* types mirror the naming of the SQL Server type they represent. This makes things easier to understand for users, who may wonder if SqlVectorFloat32 is the right .NET type for interacting with SQL Server vector.
  • If other vector types are introduced in the future, I expect they'll have to have different names (half_vector?), or possibly specify the type as a facet (vector(dim, underlying_type)) - in both cases I think you want to maintain parity between the SqlClient and database names (and also make the client-side type name shorter)
• Re SqlDataReader.GetSqlVectorFloat32():
  • Note that it's already possible to read any value type via SqlDataReader.GetFieldValue<T>() and SqlDataReader.GetFieldValueAsync<T>(), so there's no specific need to introduce GetSqlVectorFloat32().
  • GetSqlVectorFloat32() would only allow synchronous reading of a vector; to asynchronously read it, users would still have to use GetFieldValueAsync<T>, unless you also introduce GetSqlVectorFloat32Async(). There are no asynchronous type-specific getters like this on SqlDataReader: the story here is probably that the sync getters (GetInt32(), GetString()...) were originally introduced back in .NET 1.0, before .NET had generics (so GetFieldValue<T>() simply didn't exist and there was no other option). Once GetFieldValue<T>() was introduced, such type-specific getters are no longer really necessary.
  • I'll also point out that DateOnly and TimeOnly support was added to SqlClient without type-specific getters (i.e. there's no SqlDataReader.GetDateOnly()). So my recommendation would be to not introduce GetSqlVectorFloat32() either.
  • If you do introduce it, then depending on what you decide on the type naming question above you may want to rename this so SqlDataReader.GetVector() instead of SqlDataReader.GetSqlVectorFloat32().
• Null values
  • The proposed API for creating a null value is: new SqlVectorFloat32(3) (null value for a 3-dimensional vector). There's nothing indicating that this API creates a null value, as opposed to e.g. a 3-dimensional vector that can then be populated with data by the user - so I can imagine a bit of user confusion. Would this
  • There's also a static Null property for a 0-dimensional null vector, apparently for DataTable compatibility - but I can't imagine any scenario where this would be useful. Can you provide a bit more context on why this is needed?
  • Instead of the above 2 APIs, would it make sense to instead of a static factory method, e.g. CreateNull() accepting the number of dimensions? This would make it very clear what the API does etc.
  • BTW is there an example of another SQL Server type where a null value still needs to carry some facet information (like the dimensions in the vector case)? Because OTOH all other SQL Server types have a simple, single null value (which is how it ideally should be), rather than requiring this sort of extra information.

@edwardneal I'm including some thoughts on your comments below, in an answer on your comment.

Thanks @roji. I can see your answers, but to provide a little more context to the original point:

• Type names: broadly agree with a name of SqlVector. If we think it's possible that SQL Server could introduce a vector(dim[, underlying_type]) variant, we might want to have a base SqlVector<T> class or an ISqlVector<T> interface. This would let us tidily map SqlVector to vector(3), and SqlVector<int> to vector(3, int).
• Memory ownership: completely agree that SqlVector should wrap caller-supplied memory rather than perform copies.
• Nullability:
  • There's an implicit assumption in System.Data and in SqlClient that anything which implements INullable will also have a static property or field named Null. I've assumed it's the only reason why we need this field, and that this type implements INullable for consistency with the rest of SqlTypes and for compatibility with DataTables.
  • DataTable compatibility is important because it's one way for users to pass table-valued parameters to SqlClient and to perform bulk copies.
  • The closest parallel to this type might be SqlChars (which needs to encode the length of a SQL Server char(X)). This has no way to set a null value with a specific length. I'm inclined to say that if a SqlVector is null, its length should similarly be ignored. Clients will inspect the Length property to get the number of dimensions in the value, not in the schema type; the fact that for a non-null value these will always be equal is an implementation detail. If the value is null, Length should throw a SqlNullValueException. Clients should check IsNull before dealing with the value.

On one slightly broader comment: it'd be good to know if SqlVector will be added to this list of types which can appear in a CLR UDT.

• Good points about the type name.
  • It would be problematic to have a SqlVector which means "vector of floats", as opposed to SqlVector<int> which means "vector of ints".
  • On another note, binary embeddings (where the vector element is a single bit) are also a popular choice. The representation we're aligning on in .NET for this is the BitArray type. This wouldn't fit into a SqlVector<T>, in the sense that we don't want to have a SqlVector<bool> wrapping a bool[] (or ReadOnlyMemory<bool>), but rather a SqlBinaryVector which wraps a BitArray.
  • We've had very similar discussions around the .NET Embedding type hierarchy in Microsoft.Extensions.AI (which is very closely related). We ended up having an abstract Embedding superclass, with Embedding<T> extending it and wrapping ReadOnlyMemory<T>, as well as BinaryEmbedding also extending it, and wrapping BitArray.
  • In SqlClient we could do a similar thing, i.e. have an abstract SqlVector and a SqlVector<T> (for float, Half...) and SqlBinaryVector for BitArray. Though I'm not sure that in the SqlClient case the abstract base class is actually necessary.
  • So for now I'd recommend maybe just going with SqlVector<T> and only supporting SqlVector<float>. You can always support SqlVector<Half> if and when that becomes supported, plus add support for SqlBinaryVector as well. Not sure I'd have the base class right away - that can always be introduced later.
• Re nullability, I think I'd need a bit more context to fully understand this. I specifically am not sure how Null would be useful if it denotes a 0-dimension vector, which would presumably fail if you try to insert that into an actual vector column in SQL Server (which has non-zero dimensions).

SqlVector's naming sounds sensible.

I think SqlVector<float>.Null is best treated as a strongly-typed DBNull.Value - just like SqlChars, the number of dimensions shouldn't matter for a null value. In various contexts:

• Calling SqlDataReader.GetFieldValue<SqlVector<float>>() will return SqlVector<float>.Null if the SQL Server value is null, irrespective of the number of dimensions in the vector. Someone who wants to know the length of the data type should call SqlDataReader.GetColumnSchema to discover the result set schema.
• Passing a null vector(3) as a parameter would mean setting SqlParameter.Value to SqlVector<float>.Null and SqlParameter.Size to the number of dimensions, just as we'd do for a null char(3).

To feed those semantics through to the API proposal, I'd suggest removing the SqlVector(int) constructor and maintaining the convention that the default parameterless constructor creates a SqlVector<float> instance representing a null value.

Great comments here @edwardneal.

Some thoughts:

• Re Vector<T>: AFAIK in .NET, Vector<T> is specifically about CPU vectorization (SIMD). The SQL Server vector type will generally be used to represent embeddings, which are generally an opaque series of numbers that users don't perform operations on. So I'm not sure it's going to be relevant to interoperate directly between SqlClient and .NET Vector<T>. However, as long as we allow efficient extraction of the array (or ReadOnlyMemory<float>) wrapped by the new proposed vector type, users can easily get that out and then construct a Vector<T> over it, if they need to.
• Re accepting ReadOnlySpan<T>:
  • This relates to the question of whether SqlVectorFloat32 is a simple wrapper across the value provided in the constructor (e.g. the float[]), or whether it copies that value into an internal array. For performance reasons, it's IMHO very important to not force expensive copying - I think SqlVectorFloat32 should be a pure, thin wrapper over data and not copy. I'd also not have both wrapping and copying constructors, as this would lead to confusion as to which one does what: I'd lean towards a very simple type which only wraps and never copies, if possible.
  • A constructor accepting a Span (or a FromSpan) method would have to copy; SqlVectorFloat32 cannot wrap a Span, since Spans are ref structs (and SqlVectorFloat32 would have to be a ref struct itself in order to wrap it, which is impossible as it must be assignable to SqlParameter.Value).
  • However, I do believe SqlVectorFloat32 should accept a ReadOnlyMemory<float> (see my comment above).
• Re collection expressions: because of how vectors (embeddings) actually work, users generally never construct them in code, but rather generate them from natural language text, images, etc. In other words, a SqlVectorFloat32 would be created by an embedding generator, e.g. an OpenAI service. So while I'm a big fan of collection expressions, I simply don't think these would be useful in any actual real-world scenario.

Thanks @roji. I'm approaching the datatype with the view that although it's mainly intended for AI usage, at its heart vector is a way to efficiently store, insert and retrieve a fixed-size set of floats. That's interesting in its own right, and it'd be a shame if the API shape closed off options.

• Vector<T>: you're right that this is about enabling SIMD. I'm interested in making sure that this works well in both directions: we should be able to use Vector<float> to efficiently work with an array of floats and use SqlVector<float> to pass that to SQL Server without incurring any array copies; similarly, we should be able to query the database for these, read the SqlVector<float> and work with the results without having to convert them. Exposing the underlying data as a ReadOnlyMemory<float> is a good place to start for this.
• ReadOnlySpan<T>: yes, agreed. If we're going to say that SqlVector<float> only ever wraps an array, using ReadOnlyMemory<float> works perfectly well.
• Collection expressions: I'm not convinced that the type will only ever be used for embeddings, but we can always revisit the idea once vector's being used more widely and we know whether it'd actually be useful.

Re using SQL Server vector and SqlClient's SqlVector for things other than embeddings (this is related to both interop with .NET Vector<T> for SIMD, and for enabling collection expressions)... I think the key points here are:

• vector supports no operations - on the SQL Server side - aside from similarity search. You can't add, multiply, and, or even index to get a specific float value inside the vector. It's a completely opaque blob.
• vector supports floats only. Other database support types like Half, bit, and sparse vector representations, but no database supports e.g. int, long, double. In other words, the actual vector types follow what is useful for the narrow purpose of representing meaning (embeddings).

The above, by the way, isn't some SQL Server limitation - vector types in all vector databases are similarly limited, since they are generally meant to serve a very narrow purpose: these are totally opaque representations of meaning which aren't meant to be manipulated, but only searched for similarity. Note that in PostgreSQL, the pgvector extension adds new, specialized vector types, rather than reusing the existing, general full-featured array.

Of course, things may change and the world might start using vector databases (and types) in a different way. But at least at the moment, I believe we should follow the actual database support for the vector type, which is to treat it as opaque embeddings. Interoperability with Vector<T> specifically should already be OK as long as we take care to introduce a simple wrapper that doesn't copy (at that point the user can extract the array from SqlVector and use it with Vector<T> - no need for any explicit/specific integrations in SqlClient).

And as you say, we can always add additional things (e.g. collection expressions) as the need arises.

However, I'd advise tightening things up so that this property can only be set for NULL values, and otherwise simply exposes ROM.Length. In other words, in the above proposal Length is settable by the user (it's an Init property). This means that a user can create a SqlVector with mismatching Length and Memory properties, which ideally wouldn't be possible. So I'd advise making Length a read-only property, and only allowing the user to determine it when they construct a null SqlVector instance (via the constrcutor). The getter would always simply return ROM.Length, unless the instance represents a null, in which case it would return what the user passed in the constructor.

Yes, the current implementation allows Length to be set in the constructor only if the user passes it with

public SqlVector(int length);

in which case it represents a null instance of a given length and the only other constructor

public SqlVector(ReadOnlyMemory<T> memory);

initializes Length to ROM.Length after which it is immutable.

I agree with your suggestion to turn SqlVector(length) into factory method CreateNull to make it more clear.

Regarding

public int Size { get; init; }

Users are not required to interact with SqlVector.Size for any use case other than using Prepare API.
SqlVector.Size is needed because user won't know what they should set the SqlParameter.Size to in case of vector parameters when used in the context of Prepare API.
SqlVector.Size provides the correct value to be used.

OK. Re Length:

public int Length { get; init; } // The init here seems like it should be removed
public ReadOnlyMemory<T> Memory { get; init; }

Everything you wrote makes sense - I'm only point out that in the last state of the proposal, the Length property is initable. This means that the user can instantiate a SqlVector with a non-null ReadOnlyMemory, and also set its Length property to something which does not match the ReadOnlyMemory:

var vector = new SqlVector<float>([1.0f, 2.0f, 3.0f]) { Length = 4 };

This seems wrong, since a SqlVector wrapping a ROM with a length of 3 has its Length property set to 4. So I'm suggesting to remove the init; from the Length property, making it read-only.

Re Size:

SqlVector.Size is needed because user won't know what they should set the SqlParameter.Size to in case of vector parameters when used in the context of Prepare API.
SqlVector.Size provides the correct value to be used.

I see... So to prepare a statement which will send vector parameters of size 4, users are expected to do the following:

var command = new SqlCommand(...);
var parameter = new SqlParameter("vector", SqlDbType.Vector);
var dummyVector = SqlVector.CreateNull(4);
parameter.Size = dummyVector.Size;
// dummyVector will no longer be needed - was only used to calculate the byte Size for a vector of size 4
command.Parameters.Add(parameter);
command.Prepare();

Is that correct?

If so, may I ask why it was decided that for the vector type, SqlParameter.Size needs to represent the byte length as opposed to the dimension? The byte length of a vector is an internal, TDS-specific thing that users obviously know nothing about, which is why you have to provide some API - the Size property - in order for them to get it. On the other hand, it seems to make more sense for users to just provide the dimensions directly, rather than having to convert the dimensions to the byte length.

I'll point out the SqlParameter.Size already doesn't always represent the byte length; for string type parameters, Size means length in Unicode characters and not in bytes (docs). So I'm again not sure why we'd force the user to deal with the byte length in the vector case.

If there's still some reason why SqlParameter.Size must absolutely mean the vector byte length, then I'd suggest simply providing a simple static method (static SqlVector<T>.GetByteLength(int dimensions)) rather than having a property for that on SqlVector. Having a real property actually takes space which will never actually be used, is bound to create confusion between Length and Size, etc. (it's also not clear why users would want to set it - it's initable as well). A simple static method seems more than sufficient (though again, the simplest/easiest is for users to just set the dimensions directly on SqlParameter.Size).

This seems wrong, since a SqlVector wrapping a ROM with a length of 3 has its Length property set to 4. So I'm suggesting to remove the init; from the Length property, making it read-only.

You're right. I'll fix this.

If so, may I ask why it was decided that for the vector type, SqlParameter.Size needs to represent the byte length as opposed to the dimension? The byte length of a vector is an internal, TDS-specific thing that users obviously know nothing about, which is why you have to provide some API - the Size property - in order for them to get it. On the other hand, it seems to make more sense for users to just provide the dimensions directly, rather than having to convert the dimensions to the byte length.

Issue with interpreting SqlParameter.Size as number of elements in vector is that it doesn't tell us the type of the element it is unless we define multiple SqlDbTypes for every vector type such as Float32, Float16, Int32 etc. We need to know the type as well in order to calculate the correct size in bytes for transmitting the data on TDS.
Agree with your suggestion on changing the property to GetByteLength

unless we define multiple SqlDbTypes for every vector type such as Float32, Float16, Int32

Yes, I understand; it's my strong assumption that if/when other vector types are introduced in SQL Server in the future, those would have their own SqlDbType entries as well (regardless of preparing statements, I think you need a SqlDbType entry for each actual SQL Server type). Or if SQL Server itself introduces some other new facet type to represent Float32/Float16, you'd need to add that to SqlParameter next to Size, Precision and Scale. But I think that's really unlikely - just introducing a couple new types like halfvector, binaryvector would be my assumption for how things develop (you can always check with SQL Server folks if that assumption makes sense).

So to summarize, I'd indeed expect SqlDbType to be sufficient to identify the specific vector types (including float32 vs. float16), and therefore the combination of SqlDbType + Size (as dimensions) should be sufficient.

Here is the revised API for vector datatype

namespace Microsoft.Data.SqlTypes
{
 /// <summary>
 /// Represents a SQL vector of unmanaged type elements (currently supports float32 and more will be added later).
 /// </summary>
public readonly struct SqlVector<T> : System.Data.SqlTypes.INullable
 where T : unmanaged
 {
 /// <summary>
 /// Creates a vector with the specified elements.
 /// </summary>
 public SqlVector(System.ReadOnlyMemory<T> memory) { }
 /// <summary>
 /// Indicates whether this vector instance is null.
 /// </summary>
 public bool IsNull { get; }
 /// <summary>
 /// Required for maintaining the compatibility with DataTable. Points to a null value.
 /// </summary>
 public static SqlVector<T>? Null { get; }
 /// <summary>
 /// Gets the number of elements in this vector.
 /// </summary>
 public int Length { get; }
 /// <summary>
 /// Gets the vector elements as read-only memory.
 /// </summary>
 public ReadOnlyMemory<T> Memory { get; }
 /// <summary>
 /// Creates a null instance of a vector with the specified length.
 /// </summary>
 public static SqlVector<T> CreateNull(int length);
 }
}

That's correct. We are scoping the work for .NET and plan to use System.Half. For .NET Framework, there is no half-precision floating point type, so we're not planning to add support there.