Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

DataStream _bulk with duplicate _id: Unknown JSON property 'not_enabled' for type 'ResponseItem' #8697

Closed
Labels
9.xRelates to a 9.x client version Category: Bug
@MindaugasButkus

Description

Elastic.Clients.Elasticsearch version: 9.1.6

Elasticsearch version: 9.1.3

.NET runtime version: 8

Operating system version: Windows 11

Description of the problem including expected versus actual behavior:
Deserializer fails when it encounters a json property that it does not know how to interpret it.
This is a regression. It used to work with Elasticsearch version 9.0.2 and client version 9.0.6.

Steps to reproduce:

  1. Create a data stream without enabling failure_store
  2. In a single bulk operation create 2 documents with the same _id
  3. Observe UnexpectedTransportException with message Unknown JSON property 'not_enabled' for type 'ResponseItem'

I executed the same steps via kibana to see what the response json payload was.
Request:

POST /mb_test/_bulk
{ "create": { "_index": "mb_test", "_id": "1" }}{ "field1": "value1", "@timestamp": "2025-09-09T15:11:10.123" }{ "create": { "_index": "mb_test", "_id": "1" }}{ "field1": "value2", "@timestamp": "2025-09-09T15:11:10.123" }

Response:

{
 "errors": true,
 "took": 0,
 "items": [
 {
 "create": {
 "_index": ".ds-mb_test-2025年09月09日-000001",
 "_id": "1",
 "_version": 1,
 "result": "created",
 "_shards": {
 "total": 2,
 "successful": 1,
 "failed": 0
 },
 "_seq_no": 0,
 "_primary_term": 1,
 "status": 201
 }
 },
 {
 "create": {
 "_index": ".ds-mb_test-2025年09月09日-000001",
 "_id": "1",
 "status": 409,
 "failure_store": "not_enabled",
 "error": {
 "type": "version_conflict_engine_exception",
 "reason": "[1]: version conflict, document already exists (current version [1])",
 "index_uuid": "XPy8YljrTXmkF2lZoZlBSg",
 "shard": "0",
 "index": ".ds-mb_test-2025年09月09日-000001"
 }
 }
 }
 ]
}

It looks like it's failing to parse the failure_store property of the second operation.

Expected behavior
A BulkResponse with details about conflict should be returned.

Provide ConnectionSettings (if relevant):
Client created as follows:

new ElasticsearchClient(
 new ElasticsearchClientSettings(options.Uri)
 .Authentication(new ApiKey(options.ApiKey))
)

Provide DebugInformation (if relevant):

# FailureReason: Unrecoverable/Unexpected BadRequest while attempting POST on https://<node>.eastus.azure.elastic-cloud.com/<ds_name>/_bulk
 - [1] BadRequest: Node: https://<node>.eastus.azure.elastic-cloud.com/ Exception: JsonException Took: 00:00:00.1656448
# Audit exception in step 1 BadRequest:
System.Text.Json.JsonException: Unknown JSON property 'not_enabled' for type 'ResponseItem'.
 at Elastic.Clients.Elasticsearch.Core.Bulk.Json.BulkResponseItemConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Types/Core/Bulk/BulkResponseItemConverter.cs:line 114
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.<>c__11`1.<ReadCollectionValue>b__11_0(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 326
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.ReadCollectionValue[T](Utf8JsonReader& reader, JsonSerializerOptions options, JsonReadFunc`1 readElement) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 332
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.<>c.<Read>b__4_1(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
 at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, T& value)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
 at Elastic.Transport.Products.Elasticsearch.ElasticsearchResponseBuilder.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, BoundConfiguration boundConfiguration, Stream responseStream, CancellationToken cancellationToken)
 at Elastic.Transport.DefaultResponseFactory.CreateCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String contentType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
 at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, CancellationToken cancellationToken)
 at Elastic.Transport.RequestPipeline.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Auditor auditor, CancellationToken cancellationToken)
 at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)
# Inner Exception: Unknown JSON property 'not_enabled' for type 'ResponseItem'.
System.Text.Json.JsonException: Unknown JSON property 'not_enabled' for type 'ResponseItem'.
 at Elastic.Clients.Elasticsearch.Core.Bulk.Json.BulkResponseItemConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Types/Core/Bulk/BulkResponseItemConverter.cs:line 114
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.<>c__11`1.<ReadCollectionValue>b__11_0(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 326
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.ReadCollectionValue[T](Utf8JsonReader& reader, JsonSerializerOptions options, JsonReadFunc`1 readElement) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 332
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.<>c.<Read>b__4_1(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
 at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, T& value)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
 at Elastic.Transport.Products.Elasticsearch.ElasticsearchResponseBuilder.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, BoundConfiguration boundConfiguration, Stream responseStream, CancellationToken cancellationToken)
 at Elastic.Transport.DefaultResponseFactory.CreateCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String contentType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
 at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, CancellationToken cancellationToken)
 at Elastic.Transport.RequestPipeline.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Auditor auditor, CancellationToken cancellationToken)
 at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)
# Exception:
Elastic.Transport.UnexpectedTransportException: Unknown JSON property 'not_enabled' for type 'ResponseItem'.
 ---> System.Text.Json.JsonException: Unknown JSON property 'not_enabled' for type 'ResponseItem'.
 at Elastic.Clients.Elasticsearch.Core.Bulk.Json.BulkResponseItemConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Types/Core/Bulk/BulkResponseItemConverter.cs:line 114
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.<>c__11`1.<ReadCollectionValue>b__11_0(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 326
 at Elastic.Clients.Elasticsearch.Serialization.JsonReaderExtensions.ReadCollectionValue[T](Utf8JsonReader& reader, JsonSerializerOptions options, JsonReadFunc`1 readElement) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/JsonReaderExtensions.cs:line 332
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.<>c.<Read>b__4_1(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at Elastic.Clients.Elasticsearch.Json.BulkResponseConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/BulkResponse.Converters.g.cs:line 52
 at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
 at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, T& value)
 at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
 at Elastic.Transport.Products.Elasticsearch.ElasticsearchResponseBuilder.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, BoundConfiguration boundConfiguration, Stream responseStream, CancellationToken cancellationToken)
 at Elastic.Transport.DefaultResponseFactory.CreateCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String contentType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
 at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, CancellationToken cancellationToken)
 at Elastic.Transport.RequestPipeline.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Auditor auditor, CancellationToken cancellationToken)
 at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)
 --- End of inner exception stack trace ---
 at Elastic.Transport.DistributedTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, Endpoint endpoint, TResponse response, IReadOnlyCollection`1 auditTrail)
 at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)

I tried setting the UnmappedMemberHandling to Skip but it did not help. continue is missing here: https://github.com/elastic/elasticsearch-net/blob/main/src/Elastic.Clients.Elasticsearch/_Shared/Types/Core/Bulk/BulkResponseItemConverter.cs#L111

Metadata

Metadata

Assignees

No one assigned

    Labels

    9.xRelates to a 9.x client version Category: Bug

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

        AltStyle によって変換されたページ (->オリジナル) /