How to correctly represent List of objects in DynamoDbMapper · aws/aws-sdk-kotlin · Discussion #1594

sb000003
May 8, 2025

Hi,

as title suggests, I'm currently getting an error that may be related to this but I'm not sure of the cause.

The error is "The provided starting key is invalid: The provided key element does not match the schema". The error only seems to happen on scanPaginated.

My data class is as follows:
@DynamoDbItem data class Fixture( @DynamoDbPartitionKey var id: String, var club1: String? = null, var club1score: String? = null, var club2: String? = null, var club2score: String? = null, var competition: String? = null, var dateTime: String? = null, var finalScore: String? = null, var goals: List<Goal>? = null, var gW: String? = null, var isFuture: Boolean? = false, var isLive: Boolean? = false, @JsonProperty(access = JsonProperty.Access.WRITE_ONLY) var providerId: String? = null, var season: String? = null, var winner: String? = null, )

And Goal is:
@DynamoDbItem data class Goal ( @DynamoDbPartitionKey val id: String, val playerName: String, val teamId: String, val clockTime: String? )

This works when using a queryPaginated query but not when doing a scanPaginated and i receive the error above.

I have checked I've got the data class properties the exact same name and types as the attribute names in my table.

I have one secondary index on my table, do i need to reflect this in my data class somehow?

Replies: 2 comments 6 replies

ianbotsf
May 8, 2025
Maintainer

Hi @sb000003, sorry you're encountering an error. Your data class definitions look fine to me...I can't see anything that I would expect to cause problems. Can you provide any repro code to help debug?

0 replies

sb000003
May 9, 2025
Author

Yes I'm calling the get method built in this class

class DynamoDbHandler<T, PK>(
 private val schema: ItemSchema.PartitionKey<T, PK>,
 private val tableName: String,
 private val client: DynamoDbClient = DynamoDBConnector().getDynamoDbClient(),
 private val mapper: DynamoDbMapper = DynamoDbMapper(client = client),
 private val table: Table<T> = mapper.getTable(tableName, schema),
) {
 fun get(): List<T> {
 return runBlocking { table.scanPaginated{ }.items().toList() }
 }
 
 fun getByKeyValue(keyVal: String): T {
 return runBlocking {
 table.queryPaginated {
 keyCondition = KeyFilter(partitionKey = keyVal)
 }.items().first()
 }
 }
}

And calling it from this class

class FixturesCrud(
 @Value("fixtures-\${env}") private val tableName: String,
 private val dynamoDbHandler: DynamoDbHandler<Fixture, String> = DynamoDbHandler(tableName = tableName , schema = FixtureSchema)
) : AbstractCrud<Fixture, String>(dynamoDbHandler = dynamoDbHandler) {
 fun doQueryAll() = dynamoDbHandler.get()
}

Could it be something to do with the GSI on fixtures table? It doesn't seem to mind when i run a queryPaginated for one item.

6 replies

@sb000003

sb000003 May 10, 2025
Author

Thanks for your reply @ianbotsf , I did what you said and got the request in the logs but I'm not sure if it's valid or not, it looks ok to me, maybe something will stand out to you?


2025年05月10日T16:52:39.060+01:00 DEBUG 18420 --- [nio-8080-exec-2] httpTraceMiddleware : HttpRequest:
POST /
Host: dynamodb.eu-west-2.amazonaws.com
Content-Length: 440
Content-Type: application/x-amz-json-1.0
X-Amz-Target: DynamoDB_20120810.Scan
User-Agent: aws-sdk-kotlin/1.2.15 ua/2.1 api/dynamodb#1.2.15 os/macos#15.4 lang/kotlin#2.1.0 md/javaVersion#21.0.4 md/jvmName#OpenJDK_64-Bit_Server_VM md/jvmVersion#21.0.4+7-LTS 
x-amz-user-agent: aws-sdk-kotlin/1.2.15
amz-sdk-invocation-id: 2353c836-3422-4466-ba2a-f6fd1c53af34
amz-sdk-request: attempt=1; max=3
X-Amz-Date: 20250510T155239Z
{"ExclusiveStartKey":{"id":{"S":"9f164b4f-4d76-44d6-bddf-deda2670b3d8"},"club1":{"NULL":true},"club1score":{"NULL":true},"club2":{"NULL":true},"club2score":{"NULL":true},"competition":{"NULL":true},"dateTime":{"NULL":true},"finalScore":{"NULL":true},"goals":{"NULL":true},"gW":{"NULL":true},"isFuture":{"NULL":true},"isLive":{"NULL":true},"providerId":{"NULL":true},"season":{"NULL":true},"winner":{"NULL":true}},"TableName":"fixtures-dev"}
2025年05月10日T16:52:39.113+01:00 DEBUG 18420 --- [nio-8080-exec-2] httpTraceMiddleware : HttpResponse: 400: Bad Request

Possibly something to do with the nullable values?

@ianbotsf

ianbotsf May 13, 2025
Maintainer

Ah okay it appears that the ItemConverter for this table is converting the pagination keys incorrectly. That request in your logs contains an ExclusiveStartKey value with too many keys—from the description at the top of this discussion, it looks like your table has a single id key and that's all that should be in the ExclusiveStartKey value.

Is this a hand-written ItemConverter or was this created by the schema generator?