DEV Community

[画像:Maya Andersson profile picture]

Maya Andersson

Just a bored curious dev

Joined on May 19, 2026

Badges

2 Week Community Wellness Streak

1 Week Community Wellness Streak

Writing Debut

10 posts published

10 comments written

11 tags followed

I reviewed six "operator-ready" checklists for AI agents. None of them define the problem correctly.

maya_andersson_dev profile

I reviewed six "operator-ready" checklists for AI agents. None of them define the problem correctly.

#agents #llmevaluation #mlops #agentreliability

5 min read

Want to connect with Maya Andersson?

Create an account to connect with Maya Andersson. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in

We added synthetic data to our eval set. The pass rate rose, and so did our production incidents.

maya_andersson_dev profile

We added synthetic data to our eval set. The pass rate rose, and so did our production incidents.

#llm #machinelearning #ai #mlops

5 min read

I checked six LLM-as-judge tools against human labels. The scoreboard was the wrong thing to read.

maya_andersson_dev profile

I checked six LLM-as-judge tools against human labels. The scoreboard was the wrong thing to read.

#machinelearning #llm #evaluation #mlops

4 min read

LLM-as-judge tools compared: the question is not which one scores, it is which one you can trust

maya_andersson_dev profile

LLM-as-judge tools compared: the question is not which one scores, it is which one you can trust

#machinelearning #datascience #llm #ai

2 min read

Power analysis for LLM evals: how big does your eval set need to be to catch a 5% regression?

maya_andersson_dev profile

Power analysis for LLM evals: how big does your eval set need to be to catch a 5% regression?

#datascience #statistics #machinelearning #ai

2 min read

We put confidence intervals on our LLM-judge scores. The error bars ate three weeks of "trend"

maya_andersson_dev profile

We put confidence intervals on our LLM-judge scores. The error bars ate three weeks of "trend"

#datascience #statistics #machinelearning #ai

2 min read

More eval traces will not stabilize your kappa. Stratify the ones you have

maya_andersson_dev profile

More eval traces will not stabilize your kappa. Stratify the ones you have

#ai #programming #devops #agents

2 min read

Calibration set size for LLM-as-judge: when 50 traces is enough and when 200 is mandatory

maya_andersson_dev profile

Calibration set size for LLM-as-judge: when 50 traces is enough and when 200 is mandatory

#ai #development #programming #tutorial

9 min read

why Cohen's kappa drifts week to week (and what to do about it)

maya_andersson_dev profile

why Cohen's kappa drifts week to week (and what to do about it)

#ai #evaluation #machinelearning #statistics

1 min read

Your LLM-as-judge eval set is too small. Here is the math

maya_andersson_dev profile

Your LLM-as-judge eval set is too small. Here is the math

#ai #machinelearning #llm #datascience

4 min read

loading...