See https://www.psycopg.org/docs/install.html#psycopg-vs-psycopg-binary

I was able to build in 3.8 slim by adding:

• build-time dependencies:
  • libpq-dev='11.12-0+deb10u1'
  • build-essential='12.6'
• run-time dependencies:
  • libpq5='11.12-0+deb10u1'

The build-time packages can be removed from the docker image after psycopg2 python package is installed.