4.0.16 -> 4.1.10 upgrade fails with cert problems

I am stepping my teleport proxy server through major versions on the way to 4.2.11. When I take the step from 4.0.16 to 4.1.10, auth fails to come up. If I start up with a fresh /var/lib/teleport the config works fine, but upgrade with existing data doesn’t go well. Any tips appreciated.

Errors:
ERRO [PROC:1] Proxy failed to establish connection to cluster: remote error: tls: bad certificate. time/sleep.go:149
http: TLS handshake error from 127.0.0.1:59912: tls: failed to verify client’s certificate: x509: certificate signed by unknown authority

Here is scrubbed teleport.yaml:

teleport:
  advertise_ip: 1.2.3.4
  nodename: teleport.testdomain.com
  log:
    output: stderr
    severity: WARN

auth_service:
  public_addr: teleport.testdomain.com:40349
  cluster_name: teleport
  listen_addr: 0.0.0.0:40349
  tokens:
      - "node:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      - "trusted_cluster:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy"

ssh_service:
  labels:
      to_use: "Pick cluster from menu"
  listen_addr: 0.0.0.0:40348

proxy_service:
  listen_addr: 0.0.0.0:40350
  web_listen_addr: 0.0.0.0:50443
  tunnel_listen_addr: 0.0.0.0:50080