[bug?} Auth HA not working from proxy standpoint

Hi,

I have created a cluster with one proxy server and 3 auth servers (dynamo+s3).
When I shutdown first of the 3 auth servers proxy doesn’t work anymore with error message that it cannot connect to that first auth node. If i remove it from the config, then proxy proceeds to connect successfully to the teleport-auth1b. Why it doesn’t try other servers in the list if the one that is not working is not removed from the config?

My config:
teleport:
nodename: teleport-proxy
data_dir: /var/lib/teleport
pid_file: /var/run/teleport.pid
auth_token: XXX
auth_servers:

- teleport-auth1a:3025 <-- this is shut downed
- teleport-auth1b:3025
- teleport-auth1c:3025

log:
output: /var/lib/teleport/teleport.log
severity: ERROR
auth_service:
enabled: “no”
ssh_service:
enabled: “no”
proxy_service:
enabled: “yes”
listen_addr: 0.0.0.0:3023
tunnel_listen_addr: 0.0.0.0:3024
web_listen_addr: 0.0.0.0:3080
public_addr: teleport-proxy:3080
ssh_public_addr: teleport-proxy:3023
https_key_file: /etc/letsencrypt/live/teleport-proxy/privkey.pem
https_cert_file: /etc/letsencrypt/live/teleport-proxy/fullchain.pem

log:
ERRO [AUTH] Failed to dial auth server teleport-auth1a:3025: dial tcp x.x.x.x:3025: connect: connection refused. auth/clt.go:146
ERRO [AUTH] Failed to dial auth server teleport-auth1a:3025: dial tcp x.x.x.x:3025: connect: connection refused. auth/clt.go:146

This may be a bug. I believe that we have fixed a similar issue relating to etcd servers in the past.

Our recommendation is to use a load balancer for connection to the auth servers rather than specifying the servers individually, which would solve this issue. I will try to reproduce this and file a bug if appropriate, however.