Change DNS and HTTP(s) Proxy on Gravity Cluster

How to you configure/Update DNS and HTTP(s) Proxy setting for a cluster, I wanted to change them and tried to update /etc/environment and /etc/resolve.conf but it did not work,
for DNS I tried to update core dns ConfigMap but did not got success, only way it works is setup a dns and then install gravity

/etc/environment (10.100.0.1 seems to whitelist too)

http_proxy="http://ip-10-XXX-XXX-XXX:3128"
https_proxy="http://ip-10-XXX-XXX-XXX:3128"
no_proxy="10.100.0.1,localhost,127.0.0.1,::1"

/etc/resolv.conf

nameserver 10.XXX.XXX.XXX
nameserver 8.8.8.8

where 8.8.8.8 I have blocked

The answer really depends on version. For gravity 5.5+:

To update env variables such as HTTP_PROXY, is done through the resource system. So take a look at https://gravitational.com/gravity/docs/config/#runtime-environment-variables to set environment variables. This will do a rolling restart of all the cluster nodes to have the new settings take effect.

To update DNS resolvers, there are two layers:

  • planet: generates the resolv.conf on startup based on the hosts /etc/resolv.conf. So after updating the hosts resolv.conf planet will need to be drained/restarted for new settings to take effect. This only affects planet services.
  • cluster-dns: pod resolution is done through cluster DNS, which the configmap get’s written at install time (and possibly upgrade, I don’t remember), based on the installer nodes resolv.conf when the operation is started. So the coredns configmap in kube-system namespace would need to be edited with the new upstream settings as well.

/etc/environment and /etc/resolv.conf within planet are not intended to be edited by any administrator.

@knisbet thanks, this fixed the DNS part and also Http Proxy in planet but none of the pod have the proxy available.
is it possible planet and pod have different environment variables? is so then what is the best way to configure that in pods?

@knisbet does run time variables are only for planet container or for all running pods, I follow link and that set the proxy inside planet but not in any other pods although it did restart all the pods, Am I missing something?

The runtime variables only apply to planet processes. IE docker needs to use an http_proxy to reach a private docker registry to download containers.

I don’t believe pods will automatically pick up any environment variables from the host. Pod environment is meant to be configured within the PodSpec itself. See the kubernetes docs for how to configure environment variables for pods: https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/

Edit: The reason the pods restarted, is that gravity restarted planet to apply the new environment variables to the planet processes.