I am playing around with the aws-uswest2 cluster. I created a dask gateway cluster the standard way. Somehow, this got into a zombie state and it won’t shut down.
Right now, I see this
import dask_gateway
gateway = dask_gateway.Gateway()
gateway.list_clusters()
# [ClusterReport<name=icesat2-prod.e9cd49c11da94a2b9afaed9289aa44ab, status=RUNNING>,
# ClusterReport<name=icesat2-prod.c4d1d1c2644e4ddbaf6461c6f0c5be72, status=RUNNING>]
The first cluster is the zombie one. I do
cluster = gateway.connect('icesat2-prod.e9cd49c11da94a2b9afaed9289aa44ab')
cluster.shutdown()
cluster.close()
cluster.scale(0)
None of these seems to have any effect. The cluster is just sitting there (with 8 workers).
Am I doing something wrong? Or is this a bug?