Start up errors on pangeo google cloud deployments

I keep getting these error message when I start up the pangeo google cloud deployment:

It usually goes away after a few OK click or a browser refresh. It then also usually prompts the kernel selection for each open window.

Anyone have an idea what is going on there or how to fix it? Its not critical, but I thought I would bring it up here. Thanks

2 Likes

Thanks for sharing Julius. Someone will look into it.

Hi @jbusecke, we have attempted to reproduce this error but failed so far.

This remains an open question and I’m thinking on ways to debug this better.

1 Like

Related: I opened Enable users to access the jupyter server logs · Issue #684 · jupyter-server/jupyter_server · GitHub as a strategy to debug issues like this. Right now, having logs from the jupyter server when this happened would be relevant, but I don’t think you as a user can access it yourself but a jupyterhub admin would need to manually inspect the logs retroactively.

The logs may be lost as soon as the server has quit though, so without the jupyterhub admin’s having a way to persist logs, it can be tricky.

@jbusecke hmm, to workaround this, can you do the following if you experience this again?

  1. When this occurs again, note the exact day/hour/minute in UTC time format
  2. Report back to this thread, pinging me and @sgibson91
  3. Ensure your server doesn’t shut down by having a kernel running indefinitely. To do this, you can for example open a Python based notebook and write import time; time.sleep(3600*24*2) to keep your server running for two full days.

@consideRatio @sgibson91 Just had the issue again. It happened maybe a minute ago (currently 14:43:35 EST). There is actually more to that error: I usually queries me to choose a kernel for a bunch of notebooks that I havent opened in a while!


.

Is the problem maybe related to the fact that kernels do not get shut down when I leave for the day? These are older notebooks as far as I can remember. Seems like there are more kernels running than I have notebooks open atm.

I dont really have a feel for when that would be useful, I would actually prefer everything to get shut down, when I close the notebook.

Not sure if this helps at all. Happy to try further things.

@consideRatio @sgibson91 I also just experienced exactly what @jbusecke described in the previous comment:

Upon opening up a new Pangeo Cloud instance, I got the “Error Starting Kernel” message, followed by the options to “Select Kernel”. Unlike for Julius, the “Select Kernel” options popped up for notebooks that I have used recently (over the last couple days).

Specifically, there were 3 notebooks open when the JupyterHub started, and I was prompted to “Select Kernel” for the 2 that were in the background (i.e. not the one that was actively selected at the time).

This occurred probably about 2:29am UTC.

Not sure if this is helpful, but at least I can corroborate @jbusecke’s issues!

:point_down: Also, I agree with @jbusecke on this point.

I dont really have a feel for when that would be useful, I would actually prefer everything to get shut down, when I close the notebook.

Just another datapoint: Happened again to me today (Feb 3 9:43:29 EST). This time I payed attention and I specifically closed the kernels it was asking for the time before. It will literally try to open up the same notebooks, and I do not understand why, or how to reset it.

Hey all - has anybody run into this issue again since the last post on Feb 3rd?

Def had a few since my last post, but not sure about very recent occurences. Ill make sure to report again if it happens?

I just tested it and it did not happen :tada:

…now I just get a warning about the jupyterlab formatter.

That is not that big of a deal though.