European Pangeo Community

News & Announcements
1

@tinaok,@geynard, @Dan_Jones, @jlesommer
Hi everyone,

after the Dask summit, I was a little bit negatively impressed by the lack of Europeans attending the conference. I know that there are others that share the enthusiasm for Pangeo here in Europe and I would like to create some good connection. Unfortunately, the European Pangeo community meeting 2020 has been cancelled so the virtual plaza is the only way but who knows… eventually in the near future, we could be able again to physically know each other.
Moreover, I’ve been attending the BIDS’21 but, except for a super nice presentation from @fperez citing Pangeo, all the other presentation were completely ignoring the Pangeo community and tools. So I really would like to start a discussion with all the Europeans about how to efficiently ‘spread the voice’ in the old continent. (really old looking at the approaches)
Looking forward to finding someone that shares the pain and the love for EO.

Lorenzo

6 Likes
2

Yes, it’s a good point. We seem to be lacking a European Pangeo “champion” who can advocate for the project at a high level. But there are certainly many Pangeo users, particularly among the oceanography / ocean modeling community. For example: @jlesommer, @auraoupa , @apatlpo, @willirath, etc. On the met side, Alessandro Amici and collaborators at B-Open have been key partners.

One possible explanation might be that European infrastructure projects tend to be quite top-down and originating from the large funding bodies–Copernicus, ESA, etc. Pangeo is more of a bottom-up community, originating from science users. So it doesn’t get so much press from the European institutions. I think a good path forward would be to define specific goals or metrics that we would hope to achieve.

I personally think Pangeo could play a key role in the big “Digital Twin” project:

Something like Pangeo will certainly be need to deal with the data volume that will come out of this.

In the US, we have raised millions of dollars from funding agencies to support Pangeo. But I’m not aware of any funding proposals around Pangeo that have been submitted in Europe. Perhaps that would be a good place to start. @pl.marasco – would you be interesting in leading such a proposal?

3

I share with you almost all of your thoughts.

Indeed, the European market has a more top-down approach, and it definitely is not easy to be involved in the buttons room. Saying that is a long time that I’m trying to convict the closer stakeholders I have about the real innovation behind the Pangeo idea. Something has been achieved with the document you coauthored but isn’t enough.

In my vision, the first step that needs to be conducted is to ‘re/aggregate’ the European community. Without a strong community, we can’t get the visibility to start an open discussion with the main actors of the sector and get more chance to be considered by them.

About the role of Pangeo, I’ve always been thought that it would be the best option to create a common approach over the Data and Information Access Services (DIAS) and indeed, in this vision, could be a good player in the ‘Digital Twin’. Of course, the road is uphill and long, but it is worth a try.

Unfortunately, I’m aware that the most problematic point is the fragmented European vision, and it is hard to bring different interests to a shared vision, but I’ve faith, especially if we have a globally shared vision.

From my understanding, the founding ‘market’ here in Europe is entirely different from in the US, but I’m open to discussing the idea to lead a proposal more in details.

2 Likes
4

Hi everyone,

I think there were some Europeans on the Dask Summit, but maybe not in Pangeo or Xarray workshop. Unfortunately, I did not have the time/possibility to join these (this has been like this for over a year…), and only concentrate on the Dask in HPC workshop.
It’s the same for BiDS, I did a presentation in the 2019 edition, but this time not able to propose something.
And it’s the same for organizing European Pangeo community meeting…

In France, I know that a lot of people are using Pangeo related technology, for EO/climate stuf, but also other scientific subjects. I cannot really speak for other country, except you @pl.marasco, the MetOffice staff, and @willirath work, and maybe @fmaussion too?

But I totally agree with all your points above, we need some leaders to build this community in Europe, and be able to get some more visibility. I think you could be one of those! I’ll try to help as much as I can. It’s definitely the first step.

The DIAS approach is always something I wanted to look upon, at one point I was in connection with Wekeo (also with the help of @fbriol), but also OVH Cloud. Unfortunately I couldn’t pursue this activities.

About the founding, I’ve no idea how we could do this, there are so many different ways to get some funding (European or national ones), but they are often aimed at private companies more than researchers or institutions (which is more the world I’m in). I still think we could afford some little funding, maybe have some OVH credit or things like that.

2 Likes
5

You are correct in saying that other Europeans were attending the conference, and others are actively using the technology behind Pangeo. But, the point is that, from my point of view, the community needs to grow. Don’t get me wrong, and there is no need to have a pin with ‘European chapter of Pangeo’ printed over it. But, unfortunately, the lack of vision in some groups is so profound that we need to create a more solid EU community.

Overall, I’ve felt entirely alone these years, and most of the time, I had to push literally other groups to try this approach. However, if we spread the voice, most probably other groups will be more prone to test, implement or use the Pangeo approach.

From my perspective, the ocean community is much more advanced in the use of Pangeo; instead, EO scientists are a little bit behind as used to other tools. I can be wrong in this vision, and I would like to discuss this as well.

I share with you the problem about time, and definitely, I’m not in the best position to have a chair in the button rooms, but I’m sure that we can do at least something.

About the founds, again, I fully agree with you. The European system is utterly crazy in that sense. It is tough to have grants if you are not in the academic world; moreover, as Pangeo hasn’t legally constituted (at least here in Europe), it represents another problem.

About the DIAS, I’m involved in a side project that received a grant from WEKEO, but I had to suspend the test for another reason. Indeed, sharing the same env in all the DIAS would be awesome; this should be one of the first achievement and should definitely be a huge step. Unfortunatelly I have an idea about how this could be hard to obtain, but from somewhere we have to start. Thinking about the Twins, well, that’s already the moon.

I’m delighted that we started this conversation; let’s hope that others will join us!

2 Likes
6

I may have sounded a bit negative in my message above, and also too much self complaining… Sorry for that.

I’m glad you started this discussion, I totally support your proposal to try to build a strong european community. I’ll be happy to work with you to advance in this direction.

I think you’re right about the more advanced ocean community, this is clearly the case in France, I’ve almost only worked with people comming from this community when it comes to Pangeo. And most of them are using Pangeo on HPC systems. But European organisations such as EU or ESA are much more focused on EO science and cloud computing, I feel this is in this direction we could obtain some funding, but I still don’t know how.

I hope other people cited above will join the conversation, and we might be able at some point to organize a virtual meeting to discuss how to work together and achieve something.

1 Like
7

Hi Lorenzo,
this is an excellent point; I have found myself a bit lost among different approaches. I tried getting a bit more involved into Pangeo, but information about how to contribute and use Pangeo resources has proven a bit difficult for me (most probably my fault being new to the EO community). I found Wekeo as a good way to start hands on on cloud-computing and this particular forum (discourse.pangeo) is very useful.
Are you compiling a list of interested people (I’m interested!)? We could start with a short virtual meet and greet, to get a first impression of the different views; that should shed some light on the landscape we stand and hopefully we can come up with options for ways forward.
If a good hearted ambassador from our US friends could join (meet and greet) and share some toughts on their experience, would help a long way!
Best, Mario

2 Likes
8

@mechev nice to meet you and welcome on board!

The first step that I had in mind was creating an agorá to let Europeans know each other and eventually help each other.
Unfortunatelly, as you can see, the response isn’t too enthusiastic, but great things from time to time don’t start in the way one could have been expected; let’s see and in any case, never say never.

I agree with you that organising a virtual meeting would be nice and useful; eventually, we could ask @rabernat (or someone else that wants to) to give us a sort of overview of Pangeo and the vision for the future. If he or anyone from the USA would accept the invitation, we should organize in a time that’s best for him/her. During the Dask summit 2021, I woke up at 2 a.m, and it has not been a nice experience :wink: What about organising it within 2/3 weeks? Just to let people knowing the date and spread the voice. An informal virtual coffee idea would be the best; I’m not too keen on formalism and super structured events, especially in the beginning.

I agree with you that WEKEO and DIAS should be definitely part of the discussion, especially as we can’t ignore that they are a big component of the EU ecosystem. However, my personal opinion is that the new hot topic are the DigitalTwins.
Right now, I’m working to understand better where the button rooms are and how to let them be more involved in the Pangeo Community. But, definitely, we have to face the point that there are many other forums where the EO community is used to meet and, especially here in Europe, other approaches are more in used and push than the Pangeo one. But this could be another good point of the meeting.

2 Likes
9

Hi Lorenzo,

I totally agree with your proposed timming, last week of June or first week of July work well for me. I agree that the DigitalTwins is indeed a hot topic; being more specific though, it totally depends of proper model improvements e.g. via data assimilation. I see the cloud infrastructure as a mean to both fit “big models” and also to process data on the spot.
So I think we can keep it open to more than one community: DigitalTwins, Data Assimilation, Quality Control Studies, Citizen Science Experiments, etc.

I took the liberty to setup a doodle, please share it with interested parties; the proposed times are my first iteration, trying to accomodate also a decent time for the US, feedback on this or the dates is welcome!

Best, Mario

3 Likes
10

Hi there,

I have to say I’m a little disappointed by the lack of answer here… I’m sure a lot of people would like some European meeting, or have a stronger community. I’m going to cc again people!

@tinaok @apatlpo @jlesommer @auraoupa @tinaok @willirath @lesteve @fbriol @RPrudden @Theo_McCaie @jacobtomlinson

1 Like
11

At least some one else share my opinion :wink:

12

Hi all,

thanks for the heads up @geynard !

I have just filled up the doodle poll opened by @mechev. A discussion in the coming weeks would be good indeed.

Just adding my two cents on the overall context of the Pangeo Community in Europe :

  • it seems to me that we don’t have as active a community of Pangeo developers on this side of the Atlantic Ocean as compared to the US, but we have a large community of users. not sure why but this is probably a dimension worth discussing together.
  • an important element of context in Europe is indeed the implementation of the DIAS portals and the EU vision on the Digital Twins. We could think of organising a Pangeo event in this context. This would require liaising with the right people and the right projects. I am not sure where to start from though. We could discuss it during our meeting.
  • another important element of context is how HPC centers start to see the Pangeo software stack as a viable solution for broadening their user base and develop interactive computing services. I think it should be possible to organise a European Pangeo community event under the auspices of a prominent supercomputing center (as for instance one of the PRACE centers). I think there are some options in France that could be discussed.

Let’s discuss it together !

2 Likes
13

Thanks for the ping @geynard, I always forget to keep track of the various discourse boards.

I would be keen to participate in European Pangeo meetings/conferences/workshops. My focus these days since leaving the Met Office has expanded from helping the geosciences community to helping the Dask community as a whole.

Always happy to give talks, and generally help the community.

2 Likes
14

Thanks for pinging again @geynard and apologizes for the slow reply.

I concur with the general statement that the ratio developers to users appears low on the European side.
Brainstorming how we could remedy this at the European would be a valuable effort and I agree a quick chat about this prior to any other action would be helpful.
It would helpful however to assemble in a shared document a summary of the avenues to investigate (these are scattered across this growing discussion at the moment) such that we can think about it before hand.
Who would be up for the task?

My take otherwise is that while I have been able to broadcast the use of the pangeo ecosystem within my research group (students, postdocs, 1/2 engineer), I and my group have not reached the level where we actively contribute to the development of components of the ecosystem. We are able to report issues and propose very minor developments but nothing else. At the lab level, things are a bit difficult, not to say frustrating, with a lot of researcher spending most of their energy on getting high impact papers out with little investment in community tools.
I try to stay optimistic however: there is a whole culture and practices to adopt and this takes a significant amount of time (quantified maybe has units of student if not researcher turnover time?).
I am very interested in understanding where are the good resources in order to acquire/follow good practices and what are the best ways to broadcast/stimulate their adoption (if that makes any sense … :thinking: ), I am sure this has been explored by the Pangeo folks.

I won’t be able to invest myself too much in the effort but can help to some extent.

2 Likes
15

Here a draft for the topics cited up to now and that would be nice to discuss all together.

Feel free to modify, add points, reorganize …

2 Likes
16

Hello @pl.marasco . Admire your drive on this one. I’m part of the UK Met Office team and I’m keen to discuss Pangeo strategy in a Europe context.

2 Likes
17

You got me @geynard , good work! Thanks for the ping.

1 Like
18

Hi all, I would be happy to join a PANGEO - Europe discussion, the dates you proposed @mechev are all fine for me ! Talk to you soon, Aurélie

1 Like
19

Hi everyone, and thank you for pinging me. It was unfortunate that we couldn’t organise Pangeo European meeting on 2020. Thank you very much @pl.marasco for your great initiative! I support the idea on finding the funding as well. I’ll try to ping some people who may be interested in this virtual meet up. Looking forward to seeing you all.

1 Like
20

Thank you @tinaok for sharing the info. I’d be happy to be part of the discussion.
Kevin