Thank you for the idea @jlesommer an HPC workshop sounds like a great idea. I wonder if @kmpaul or @andersy005 know anyone that might have bandwidth to organize a group.
Regarding timezones, one of the objectives of workshops is that they can be organized and scheduled independently.
Actually, there might be enough interest in a Dask-in-HPC-in-France workshop that could be in a convenient timezone and in french language if people preferred.
The objective of workshops is to provide remote conference infrastructure to groups to self-organize. Dask has many small sub-communities. Our hope is that workshops provide a convenient context for those communities to organize.
Just noting that the pangeo workshop is hoping to have a section on Dask / Pangeo on HPC. It’ll only be ~12-15 minutes of the workshop though, and there’s likely enough interest in HPC to warrant spending more time on this in a second workshop.
@geynard Unfortunately I don’t have yet something presentable or I at least I don’t think is so interesting. Next time I’ll be in for sure.
I concur with Tom. @geynard,it sounds like there’s enough interest to justify having a dedicated Dask-in-HPC workshop, and I am happy to chair/co-chair this session.
@geynard, since the call for proposals will close this Sunday, March 21, do you have time (~30minutes) for us to meet today and decide on the theme of the workshop and format? (I will take care of the proposal submission over the weekend, and the logistics of the workshop.). If anyone else wants in on organizing, please let me know.
I confirm that @auraoupa and I would happily contribute to this workshop on Dask/HPC with a presentation (typically 15’). I should also be able to comment on a shared document describing the session over the week-end if you want.
1 Like
Thank you, @jlesommer! Here’s a link to a Google Doc: Dask-in-HPC workshop - Google Docs
@geynard, I just realized it’s past work hours in France. So, we don’t have to meet if you’ve started your weekend already :). Let’s coordinate via the online Google doc.
1 Like
I would not be surprised if the conference organizers extended the call for proposals by a week …
1 Like
Yeah sorry I was moving for the weekend so was not available. It would be good to have one more week to work on this! I’ll try to have a look at the document. Thanks @andersy005 and @jlesommer for your motivation!
Great, the submissions have been extended I’m getting a fairly good response both internally and externally to CSIRO. Just wanted to check could a 1-day hackathon of a specific topic be part of this?
On the CNES HPC, the feedback we can give is on using Dask with the tools developed for SWOT. We use Dask with the cluster to scale up some processing, such as generating simulated products with high-resolution models (LLC4320, NATL60), spectral analysis, and studying CalVal diagnostics. Dask is helping us greatly to scale up our processing.
@andersy005 I dropped a few thoughts in the Google sheet. I think there is a lot of potential material for this workshop! I like the format you propose with talks and a good part of discussion, even if I don’t know yet how this discussion will be driven and I don’t have experience with this.
I think the goal of this workshop would be to share experience, identify common problems and how to solve them, identify larger issues and most wanted features.
I’ve added a “Proposed talks” part on the Google sheet if anyone has something he’d like to share!
If you are referring to the workshop proposal, the last time I checked we didn’t have much content in our Pangeo workshop proposal (very rough draft) -;). I will check again.
@mrocklin, are you aware of any GPU-focused workshop at the Dask summit being organized? I’ve been thinking that in case there’s no dedicated GPU workshop, It’d be great to invite RAPIDS folks or people at Oak Ridge doing work with Dask on Summit to share some of their work in our Dask in HPC workshop. Let me know if you have any suggestions of people I should reach out to. Thanks!
I believe that there will be a RAPIDS workshop. It might still be good to reach out to the folks at Summit though, there is a lot of GPU work going on and I’m not sure if that group in particular is engaged.