Strange error in icepyx

Science ICESat-2
1

I have been using Icepyx for sometime and it has been working great. Thanks to the efforts of all the developers and contributers.

Today I came across a strange error while choosing a specific ground track for download using icepyx. I hope someone can look into this; I am really sorry if this turns out to be something silly that I have missed but several trial-and-error attempts have helped me understand there is some problem with the date_range and tracks combination.

Code:

short_name = ‘ATL03’
spatial_extent = ‘/home/anirudha/Desktop/antarctic_gl_timeseries/data/fimbul_wgs84.shp’

Case1 : (Working)

region_a = ipx.Query(short_name, spatial_extent, date_range=[‘2018-02-20’, ‘2019-02-28’], tracks=‘1349’ )

Case2 : (Not working)

region_a = ipx.Query(short_name, spatial_extent, date_range=[‘2018-02-20’, ‘2019-04-28’] , tracks=‘1349’)

Case 3 : (Working)

region_a = ipx.Query(short_name, spatial_extent, date_range=[‘2018-02-20’, ‘2019-04-28’]) #No tracks mentioned

Screenshot from 2020-10-07 17-29-46
Screenshot from 2020-10-07 17-29-461398×557 92 KB

As we can see in the screenshot, the first case works well with a specific date range and 1 track (1349). The 2nd case breaks when the date range is increased by 2 months, track remains the same.

In the third case, the date range is same as second, but no filter for tracks is mentioned (tracks=None). And this works as well.

I wonder what is going wrong in case 2. Can someone help?

To be able to recreate the same situation, I am attaching the shapefile that I have used.

~ Anirudha

2

Hi @anirudhavm, thanks for posting such a detailed report! I’ve managed to reproduce the exact same error on my local machine. My initial thought was that there’s a timeout from fetching too much data, but it’s strange that Case 3 works when a reference ground track is not specified.

After some digging into the granules.py file, it appears to be an error from the json response that wasn’t surfaced correctly:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/icepyx/icepyx/core/granules.py in get_avail(self, CMRparams, reqparams)
    185             try:
--> 186                 if len(results["feed"]["entry"]) == 0:
    187                     # Out of results, so break out of loop

KeyError: 'feed'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-5-f8cc1c75e464> in <module>
      1 region_a = ipx.Query(
----> 2     short_name, spatial_extent, date_range=["2018-03-20", "2019-04-28"], tracks="1349"
      3 )

~/icepyx/icepyx/core/query.py in __init__(self, dataset, spatial_extent, date_range, start_time, end_time, version, cycles, tracks, orbit_number, files)
    167             self._reqparams.build_params()
    168             # update the list of available granules
--> 169             self.granules.get_avail(self.CMRparams, self.reqparams)
    170 
    171     # ----------------------------------------------------------------------

~/icepyx/icepyx/core/granules.py in get_avail(self, CMRparams, reqparams)
    188                     break
    189             except KeyError:
--> 190                 raise ValueError(results)
    191 
    192             # Collect results and increment page_num

ValueError: {'errors': ["orbit_number must be an number or a range in the form 'number,number'.", '[1550,2937] is not a valid class java.lang.Double']}

Perhaps @JessicaS11 has more thoughts on what this means and how to solve it? We probably need to file a bug report at https://github.com/icesat2py/icepyx/issues/new for this. In the meantime, perhaps you could pull out the one track you need using the Case 3 code somehow.

If you don’t mind, what might be helpful is if you can describe the scientific goal you’re trying to solve and we could suggest a better answer to your problem, and/or improve the usability of icepyx for users like you!

3

Thanks @weiji14 for confirming the error/bug.

In the case posted above, my intention was to download data from different time periods only for a particular ground reference track, and plot a time series.

Since this didn’t work, I have downloaded it directly from the NSIDC portal. Therefore, I have got what I wanted to achieve. But the primary intention of the post was to ensure that it was a legitimate error and report it in case it was one.

4

Thanks for posting your issue here @anirudhavm, and I’m glad that you found a short-term solution and appreciate you letting us know about it. Thanks for reproducing/digging into it @weiji14!

@tsutterley, have you ever seen this? I haven’t worked much with making requests using tracks. What doesn’t make sense to me is how the date range itself seems to be influencing the appearance of the error, but the error itself seems to stem from the formatting of the request.

5

This is interesting and I’m pretty sure it comes down to me being unable to read. The CMR orbit_number parameter takes either a single value or a range (not a list of orbits). Not sure how this was working previously in the year and isn’t working now. I’ll have to rethink how to do this if people still want it as a feature.

I wrote NASA Earthdata to see if the parameter could be changed to have “AND” support (that way it doesn’t break their range functionality but adds our list functionality).

6

Got the email today that the CMR Operations team may open the orbit_number parameter to accept a list of entries.

7

Awesome - thanks for keeping us updated. Please let us know once you hear more from CMR Operations in case we need to revisit on the icepyx end. Either way it sounds like we should add a few tests. We can probably take @anirudhavm’s examples as a start. Anyone up for adding those tests? I’d be happy to help you through the process if you’re new to unit testing!