[Feature]: Consider adding vertical regridding feature to reconstruct pressure from hybrid #446
Comments
|
Hey @jasonb5, can you follow the "Describe the solution you'd like" and see what we should do?
|
|
@tomvothecoder I'll check this out and see if I can find anything existing otherwise I'll go off the original plan I had for this in #388. |
|
Thanks @jasonb5! |
|
I would happily merge this in cf-xarray. We already support a few ocean parametric coordinates. The code would go here https://github.com/xarray-contrib/cf-xarray/blob/c36a3687b3820176c394580f86a67ca53b9cadbd/cf_xarray/accessor.py#L2507 |
|
Hey @dcherian, yes @jasonb5 came across that feature in cf-xarray! He mentioned it might be a good opportunity to make a PR to cf-xarray to support decoding other parametric vertical coordinates. Ideally, we'd like to have this feature out relatively soon so we can start refactoring some other packages with it. |
Yes, at the moment I just release when there's some nice new feature. |
|
Hey @jasonb5 and @chengzhuzhang, I found these features from
Also MetPy has the |
|
In testing
This is a clunky function (though it so far appears to be working on CMIP data...). I am absolutely sure you developers can come up with a better method for interpreting the hybrid formula on the fly, but I thought I'd put it here in case it is useful. I also recognize using def interprete_formula(f):
operator_chars = ['*', '(', ')', '+', '-', '/']
# first find/remove all parenthesis that are specifying axes
# p(n,k,j,i) = ap(k) + b(k)*ps(n,j,i) --> p = ap + b*ps
# get formula indices corresponding to a left hand parenthesis (
pinds = [i for i in range(len(f)) if f[i] == '(']
# initialize list to replace sections of formula (e.g., '(i,j,k)' --> '')
replace = []
# loop over left hand parenthesis instances
for p in pinds:
pref = '' # initialize parenthesis string
stopsearch = False # set stop flag to false
# loop from left hand parentheis to end of formula
for i in range(p, len(f)):
# if stopsearch, continue
if stopsearch:
continue
# get indexed character
c = f[i]
# check for close parenthesis )
# if closed, stopsearch
if c == ')':
pref += c # add character to parenthesis string
stopsearch = True
else:
pref += c # add character to parenthesis string
# check for arithmetic operators inside parenthesis string
ocs = [c for c in pref[1:-1] if c in operator_chars]
# if no arithmetic operators, queue parenthesis string for removal
if len(ocs) == 0:
replace.append(pref)
# remove all parenthesis strings with no arithmetic operators
for r in replace:
f = f.replace(r, '')
# further process formula
fout = '' # initialize blank formula
vid = '' # initialize blank variable
lhs = f.split('=')[0] # get lhs
f = f.split('=')[1] # get rhs
# loop over each char
for s in f:
# ignore blank spaces
if s == ' ':
continue
# if arithmetic operator, need to add to formula
if s in operator_chars:
# if a variable is not empty, write it out first
if len(vid) > 0:
fout = fout + 'ds["' + vid + '"]'
vid = '' # reset variable accumulator
# write out arithmetic operator
fout = fout + s
else:
# char must be part of a variable - add to variable accumulator
vid += s
# finished looping over all characters
# write out variable to formula
fout = fout + 'ds["' + vid + '"]'
# add lhs
fout = lhs + ' = ' + fout
# return formula
return fout |
tomvothecoder
modified the milestones:
FY24Q1 (10/01/23 - 12/31/23),
FY24Q2 (01/01/24 - 03/31/24)
tomvothecoder
modified the milestones:
FY24Q1 (10/01/23 - 12/31/23),
FY24Q2 (01/01/24 - 03/31/24)
tomvothecoder
modified the milestones:
FY24Q4 (07/01/24 - 09/30/24),
FY24Q3 (4/01/24 - 06/30/24)
tomvothecoder
modified the milestones:
FY24Q3 (4/01/24 - 06/30/24),
FY24Q4 (07/01/24 - 09/30/24)
|
@pochedls @tomvothecoder Created PR in the cf_xarray repo xarray-contrib/cf-xarray#528. |
|
I think I have this started correctly. @jasonb5 – would I pulled your PR (in my normal xcdat environment) import cf-xarray and Syntax for testers (i.e., note to self): |
SummaryI made some progress on this. The main finding is that models with the formula I've broken the problem into two main chunks ("Decode Error" and "Silent Failures"): Decode ErrorThe following models produce an error ( UKESM* and HadGEM3-GC31-LL produce Silent failuresA number of models silently fail (i.e., CESM2-WACCM and CESM2 also silently fail with IPSL-CM6A-LR also silently fails. It has vertical coordinate presnivs with Dataset ListingsDatasets with no apparent problem
Dataset with an error on
|
|
Now that is amazing. Given
perhaps we can check units to decide which representation to use. |
|
FYI – I had focused on testing atmospheric vertical coordinates. Ideally we could get some help testing ocean coordinates (@durack1?); otherwise, this might take awhile to stress test. Although I think CMIP data I've used is mostly on depth coordinates: are there CMIP ocean variables that are typically on coordinate systems that would need to be "decoded" (ocean sigma, s-coordinate, etc.)? |
tomvothecoder
modified the milestones:
FY24Q4 (07/01/24 - 09/30/24),
FY25Q1 (10/01/24 - 12/31/24)
|
@pochedls Can you check these files again, the current implementation will check that the dataset contains either one or the other form of As for xCDAT's support, currently |
|
@jasonb5 - I am wondering if I am looking at the right thing. I grabbed the latest cf-xarray and nothing is working (with the same code). One example, from a model that worked before: I'm working with cf-xarray main (15259).
|
|
@pochedls That error looks like it's from an old version of the code prior to my PR. That line is no longer there on |
|
@jasonb5 – sorry about that. I think I was working with a imported/cached cf-xarray import. The code above is working, but
|
tomvothecoder
modified the milestones:
FY25Q1 (10/01/24 - 12/31/24),
FY25 Q2 (01/01/25 - 03/31/25)
tomvothecoder
added
project: seats-fy25
|
@jasonb5 – regarding the resulting
It's almost like But I think we expect a dataarray object that has the form |
|
@pochedls That's a bug, I'll put a fix into cf_xarray for that. @tomvothecoder @lee1043 @chengzhuzhang As for integrating this into xCDAT, currently it's on the user to decode the variable then pass it to the method. We could add an option for Does that seem useful or fit into any workflow. I'm open to any other ideas of bringing this into xCDAT. |
|
@pochedls I've put in a fix for issue above (xarray-contrib/cf-xarray#561). |
|
Issued a release with that fix. Thanks @jasonb5 |
|
Thanks for getting this upgraded @jasonb5 and @dcherian! This version appears to be more robust with 10 datasets working that were not working before. Residual issues:
|
|
@pochedls Great to hear! I'll look at the remaining failures. |
tomvothecoder
modified the milestones:
FY25Q2 (01/01/25 - 03/31/25),
FY25Q3 (04/01/25 - 06/30/25)
Is your feature request related to a problem?
Related to #45 and #388.
Discussed in #440
Originally posted by tomvothecoder March 27, 2023
Hi @xCDAT/core-developers or anybody in the xCDAT community. Have any of you come across a package that implements an xarray-based function similar to
cdutil.vertical.reconstructPressureFromHybrid? I am working on refactoringe3sm_diagsande3sm_to_cmipto use xarray/xCDAT and there are references to this function that I need to replace. Thanks!Docstring:
Examples Usages:
Describe the solution you'd like
First, check of any other xarray-based package includes this feature.
cdutil.vertical.reconstructPressureFromHybrid)Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: