Metview regrid failing

[deepgabani8]

What happened?

Actual Results:
Upon running this:

import metview as mv

fs = mv.bindings.Fieldset(path='201201010000_FH-0.gb')
fieldset = mv.regrid(data=fs, grid=[0.1, 0.1])  # Error line.

Getting this:

Expected Results:
Create a fieldset with regulat_ll grid.

Additional Information:

• Here is the data-file '201201010000_FH-0.gb' dataset:
  

• The data is ECMWF HRES historical of 2012. It is in reduced_gg grid-type with 640 GRIB_N.

• metview's regrid is not able to convert this into a regular latlon grid.

• Interestingly, when I do the same with 2017 year's data, which is with 1280 GRIB_N, metview regrid is working as expected. Here is a data-file of 201701010000_FH-1.gb:
  

What are the steps to reproduce the bug?

Running this:

import metview as mv

fs = mv.bindings.Fieldset(path='201201010000_FH-0.gb')
fieldset = mv.regrid(data=fs, grid=[0.1, 0.1])  # Error line.

Version

v5.19.1

Platform (OS and architecture)

Linux test-weather-sp 5.10.0-22-cloud-amd64 #1 SMP Debian 5.10.178-3 (2023-04-22) x86_64 GNU/Linux

Relevant log output

No response

Accompanying data

No response

Organisation

Google

[iainrussell]

Hello @deepgabani8,

Thanks for the report. I tried this with an N640 GRIB from 2012 and it worked ok for me.

In your report, you show an xarray - is your original file a GRIB file or a NetCDF? I notice that in the xarray dump, the longitudes go up to 520, which looks suspicious to me. Did you just convert to xarray in order to show the metadata?

If you are using GRIB, you could set this env var before starting your Python program (or set it in your Python program before importing metview):

export METVIEW_PYTHON_DEBUG=1

This should give more output.

It's also good to do this:

print(mv.version_info())

Thanks,
Iain

[deepgabani8]

Thank you @iainrussell,

The original file is GRIB.
Logging more info doesn't tell much about the cause.

> mv.regrid(data=fs, grid=[0.1, 0.1])
event - INFO   - 20230630.071324 - Got connection
event - INFO   - 20230630.071324 - incoming host is localhost (127.0.0.1)
event - INFO   - 20230630.071324 - Register: /Process@638197/PythonScript, line 0:3 jupyter test-weather-sp 638197 as ref 3
event - INFO   - 20230630.071324 - Alive value for /Process@638197/PythonScript, line 0:3 : on 
event - INFO   - 20230630.071324 - Starting service: regrid
event - INFO   - 20230630.071324 - With command    : " /home/jupyter/anaconda3/envs/weather-tools/lib/metview-bundle/bin/Regrid"
event - INFO   - 20230630.071324 - Got connection
event - INFO   - 20230630.071324 - incoming host is localhost (127.0.0.1)
event - INFO   - 20230630.071324 - Register: regrid jupyter test-weather-sp 638249 as ref 5
event - INFO   - 20230630.071324 - regrid : Timeout = 300 Alive = 0, Max = 1
event - INFO   - 20230630.071324 - Maximum value for regrid : 1 
event - INFO   - 20230630.071324 - Got connection
event - INFO   - 20230630.071324 - incoming host is localhost (127.0.0.1)
event - INFO   - 20230630.071324 - Register: regrid@638250 jupyter test-weather-sp 638250 as ref 6
event - INFO   - 20230630.071324 - Starting service: pool
event - INFO   - 20230630.071324 - With command    : " /home/jupyter/anaconda3/envs/weather-tools/lib/metview-bundle/bin/pool"
event - INFO   - 20230630.071324 - Got connection
event - INFO   - 20230630.071324 - incoming host is localhost (127.0.0.1)
event - INFO   - 20230630.071324 - Register: pool jupyter test-weather-sp 638252 as ref 8
event - INFO   - 20230630.071324 - pool : Timeout = 86400 Alive = 0, Max = 0
pool - INFO   - 20230630.071324 - [regrid@638250] - Object /Process@638197/PythonScript, line 0:2 [REGRID] not cached
Regrid::serve in
Cache file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat does not exist
Creating cache file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat
CacheManager creating file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat
event - INFO   - 20230630.071325 - Closing service regrid@638250
event - INFO   - 20230630.071325 - 1 outstanding replies
event - INFO   - 20230630.071325 - Service regrid@638250 died before completing the request
event - INFO   - 20230630.071325 - AbortIfBatch: not batch, no abort!
Python - ERROR  - 20230630.071325 - Line 0 in 'PythonScript': Service regrid@638250 died before completing the request
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 mv.regrid(data=fs, grid=[0.1, 0.1])

File ~/anaconda3/envs/weather-tools/lib/python3.8/site-packages/metview/bindings.py:1274, in make.<locals>.wrapped(*args, **kwargs)
   1271     pass  # throw Exception
   1273 val = lib.p_result_as_value()
-> 1274 return value_from_metview(val)

File ~/anaconda3/envs/weather-tools/lib/python3.8/site-packages/metview/bindings.py:1228, in value_from_metview(val)
   1226 retval = vr.translate_return_val(val)
   1227 if isinstance(retval, Exception):
-> 1228     raise retval
   1229 return retval

Exception: Metview error: Service regrid@638250 died before completing the request

[iainrussell]

Thanks - I agree, it doesn't help too much! Could you additionally set the environment variable MIR_DEBUG=1 before importing metview? That should add more detail to the output.

[deepgabani8]

setting MIR_DEBUG=1 before import metview also gives the exact same output as above :(

[iainrussell]

Sorry, you may have to also set MARS_DEBUG=1 in order to see it.

[deepgabani8]

Yes. Now getting a detailed output:

> mv.regrid(data=fs, grid=[0.1, 0.1])
Python - DEBUG  - 20230703.065537 - Calling "localhost" port 43415
Python - DEBUG  - 20230703.065537 - Connected to address 127.0.0.1
Python - DEBUG  - 20230703.065537 - Connected to server
Python - DEBUG  - 20230703.065537 - User is [jupyter]
event - INFO   - 20230703.065537 - Got connection
Python - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REGISTER ?
Python - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
Python - DEBUG  - 20230703.065537 - encode_request : xdr_netblk ALIVE ?
event - INFO   - 20230703.065537 - incoming host is localhost (127.0.0.1)
Python - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
Sending to regrid
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
REGRID,
    DATA       = (GRIB,
    PATH       = '/home/jupyter/weather-tools/201201010000_FH-0.gb',
    TEMPORARY  = 0,
    OFFSET     = 0/4284072/8568144/12852216/17136288/21420360,
    LENGTH     = 4284072/4284072/4284072/4284072/4284072/4284072,
    _CLASS     = GRIB,
event - DEBUG  - 20230703.065537 - decode_request : --------> REGISTER ?
    _NAME      = '/Process@698049/PythonScript, line 0:0',
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
    _PATH      = '/home/jupyter/weather-tools/201201010000_FH-0.gb'
REGISTER,

    USER       = jupyter,
),
    NAME       = '/Process@698049/PythonScript, line 0:3',
    GRID       = 0.1/0.1,
    HOST       = test-weather-sp,
    _VERB      = REGRID,
    PID        = 698049

    _APPL      = Python,
    _NAME      = '/Process@698049/PythonScript, line 0:2',
event - INFO   - 20230703.065537 - Register: /Process@698049/PythonScript, line 0:3 jupyter test-weather-sp 698049 as ref 3
    _CLASS     = REGRID,
    _MACRO     = '(null)',
    _PATH      = '/home/jupyter/weather-tools',
    _CWD       = '/home/jupyter/weather-tools'

Python - DEBUG  - 20230703.065537 - encode_request : xdr_netblk SERVICE REGRID
Python - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> ALIVE ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
ALIVE,
    ALIVE      = 1

event - INFO   - 20230703.065537 - Alive value for /Process@698049/PythonScript, line 0:3 : on 
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> SERVICE REGRID
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
SERVICE,
    USER_REF   = 94780229717160,
    TARGET     = regrid

REGRID,
    DATA       = (GRIB,
    PATH       = '/home/jupyter/weather-tools/201201010000_FH-0.gb',
    TEMPORARY  = 0,
    OFFSET     = 0/4284072/8568144/12852216/17136288/21420360,
    LENGTH     = 4284072/4284072/4284072/4284072/4284072/4284072,
    _CLASS     = GRIB,
    _NAME      = '/Process@698049/PythonScript, line 0:0',
    _PATH      = '/home/jupyter/weather-tools/201201010000_FH-0.gb'

),
    GRID       = 0.1/0.1,
    _VERB      = REGRID,
    _APPL      = Python,
    _NAME      = '/Process@698049/PythonScript, line 0:2',
    _CLASS     = REGRID,
    _MACRO     = '(null)',
    _PATH      = '/home/jupyter/weather-tools',
    _CWD       = '/home/jupyter/weather-tools',
    _CALLED_FROM_MACRO = 1

event - INFO   - 20230703.065537 - Starting service: regrid
event - INFO   - 20230703.065537 - With command    : " /home/jupyter/anaconda3/envs/weather-tools/lib/metview-bundle/bin/Regrid"
Regrid - DEBUG  - 20230703.065537 - Created PPROC backend MIR
Regrid - DEBUG  - 20230703.065537 - Post processing backend is MIR
Regrid - DEBUG  - 20230703.065537 - Calling "localhost" port 43415
Regrid - DEBUG  - 20230703.065537 - Connected to address 127.0.0.1
Regrid - DEBUG  - 20230703.065537 - Connected to server
event - INFO   - 20230703.065537 - Got connection
Regrid - DEBUG  - 20230703.065537 - User is [jupyter]
Regrid - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REGISTER ?
event - INFO   - 20230703.065537 - incoming host is localhost (127.0.0.1)
Regrid - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
Regrid - DEBUG  - 20230703.065537 - encode_request : xdr_netblk MAXIMUM ?
Regrid - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : --------> REGISTER ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
REGISTER,
    USER       = jupyter,
    NAME       = regrid,
    HOST       = test-weather-sp,
    PID        = 698102

event - INFO   - 20230703.065537 - Register: regrid jupyter test-weather-sp 698102 as ref 5
event - INFO   - 20230703.065537 - regrid : Timeout = 300 Alive = 0, Max = 1
event - DEBUG  - 20230703.065537 - encode_request : xdr_netblk SERVICE REGRID
event - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
Regrid - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : --------> MAXIMUM ?
Regrid - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
MAXIMUM,
    MAXIMUM    = 1

event - INFO   - 20230703.065537 - Maximum value for regrid : 1 
Regrid - DEBUG  - 20230703.065537 - decode_request : --------> SERVICE REGRID
Regrid - DEBUG  - 20230703.065537 - get service:
SERVICE,
    USER_REF   = 94780229717160,
    TARGET     = regrid,
    REQ_ID     = 4,
    SOURCE     = '/Process@698049/PythonScript, line 0:3',
    SOURCE_REF = 3

REGRID,
    DATA       = (GRIB,
    PATH       = '/home/jupyter/weather-tools/201201010000_FH-0.gb',
    TEMPORARY  = 0,
    OFFSET     = 0/4284072/8568144/12852216/17136288/21420360,
    LENGTH     = 4284072/4284072/4284072/4284072/4284072/4284072,
    _CLASS     = GRIB,
    _NAME      = '/Process@698049/PythonScript, line 0:0',
    _PATH      = '/home/jupyter/weather-tools/201201010000_FH-0.gb'

),
    GRID       = 0.1/0.1,
    _VERB      = REGRID,
    _APPL      = Python,
    _NAME      = '/Process@698049/PythonScript, line 0:2',
    _CLASS     = REGRID,
    _MACRO     = '(null)',
    _PATH      = '/home/jupyter/weather-tools',
    _CWD       = '/home/jupyter/weather-tools',
    _CALLED_FROM_MACRO = 1

Regrid - DEBUG  - 20230703.065537 - Calling "localhost" port 43415
Regrid - DEBUG  - 20230703.065537 - Connected to address 127.0.0.1
Regrid - DEBUG  - 20230703.065537 - Connected to server
Regrid - DEBUG  - 20230703.065537 - Cached user is [jupyter]
event - INFO   - 20230703.065537 - Got connection
Regrid - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REGISTER ?
Regrid - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - INFO   - 20230703.065537 - incoming host is localhost (127.0.0.1)
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
Regrid - DEBUG  - 20230703.065537 - encode_request : xdr_netblk FOLLOWUP ?
Regrid - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : --------> REGISTER ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
REGISTER,
    USER       = jupyter,
    NAME       = 'regrid@698103',
    HOST       = test-weather-sp,
    PID        = 698103

event - INFO   - 20230703.065537 - Register: regrid@698103 jupyter test-weather-sp 698103 as ref 6
Regrid - DEBUG  - 20230703.065537 - encode_request : xdr_netblk SERVICE FETCH
Regrid - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> FOLLOWUP ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
FOLLOWUP,
    USER_REF   = 94780229717160,
    TARGET     = regrid,
    REQ_ID     = 4,
    SOURCE     = '/Process@698049/PythonScript, line 0:3',
    SOURCE_REF = 3

event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> SERVICE FETCH
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
SERVICE,
    USER_REF   = 0,
    WAITMODE   = 1,
    TARGET     = pool

FETCH,
    NAME       = '/Process@698049/PythonScript, line 0:2',
    CLASS      = REGRID

event - INFO   - 20230703.065537 - Starting service: pool
event - INFO   - 20230703.065537 - With command    : " /home/jupyter/anaconda3/envs/weather-tools/lib/metview-bundle/bin/pool"
pool - DEBUG  - 20230703.065537 - Created PPROC backend MIR
pool - DEBUG  - 20230703.065537 - Post processing backend is MIR
pool - DEBUG  - 20230703.065537 - Calling "localhost" port 43415
pool - DEBUG  - 20230703.065537 - Connected to address 127.0.0.1
pool - DEBUG  - 20230703.065537 - Connected to server
event - INFO   - 20230703.065537 - Got connection
pool - DEBUG  - 20230703.065537 - User is [jupyter]
pool - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REGISTER ?
event - INFO   - 20230703.065537 - incoming host is localhost (127.0.0.1)
pool - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> REGISTER ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
REGISTER,
    USER       = jupyter,
    NAME       = pool,
    HOST       = test-weather-sp,
    PID        = 698105

event - INFO   - 20230703.065537 - Register: pool jupyter test-weather-sp 698105 as ref 8
pool - DEBUG  - 20230703.065537 - encode_request : xdr_netblk WANT ?
pool - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - INFO   - 20230703.065537 - pool : Timeout = 86400 Alive = 0, Max = 0
event - DEBUG  - 20230703.065537 - encode_request : xdr_netblk SERVICE FETCH
event - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
pool - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : --------> WANT ?
pool - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
WANT,
    NAME       = STATUS,
    TYPE       = MESSAGE

pool - DEBUG  - 20230703.065537 - decode_request : --------> SERVICE FETCH
pool - DEBUG  - 20230703.065537 - get service:
SERVICE,
    USER_REF   = 0,
    WAITMODE   = 1,
    TARGET     = pool,
    REQ_ID     = 7,
    SOURCE     = 'regrid@698103',
    SOURCE_REF = 6

FETCH,
    NAME       = '/Process@698049/PythonScript, line 0:2',
    CLASS      = REGRID

pool - INFO   - 20230703.065537 - [regrid@698103] - Object /Process@698049/PythonScript, line 0:2 [REGRID] not cached
pool - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REPLY ?
pool - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
event - DEBUG  - 20230703.065537 - decode_request : --------> REPLY ?
event - DEBUG  - 20230703.065537 - Service::ProcessIncoming(), dispatch request:
REPLY,
    USER_REF   = 0,
    WAITMODE   = 1,
    TARGET     = pool,
    REQ_ID     = 7,
    SOURCE     = 'regrid@698103',
    SOURCE_REF = 6

event - DEBUG  - 20230703.065537 - encode_request : xdr_netblk REPLY ?
event - DEBUG  - 20230703.065537 - encode_request : xdrrec_endofrecord
Regrid - DEBUG  - 20230703.065537 - decode_request : xdrrec_skiprecord
Regrid - DEBUG  - 20230703.065537 - decode_request : xdr_netblk
Regrid - DEBUG  - 20230703.065537 - decode_request : --------> REPLY ?
Regrid - DEBUG  - 20230703.065537 - get service:
REPLY,
    USER_REF   = 0,
    WAITMODE   = 1,
    TARGET     = pool,
    REQ_ID     = 7,
    SOURCE     = 'regrid@698103',
    SOURCE_REF = 6

Regrid::serve in
Cache file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat does not exist
Creating cache file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat
CacheManager creating file /tmp/cache/mir/weights/15/linear/R640-296ba3f6fbb8b96514a92b95da48e127-89.8924:-180:-89.8924:179.918/LL-0.1x0.1-90:0:-90:359.9-754582ce2e1a0b206b7c62a006a47cfb.mat
event - DEBUG  - 20230703.065538 - decode_request : xdrrec_skiprecord
event - DEBUG  - 20230703.065538 - decode_request : xdr_netblk
event - INFO   - 20230703.065538 - Closing service regrid@698103
event - INFO   - 20230703.065538 - 1 outstanding replies
event - INFO   - 20230703.065538 - Service regrid@698103 died before completing the request
event - INFO   - 20230703.065538 - AbortIfBatch: not batch, no abort!
event - DEBUG  - 20230703.065538 - encode_request : xdr_netblk REPLY ?
event - DEBUG  - 20230703.065538 - encode_request : xdrrec_endofrecord
event - DEBUG  - 20230703.065538 - encode_request : xdr_netblk MESSAGE DEATH
event - DEBUG  - 20230703.065538 - encode_request : xdrrec_endofrecord
Python - DEBUG  - 20230703.065538 - decode_request : xdrrec_skiprecord
Python - DEBUG  - 20230703.065538 - decode_request : xdr_netblk
pool - DEBUG  - 20230703.065538 - decode_request : xdrrec_skiprecord
pool - DEBUG  - 20230703.065538 - decode_request : xdr_netblk
pool - DEBUG  - 20230703.065538 - decode_request : --------> MESSAGE DEATH
Python - DEBUG  - 20230703.065538 - decode_request : --------> REPLY ?
pool - DEBUG  - 20230703.065538 - get service:
Python - DEBUG  - 20230703.065538 - get service:
MESSAGE
REPLY,

    USER_REF   = 94780229717160,
DEATH,
    TARGET     = regrid,
    NAME       = 'regrid@698103',
    REQ_ID     = 4,
    USER       = jupyter,
    HOST       = test-weather-sp,
    SOURCE     = '/Process@698049/PythonScript, line 0:3',
    PID        = 698103
    SOURCE_REF = 3,

    ERR_CODE   = -42,
    ERROR      = 'Service regrid@698103 died before completing the request'

Got reply from regrid
Python - ERROR  - 20230703.065538 - Line 0 in 'PythonScript': Service regrid@698103 died before completing the request
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 mv.regrid(data=fs, grid=[0.1, 0.1])

File ~/anaconda3/envs/weather-tools/lib/python3.8/site-packages/metview/bindings.py:1274, in make.<locals>.wrapped(*args, **kwargs)
   1271     pass  # throw Exception
   1273 val = lib.p_result_as_value()
-> 1274 return value_from_metview(val)

File ~/anaconda3/envs/weather-tools/lib/python3.8/site-packages/metview/bindings.py:1228, in value_from_metview(val)
   1226 retval = vr.translate_return_val(val)
   1227 if isinstance(retval, Exception):
-> 1228     raise retval
   1229 return retval

Exception: Metview error: Service regrid@698103 died before completing the request```

[iainrussell]

Thanks for the output. I still can't replicate it, so could you make your GRIB file available to me please, then I can try it out and get to the bottom of this!

[deepgabani8]

Thank you @iainrussell .
Here - https://github.com/deepgabani8/temp/blob/master/grib-with-metview-error/201201010000_FH-0.gb is the grib file that I am trying.

[iainrussell]

Hi @deepgabani8, many thanks for the GRIB file. I had a little time to look at it, and I can reproduce the error. It looks as though this file has been manipulated from what ECMWF produced. When I retrieve the equivalent file from our MARS archive, the longitudes go from 0 to 360 (approx), whereas your file goes from approx -180 to +180, and even the bounding latitudes are different at the last decimal place.

Now, I don't think the file is incorrect, but it does seem that our software cannot handle it properly - I'm still looking in the debugger to understand what's going on. In the meantime, if you can tell me anything about where the file came from and what sort of processing it's received, that could be helpful.

Thanks,
Iain

[deepgabani8]

Hey @iainrussell , The file hasn't received any processing. (The grib data file is in the Gaussian grid and I want to re-grid it into a regular lat-lon.)

The actual big grib file has data of an entire month, and I've splitted it by one timestamp & forecast-hour but I'm sure it just divided the monthly data into smaller chunks. (I've also tried the mv.regrid on the monthly grib file, but got the same error.)

[iainrussell]

Hi @deepgabani8,

OK, then it remains a little bit of a mystery! In any case, I did track it down the problem to one particular library in our software stack, and it is possible to avoid it by choosing a different interpolation method. The default method for most parameters is 'linear', but if you choose something like 'nearest_neighbour' or 'k_nearest_neighbours', the code path will be different, and you will get a decent result. Here's an example:

fieldset = mv.regrid(
    grid          = [0.1,0.1],
    interpolation = "k_nearest_neighbours",
    data          = fs
    )

It will be a bit slow on the first field it interpolates, but much much faster on subsequent fields because it caches the results of the heavy computation.

I hope this will help you get further!

[deepgabani8]

Yes, this works.
Thank you @iainrussell . You are a life saver :)

[deepgabani8]

Closing this issue for now. Iain, maybe you can create a new issue for the fix referencing this issue.