PAVICS Web Processing Services

PAVICS allows access to a number of different WPS services via Birdhouse * Each ‘bird’ groups a set of processing tools

[1]:
from owslib.wps import WebProcessingService
import requests
from lxml import etree
import owslib
owslib.__version__

def parseStatus(execute):
    o = requests.get(execute.statusLocation)
    t = etree.fromstring(o.content)
    ref = t.getchildren()[-1].getchildren()[-1].getchildren()[-1].get('{http://www.w3.org/1999/xlink}href')

    return ref

One suite of WPS tools for netcdf files resides in ‘Hummingbird’

For metadata use GetCapabilities

[2]:
# Hummingbird WPS url
wps_url = 'https://pavics.ouranos.ca/twitcher/ows/proxy/hummingbird/wps'
# connection
wps = WebProcessingService(url=wps_url)
# print wps title
print(wps.identification.title)
Hummingbird 0.5_dev

PAVICS/Hummingbird has lots of WPS services

Let’s keep it simple with ‘ncdump’

  • Print info on WPS inputs needed
[4]:
# ncdump
proc_name = 'ncdump'
process = wps.describeprocess(proc_name) # get process info
print(process.abstract)
print("Inputs:")
for inputs in process.dataInputs:
    print(' * ', inputs.identifier)
Run ncdump to retrieve NetCDF header metadata.
Inputs:
 *  dataset
 *  dataset_opendap

Create WPS input - Python list

[6]:
myinputs = []
myinputs.append(('dataset_opendap',nc_url)) # inputs : use opendap link of a single netcdf file from catalogue search to erun ncdump
print(myinputs)
[('dataset_opendap', 'https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/nrcan/nrcan_canada_daily/tasmin/nrcan_canada_daily_tasmin_2013.nc')]

Execute the WPS

The execution is asynchronous, meaning that it does not automatically return the output. The response of the server is only a message saying that the request was accepted.

[7]:
print(proc_name)
execute = wps.execute(proc_name, myinputs)
ncdump
[8]:
from lxml import etree
print(etree.tostring(execute.response).decode())
<wps:ExecuteResponse xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsExecute_response.xsd" service="WPS" version="1.0.0" xml:lang="en-US" serviceInstance="https://pavics.ouranos.ca:443/wps?service=WPS&amp;request=GetCapabilities" statusLocation="https://pavics.ouranos.ca:443/wpsoutputs/hummingbird/94d27a14-c36a-11e9-be78-0242ac120014.xml">
  <wps:Process wps:processVersion="4.4.1.1">
    <ows:Identifier>ncdump</ows:Identifier>
    <ows:Title>NCDump</ows:Title>
    <ows:Abstract>Run ncdump to retrieve NetCDF header metadata.</ows:Abstract>
  </wps:Process>
  <wps:Status creationTime="2019-08-20T16:50:14Z">
    <wps:ProcessSucceeded>PyWPS Process NCDump finished</wps:ProcessSucceeded>
  </wps:Status>
  <wps:ProcessOutputs>
    <wps:Output>
      <ows:Identifier>output</ows:Identifier>
      <ows:Title>NetCDF Metadata</ows:Title>
      <ows:Abstract>NetCDF Metadata</ows:Abstract>
      <wps:Reference xlink:href="https://pavics.ouranos.ca:443/wpsoutputs/hummingbird/94d27a14-c36a-11e9-be78-0242ac120014/nc_dump_DP5lRn.txt" mimeType="text/plain"/>
    </wps:Output>
  </wps:ProcessOutputs>
</wps:ExecuteResponse>

Get the result

To actually parse the output, we must first make sure that the process completed server-side. execute.checkStatus() will poll the server and update its response.

[21]:
execute.checkStatus()
print("Status: ", execute.status)
print(execute.statusLocation)
Status:  ProcessSucceeded
https://pavics.ouranos.ca:443/wpsoutputs/hummingbird/6ca0018a-bb58-11e8-91d6-0242ac12000d.xml

Then we can check the actual output of the process, stored as a list in the processOutputs attribute. In the case where the output is a reference to a file, we can get it using the reference attribute. The method retrieveData let’s us fetch and retrieve the content of the file.

[23]:
ref = parseStatus(execute)
print('Output reference :\n*', ref)

r = requests.get(ref)
print('\nNCDUMP results :\n',r.text)

#out = execute.processOutputs[0]
#print("Output reference: ", out.reference)
#data = out.retrieveData()
#print("Data: ", data.decode())
Output reference :
* https://pavics.ouranos.ca:443/wpsoutputs/hummingbird/6ca0018a-bb58-11e8-91d6-0242ac12000d/nc_dump_8MSe6y.txt

NCDUMP results :
 netcdf nrcan_canada_daily_tasmin_2013.nc {
dimensions:
        time = UNLIMITED ; // (365 currently)
        lat = 510 ;
        lon = 1068 ;
        ts = 3 ;
variables:
        float lon(lon) ;
                lon:units = "degrees_east" ;
                lon:long_name = "longitude" ;
                lon:standard_name = "longitude" ;
                lon:axis = "X" ;
                lon:_ChunkSizes = 1068 ;
        float lat(lat) ;
                lat:axis = "Y" ;
                lat:units = "degrees_north" ;
                lat:long_name = "latitude" ;
                lat:standard_name = "latitude" ;
                lat:_ChunkSizes = 510 ;
        short ts(ts) ;
                ts:_FillValue = -32767s ;
                ts:_ChunkSizes = 3 ;
        short time(time) ;
                time:axis = "T" ;
                time:units = "days since 1950-01-01 00:00:00" ;
                time:long_name = "time" ;
                time:standard_name = "time" ;
                time:calendar = "gregorian" ;
                time:_ChunkSizes = 1 ;
        short time_vectors(time, ts) ;
                time_vectors:_ChunkSizes = 1, 3 ;
        float tasmin(time, lat, lon) ;
                tasmin:long_name = "air_temperature" ;
                tasmin:standard_name = "air_temperature" ;
                tasmin:units = "K" ;
                tasmin:_FillValue = 9.96921e+36f ;
                tasmin:_ChunkSizes = 31, 102, 267 ;

// global attributes:
                :Conventions = "CF-1.5" ;
                :title = "NRCAN 10km Gridded Climate Dataset" ;
                :history = "2016-01-05T16:30:06: Convert from original format to NetCDF" ;
                :institution = "NRCAN" ;
                :source = "ANUSPLIN" ;
                :redistribution = "Redistribution policy unknown. For internal use only." ;
                :DODS_EXTRA.Unlimited_Dimension = "time" ;
}