Accessing PAVICS THREDDS Server

The THREDDS data storing netCDF file on PAVICS has some public and private directories. Data from public directories can be accessed anonymously, while data from private directories require authentication. This notebook shows how to access public and private data on the THREDDS server.

The PAVICS THREDDS server has a testdata/ folder, in which we store test datasets to validate process requests. Within that directory is a secure/ folder whose file access requires authentication (to be done).

First let’s just open an unsecured link.

[1]:
import xarray as xr
xr.set_options(display_style="text")  # comment out for html style, text style simpler for automated testing

url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/testdata/ta_Amon_MRI-CGCM3_decadal1980_r1i1p1_199101-200012.nc"
ds = xr.open_dataset(url)
ds
[1]:
<xarray.Dataset>
Dimensions:    (bnds: 2, lat: 160, lon: 320, plev: 23, time: 120)
Coordinates:
  * time       (time) datetime64[ns] 1991-01-16T12:00:00 ... 2000-12-16T12:00:00
  * plev       (plev) float64 1e+05 9.25e+04 8.5e+04 7e+04 ... 200.0 100.0 40.0
  * lat        (lat) float64 -89.14 -88.03 -86.91 -85.79 ... 86.91 88.03 89.14
  * lon        (lon) float64 0.0 1.125 2.25 3.375 ... 355.5 356.6 357.8 358.9
Dimensions without coordinates: bnds
Data variables:
    time_bnds  (time, bnds) datetime64[ns] ...
    lat_bnds   (lat, bnds) float64 ...
    lon_bnds   (lon, bnds) float64 ...
    ta         (time, plev, lat, lon) float32 ...
Attributes:
    institution:                     MRI (Meteorological Research Institute, ...
    institute_id:                    MRI
    experiment_id:                   decadal1980
    source:                          MRI-CGCM3 2011 atmosphere: GSMUV (gsmuv-...
    model_id:                        MRI-CGCM3
    forcing:                         GHG, SA, Oz, LU, Sl, Vl, BC, OC (GHG inc...
    parent_experiment_id:            N/A
    parent_experiment_rip:           N/A
    branch_time:                     0.0
    contact:                         Seiji Yukimoto (yukimoto@mri-jma.go.jp)
    history:                         Output from /sharex3/cmip5/decadal1980/r...
    references:                      Model described by Yukimoto et al. (Tech...
    initialization_method:           1
    physics_version:                 1
    tracking_id:                     ce91e727-5f22-44fc-b24d-5bb53393ac69
    product:                         output
    experiment:                      10- or 30-year run initialized in year 1980
    frequency:                       mon
    creation_date:                   2011-08-12T05:05:34Z
    Conventions:                     CF-1.4
    project_id:                      CMIP5
    table_id:                        Table Amon (26 July 2011) 976b7fd1d9e1be...
    title:                           MRI-CGCM3 model output prepared for CMIP...
    parent_experiment:               N/A
    modeling_realm:                  atmos
    realization:                     1
    cmor_version:                    2.7.1
    DODS_EXTRA.Unlimited_Dimension:  time

Now let’s do the same with a secured link.

[2]:
secured_url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/testdata/secure/tasmax_Amon_MPI-ESM-MR_rcp45_r2i1p1_200601-200612.nc"

# This should fail but doesn't at the moment.
ds = xr.open_dataset(secured_url, decode_cf=False)

To open a secured link, we need to open a session. We’ve created a authtest user to facilitate testing.

[3]:
import requests
from requests_magpie import MagpieAuth

secured_url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/dodsC/birdhouse/testdata/secure/tasmax_Amon_MPI-ESM-MR_rcp45_r2i1p1_200601-200612.nc"
auth = MagpieAuth("https://pavics.ouranos.ca/magpie", "authtest", "authtest")

# Open session
session = requests.Session()
session.auth = auth

# Open a Pydap data store and pass it to xarray
store = xr.backends.PydapDataStore.open(secured_url, session=session)
ds = xr.open_dataset(store, decode_cf=False) # Attributes are problematic with this file.
ds
[3]:
<xarray.Dataset>
Dimensions:             (bnds: 2, lat: 96, lon: 192, time: 12)
Coordinates:
  * time                (time) float64 5.699e+04 5.702e+04 ... 5.733e+04
  * lat                 (lat) float64 -88.57 -86.72 -84.86 ... 84.86 86.72 88.57
  * lon                 (lon) float64 0.0 1.875 3.75 5.625 ... 354.4 356.2 358.1
Dimensions without coordinates: bnds
Data variables:
    time_bnds           (time, bnds) float64 ...
    lat_bnds            (lat, bnds) float64 ...
    lon_bnds            (lon, bnds) float64 ...
    latitude_longitude  |S128 ...
    tasmax              (time, lat, lon) float32 ...
Attributes:
    institution:                     Max Planck Institute for Meteorology
    institute_id:                    MPI-M
    experiment_id:                   rcp45
    source:                          MPI-ESM-MR 2011; URL: http://svn.zmaw.de...
    model_id:                        MPI-ESM-MR
    forcing:                         GHG,Oz,SD,Sl,Vl,LU
    parent_experiment_id:            historical
    parent_experiment_rip:           r1i1p1
    branch_time:                     56978.0
    contact:                         cmip5-mpi-esm@dkrz.de
    history:                         Model raw output postprocessing with mod...
    references:                      ECHAM6: n/a; JSBACH: Raddatz et al., 200...
    initialization_method:           1
    physics_version:                 1
    tracking_id:                     e7e78b40-458b-48bd-963a-aa377a89f90b
    product:                         output
    experiment:                      RCP4.5
    frequency:                       mon
    creation_date:                   2011-10-11T11:27:30Z
    Conventions:                     CF-1.4
    project_id:                      CMIP5
    table_id:                        Table Amon (27 April 2011) a5a1c518f52ae...
    title:                           MPI-ESM-MR model output prepared for CMI...
    parent_experiment:               historical
    modeling_realm:                  atmos
    realization:                     1
    cmor_version:                    2.6.0
    DODS_EXTRA.Unlimited_Dimension:  time