PAVICS is built to work within POSIX and POSIX-like systems (e.g. Unix/Linux). For Windows users who want to administer a PAVICS data server, you may need to either install PuTTY, a Unux/Linux virtual machine or use a POSIX API environment like the Windows Subsytem for Linux or Cygwin to interface with the PAVICS server system.
PAVICS can be installed from source on GitHub or by using Docker. Git/Github is a versioning software and platform that can fetch development branches of pavics while Docker is a virtualization tool for running isolated service images built with specific library environments. Depending on your needs and interests (e.g. debugging vs deploying) either platform may suit your needs.
Proposing changes to PAVICS or Contributing requires free a GitHub account but anyone can download the source code for PAVICS withour registering. Git is normally in most standard Linux software repositories and can be installed (using Debian/Ubuntu-based systems) with the following:
# Enable root access sudo su # Updating software catalogues and and installed libraries apt-get update; sudo apt-get upgrade # Installing git will also install dependent libraries apt-get install -y git # Configuring git for code commits using your user credentials exit # Leave superuser git config --global user.name "Your Name" git config --global user.email "email@example.com"
For Docker installations, begin by following the guide for installing
Docker Community Edition on your home machine from the Docker Installation Page. Docker Community Edition is a set of command line tools for creating and launching container-based applications.
Depending on your Linux distribution, you can either download and launch the installer directly or you may need to add a new ppa/apt/yum/dnf/etc. software repository to your system to install the most recent version. For Debian/Ubuntu-based systems:
# Enable root access sudo su # Install base library requirements apt-get install -y apt-transport-https ca-certificates wget software-properties-common # Install the Docker suite of tools apt-get install -y docker docker.io docker-engine
PAVICS and Dockerhub images¶
All code produced for the PAVICS project is Open Source and hosted publicly through GitHub repositories. Among our members/users, the preferred method for packaging and deployment relies on Docker images. As such, we use the Docker Image building and hosting features of DockerHub. This feature automatically builds a new image for all major releases to the GitHub repository, for all repositories that include a
Dockerfile. The resulting images are all publicly available for distribution and deployment. Any PAVICS Docker image can be obtained using the following command:
docker pull pavics/[image_name]
Most of the code base for PAVICS is forked from the Birdhouse Project which already has multiple DockerHub build processes. Using our own DockerHub build gives us the advantage of having our own upstream build process for the code being modified by CRIM / Ouranos. Many of the modifications to these birdhouse components are merged upstream, but some are specific to the PAVICS project and we felt it was worthwhile to maintain an independent build process.
PAVICS installation with docker-compose¶
Base PAVICS installation is incomplete. The following lines refer to Phoenix instance. Need to specify which birds are needed for a bare installation of PAVICS: Phoenix, FlyingPigeon, Malleefowl, Emu, etc.
First mount or create a symlink for the datasets storage at
Mount or make a symlink for the geoserver data storage so that
/geoserver_data could be used (read/write) by geoserver.
To install the suite of docker images you must use docker-compose. Docker-compose is a docker helper for coordinating multiple docker images at once. Docker-compose exists in most Linux software repositories but can also be installed using
conda. Depending on your Linux distribution and whether you have Anaconda/miniconda installed, run any of the following:
# For standard libraries (Ubuntu/Debian) sudo apt-get install docker-compose # For system-installed Python2 or Python3 (requires sudo) sudo apt-get install -y python-dev python3-dev sudo -H pip install docker-compose # For Anaconda/miniconda (Python2 or Python3) conda install docker-compose
Presently, the main
PAVICS repo is closed to the public while it is under heavy development
After installing docker-compose, clone the PAVICS repository and navigate to the docker build recipe within PAVICS:
# Cloning the PAVICS repository git clone https://github.com/Ouranosinc/PAVICS.git cd pavics-sdi/birdhouse # Open the docker-compose.yml using a text editor nano docker-compose.yml
docker-compose.yml contains many of the setup configurations needed to successfully launch Birdhouse. Be sure to read the Birdhouse service configuration and the Load balancing suggestions before continuing the installation.
After modifying the necessary variables in the
docker-compose.yml file, simply run the following command, taking care to select an appropriate host name:
HOSTNAME='<public-ip>' bash -c 'docker-compose up -d'
This installation will run on a single server instance, but there are instructions for Load balancing.
Updating containers to the latest version¶
To synchronize a deployment with the latest container available on dockerhub:
# Set working directory where the docker-compose.yml is located. Usually : cd ~/PAVICS/birdhouse # Docker requires root privileges sudo su # Pull the latest containers (container_name is optional but can limit the operation to only one container rather than applying to all containers) HOSTNAME='<public-ip>' bash -c 'docker-compose pull [container_name]' # Stopping containers HOSTNAME='<public-ip>' bash -c 'docker-compose stop [container_name]' # Start again containers (-d is for detached, avoid it to get all output to the command line) HOSTNAME='<public-ip>' bash -c 'docker-compose up -d [container_name]'
container_name is the name chosen in the docker-compose.yml, not the name of the docker image.
Resetting Bird services¶
If a Bird Service becomes unusable, the docker containers can easily be reset to default settings:
# Docker requires root privileges sudo su # Stopping and removing containers (this will flush their states) HOSTNAME='<public-ip>' bash -c 'docker-compose down' # Start again containers (-d is for detached, avoid it to get all output to the command line) HOSTNAME='<public-ip>' bash -c 'docker-compose up -d'
Some manual tasks are required after resetting the birdhouse environment : See the manual steps under the Phoenix configuration
Restarting Bird services¶
If the dockers containers need to be stopped (including the docker service, if required) these steps will preserve the docker state and all configurations set via Bird Service web portals:
# Docker requires root privileges sudo su # Stopping running containers HOSTNAME='<public-ip>' bash -c 'docker-compose stop' # Stopping the docker service service docker stop
Restarting Bird Services (required after restarting the host vm):
# Docker requires root privileges sudo su # Starting the docker service service docker start # Starting the birdhouse containers HOSTNAME='<public-ip>' bash -c 'docker-compose start'
docker-compose.yml config file exposes ports for each docker container (left part is the public one, right part the container internal one). We try to respect the following convention:
8xxx : port usually responding for the service (The HTTP port) 28xxx: https port 38xxx: the output port (To be documented) 48xxx: the supervisor port of the container
Container xxx value:
Phoenix : 443 (With 8081 as http and 8443 as https) Malleefowl : 091 Flyingpigeon : 093 Emu : 094 Solr : 983 (No https or output ports) ncWMS2 : 080 (No https or output ports) thredds : 083 (No https or output ports) pavics-catalog : 086 (No https or output ports) geoserver : 087 (No https or output ports)
The exception is the Pavics-frontend, which has port 3000.
To find which shell process uses a given port, use:
netstat -nlp | grep :<port #>
Update the installation and config with security changes