Platform installation

In this post we try to describe the different steps required to install a Thing’In platform, that is all the differents servers, databases, and their software requirements.

Virtual Machine (VM) setup

All the thingin servers are hosted in a openstack based IaaS provided by Openwatt which is an Orange Cloud Provider. So first of all we will verify/ask for more resources.

Resources configuration

Only the tenant admin user can manage the resources allocated (compute, memory, storage).

  • Click on ???
  • Select the number of Virtual CPU (Vcpu) needed
  • Select the types and size of storage pool needed (e.g. ceph sas)
  • Select amount of memory needed. Note that a tenant is limited to 50GB of RAM. One needs to open a ticket to go beyond that limit. In our case we asked for 100GB via the ticketing service.

Generally to access more resources, the ticketing system is the way to go.

Network configuration

We need to set a private network for our machine to machine communication.

  • Click on ’’Network > network’’ in the left panel, then on ’’Create Network’’ in the right panel,
  • Set the name of the network (e.g. internal-thingin-dev-np1),
  • Add network to the tenant so that this network can be later added to created VMs),
  • Set subname (e.g. subnet-internal-thingin-dev-np1),
  • Set network address (e.g. 192.168.0.0/24),
  • Set IP version (e.g. IPV4),
  • Set gateway ip (e.g. 192.168.0.1).

Virtual machine instanciation

Here we will create the a virtual machine, attach to it some networks and later we will attache storage volume.

  • Click on ’’???’’ in the left panel, then on ’’???’’ in the right panel.
  • set instance name
  • set Avaiability Zone (AZ)
  • set the Operating System (OS). We use Ubuntu as OS
  • set the flavor of the VM
  • Add the networks one at a time. First the network we created earlier, then the Internal_GINIS_IntercoNonProd network. We can add the Internal_ST_NonProd network which a a storage network allowing to setup shared storage volumes.
  • Set the security groups
  • Add user key pairs
  • Associate a floating IP on the Internal_GINIS_IntercoNonProd network

Storage volume instanciation

For each volume to create:

  • Set the name
  • Set the description
  • Set the type (e.g. ceph_sas)
  • Set the size
  • Set the AZ (.e.g. nova)
  • When the volume is create, click on management attachment and select instance
  • Optionnaly, you can set the device name (e.g. /dev/vdb)
  • Click on start instance to start the VM created

If the create instance is missing a network interface, find the disable interface with the command ’’lshw’’
You can declare the network interface in ’’auto’’ mode with a DHCP in the file /etc/network/interface.d/cloud_init.cfg as follows

auto ens4
iface ens4 inet dhcp

Prefer a ’’hard reboot’’ via the web interface to restart the VM.

Proxies configuration

Before diving in the software and dependencies requirements, we need to configure the openwatt proxies

Java installation

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

’’Storage partition creation for OrientDB’’

  • Create the partition on the external volume /dev/vdb using the command fdisk
  • Format the partition in ext4 with the command mkfs.ext4
  • Give a label to the partition using the command e2label
  • Add auto mount to the partition in the folder /opt/orientdb/data by configuring it in the /etc/fstab file

OrientDB installation

We use the 2.2.30 version of OrientDB. install and configure OrientDb as follows

wget -O orientdb-community-2.2.30.tar.gz http://orientdb.com/download.php?file=orientdb-community-2.2.30.tar.gz&os=linux
tar -zxvf orientdb-community-2.2.30.tar.gz
sudo mv ~/orientdb-community-2.2.30 /opt/orientdb
cd /opt/orientdb
ln -s orientdb-community-2.2.30 orientdb-community
mv orientdb-community/databases data
ln -s orientdb-community/databases /opt/orientdb/data/

Install the plugin orientdb-spatial and copy it in the lib folder of orientdb /opt/orientdb/orientdb-community/lib

Create a user with the login/password root:root in /opt/orientdb/orientdb-community/config/orientdb-server-config.xml and change the permission of that file to be 600 with the command chmod

In the file /opt/orientdb/orientdb-community/bin/server.sh, configure the memory options. Set a heap size of 6GB, a Direct Memory Access of 14GB and activate the garbage collector log.

You can launch the server by executing the following command
sudo /opt/orientdb/orientdb-community/bin/server.sh

The web interface of orientdb can be accessed on the port 2480 of the server. Note that you need to open the port in the security policy to do so.

’’add a user orientdb’’

useradd -r orientdb -s /sbin/nologin

give ownership of the orientdb folder to the user orientdb

chown -R orientdb:orientdb opt/orientdb

configure systemd for service control on orientdb

  • Configure the installation path and user in /opt/orientdb/orientdb-community/bin/orientdb.sh
  • Add the orientdb username and group in the file /opt/orientdb/orientdb-community/bin/orientdb.service and configure the ExecStart
  • Copy /opt/orientdb/orientdb-community/bin/orientdb.service in /etc/systemd/systemd

Rabbitmq Installation

echo "deb https://dl.bintray.com/rabbitmq/debian {distribution} main" | sudo tee /etc/apt/sources.list.d/bintray.rabbitmq.list
wget -O- https://www.rabbitmq.com/rabbitmq-release-signing-key.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install rabbitmq-server

Zookeeper installation

sudo apt-get install zookeeperd zookeeper zookeeper-bin

To be able to use the Command Line tool zkCli.sh to manage Zookepper, add the folder /usr/share/zookeeper/bin/ in the PATH environment variable

Elastic Search and OLS installation

Currently, we provide a Ansible playbook to cope with the installation of ’’Elastic Search’’ (ES) and ’’Ontology Lookup Service’’ (OLS).
Nevertheless for ES, a external volume is requiered and for better performance, it uses a XFS and mounted at /opt/elasticsearch.
The playbook installs a ’’nodeJS’’ server to provide the Swagger; and ’’Supervisor’’ to ensure the run.
A Ngnix is also needed to reverse-proxify the NodeJS and the OLS API.

Influx installation

Influxdb needs a volume to store its data. The volume is mounted with a EXT4 file system at /opt/influxdb.
The installation doc could be find in the influx web site.
We modified the /etc/influxdb/influxdb.conf to target the new storage location /opt/influxdb .
Today, Influx becomes "not free" for a cluster mode instanciation. ’’We should find an alternative. ’’

Mongodb installation

Mongodb needs a volume to store its data. The volume is mounted with a XFS file system at /opt/mongodb.

sudo apt-get install mongodb-org

We need to configure mongo to set the new data directory.

Thing’in Server

get the jar

artifactoy : 10.226.227.130:8081 / admin:admin /

com / mycompany / ziggy server

wget url

put jar in /opt/thingin-server

copy fichier de conf depuis gitlab -> / /opt/thingin-server

configure

Thing’in API

git clone ziggy-api ; mv ziggy-api -> thing’in api

sudo apt-get install python-pip

sudo pip install virtualenv

virtualenv -p python2.7 venv

sudo vim /etc/supervisor/conf.d/thing-api-supervisoir.conf

sudo pip install gunicorn

/search for -> gunicorn supervisor flask virtualenv

useradd thingin -M -s /usr/sbin/nologin

modify main de l’api pour l’emplacement du fihier de conf en le mettant dans le répertoire local

config : non des fichiers, et proxy

supervisor lance gunicorn

conf ngnix, api sur 8080

install flask with ziggy patch

en root, dans le virtual env

pip install git+https://gitlab.forge.orange-labs.fr/dhbt3263/ziggy-flask-restplus/

pip install -r requieremnts