Quinta Usage

From Navigators

(Difference between revisions)
Jump to: navigation, search
Line 43: Line 43:
4) Verify which images of operating systems are available to deploy in the farm (see [[#Kaenv|kaenv3]]):
4) Verify which images of operating systems are available to deploy in the farm (see [[#Kaenv|kaenv3]]):
  $ kaenv3 -l
  $ kaenv3 -l
 +
Name                Version    User      Description
 +
####                #######    ####      ###########
 +
debian-8-jessie    1          root      Basic Debian Jessie (8.X) image.       
 +
debian-xen          1          root      Xen image based on Debian Lenny.       
 +
ubuntu-14.04        1          root      Ubuntu 14.04.1 LTS (Trusty Tahr) image.
 +
ubuntu-java        1          root      Ubuntu Lucid Lynx image with Sun Java and Build Essentials (GCC, G++, etc) installed.
 +
us-12.04-kvm        2          root      Ubuntu Server 12.04 (Precise - updated) with KVM.
5) Deploy the chosen image in those selected machines (see [[#Kadeploy|kadeploy3]]):
5) Deploy the chosen image in those selected machines (see [[#Kadeploy|kadeploy3]]):

Revision as of 21:01, 16 May 2016

Contents

Instructions to Use Quinta, the Navigators Farm

1) Access the front-end machine with your username using SSH:

$ ssh user@quinta.navigators.di.fc.ul.pt

2) Verify the power status of all machines in the cluster:

$ quinta status -a
HOST    POWER   OWNER          IMAGE          SINCE          DEPLOY
r1       off      user1            ubuntu-base      May 23, 2012     deployed 
r2       on       user2            ubuntu-base      May 23, 2012     deployed 
r3       off      user1            ubuntu-java      May 04, 2012     deployed 
r4       on       user5            ubuntu-java      May 04, 2012     deployed 
r5       off      user5            ubuntu-java      May 04, 2012     deployed 
r6       on       user5            ubuntu-java      May 04, 2012     deployed 
r7       on       user1            debian-base      May 04, 2012     deployed 
r8       on       user3            ubuntu-java      May 04, 2012     deployed 
r9       on       user3            ubuntu-java      May 04, 2012     deployed  
r10      on       user1            ubuntu-java      May 04, 2012     deployed 
r12      on       user9            debian-xen       May 04, 2012     aborted  
s1       on       user8            ubuntu-java      May 04, 2012     deployed 
s2       on       user1            ubuntu-12.04     May 04, 2012     deployed 
s3       on       user8            ubuntu-java      May 04, 2012     deployed 
s4       on       user1            ubuntu-java      May 04, 2012     deployed 
s5       on       user9            ubuntu-java      May 04, 2012     deployed 
s6       on       user1            debian-xen       May 30, 2012     deploying 
s7       off      user7            debian-xen       May 04, 2012     deployed 
s8       off      user1            ubuntu-java      May 04, 2012     deployed 
s9       off      user3            ubuntu-java      May 04, 2012     deployed 
s10      off      user5            ubuntu-base      May 23, 2012     deployed 
s11      on       user2            ubuntu-java      May 04, 2012     deployed 
s12      on       user6            ubuntu-java      May 04, 2012     deployed 
s13      on       user7            ubuntu-java      May 04, 2012     deployed 
s14      on       user7            ubuntu-java      May 04, 2012     deployed 
s15      off      user1            ubuntu-base      May 23, 2012     deployed
s16      on       user1            ubuntu-12.04     May 23, 2012     deployed
s17      off      user6            ubuntu-base      May 23, 2012     deployed
s18      on       user3            ubuntu-12.04     May 23, 2012     deployed
s19      on       user5            ubuntu-base      May 23, 2012     deployed

3) Choose the machine you want to use (note only those turned off are deployable).

E.g., s9

4) Verify which images of operating systems are available to deploy in the farm (see kaenv3):

$ kaenv3 -l
Name                Version     User       Description
####                #######     ####       ###########
debian-8-jessie     1           root       Basic Debian Jessie (8.X) image.        
debian-xen          1           root       Xen image based on Debian Lenny.        
ubuntu-14.04        1           root       Ubuntu 14.04.1 LTS (Trusty Tahr) image. 
ubuntu-java         1           root       Ubuntu Lucid Lynx image with Sun Java and Build Essentials (GCC, G++, etc) installed.
us-12.04-kvm        2           root       Ubuntu Server 12.04 (Precise - updated) with KVM.

5) Deploy the chosen image in those selected machines (see kadeploy3):

$ kadeploy3 -e os-image-name -m machinename
E.g., kadeploy3 -e ubuntu-14.04 -m s9

At the end of the deployment, kadeploy will present which nodes were correctly deployed and which weren't. For example:

Nodes correctly deployed on cluster r
s9

6) Login into your machine:

$ ssh root@s10
$ root@s9's password: quinta

7) Once you have finished your experiments, turn off your machines:

$ quinta off s9

Allocation Rules

For now, there is neither scheduler nor reservation tool in the farm, therefore we ask users to follow some deployment rules:

  • Only use machines that are turned off. Chances are that nobody is using them.
  • Above all, don't deploy an image on machines with the power status on. It is very likely that someone is using them.
  • Copy everything important to your home in the front-end machine before turning off your nodes (via SCP or NFS).
  • Turn off all your machines after using them.

The quinta script

We developed quinta, a simple front-end script to the ipmitool, that is used to control the computers in a cluster via IPMI (if you have any comments on the script, please tell us). The quinta script has the following usage:

quinta command [ options ] [ hostname ... ]
  commands:
    status     Show current machine power status
    update     Update the ownership of one or more machines (use with -u <username> in options)
    reboot     Perform a soft reset of OS via ACPI
    hard       Perform a hard reset
    on         Power up the machine
    off        Power down the machine
    pxe        Set PXE as boot device
    hd         Set hard disk as boot device
    cd         Set optical driver as boot device
    wait       Wait the machine answer a ping
  options:
    -a: all hosts
    -v: verbose

The Ka-Tools Commands

Kaconsole

Kaconsole provides user access to the consoles of the nodes on which the user has deployment rights. In Quinta's case, it's just a connection via SSH to the node:

$ kaconsole3 -m r1

Kadeploy

Kadeploy is the main tool of Ka-Tools set. It's responsible for deploying a registered environment in one or more nodes. If you intend to deploy in one machine only, you can specify it with:

$ kadeploy3 -e os-image-name -m node

If you want to deploy in a list of nodes, create a file with one node per line, and execute:

$ kadeploy3 -e os-image-name -f nodefile

If you intend to deploy a non-registered image, you can specify it as the following:

$ kadeploy3 -a os-image-name-descriptor.dsc3 -m node

Besides the basic usage, Kadeploy allow users to run passive jobs, through a script passed as an argument:

$ kadeploy3 -e os-image-name -f nodefile -s scriptfile

In addition, it's possible to specify a SSH-Key to be used in image's root authorized_keys, which allows users to execute privileged tasks in passive jobs:

$ kadeploy3 -e os-image-name -f nodefile -s scriptfile -k ssh-key-file

Kaenv

Kaenv manages users' environments. Currently, Quinta contains the following images:

$ kaenv3 -l

Name Version User Description

        1. ####### #### ###########

debian-5-lenny 1 root Basic Debian Lenny (5.X) image. [DISCONTINUED] debian-8-jessie 1 root Basic Debian Jessie (8.X) image. debian-xen 1 root Xen image based on Debian Lenny. ubuntu-10.04 1 root Ubuntu 10.04 (Lucid Lynx) image. [DISCONTINUED] ubuntu-12.04 1 root Ubuntu 12.04 (Precise Pangolin) image. ubuntu-14.04 1 root Ubuntu 14.04.1 LTS (Trusty Tahr) image. ubuntu-java 1 root Ubuntu Lucid Lynx image with Sun Java and Build Essentials (GCC, G++, etc) installed. us-12.04-kvm 2 root Ubuntu Server 12.04 (Precise - updated) with KVM.

Kaenv allows users to register their own images (see how to create your own image) through the following command:

$ kaenv3 -a descriptorfile.dsc3

Additionally, Kaenv allows users to delete their images:

$ kaenv3 -d os-image-name

Kanodes

Kanodes presents the current deployment state of nodes.

$ kanodes3 -d
r1,  aborted,   user, lenny-base, 1, root
r2,  aborted,   user, lenny-base, 1, root
r3,  aborted,   user, lenny-base, 1, root
r4,  aborted,   user, lenny-base, 1, root
r5,  aborted,   user, lenny-xen,  1, root
r6,  deployed,  user, lenny-xen,  1, root
r7,  deployed,  user, lenny-xen,  1, root
r8,  deployed,  user, lenny-base, 1, root
r9,  deployed,  user, lenny-base, 1, root
r10, deployed,  user, lenny-base, 1, root
r12, deployed,  user, lenny-base, 1, root
s1,  deploying, user, lenny-xen,  1, root
s2,  deploying, user, lenny-xen,  1, root
s3,  deploying, user, lenny-xen,  1, root
s4,  deploying, user, lenny-xen,  1, root
s5,  aborted,   user, lenny-base, 1, root
s6,  deployed,  user, lenny-base, 1, root
s7,  deployed,  user, lenny-base, 1, root
s8,  deployed,  user, lenny-base, 1, root
s9,  deployed,  user, lenny-base, 1, root
s10, deployed,  user, lenny-base, 1, root
s11, aborted,   user, lenny-base, 1, root
...

Kastat

Kastat shows several statistics about the deployments.

$ kastat3 -d

How to Create Your Own Images

There are two main ways to create your own images in Quinta: creating a new one from scratch or improving an existent environment. In the first case you will need physical access to Quinta cluster. The second one consists in deploying an existent environment, improving its installation or configuration and creating a new "snapshot" of this environment. In both cases it's suggested to use the TGZ-G5K script to create the snapshots.

The TGZ-G5K Command

TGZ-G5K is a script to create system deployment images that don't posses site-flavours. In Quinta's case you can deploy any existent image in one machine, install new software, configure others and there are two steps at the end:

1. Clean the interfaces rules file, to the image work in any node:

$ echo "" > /etc/udev/rules.d/70-persistent-net.rules

2. Create your own image with tgz-g5k command:

$ tgz-g5k user@frontal:/.../os-image.tgz

This command is executed inside the node you want to make the "snapshot" and will create a TGZ file with your environment on your home in front-end machine.

The Environment Descriptor File

To register your new environment and make it deployable, do you need an image descriptor file, which can be similar to:

name : debian-base
version : 1
description : Basic Debian Lenny image.
author : Author Name (Author e-mail)
tarball : /home/deploy/images/debian-base.tgz|tgz
postinstall : /home/deploy/postinstalls/post4all.tgz|tgz|traitement.ash
kernel : /boot/vmlinuz-2.6.32-bpo.2-amd64
initrd : /boot/initrd.img-2.6.32-bpo.2-amd64
fdisktype : 83
filesystem : ext3
environment_kind : linux
visibility : private

Further Information

For further information about the tools used in this tutorial, please visit their sites or their documentations:

Ka-Tools: oficial installation, configuration and use guide.

TGZ-G5K documentation in Grid'5000 Wiki

Download TGZ-G5K files

Personal tools
Navigators toolbox