Loading...
 
ESA > Join & Share > SSEP support > CloudToolbox FAQ
Print

CloudToolbox FAQ

- How to apply?

You can request for a CloudToolbox instance directly from the CloudToolbox Service website.

Once you have logged in, you can press the Request VM button in the left bar, select the desired template and fill the request page with your desired machine resources. You can put additional notes (for example request for additional software installation or special hardware configuration) on the Notes field and submit the request by pressing the "Submit request" button.

The request is going to be evaluated by ESA and RSS (service operators), and you will be contacted with the outcome of your request. If the request is approved, a new machine is created for you and you will be able to access it from the My VMs page.

- How long does it take to have a VM?

Supposing that all the needed information is provided, evaluation of the request for a new VM usually takes about three days (one week maximum). Since the request for a new VM or an upgrade to an existing VM is approved, it usually takes 1 day for the VM to be avaiable for the user.

- What VM means?

VM stands for Virtual Machine and it usually defines a virtual server, thus a software implementation abstraction of a phisical server hardware. Virtual Machines permit to have flexibility in the virtual hardware (easily adding/removing CPU, RAM, disks, etc...) and in the resource management (sharing the phisical CPU between multiple machines and using it only when needed).

In this context, the VM term is used to identify a particular CloudToolbox instance (which is effectively a virtual machine with a custom OS running on a Cloud infrastructure)

- How I can get the credentials to access the CloudToolbox Service?

CloudToolboxservice credentials are your EO-SSO credentials. If you do not have any, you can register at this link (or by pressing the Register button in the CloudToolbox page).

- How can I request for machine hardware or software upgrades?

You can request changes into the VM hardware (ex. more RAM, more CPU, more disk, etc...) or software (installation of additional software, software licenses, etc...) from the My VMs page.

Just click on your VM, fill the "Notes", "CPU", "RAM" or "Disk" fields with your request and press the "Request machine update" button.

- How can I increase the number of CPU and RAM?

See "How can I request for machine hardware or software upgrades?".

- How can I start my VM?

You can start your CloudToolbox VM from the My VMs page. Just press the "Start" button next to the VM name (or in the VM details page)

Take note that your request will go in a queue, and will take from one to ten minutes to complete. You can monitor the status of the request from the My VMs page. A PENGING status (yellow button) means that your request is processed. A STARTED status (green button) means that the machine is correctly started and you can now access it.

- I cannot Start a VM. I cannot see the Start button in the My VMs page.

If you cannot see the Start button, this is probably because your machine access expired (you see the "Expired" message in the My VMs page) or the machine is already started (you see a green button in the left of the VM name in the My VMs page).

If the above is not the case, please contact us at eo-gpod at esa.int .

- How can I connect to my VM?

Connection to the VM machine is primarly done via SSH (command line shell). If you'd like to have a Desktop access, you can use X11 forwarding or VNC access.

Information about the connection (IP, credentials, etc...) are present in the My VMs details for your VM.

- How can I find the IP to connect to my VM?

The IP of your VM is reported in the My VMs details for your VM. To access that page, just click on the My VMs page then click on your VM name.

- How can I find the credentials to connect to my VM?

The credentials to access your VM are reported in the My VMs details for your VM in the "Notes" field. To access that page, just click on the My VMs page then click on your VM name.

- How can I connect to my VM command line shell?

You can connect to your CloudToolbox VM via the SSH protocol over the port 22 or 443, using any SSHv2 enabled clients (see the FAQs below for some recommanded clients for Windows, Linux, Mac and other systems).

Your user access credentials are provided in the My VMs page (in the VM details notes).

Once you have logged in, you can use all the standard Linux tools, for example SCP for files transfer, X11 forwarding and VNC via SSH for desktop access.

- How can I connect to my VM command line shell (from Windows)?

From a Windows, you can use the free PuTTY client. Just download the client here, run it and add your VM IP and port (we recommend to use 443 as port, as in the sample here), then press the "Open" button.

At the first time, you will be prompted with a message to accept the autenticity of the connection, click yes. Then, you will be requested with your VM credentials, which you can find in the My VMs page (in the VM details notes).

- How can I connect to my VM command line shell (from Linux or Mac)?

From a Linux or Max, you can use the free SSH tools provided by the Linux Terminal. Just open a new Terminal window and type

  ssh -p 443 @

where is the IP of the machine reported in the My VMs page and  is your credentials username reported in the My VMs page.

After running the command, the first time, you will be prompted with a message to accept the autenticity of the connection, type yes. Then, you will be requested with your credentials password, which you can find in the My VMs page (in the VM details notes).

- How can I connect to my VM command line shell (from my tablet/cellular phone)?

From a Android/iOS tablet/cellular phone you can access your VM with any SSHv2 client. Just type your machine IP and credentials as eported in the My VMs page.

- I am on the VM, how I can access the VA4 data?

VA4 and the CloudToolbox VMs relies on the same cloud infrastructure, thus VA4 data access is very fast and stable.

From your VM, you can access the VA4 data using both command-line and graphical tools.

From the Desktop, you can click on the "Virtual Archive 4" link on the desktop, browse the Virtual Archive using the graphical tools and download the files directly to your PC (using your EO-SSO credentials)

From the command-line, you can perform queries to the VA4 catalogue using the 'ciop-catquery' command (run the command from command line or refer to the "What is the ciop-catquery command and how I can use it?" FAQ for more information) or the VA4 web interface from your local PC and then download the entire list of files using the 'secp' tool (run the command from command line or refer to the "What is the ciop-catquery command and how I can use it?" FAQ for more information)

Note that you can get the full list of files from the VA4 web interface for secp to download using the "Chain" link in the search results (as shown here)

- How can I download in bulk the results of a VA4 query?

If you have performed a search in VA4 for data and you want to download all the result into your VM you can:

  1. Click on the (i) icon on the result query and then right-click on the "Chain" icon and copy the link address (as shown in the figure here)

  2. Run the following command in your VM from any terminal

 #Download the list of files to download
 curl -k '' -o search_results.url
 #Download the files into the folder
 secp -O -F search_results.url

For example, to download all the products over Etna, you can run

 curl -k 'http://eo-virtual-archive4.esa.int/search/ASA_IM__0P/txt//?startIndex=0&q=Etna&start=1992-01-01&stop=2013-04-17&count=*' -o search_results.url
 secp -O ./ -F search_results.url

- What is the secp command and how I can use it?

The secp is a command-line tool to download files from multiple sources, managing automatically authorization credentials and download session (with ESA EO-SSO support).

secp tool is provided under GPLv3 as part of the G-POD Grid-Cache System software.

To use secp, you need just to place all the urls you want to download in the command line, for example,

  secp https://eo-virtual-archive4.esa.int/supersites/ASA_IM0CNPAM20110520_090307_000000173102_00352_48211_1866.N1

will download the ASA_IM__0CNPAM20110520_090307_000000173102_00352_48211_1866.N1file in the local folder.

You can give to secp an entire list of files to download. For example, if you need to download all the files in search_results.url, you can run the dollowing command:

For example, to download all the products over Etna, you can run

  secp -F search_results.url

secp provides many other options to customize your download (for example, authomatic extraction of compressed files, authomatic retries on download failure, etc...). For a complete list of features you can run

  secp --help

from any terminal.

- What is the ciop-catquery command and how I can use it?

The ciop-catquery tool is a command-line tool to query OpenSearch EO catalogues (like the VA4)

ciop-catquery is provided as part of the Terradue's CIOP tools software.

With ciop-catquery you can search for files in VA4 , for example, to obtain all the VA4 files over Etna you can run

  ciop-catquery -ol 99999 --outputfields=dc:identifier,ical:dtstart,dct:spatial,ws:HTTPS -tq 'begin=2010-01-01;end=2012-01-01' -se ASA_IM__0P http://eo-virtual-archive4.esa.int/search

to get the list of the products belonging to the ASAR Image Mode Level 0 dataset (ASA_IM__0P) with start time between 2010-01-01 and 2012-01-01.

ciop-catquery provides many other options to customize your download (for example, authomatic extraction of compressed files, authomatic retries on download failure, etc...). For a complete list of features you can run

  ciop-catquery --help

from any terminal.

- How I can access the G-POD data?

The G-POD data access is usually limited to processing services only to registered G-POD users. Access to the data for download is allowed only to particular datasets (like ASAR, via the EO-Data gateway application) or to all the datasets previous user request approval.

The CloudToolbox VMs shares dedicated fast access to the G-POD storages, so access to the G-POD data is possible via the Desktop and command line tools present in the VM (see "I am on the VM, how I can access the VA4 data?" FAQ for the tools list)

To request authorization to access the G-POD stores from your VM, you can send an email to eo-gpod at esa.int.

- I need more EO data, how I can get it?

G-POD (via the CloudToolbox and other projects) has access to wide range of ESA and thirty party mission data. Access to this data may be granted on user request.

The general rule is "if you need, you ask", we will try to do our best to fulfill any request. You can contact us at the eo-gpod at esa.int

- How can I can move files from/to my VM?

The easiest way to share files from your PC to the CloudToolbox VM is to use the SCP protocol. To do so, you just need your VM credentials (see the "How can I find the credentials to connect to my VM?" FAQ) and an SCP client.

For Windows, you can use the free WinSCP software.

For Linux and Mac, you can use the scp software (you can run it from your terminal)

From your Andorid/iOS tablet you can use any SCP client application.

NOTE: The details of the connection are the same as the SSH connection (see "How can I connect to my VM command line shell?" FAQ)

- How can I can access the desktop of my VM?

You can access the desktop of your VM via VNC from any VNC client. VNC access is not allowed directly, but, for security reason, it needs to be tunnelled via SSH (so you need always to estabilish an SSH connection prior the VNC)

The VNC access details and credentials are provided in the My VMs page. For most of the toolbox, anyway, VNC server is available on local port 5902.

More details about the VNC connection from Windows, Linux and Mac are reported in the

- How can I can access the desktop of my VM (from Windows)?

To access your VM desktop you need a VNC and SSH client. You can download the PuTTY SSH client from here and the UltraVNC VNC client from here (we suggest to download the Portable edition)

Once you have uncompressed and installed the client, you need first to connect via SSH and enable the SSH tunnel (as shown here) and insert your machine credetials. After you get the SSH command line, you can open the VNC Viewer (use vncviewer.exe application from your UltraVNC installation) and connect to "localhost:5902".

- How can I can access the desktop of my VM (from Linux)?

On Linux, you can use the TigerVNC client (or any other VNC client) and the default linux SSH client.

To install TigerVNC you can run "yum install tigervnc" (for RedHat based systems) or "apt-get install tigervnc" (for Ubuntu/Debian based systems)

Once you have installed your desired VNC client, you need to establish the SSH connection to the VM machine enabling the VNC tunnel, you can do so by opening a terminal and running

  ssh -L 5902:localhost:5902 -p 443 @

where is the IP of the machine reported in the My VMs page and  is your credentials username reported in the My VMs page.

After you get the SSH command line, you can open the VNC Viewer (type vncviewer on your terminal or open it directly from your Desktop) and connect to "localhost:5902".

- How can I can access the desktop of my VM (from Mac)?

On Mac, you can use the Safari VNC client (or any other VNC client) and the default Mac SSH client.

Once you have installed your desired VNC client, you need to establish the SSH connection to the VM machine enabling the VNC tunnel, you can do so by opening a terminal and running

  ssh -L 5902:localhost:5902 -p 443 @

where is the IP of the machine reported in the My VMs page and  is your credentials username reported in the My VMs page.

After you get the SSH command line, you can open the Safari browser and browse to vnc://localhost:5902 .

- Can open graphic application installed on the VM directly on my PC?

You can use X11 forwarding to open Linux graphic application over an X Server installed on your local PC (so you will actually see the application graphic interface on your local PC screen).

To do so, you need to enable X11 forwarding in your SSH client (from Windows, using PuTTY, you can tick the "Enable X11 forwarding" switch in the "Connection" -> "SSH" -> "X11" panel, from Linux or Max you can add the -X switch to the ssh command) and then install and run a X11 server (for Linux, you have it already installed, for Mac, you can install XQuartz, for Windows, you can install XMing).

When X11 forwarding is enabled and your X11 server is running, if you run a graphic application from the command line, you should see the graphic interface on your local PC)

- What is SSH?

SSH stands for Secure SHell and it is a cripthographic network protocol to ensure secure access to a remote linux server. For more information, you can check the Wikipedia SSH page.

- What is VNC?

VNC stands for Virtual Network Computing and it is a protocol to access a remote desktop environment via the network. Since VNC is not a secure protocol, to ensure security it is usually tunnelled via SSH. For more information, you can check the Wikipedia VNC page.

- What is X11 forwarding?

X11 stands for the X Windows System (currently at major version 11), which is the standard protocol to provide graphical user interfaces for the Linux OS. X11 can be tunnelled via the network (for example using SSH forwarding) effectively displaying the GUI of an application over a system different from the one where the application code is running. To do so, the client need to run an X Windows System Server (X11 servers are avaiable for the most used operating systems, like Windows or Mac). For more information, you can check the Wikipedia X11 page.

- What is SCP?

Secure Copy (SCP) is a mean to transfer computer files using the Secure SHell (SSH) protocol. SCP clients are available for most of the operating systems (like Windows, Linux, Max, Android or iOS). For more information, you can check the Wikipedia SCP page.

- I cannot connect to my VM via SSH.

Ensure that your machine is correctly started by checking the My VMs page(status of the machine must be STARTED, with a green icon on the left of the machine name)

Ensure that you are using the correct VM IP and credentials from the My VMs page (IP can change without notice).

Ensure that the networks you are connected allows SSH protocol (you can use port 443 as SSH port, which is allowed by most of the internet network providers).

If you still cannot access to your VM, please send the details of the problem to our support at eo-gpod at esa.int.

- I cannot connect to my VM via VNC.

Ensure that you can login via SSH to the machine and, if not, follow the "I cannot connect to my VM via SSH" FAQ instructions.

Ensure that you have correctly set-up an SSH tunnel for VNC on port 5902, following the "How can I can access the desktop of my VM" for your particular operative system.

If you still cannot access to your VM, please send the details of the problem to our support at eo-gpod at esa.int.

- How can I stop my VM?

Once you are logged in into your VM, you can stop it by typing "poweroff" in any terminal (or from the command line shell) or by a double click on the "Poweroff" button on the Desktop.

If you have any problem in the shutdown, you can force termination of the instance by pressing the "Stop" button in the My VMs page.

Please note that the machine will usually takes from one to ten minutes to terminate its operations. The machine can be restarted only when it is successfully stopped (red icon on the My VMs page)

- Why my machines stops automatically after some minutes?

In order to save processing time, the CloudToolbox VMs are configured to shutdown after 30 minutes of inactivity (no background processes running and no buttons pressed by the user).

If this behaviour is causing problems in your processing, you can temporary disable it (until the next shutdown) by running the 'disable-autoshutdow' command from the machine terminal.

After a machine is shut down, you can start it from the My VMs page (for more information, see the "How can I start my VM" FAQ).

- How I can disable/enable the machine auto shutdown?

You can disable the machine automatic shutdown (after 30 minutes of inactivity) system by running from any terminal (or your SSH access) the command

  disable-autoshutdown

To re-enable the automatic shutdown you can execute from any terminal the command

  enable-autoshutdown

- I cannot access my VM. How can I check its status?

You can monitor the VM status from the My VMs page. The machine can be in the following status:

 * STOPPED (Black icon): The machine is off and it is not reachable via SSH

 * STARTED (Green icon): The machine is on and it is reachable via SSH

 * PENDING (Yellow icon): The machine is starting up or shutting down, no operation or SSH connection is possible

 * ERROR (Red icon): The machine is in error state, please contact the service administrator.

- How to monitor usage and quota?

You can monitor the machine usage from the My VMs page. If you click on the VM name, you get the machine uptime (in seconds) and the status of your qouta (Expire time)

- My machine has expired, is my data lost?

After your machine expires, you data is still stored on the Cloud for at least one week. If you need to access some data, you can send a request to the system administrator (using the My VMs "Request machine update" form or an email to eo-gpod at esa.int) which will enable your machine access for the time needed for you to download all the files.

Please note that, since data storage on the cloud ensures a cost, it is not guaranteee that your data will be stored for more than one week after the expiration of your VM.

- My machine has expired, how I can regain access?

If you need additional time access to your VM, you can request for it using the My VMs "Request machine update" form or an email to eo-gpod at esa.int .

- Who can help me?

For any clarification or other request, you can contact us via the Contact Us button at the left of this page or via an email to eo-gpod at esa.int .


Contributors to this page: .

Page last modified on Tuesday 30 of April 2013 13:01:42 CEST by .