About Author: Christian Strijbos

Posts by Christian Strijbos

0

vRO and the OpenStack Admin API

In this post we will start to interact with the OpenStack admin API. If you already followed my first two post, many things here are similar. The first two post can be found here:

Part 1:

http://www.vcoportal.de/2015/02/vro-and-openstack-thoughts-on-the-orchestrator/

Part 2:

vRO and the OpenStack Public API

So before we can start to interact with the Admin API we have to create a new REST Host entry in vRO.

The approach to create the Rest Host is the same then described in the Posts before. The difference here is the used Port which we take to access the Rest Host. If you have created different IPs for your OpenStack environment and access the Admin API over a different IP (which can be configured in the files for the service)  you have to adjust your URL. Like before we doesn’t use authentication for the Rest Host.

After we have created the REST Entry we have to take a look at the Authentication. For the Administration OpenStack uses an Admin Token which is created during installation. We can use this Admin Token for Authentication. You can find this token in the file /etc/keystone/keystone.conf

We can use that token to operate the Admin Interface. From Security prospective we must be aware, that everyone who can execute the Workflow with the Token has administration rights. So if you use the Workflow in production limit the access to the Workflow!

So let’s create a new Action element. I named it OpenStackCreateUser. In the Action we need some Inputs.

After we have created the Inputs we have to insert the code for our need:

// We need a Json fomated String for Authentication. We create the string with this Workflow

var content = '{"user": {"id": "' + TenantID + '","name": "' + Username + '","email": "' + Email + '","enabled": ' + UserEnabled + '}}';

//Authenticate the request with the Admin-Token

var SessionRequest = RestHost.createRequest("POST", "/v2.0/users", content);

SessionRequest.setHeader("X-Auth-Token", SessionID);

SessionRequest.contentType = "application/json";

var SessionResponse = SessionRequest.execute();

// Show the Output

System.log("Session Response: " + SessionResponse.contentAsString);

I insert some Comments in the Code for explanation. Take a look at the

var content = '{"user": {"id": "' + TenantID + '","name": "' + Username + '","email": "' + Email + '","enabled": ' + UserEnabled + '}}';

code. Be aware that if you have to use a type of Boolean you are not allowed to use double quotes for the value.

After we finished our Action element we can build up our Workflow. I created a new Workflow with the Name “OpenStackAddUser”

In this Workflow we add the Action Element OpenStackGetTenantID which we created before.

Here we use the Visual Binding Editor to create the In- and Outputs. After we are finished the Visual Bindings should like this:

Next we insert the Action element “OpenStackCreateUser” which we build up before.

Also here we use the Visual Binding Editor to create the needed In- and Outputs.

As you can see we doesn’t have an output from the action since the OpenStack API doesn’t provide one for that operation. We can see the success in the log files which we write to System.log.

After Validation of the Workflow we can start with a first run.

Some notes for the run:

–          We use the Rest Host Admin API Connection.

–          We Use the Session ID from the admin_token value in /etc/keystone/keystone.conf

–          We provide a Username and an Email Address.

–          The TenantID is catched from the Tenant Name dynamically

If everything went well you should see and output like mine.

 

 

So now let’s verify this on a console. As we can see the user is created

 

 

This is only a short example hot to work with vRO and OpenStack. As you could see a lot is possible and could be done in an easy way.

I have uploaded the Actions and Workflows with I showed in the Blogpost on Flowgrab (https://flowgrab.com/). From there you can install the vCO Plugin and download the content directly into your vCO.

 

 

I you have feedback, comments or when I showed / explained something wrong please comment.

Have fun and Orchestrator the World 😉

0

vRO and the OpenStack Public API

This is the second post regarding the interaction of vRealize Orchestrator and OpenStack. The first Post can be found here:

vRO and OpenStack thoughts on the Orchestrator

As I already mentioned in the first post it is possible to interact from vRO with OpenStack. Before we start with an example I would point you to the OpenStack API structure.

OpenStack knows two different APIs.

–          The Public API

–          The Admin API

The APIs are accessible over different Ports. The Standard Port for the Public API is port 5000. The Admin API is accessible over Port 35357. The Ports can be changed in the configuration files. I used the standard ports in my environment. So when you try to build the workflows for our environment take a look at the used ports. I created my OpenStack Installation on a single host. This isn’t the best configuration and most installations will have different servers for the services. This is also something which a take care when you start to create the Demos.

Now let’s come back to the admin and the public API. The public API offers some functions but not all. For example you cannot create a user on the public API. Therefore you have to use the Admin API.

For the two APIs you have to use different authentication ways but we will come to that later.

OpenStack offers a lot of documentation regarding the functions which are available on the API. Here is a links which are useful for the development of the workflows.

http://docs.openstack.org/api/openstack-identity-service/2.0/content/preface.html

Every service in OpenStack has such a documentation. In my examples I will work with the Identify services were the link above pointed to.

First thing we have to do is to add the Rest Host to vRO. Due the circumstance, that you must use different ports to access the public and admin API you have to create to REST Hosts with different ports.

Just insert the needed information’s in the workflow for the host. If you server provide the possibility to use HTTPS just change the URL to use the secure Protocol to access the API. This is recommended!

I doesn’t need a proxy server so I can go  fast over this screen.

The last page is for Authentication. We provide the user information in our workflow so we doesn’t need the Host Authentication and set it to “none”.

After we have insert the REST Host we can start with the first workflow. Before we can “access” public API functions we have to authenticate to the Identity service.

The authentication is done with JSON formatted call to the API. The API response is also in JSON. To create the Authentication Token we have to make the Call to the OpenStack Server. The call is done as a PUT command to the /v2.0/tokens extension.

Here a quote from the API documentation mentioned above.

Client authentication is provided through a ReST interface by using the POST method with v2.0/tokens supplied as the path. Include a payload of credentials in the body.

The Identity API is a ReSTful web service. It is the entry point to all service APIs. To access the Identity API, you must know its URL.

Each ReST request against the Identity Service requires the X-Auth-Token header. Clients obtain this token, along with the URL to other service APIs, by first authenticating against Identity with valid credentials.

So, let’s use this information for a first workflow or better a first action which handles the authentication. My Action is named “OpenStackCreateAuthToken”. In vRO we start with a new action element. The Action itself need some parameter.

This parameter are required to specific the REST Host with parameter, Username, Password and so on. In the Scripting Section we insert this code:

// We need a Json formatted String for Authentication. We build together the JSON String with the information we request for the Workflow.

var content = '{"auth": {"passwordCredentials": {"username": "' + authUserName + '","password": "' + authPassword + '"}}}';

// After we have built the Json String we create the Session and insert the Answer in the Response

var SessionRequest = RestHost.createRequest("POST", "/v2.0/tokens", content);

SessionRequest.contentType = "application/json";

var SessionResponse = SessionRequest.execute();

// Show the Content

System.log("Session Response: " + SessionResponse.contentAsString);

//Split the response as Json

var JsonContent = JSON.parse(SessionResponse.contentAsString);

// Here we get the ID with is used for Authentication in the following sections.

var SessionIDKey = JsonContent.access.token.id

// Just for information

System.log(" Session Key: " + SessionIDKey);

// The Session ID Key were will work with

return SessionIDKey

I provided required information in the code itself for explanation.

After we have created our action, we create a New Workflow. I put it in a OpenStack Folder with the name “OpenStackGetTenants”.

On the Schema Tab I added the Action which I created before.

Now we need to Create Inputs for the Action. You can do this via Drag and Drop on the Visual Binding.

In the Attributes, I already insert some Values for the connection.

These values are required for the connection. Now let’s start with a first run.

When everything worked fine you become an Output similar to mine.

As we can see we got a session key which we can use in another workflow or action.

Now let’s create a second Action. I named it “OpenStackGetTenantID”. Like before we need some Inputs.

A lot of them we already used before.

Know lets create the script.

//Authenticate the request with the X-Auth-Token

var SessionRequest = RestHost.createRequest("GET", "/v2.0/tenants", null);

SessionRequest.setHeader("X-Auth-Token", SessionID);

var SessionResponse = SessionRequest.execute();

// Log which answer did we get from the API Request

System.log("Session Response: " + SessionResponse.contentAsString);

//Let's split the API Response to filter for our wanted information

var JsonContent = JSON.parse(SessionResponse.contentAsString);

var Tenants = JsonContent.tenants

// Now we want search for our specified tenant (Name)

for (var idx=0; idx < Tenants.length ; idx++) {

if (Tenants[idx].name.toLowerCase() == TenantName.toLowerCase()){

var TenantID = Tenants[idx].id;

}

}

// Let's provide the Output when it is not null

if (TenantID != null) {

System.log("Tenant Name: " + TenantName + " found");

System.log(" Session Key: " + TenantID);

return TenantID;

}

else{

System.log("Tenant not found!");

}

After we finished the Action we extend our Workflow we created before.

First we create an additional attribute to output the Tenant ID which can be used in following workflows.

After that we use the “Visual Binding” Editor to bind the parameter.

Validate the workflow and let’s run it.

As you can see I have an additional input. I use this input to filter the response for a specific Tenant. As you can see we found the tenant.

 

So this was an example of how to use the OpenStack Public API. In the next post we interact with the Admin API.

Have fun and orchestrate the World 😉

0

vRO and OpenStack thoughts on the Orchestrator

Some of you are maybe playing with OpenStack. Some of you with the native OpenStack of in a Distribution of your choice others with the VMware Integrated OpenStack (VIO) Beta. The VIO Beta is actual available as Beta. Information’s about VIO and other OpenStack Staff in context of VMware can be found here http://blogs.vmware.com/openstack/

Some general information about OpenStack and the different Modules are documented in the OpenStack Documentation. I want to include some Information from here: http://docs.openstack.org/juno/install-guide/install/yum/content/ch_overview.html

About the different modules:

Overview

The OpenStack project is an open source cloud computing platform that supports all types of cloud environments. The project aims for simple implementation, massive scalability, and a rich set of features. Cloud computing experts from around the world contribute to the project.

OpenStack provides an Infrastructure-as-a-Service (IaaS) solution through a variety of complemental services. Each service offers an application programming interface (API) that facilitates this integration. The following table provides a list of OpenStack services:

Table 1.1. OpenStack services

Service Project name Description
Dashboard Horizon Provides a web-based self-service portal to interact with underlying OpenStack services, such as launching an instance, assigning IP addresses and configuring access controls.
Compute Nova Manages the lifecycle of compute instances in an OpenStack environment. Responsibilities include spawning, scheduling and decommissioning of virtual machines on demand.
Networking Neutron Enables Network-Connectivity-as-a-Service for other OpenStack services, such as OpenStack Compute. Provides an API for users to define networks and the attachments into them. Has a pluggable architecture that supports many popular networking vendors and technologies.

Storage

Object Storage Swift Stores and retrieves arbitrary unstructured data objects via a RESTful, HTTP based API. It is highly fault tolerant with its data replication and scale out architecture. Its implementation is not like a file server with mountable directories.
Block Storage Cinder Provides persistent block storage to running instances. Its pluggable driver architecture facilitates the creation and management of block storage devices.

Shared services

Identity service Keystone Provides an authentication and authorization service for other OpenStack services. Provides a catalog of endpoints for all OpenStack services.
Image Service Glance Stores and retrieves virtual machine disk images. OpenStack Compute makes use of this during instance provisioning.
Telemetry Ceilometer Monitors and meters the OpenStack cloud for billing, benchmarking, scalability, and statistical purposes.

Higher-level services

Orchestration Heat Orchestrates multiple composite cloud applications by using either the native HOT template format or the AWS CloudFormation template format, through both an OpenStack-native REST API and a CloudFormation-compatible Query API.
Database Service Trove Provides scalable and reliable Cloud Database-as-a-Service functionality for both relational and non-relational database engines.

This guide describes how to deploy these services in a functional test environment and, by example, teaches you how to build a production environment. Realistically, you would use automation tools such as Ansible, Chef, and Puppet to deploy and manage a production environment.

 Conceptual architecture“

As graphical view this is a good oversight about the different modules and how they work together. The drawing this from the OpenStack documentation. The original can be found here: http://docs.openstack.org/juno/install-guide/install/yum/content/ch_overview.html

Independent which “Version” of OpenStack you use, the APIs and components are always the same. From automation perspective the Project Heat is interesting.

Quote (https://wiki.openstack.org/wiki/Heat):

OpenStack Orchestration

The mission of the OpenStack Orchestration program is to create a human- and machine-accessible service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds.

Heat

Heat is the main project in the OpenStack Orchestration program. It implements an orchestration engine to launch multiple composite cloud applications based on templates in the form of text files that can be treated like code. A native Heat template format is evolving, but Heat also endeavours to provide compatibility with the AWS CloudFormation template format, so that many existing CloudFormation templates can be launched on OpenStack. Heat provides both an OpenStack-native ReST API and a CloudFormation-compatible Query API.

Why ‘Heat’? It makes the clouds rise!

How it works

  • A Heat template describes the infrastructure for a cloud application in a text file that is readable and writable by humans, and can be checked into version control, diffed, &c.
  • Infrastructure resources that can be described include: servers, floating ips, volumes, security groups, users, etc.
  • Heat also provides an autoscaling service that integrates with Ceilometer, so you can include a scaling group as a resource in a template.
  • Templates can also specify the relationships between resources (e.g. this volume is connected to this server). This enables Heat to call out to the OpenStack APIs to create all of your infrastructure in the correct order to completely launch your application.
  • Heat manages the whole lifecycle of the application – when you need to change your infrastructure, simply modify the template and use it to update your existing stack. Heat knows how to make the necessary changes. It will delete all of the resources when you are finished with the application, too.
  • Heat primarily manages infrastructure, but the templates integrate well with software configuration management tools such as Puppet and Chef. The Heat team is working on providing even better integration between infrastructure and software.

What does this mean for a VMware Administrator which want to orchestrate and automate his VMware and OpenStack environment?  At the Moment Heat is primary focused on the OpenStack environment. When you want integrate your Automation solutions you have to use other Automation Tools. I would speak from Infrastructure Orchestrator and Application Orchestration.

In this case Heat is the Infrastructure Orchestrator which offers the possibility to create new virtual machines. For the communication from Heat to VMware and API must be used to provide the Information from Heat to vRO. That is a possible way to Automate and Orchestrate the OpenStack Environment. When we take VMware into the Account, the drawing becomes somewhat more complex.

vCO is used to provision virtual Machines on the VMware Site. Heat is used to provision virtual Machines on the OpenStack Site.

I want to keep my Orchestration and Automation part as simple as possible so why we don’t only use vCO for provisioning new VMs?

From OpenStack site, you lose somewhat flexibility because can cannot use Tosca as description Language any more but from Orchestration Site it is simpler.

In my next Blog Post I will show you the way how vCO can talk with OpenStack.

Have fun and Orchestrate the World 😉

0

New Version of OVF Transfer Plugin for vCO released

After a long time of development I am pleased to announce the availability of the New Version of the OVF Transfer Plugin. We reviewed the whole code of the old Plugin and rewrote it. With this Version we also improved the available feature.  The Plugin Description reads as flowing:

The OVA/F Transfer plug-in allows you to import and export virtual machines as OVF/OVA template to/from the VMware vCenter via the VMware vCenter/vRealize Orchestrator.

This plug-in provides actions and workflows to use the OVF/A functionalities directly in your own workflows.

Here are some functions of the new Plugin:

This plug-in includes amongst others following features:

  • [Export]  Export of virtual machines as OVF templates
  • [Import]  Import of virtual machines as OVF or OVA templates
  • [Import]  Support for vSS and vDS PortGroups
  • [Import]  Support for multiple vNics
  • [Import]  Support for OVF properties, e.g. used for virtual appliance imports
  • [Import/Export] Supported sources/destinations: Locale file, HTTP, HTTPS, FTP, CIFS

We included a lot of new functions which some of you required and fixed all Bugs which were reported.

For this Plugin the following types of sources are supported:

–       Local file (local means on vCO Server e.g. vCO appliance or vCO Windows installable)

–       HTTP, HTTPS

–       FTP

CIFS/Network Share (only with vCO Windows installable version

The Plugin is available at the VMware Solution Exchange https://solutionexchange.vmware.com/store/products/vmware-vcenter-orchestrator-ovf-transfer-plug-in  and is free of charge.

Many thanks to my colleague Sascha Bitzer who wrote and maintains the Plugin!

0

The Standalone VMRC Console and permissions…….

This Week I had a Customer requirement were the usage of the VMRC Plugin could be a good solution to resolve the issue. The customer had the requirement to use virtual machines as VPN gateways for connections outside his network. Therefore virtual machines were created and placed into a DMZ Zone. The customer Idea was, that his service staff could use the virtual machines to make VPN connections to external customers. One of the problems here is, that most VPN connections cut all other connections when the VPN connection is established.  These leads to the problem that RDP or VMware View connections could not be used because they are dropped when the tunnel came up……The Service people should not use a “full” Web Client therefore I made the suggestion to use the Standalone VMRC Console.

For those who are not familiar with the Standalone VMRC Console William Lam made a good post on it which could be found here: http://www.virtuallyghetto.com/2014/10/standalone-vmrc-vm-remote-console-re-introduced-in-vsphere-5-5-update-2b.html

After installing and configuration of the VMRC Console I created a new role on the vCenter Server. This role had the permissions to interact with the Client Console and some other stuff like stopping and starting the VM.

After that, I created used a group which was provided with the created role and permissions.

When I was finished, I made a test with the VMware Web Client and could access the console without any problems. I created links for the user so that could connect directly with the virtual machine without starting the VMware Web Client. The links I created was from type: vmrc://[VC]/?moid=[VM-MOREF-ID]

I didn’t used a Username in the call so every user could start the VM Console with his user and password. For me as administrator of the environment everything worked link expected…..

We rolled out the solution to the user. The user were requested to provide a Username and password but then get and error message.

I did some research and that but could find an answer in the first step…..after a chat with Joerg Lew  he pointed me to the right direction……Thanks Jörg!

When you use the Standalone VMRC Console with Username and Password the VMRC Console redirect the connections after the initial connection with the vCenter Server to the ESXi host. So I had to include the group with the permissions on the ESXi Hosts.

After that, the VMRC Console worked like a charm and we could solve the customer requirement without the usage of the VMware Web Client.

So have fun and orchestrate the World 😉