Integrate vCO

0

VMware vSphere Autodeployment (virtuallyGhetto)

Did I already mention that I am a huge fan of the Autodeploy Script from William Lam? William created a bunch of script which help you to deploy a nested VMware environment in an automated way. The environment itself is built with a VSAN based datastore.

You can find the Blog Post from William here:

vGhetto Automated vSphere Lab Deployment for vSphere 6.0u2 & vSphere 6.5

 

and his GIT repository with the scripts here:

https://github.com/lamw

When you look at the Blog Post from William you can see that he uses a single ESX Hosts for his deployment. In case that you maybe have more ESX host there is a small issue in case that you have a VSAN based cluster. For the deployment of a nested VSAN environment on a VSAN bases Cluster you must set a VSAN Parameter (not recommended for production usage!). William makes these settings only for one host as you can see in this script


if($datastore.Type -eq "vsan") {
My-Logger "VSAN Datastore detected, enabling Fake SCSI Reservations ..."
Get-AdvancedSetting -Entity $vmhost -Name "VSAN.FakeSCSIReservations" | Set-AdvancedSetting -Value 1 -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile
}

 

In my environment a have a four node cluster hosts with HA and DRS configured. In some cases, DRS kicks-in during the enrollment of the nested environment. When a virtual ESX Hosts ends up on a Hosts were the settings was node made, the deployment fails.

In my case I made an improvement (for my usage) to the script where I configure the Fake SCSI settings for all Hosts of a cluster.

 

if($datastore.Type -eq "vsan") {
$FakeVSANhosts = $cluster | Get-VMHost | ForEach-Object {
My-Logger "VSAN Datastore detected, enabling Fake SCSI Reservations for Host $_ ..."
Get-AdvancedSetting -Entity $_ -Name "VSAN.FakeSCSIReservations" | Set-AdvancedSetting -Value 1 -Confirm:$false | Out-File -Append -LiteralPath $verboseLogFile
}

 

So, if someone of you runs into the same problem, just change the lines in the script from William and everything should work like expected.

0

Honeypot as a Service (HaaS) Part 3

This is the third Post of the Haas Series…..

After we have everything up and running would should take a deeper look at the vRealize Log Insight.  There we get our notifications in the interactive analytics. In normal cases nobody will monitor the whole time the interactive analytics therefore we create a Dashboard. The create dashboard is only for your custom view. The log Insight documentation state it like this “You can add, modify, and delete dashboards in your Custom Dashboards space.”

So, in the first step we create a dashboard for your personal view. The creating is quite easy. So, if you have an alert in the interactive analytics you can create the Dashboard from there.

 

 

Just click in the New Dashboard Icon on the right site. In the Wizard you create a new dashboard. Provide a Name and if you want share the Dashboard with other Users in the environment.

I use the chart graph for my Dashboard. After you are finished you will find your Dashboard in on the Dashboard View under Shared Dashboards.

Quite Easy or? But would It be better to get an Alert Notification from Log Insight?

This is also very easy. When you go back in the Interactive Analytics just take the Alerts Button.

In the Windows we choose “Create Alert from Query”.

There you provide the required Details for the Alert. Be sure that you configured your SMTP Server before when you use E-Mail as alert notification.

That was cool so far, or?

In the last months I made a couple of NSX Implementations. NSX provides us some really cool feature like the possibility to move VMs into Quarantine to isolate them from the communication with other VMs. So why wo should not use these feature, to migrate the VM were the Access violation was made, into a quarantine location to research what is happening on this VM? When the access was done from a physical machine, we can also create a firewall rule which deny the access to the virtual environment? From my point of view this is security in an automated way.

In this blog Post I will not show how to install and configure the NSX Part of this series. There are a lot of useful information available.

To archive these goal, we can use a REST API Call to the vRO Server to call a workflow which we develop to migrate the VMs or create the firewall rule. One of the first things which we need, is the possibility to interact with NSX from the VMware Orchestrator. Therefore, will install the VMware Orchestrator NSX Plugin. The plugin can be found on the VMware Page:

https://my.vmware.com/web/vmware/details?downloadGroup=NSXV_VROPLUGIN_120&productId=417

We also need two additional Plugins The first plugin is needed to decrypt the BASE64 encrypt string from the Shim in vRO. The Plugin can be found here: https://communities.vmware.com/docs/DOC-24991

The next Plugin is not really necessary but from my point of view the JSON Implementation in vRO is not the best in the world…. therefore, I use the jsonPath for Orchestrator Plugin frim Soeldner-Consulting. The documentation and the Link to the VMware Solution Exchange can be found here:

jsonPath for Orchestrator

After we download the all plugins we can install them via the vRO Control center.

After we are finished with that, we have to create our Workflow that must start when an alert is triggered. The triggering can be done via email or REST API Call. I prefer the REST-API Call du the circumstance that this more flexible, but wait…….here we have a problem. The VMware vRealize Log Insight doesn’t offer any option to create a REST API Call. In the Web GUI only Webhook Notification is available. Therefore, we need a translation between the Webhook and the REST API Call which we need for the vRealize Orchestrator. As always there is an solution available…..Steve Flanders create a Shim which can act as Proxy for the Webhook notification in direction of the VMware vRealize Orchestrator. Steve has a GIT Repository with a good documentation about the different available versions and the installation. I will not go further on the installation as already everything is explained here: https://github.com/vmw-loginsight/webhook-shims

Another useful post can you find here:

https://blogs.vmware.com/management/2017/03/webhook-shims-now-available-on-docker-hub.html

After we installed and configured the Shim, we are ready to make our next steps.

I created a vRO Workflow which used as Input parameter the name of the Attacker VM. The workflow checks the Attacker VM against the configured Datacenter. When it is a virtual machine, then an NSX Security Profile is attached to the virtual machine and all communication is denied. When it is not a virtual machine, then an IPSet is created and this IPSet is added to a Security Group

For the workflow it is necessary, that the NSX Manager is configured within vRO. Also, that a security Policy exist which we can use in the workflow. Also, the Datacenter which host our virtual machines in our environment must be provided.

Now let’s have a look at the Workflow:

The Workflow consists from different areas. In the First Part of the Workflow (the yellow box) we have to decode the Base64 string and parse the VM Name from the Log Insight Message.

When we are finished with that, we check if the Attacker Machine is a virtual machine (the red box).  When the machine is a VM, we apply the security Group to the VM (the light brown box) which denies all communication via the Security Policy.

When the Attacker VM is not a virtual machine then the machine will be checked against DNS (the green box) to gather the IP. The DNS checking is done via vRO DNS query so the Orchestrator Appliance must be configured correctly with DNS Server. The DNS Check is done via Name and FQDN. When the name could be resolved and we have the IP Address, then an IPSet is created and the Attacker machine is applied to the security Policy. This only protects the virtual environment!

In all cases a email is created (the blue box) and send to an Administrator.

The Workflow itself has some Attributes which must be filled:

  1. Datacenter –> The Datacenter were to check against the Attacke Machine if it is a VMs
  2. SecurityGroupID –> The Securitygroup ID. The ID is not the name but should look like this: securitygroup-98765. If you don’t know the SecurityGroupID, you can Browse within vRO in the NSX Manager Connection. There you can find the correct name
  3. NSXManager –> Your corresponding NSX Manager (which must be configured before in vRO)
  4. smtpHost –> Your Mail Server
  5. fromName –>  Email Sender Name
  6. fromAddress –>  Email Sender Address
  7. toAddress –> Were the Mail should arrive
  8. DNSDomain –> Your DNS Name to check against FQDN
  9. Optional: SMTPUsername –> The Mail Server Username, if required
  10. Optional: SMTPPassword –> Password for the Mail Server user

For the Workflow itself there is room for optimizing. A loop to check against different Datacenter, or a loop to check against different Domain names……For me and this Demo the Workflow makes his job….

The Workflow can be found here: http://www.vcoportal.de/download/workflow/de.vcoportal.HaaS_.package

 

Honeypot as a Service (HaaS) Part 1 Link: http://wp.me/p7tsEp-C1
Honeypot as a Service (HaaS) Part 2 Link: http://wp.me/p7tsEp-Cb

0

Honeypot as a Service (HaaS) Part 2

In the second Post of this series, we create the Log File Reporting which we will send to vRealize Log Insight.

 

After our samba server is configured (in Part 1) we have to deal with our logfiles and report them to our vRealize Log Insight.

In our Samba Server we already configured the logging which is done in the path /var/log/samba. There files are created for every client which tries to access our server. As long as no client tried to access the file the director should log like this:

When you already accessed your server, you will have a log entry like log.192.168.157.10 or similar.

I will create an event every time a client is accessing the share. Therefore, I will use the inotify-tools. unfortunately, these program is not included the standard CentOS Repository so we have to add an additional repo. I use the fedoraproject repo where we download the Repo configuration and install it.


wget http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-10.noarch.rpm

rpm -ivh epel-release-7-10.noarch.rpm

After that we can install the inotify-tools with yum.


yum -y install inotify-tools-devel.x86_64

After that we create the string to monitor the samba server for client access.


inotifywait -mrq -e create -e modify -e delete /var/log/samba

In this monitoring we ignore changes to the log.smbd file, due the circumstance that this a a standard file form the samba service. The output of this monitoring locks like this:


/var/log/samba/ MODIFY log log.client1-vcoportal

As we want the information on a remote server, we have to extend our logging string with logger to send the information to a remote server.


inotifywait -mrq -e create -e modify -e delete /var/log/samba  | while read file; do ( logger -n loginsight.vcoportal.de-t securitybreak $file ) done

I send my string to a vRealize Log Insight server and tag the log entry with the word “securitybreak”. In Log Insight we can filter for that word……

So, if you use the interactive analyses you can search for your string.

For me the is still room for improvement. The only relevant information for me is the client name. So, let’s modify the monitoring string.


inotifywait -mrq -e create -e modify -e delete /var/log/samba/host | while read file; do hostname=$(echo $file | cut -d '.' -f2,3,4,5 ); (logger -n loginisght.vcoportal.de -t securitybreak $hostname) done

 

We strip done the string we send to our Log Insight Server to just the Client name. I guess there is a better way to create the string but for me this one work fine.

Now we get only the Client name reported in our Log Insight

How cool is that?

The last thing we have to do is to create a startup script for our monitoring string.

In our case, we will want to run inotifywait as a service and create an init script. First we will create our configuration file:


# specify log file

LOGFILE=/var/log/inotify.log

# specify target directory for monitoring

MONITOR=/var/log/samba

# specify target events for monitoring ( comma separated )

# refer ro man inotifywait for kinds of events

EVENT=create,delete,modify

# Log Insight or SNMP Server

SNMPSERVER=loginsight.vcoportal.de

# Security tag

SECURITYTAG=securitybreak

 

Next, we will create the init script where we use the config file we created before as input for the variables.


#!/bin/bash

# inotifywait: Start/Stop inotifywait

#

# chkconfig: 2345 10 90

# description: inotifywait waits for changes to files using inotify.

#

# processname: inotifywait

. /etc/rc.d/init.d/functions

. /etc/sysconfig/network

. /etc/inotifywait.conf

LOCK=/var/lock/subsys/inotifywait

RETVAL=0

start() {

echo -n $Starting inotifywait:

# Clear Log File

if [ -e $LOGFILE ];then /usr/bin/truncate -s0 $LOGFILE;fi

# Start the Monitoring

/usr/bin/inotifywait -e $EVENT -dmrq $MONITOR --exclude ‘\(log.smbd\)*’ -o $LOGFILE | cut -d '.' -f2,3,4,5 $LOGFILE | while read file; do (/usr/bin/logger -n $SNMPSERVER -t $SECURITYTAG $file) done

RETVAL=$?

echo [ $RETVAL -eq 0 ] && touch $LOCK return $RETVAL

}

stop() {

echo -n $Stopping inotifywait:

killproc inotifywait

RETVAL=$?

echo [ $RETVAL -eq 0 ] && rm -f $LOCK return $RETVAL

}

case $1 in start)

start;;

stop)

stop;;

status)

status inotifywait;;

restart)

stop

start;;

*)

echo $Usage: $0 {start|stop|status|restart}

exit 1

esac

exit $?

 

The permissions need to be 755:


chmod 755 /etc/rc.d/init.d/inotifywait

 

We can start the service now:


/etc/rc.d/init.d/inotifywait start

 

The result will be like this:


[root@backup ~]# /etc/rc.d/init.d/inotifywait start

Starting inotifywait (via systemctl):                      [  OK  ]

 

We need to ensure that the service starts when the server boots, so we need to add it to chkconfig


chkconfig --add inotifywait

chkconfig inotifywait on

 

The log entry is written in the /var/log/inotify.log file and from there they were cut down to post the to the log insight server. We have to use this loop due the circumstance, that in our start script inotiywait is used in daemon mode. There a logfile is mandatory.

That’s it for Part 2

 

Honeypot as a Service (HaaS) Part 1 Link: http://wp.me/p7tsEp-C1
Honeypot as a Service (HaaS) Part 3 Link: http://wp.me/p7tsEp-Cl

0

Honeypot as a Service (HaaS) Part 1

Disclaimer:

This series will show some Ideas which include “homemade” Security Functions. For professional environments, I recommend to talk to the security Vendor of your choice to build up a secure environment. This series will show examples which include different VMware products and their Interaction and teach things which can be done with them.

Now, let’s start.

Today I will start a series which I call Honeypot as a Service (HaaS). What is the Idea behind the HaaS?

In most environments hackers are already a long time present bevor the get localized. When the hacker has access on one of the server, he will start to search for other systems where he can get access and grab sensitive information (credit card data, intellectual property). So, which are system / functions a hacker is locking for?

So, most Hacker are looking for routing information and will start a “slow and silence network scan” for systems which he can attack and get into his hands. When we now start to place virtual machines in our network, which doesn’t have any functions for our network nobody should access them in normal case. From the standpoint of an attacker, he doesn’t know if a system is relevant or not. That’s will be the point for us, to provide the hacker a system with an old infectible SMB Server which he can attack. So, if somebody / somewhat tries to access such an “no functional VM”, we generate a log entry which is forwarded to an Log system. There we can parse the logs and generated “automated” rules to isolate VMs or VM communications.

In this example, I will use a lot of different VMWare products to archive these HaaS environment and the actions we will use.

This series will include:

  • VMware vRealize Log Insight as log host
  • VMware vRealize Orchestrator as “workhorse for a lot of task”
  • VMware NSX for VM isolation and Firewall Rules
  • A small Linux System as Honeypot with an SAMBA Service

So, let start to build up a Linux System which we can use in our vRA Blueprint as Honeypot. I prefer a CentOS Distribution. I choose a 7.x Release of CentOS. The ISO can be downloaded here http://ftp.hosteurope.de/mirror/centos.org/7/isos/x86_64/

I will use the minimal Installation ISO and Install Samba and SSH afterwards.

We will change the Hostname and IP Settings for a later time in vRA so during installation we keep the hostname and network on default settings. We have one important things which we take care about during installation. By default, the NICs are not automatically connected during installation, so that should be changed.

After our Installation has finished we will install the following packages:

  • Perl
  • Samba
  • DNS tools (just for Troubleshooting)
  • Network Manager (Text Version if you need to reconfigure your Config)

This can be done via this command:

yum -y install perl samba system-config-network-tui.noarch dns-utils

When you don’t have a DHCP Server in place, then you need to configure your network settings on Command line: $ sudo ifconfig eth0 192.168.1.50 netmask 255.255.255.0

Also, you need to set your default route:

$ sudo route add default gw 192.168.1.1 eth0

As last step you need some DNS Server entries:

$ sudo vi /etc/resolv.conf

Modify or enter nameserver as follows:

nameserver 8.8.8.8

After we have installed the need packages we can start with the installation from the VMware tools. This is documented well in the VMware documentation so I will not write it down here.

The next Step is to configure our Samba Server for a SMB Share which is available for the attacker.

Now, let us create a fully accessed anonymous share for the users. Everybody can read/write in this share.

We create a directory called /samba/backup and set full permission. The reason (for me) to set the name to backup is, to mark the share as important and a useful target which information or potential interesting thinks in it. I also will create some additional folders to have a interesting target. I use a linux bash script to create the folder and set the required permissions.

#!/bin/bash
echo "This script creates a bunch of folder which were used in a samba share example on vcoportal.de"

dirname=/samba/backup

echo "$dirname"

if [ ! -d "/$dirname" ]

then

echo "Folder doesn't exist. Creating now"

mkdir --p "/$dirname"

echo "Create some other Folders ....."

mkdir --p "/$dirname/HR"

mkdir --p "/$dirname/IT"

mkdir --p "/$dirname/Dev"

mkdir --p "/$dirname/Public"

mkdir --p "/$dirname/Files"

mkdir --p "/$dirname/Userhome"

echo "Set some permissions for the parent Folder"

chmod -R 0775 /samba/backup

echo "Set nobody as Owner for the Folder so that everybody can read see and read it"

chown  nobody:nobody /samba/backup

echo "Change SE Linux Permissions on the Folder"

chcon -t samba_share_t /samba/backup

else

echo "Parent Directory exists"

fi

Just create a file and make it executable with the script content in it. You can change the directory names to whatever you want / need.

After we created the directory we must edit the Samba configuration file.

vi /etc/samba/smb.conf
# See smb.conf.example for a more detailed config file or
# read the smb.conf manpage.
# Run 'testparm' to verify the config is correct after
# you modified it.
 
[global]
        workgroup = vcoportal
        security = user
 
        passdb backend = tdbsam
 
        unix charset = UTF-8
        dos charset = CP932
 
        hosts allow = 127. 192. 172. 10.
 
        max protocol = SMB2
 
 
        map to guest = Bad User
 
        # log files split per-machine (the host logs are stored on a non-default path):
        log file = /var/log/samba/log.%m
Log level = 2
        # maximum size of 50KB per log file, then rotate:
        max log size = 50
 
 
[homes]
        comment = Home Directories
        valid users = %S, %D%w%S
        browseable = No
        read only = No
        inherit acls = Yes
 
 
[Backup]
comment = Backup Share
path = /samba/backup
browsable =yes
writable = yes
guest ok = yes
read only = no
force user = nobody

Now we can test the Samba server configuration

We can test the Samba server configuration syntax errors using the command ‘testparm’.

testparm

When everything looks good you can browse to your samba share.

 

 

 

 

That’s it for Part 1

 

Honeypot as a Service (HaaS) Part 2 Link: http://wp.me/p7tsEp-Cb
Honeypot as a Service (HaaS) Part 3 Link: http://wp.me/p7tsEp-Cl

0

vRO and the OpenStack Admin API

In this post we will start to interact with the OpenStack admin API. If you already followed my first two post, many things here are similar. The first two post can be found here:

Part 1:

http://www.vcoportal.de/2015/02/vro-and-openstack-thoughts-on-the-orchestrator/

Part 2:

vRO and the OpenStack Public API

So before we can start to interact with the Admin API we have to create a new REST Host entry in vRO.

The approach to create the Rest Host is the same then described in the Posts before. The difference here is the used Port which we take to access the Rest Host. If you have created different IPs for your OpenStack environment and access the Admin API over a different IP (which can be configured in the files for the service)  you have to adjust your URL. Like before we doesn’t use authentication for the Rest Host.

After we have created the REST Entry we have to take a look at the Authentication. For the Administration OpenStack uses an Admin Token which is created during installation. We can use this Admin Token for Authentication. You can find this token in the file /etc/keystone/keystone.conf

We can use that token to operate the Admin Interface. From Security prospective we must be aware, that everyone who can execute the Workflow with the Token has administration rights. So if you use the Workflow in production limit the access to the Workflow!

So let’s create a new Action element. I named it OpenStackCreateUser. In the Action we need some Inputs.

After we have created the Inputs we have to insert the code for our need:

// We need a Json fomated String for Authentication. We create the string with this Workflow

var content = '{"user": {"id": "' + TenantID + '","name": "' + Username + '","email": "' + Email + '","enabled": ' + UserEnabled + '}}';

//Authenticate the request with the Admin-Token

var SessionRequest = RestHost.createRequest("POST", "/v2.0/users", content);

SessionRequest.setHeader("X-Auth-Token", SessionID);

SessionRequest.contentType = "application/json";

var SessionResponse = SessionRequest.execute();

// Show the Output

System.log("Session Response: " + SessionResponse.contentAsString);

I insert some Comments in the Code for explanation. Take a look at the

var content = '{"user": {"id": "' + TenantID + '","name": "' + Username + '","email": "' + Email + '","enabled": ' + UserEnabled + '}}';

code. Be aware that if you have to use a type of Boolean you are not allowed to use double quotes for the value.

After we finished our Action element we can build up our Workflow. I created a new Workflow with the Name “OpenStackAddUser”

In this Workflow we add the Action Element OpenStackGetTenantID which we created before.

Here we use the Visual Binding Editor to create the In- and Outputs. After we are finished the Visual Bindings should like this:

Next we insert the Action element “OpenStackCreateUser” which we build up before.

Also here we use the Visual Binding Editor to create the needed In- and Outputs.

As you can see we doesn’t have an output from the action since the OpenStack API doesn’t provide one for that operation. We can see the success in the log files which we write to System.log.

After Validation of the Workflow we can start with a first run.

Some notes for the run:

–          We use the Rest Host Admin API Connection.

–          We Use the Session ID from the admin_token value in /etc/keystone/keystone.conf

–          We provide a Username and an Email Address.

–          The TenantID is catched from the Tenant Name dynamically

If everything went well you should see and output like mine.

 

 

So now let’s verify this on a console. As we can see the user is created

 

 

This is only a short example hot to work with vRO and OpenStack. As you could see a lot is possible and could be done in an easy way.

I have uploaded the Actions and Workflows with I showed in the Blogpost on Flowgrab (https://flowgrab.com/). From there you can install the vCO Plugin and download the content directly into your vCO.

 

 

I you have feedback, comments or when I showed / explained something wrong please comment.

Have fun and Orchestrator the World 😉