VM stuck in invlaid state after export

After trying to export a VM and cancelling the export it could happen that the VM is nolonger reponsive (no start, unregister, delete, … possible)

The hostd.log (in the /var/log directory on the ESXi) will show an error similar to:

2021-09-07T10:55:19.583Z error hostd[2099544] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/XXXXXXXX-05c01598-574e-88d7f6d5ef52/myvm/myvm.vmx opID=esxui-6c72-aafe user=root] Invalid transition requested (VM_STATE_EXPORTING -> VM_STATE_DELETING): Invalid state
2021-09-07T10:55:19.583Z warning hostd[2099544] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/XXXXXXXX-05c01598-574e-88d7f6d5ef52/myvm/myvm.vmx opID=esxui-6c72-aafe user=root] Method fault exception during VM destroy: Fault cause: vim.fault.InvalidPowerState

It seems the VM is somehow stuck in the Exporting-state and therefore no other operation is possible on the VM .

As a workaround log in to the ESXi Host and restart the Management Agents (https://kb.vmware.com/s/article/1003490)

Bash – Monitor directory for sizechange

A simple bash-script to easily monitor if a directory has grown or shrunk in size:

while [ 1 ]; do result=$(du -s * | egrep "bitcoin-0.21.1$"); echo -e "\e[95m$result\e[0m"; curSize=$(echo $result | cut -d" " -f1); if [ $curSize -lt $oldSize ]; then echo -e
 "\e[92mShrunk: $curSize\e[0m"; else echo -e "\e[91mGrown: $curSize\e[0m"; fi; oldSize=$curSize; sleep 5; done

Script needs to be executed in the parrent directory of the monitored dir and directory name must be adapted: bitcoin-0.21.1$ -> to whatever you want to grep for

Start PS-Admin session from an unprivileged user

 runas  /user:administrator 'powershell -command Start-Process powershell -verb runas' 

PowerCLI batch revert to Snapshot

The following script-let can be used to revert a bunch of VMs/all VMs in a folder back to the first snapshot made for the VM

$vms = Get-Folder "MY_VM_FOLDER_NAME" | Get-VM
 
foreach($vm in $vms)
{
    $vm.name
    $snap = Get-Snapshot -VM $vm | Sort-Object -Property Created -Descending | Select -First 1
    Set-VM -VM $vm -SnapShot $snap -Confirm:$false
    echo "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
}

MIkrotik packet loss (Ping <70%)

Today I experienced an interesting story with my Mikrotik router at home. While updating the PiHole instance the system hat quite some problems obtaining either system updates but also the PiHole update packages. A ping on 9.9.9.9 showed that the Raspberry – on which the PiHole was hosted – had somewhat between 70-80% packet loss. Pinging the same IP from a Windows machine resulted in 0% packet.

All this happened only on wired connections but did not cause any problmes when connected via wifi.

As the Raspberry is directly attached to the Mikrotik router I also tried to connect it via a switch in between as that’s the setup for the windows machine. Same behavior.
Running pings from 3 Linux systems in the network and two Windows systems (even with exchanging network connections to be directly connected to the Mikrotik and connecting through the switch) came up with an interesting result:
All Windows machines had hardly any packet loss in 10 mins (<3%) and all the Linux systems had somewhat between 70%-80% packet loss (tested with a ping).
Any ping that involved the Mikrotik routers L2 functionality seemed to result in packet loss on the Linux machines.
Pinging any other machine on the same subnet worked without problems, but as soon as there was one hob in between the problem occurred.

Interestingly the problem vanished as far as the Torch tool was activated and no more packet loss occurred on any of the systems.

After some additional troubleshooting time (and disabling nearly all Mikrotik-configuration -> Firewall-Rules/Interfaces) the problem seemed to be with the Bridge interface used. It seems that the deactivation of the IP Firewall for the bridge interface caused the problem. After enabling it the behavior vanished and all systems no longer had any packet loss issues.

[admin@MikroTik] /interface bridge settings> /interface bridge settings 
[admin@MikroTik] /interface bridge settings> set use-ip-firewall yes    

Monitoring Tasmota with Zabbix

The repository contains a python-script which can be used to subscribe to an MQTT-server (e.q. mosquitto) and forward all MQTT-message to Zabbix. There the Template can be used to monitore the values sent via MQTT. Currently the Template is mainly intended as an example and requires adaption to your current setup/configuration of tasmote.

https://gitlab.com/fawcs/zabbix_tasmota_mqtt

CVSS-Regex

Just a little regex to validate CVSSvector strings

^CVSS\:\d\.\d\/AV\:[N,A,L,P]\/AC\:[L,H]\/PR\:[N,L,H]\/UI\:[N,R]\/S\:[U,C]\/C\:[N,L,H]\/I\:[N,L,H]\/A\:[N,L,H]$

Additional JS-Scripts for CVSS-cacluation:

https://www.first.org/cvss/v3.1/use-design

Batch import beats dashboards

cd /usr/share/
for BEAT in $(ls | grep -e "beat$"); do
echo -e "\e[92mBEAT: $BEAT\e[0m"
./$BEAT/bin/$BEAT setup --pipelines -c /etc/$BEAT/$BEAT.yml.* -path.home /usr/share/$BEAT/
./$BEAT/bin/$BEAT setup --dashboards -c /etc/$BEAT/$BEAT.yml.* -path.home /usr/share/$BEAT/
done

Zabbix SELinux policy generation

Commands to query the auditlog for Zabbix relevant queries and create/import a compiled policy file within Zabbix

Could be adapted to generate policies for any other system.

The suggestion is to set SELinux to permissive (setenforce=0) execute the action and afterwards create the policy based on the logged events. If the policy does not work on the first try after re-enabeling SELinux again it it could happen that a call was blocked (which is also logged within the auditlog) that was not blocked with SELinux in permissive mode. Therefore it could help creating a new human readable policy (.te-file) and checking the first version vs. the second version + merging them. 

filename=zabbix-server
cat /var/log/audit/audit.log  | grep zabbix | audit2allow -m $filename >> $filename.te
checkmodule -M -m -o $filename.mod $filename.te
semodule_package -o $filename.pp -m $filename.mod
semodule -i $filename.pp
 
 
#restorecon -R -v /run/zabbix/zabbix_server_alerter.sock    #suggested by the policygenerator

Ansible handlers (within roles) – run multiple tasks

Sometimes it could happen that we want to run multiple tasks after configuration file has changed instead of just one.

My specific usecase is, that I’m having a role that configures a SSL-certificate and additional SSL settings for an Apache webserver which could run standalone on a server or as a pacemaker resource.

If it is running directly on the server it’s quite simple and a handler is sufficient to restart the Apache service after the role ran through. In case the Apache is running as a pacemaker resource the resource should be restarted instead of the whole service to make sure pacemaker does not get confused. Therefore it is necessary to first check if we have the service running as a pacemaker resource first and execute the corresponding task afterwards so a single task (within our handler) is not sufficient.
Using the block statement will also not lead to success but fail with : ERROR! The requested handler 'Apply Apache Config' was not found in either the main handlers list nor in the listening handlers list

My approach to tackle this issue was to “missuse” the handler to only set a variable if something changed.
At the end of the role I’m checking if the variable is ture and if so, I’m including/executing my “handler-block”.

defaults/main.yml

common_linux_zabbix_server_web_ssl_path_private_apply_apache_config: no

handlers/main.yml

---
# handlers file for common_linux_zabbix_server_web_certificate

#set a fact which is checked at the end of the role-tasks
- name: "Apply Apache Config"
  set_fact:
    common_linux_zabbix_server_web_ssl_path_private_apply_apache_config: yes

tasks/main.yml

---
...
all your other tasks are executed before these following tasks
...


# workaround to run multiple tasks within a handler -> run the handlers if any of the above tasks did change something to notify the handler
- meta: flush_handlers

- name: "Run multiple tasks as a handler"
  include_tasks: ./restart_apache_resource_or_service.yml

tasks/restart_apache_resource_or_service.yml


---
- name: "Check if cluster resource exists"
  shell: "pcs resource | grep zbx_srv_httpd"
  ignore_errors: yes
  register: check

- name: "Restart cluster resource:"
  shell: "pcs resource restart zbx_srv_httpd"
  register: resource_restart
  when: check.rc == 0

- fail:
    msg: "An error occured when restarting the cluster resurce!"
  when: resource_restart.rc != 0

#Just restart the httpd service if no cluster resource was found
- name: "Restart httpd service"
  service:
    name: "httpd"
    state: restarted
  when: check.rc != 0[