Tag Archives: Linux

Kiwix – Make Wikipedia anD other websites availAble offline

In some cases it might be handy to have a website available offline for cases where no internet connection is available.

With Kiwix and the ZIM-package format it’s quite easy to do so. It can easily be run on a Raspberry and made accessible on the local network.

To automate updates of ZIM packages I wrote some little scripts which are available in the following Github repo: https://github.com/fawcs-at/zim-downloader

Information on how to use the scripts can be found in the readme in the GIT repo.

To automate the process of updating ZIM packages once a month the “updateZim.sh” should be added as a cronjob to your crontab:

e.g.:

#cat /etc/crontab
45 2    1 * *  <username> /<path_to_script>/updateZim.sh

Will start an update on ever 1st day of the month at quarter to 3 in the morning.

iPXE Network boot that supports Virtualbox VMs

iPXE is pretty nice when it comes to network booting computers as it offers lots of scripting functionality at a very early stage of the deployment as it could be configured to load an iPXE script from an webserver. The script provided by the webserver itself can be created dynamically with any scripting language of your choice depeding on parameters that get handed over.

That provides the possibilityty, to automatically roll out systems that have been specified in an inventory. In case a machine can not be found in the inventory you could provide a menu where an users can manually choose what shall be done and lots more.

Not a problem for bare metal machines and also most VMs. However – Oracle with Virtualbox – also seem to have discovered the advantages of virtual box for their virtual machines and every Virtualbox VM will initially load iPXE.
As that’s an iPXE binary with very little capabilities, this could cause issues when trying to PXE boot an Oracle Virtualbox VM via iPXE as the dhcp-server used for iPXE will get the iPXE identifier form Oracls built in iPXE binary instead of the one that will be initially provided via the DHCP-server.

If the iPXE-script, which is loaded in the second stage, uses the console-command, the deployment will halt as that functionality is not supported by Oracles iPXE binary.

To work around this problem we can modify the user-class identifier provided by our iPXE binary to provide something different than the default “iPXE”-string and use that to make sure that our dhcp-server will always provide our ipxe binary in case a new client tires to PXE boot.

To change the “iPXE” userclass string to a custom string, you have to open the “src/net/udp/dhcp.c” file, once the iPXE repository was checked out.

The interesting part is somewhere down at line 90 in the file:

If you don’t want to change to much code – just change any character to somthing else:

e.g.

DHCP_USER_CLASS_ID, DHCP_STRING ( 'i', 'P', 'X', 'E' ),
TO
DHCP_USER_CLASS_ID, DHCP_STRING ( 'x', 'P', 'X', 'E' ),
or
DHCP_USER_CLASS_ID, DHCP_STRING ( 'C', 'S', 'T', 'M' ),

Whatever is defined there will be the new user-class identifier that can be used to determine if our custom iPXE was loaded or if the iPXE binary from another vendor is used.

Once the change was done the ipxe file needs to be recompiled and copied to the tftp servers directory.

The check for the custom user-class identifier in dnsmasq would look like: (CSTM as the userclass identifier)

...

# Boot for iPXE. The idea is to send two different
# filenames, the first loads iPXE, and the second tells iPXE what to
# load. The dhcp-match sets the ipxe tag for requests from iPXE.
dhcp-boot=ipxe.efi
dhcp-userclass=set:ipxe,CSTM
dhcp-boot=tag:ipxe,http://10.16.96.16/script.ipxe

...

A nice example on how dnsmasq can be configured for iPXE can be found at in the iPXE Forum.

Simple script to test if the chan works properly:

#!ipxe
console --x 1024 --y 768
dhcp
config

Bash – Monitor directory size for change

A simple bash-script to easily monitor if a directory has grown or shrunk in size:

while [ 1 ]; do result=$(du -s * | egrep "bitcoin-0.21.1$"); echo -e "\e[95m$result\e[0m"; curSize=$(echo $result | cut -d" " -f1); if [ $curSize -lt $oldSize ]; then echo -e
 "\e[92mShrunk: $curSize\e[0m"; else echo -e "\e[91mGrown: $curSize\e[0m"; fi; oldSize=$curSize; sleep 5; done

Script needs to be executed in the parrent directory of the monitored dir and directory name must be adapted: bitcoin-0.21.1$ -> to whatever you want to grep for

MIkrotik packet loss (Ping <70%)

Today I experienced an interesting story with my Mikrotik router at home. While updating the PiHole instance the system hat quite some problems obtaining either system updates but also the PiHole update packages. A ping on 9.9.9.9 showed that the Raspberry – on which the PiHole was hosted – had somewhat between 70-80% packet loss. Pinging the same IP from a Windows machine resulted in 0% packet.

All this happened only on wired connections but did not cause any problmes when connected via wifi.

As the Raspberry is directly attached to the Mikrotik router I also tried to connect it via a switch in between as that’s the setup for the windows machine. Same behavior.
Running pings from 3 Linux systems in the network and two Windows systems (even with exchanging network connections to be directly connected to the Mikrotik and connecting through the switch) came up with an interesting result:
All Windows machines had hardly any packet loss in 10 mins (<3%) and all the Linux systems had somewhat between 70%-80% packet loss (tested with a ping).
Any ping that involved the Mikrotik routers L2 functionality seemed to result in packet loss on the Linux machines.
Pinging any other machine on the same subnet worked without problems, but as soon as there was one hob in between the problem occurred.

Interestingly the problem vanished as far as the Torch tool was activated and no more packet loss occurred on any of the systems.

After some additional troubleshooting time (and disabling nearly all Mikrotik-configuration -> Firewall-Rules/Interfaces) the problem seemed to be with the Bridge interface used. It seems that the deactivation of the IP Firewall for the bridge interface caused the problem. After enabling it the behavior vanished and all systems no longer had any packet loss issues.

[admin@MikroTik] /interface bridge settings> /interface bridge settings 
[admin@MikroTik] /interface bridge settings> set use-ip-firewall yes    

Zabbix SELinux policy generation

Commands to query the auditlog for Zabbix relevant queries and create/import a compiled policy file within Zabbix

Could be adapted to generate policies for any other system.

The suggestion is to set SELinux to permissive (setenforce=0) execute the action and afterwards create the policy based on the logged events. If the policy does not work on the first try after re-enabeling SELinux again it it could happen that a call was blocked (which is also logged within the auditlog) that was not blocked with SELinux in permissive mode. Therefore it could help creating a new human readable policy (.te-file) and checking the first version vs. the second version + merging them. 

filename=zabbix-server
cat /var/log/audit/audit.log  | grep zabbix | audit2allow -m $filename >> $filename.te
checkmodule -M -m -o $filename.mod $filename.te
semodule_package -o $filename.pp -m $filename.mod
semodule -i $filename.pp
 
 
#restorecon -R -v /run/zabbix/zabbix_server_alerter.sock    #suggested by the policygenerator

Install Zabbix on Raspberry PI 2

Wouldn’t it be cool to monitor your home? For example all your devices, but also temperature and other sensors an have all that data accessible via a web interfaces?

I think it would so, i thought about setting up Zabbix for home monitoring, but on the RaPi B and B+ it’s not the most performant setup, So i decided to try it again with the PI2.

This post provides a short log on how I set it up.

At first we have to download the source from Zabbix’ SF-page because there is no official package for the ARM-architecture available.

[pastacode lang=”bash” message=”zabbix installation” highlight=”” provider=”manual”]

cd /opt
wget http://downloads.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/2.4.6/zabbix-2.4.6.tar.gz?r=http%3A%2F%2Fwww.zabbix.com%2Fdownload.php&ts=1441447329&use_mirror=skylink^C
mv zabbix-2.4.6.tar.gz\?r\=http\:%2F%2Fwww.zabbix.com%2Fdownload.php zabbix-2.4.6.tar.gz
tar xfvz zabbix-2.4.6.tar.gz
cd zabbix-2.4.6/


#With ./configure --help we can see all the availalbe switches which can be used to compile zabbix.
root@raspberrypi /opt/zabbix-2.4.6 # groupadd zabbix
root@raspberrypi /opt/zabbix-2.4.6 # useradd -g zabbix zabbix
root@raspberrypi /opt/zabbix-2.4.6 # ./configure --help


#I used the follwoing switches to compile the zabbix server and agent, use a MySQL-DB, enable jabber-support, lib-xml2 - which is needed for webmonitoring, net-snmp, ssh and curl which is alos needed for webmonitoring. IPMI can be useful if you also hav a realy server with a BMC to monitor. But for most homeusers the IPMI-option is not neede if you only want to monitor your home and thats it. If you have a LDAP/AD-environment where you want to integrate zabbix you also should use the ldap-switch, but I think most home users also do not have a directory service running at home. ;)

#If this command is run there will ocure some erroes in most cases because there are missing dependencies

./configure --enable-server --enable-agent --with-mysql --with-jabber --with-libxml2 --with-net-snmp --with-ssh2 --with-libcurl


apt-get install apache2 php5-mysql mysql-server mysql-common mysql-utilities libiksemel-dev libiksemel-utils libxml2-dev libxml2-utils libxml2 snmp libsnmp-dev libsnmp-perl libssh2-1-dev libssh2-1 libcurl3 libghc-curl-dev libmysql++-dev php5-gd

#now all dependencies should be resolved
./configure --enable-server --enable-agent --with-mysql --with-jabber --with-libxml2 --with-net-snmp --with-ssh2 --with-libcurl

#copy init scripts
cp /opt/zabbix-2.4.6/misc/init.d/debian/* /etc/init.d/
#copy webfrontend
cp -r /opt/zabbix-2.4.6/frontends/php/* /var/www/zabbix/
chown -R www-data:www-data /var/www/zabbix/


#create the database
#at first log in to your mysql-server as a root useradd and runn the following commands
mysql -uroot -p
create database zabbix character set utf8 collate utf8_bin;
CREATE USER 'zabbix'@'localhost' IDENTIFIED BY 'zabbix';
GRANT ALL PRIVILEGES ON zabbix.* TO 'zabbix'@'%' WITH GRANT OPTION;
mysql -uzabbix -pzabbix zabbix < /opt/zabbix-2.4.6/database/mysql/schema.sql
mysql -uzabbix -pzabbix zabbix < /opt/zabbix-2.4.6/database/mysql/images.sql
mysql -uzabbix -pzabbix zabbix < /opt/zabbix-2.4.6/database/mysql/data.sql

#adapt configuration files at /usr/local/etc/ like in the attached examples
#create dircetories for logfiles:
mkdir -p /var/log/zabbix
chown -R zabbix:zabbix /var/log/zabbix/

#create dirs for alert & external scripts 
mkdir -p /var/zabbix/alertscripts
mkdir -p /var/zabbix/externalscripts
chown -R zabbix:zabbix /var/zabbix/


#configure php-settings
vim /etc/php5/apache2filter/php.init
post_max_size = 16M
max_execution_time = 300
max_input_time = 300
#select timezone from http://php.net/manual/en/timezones.php and set:
date.timezone = 

#restart/reload webserver to accept changes
service apache2 restart
service zabbix-server restart
service zabbix-agent restart

#open http:///zabbix in browser and finish installation



[/pastacode]

 

zabbix-conf

UNIX/DOS-encoding – ^M

Ever had the problem that you tried running a script on a linux-machine and got an error message like the following one?
-bash: ./getRaidFromIrmc.php: /usr/bin/php^M: bad interpreter: No such file or directory

The ^M indicatees that e file you are trying to run is DOS-encoded. This means that its using a CHAR 13 instead of CHAR 10 for a line break and Linux does not like that kind of line break. That often happens if you are writing a script or config file on a Windows machine and transfer it to a Linux machine.
If you try to run/parse the file in linux -> wrong encoding and BAM

The first time I ran into this problem was while deploying a RHEL machine with a faulty kickstart file, but if you know what the problem is it’s quite easy to fix it. Just run “dos2unix” over your file and everything should work again as it should work.

If you often have to modify Linux-files on widows machines I’ld recommend you to use Notepadd++, because it has a feature implemented to set the correct EOL-conversion and lots of more cool and useful plugins like the built in FTP/SFTP-plugin.