UCS 4

Communication between the different systems in a UCS domain is largely encrypted via TLS (formerly SSL). A root certificate and host certificate for each computer are required for the TLS encryption. The root certificate is only valid for a specified period of time, as are the host certificates created with the root certificate. Once this period of time elapses, services which encrypt their communication with TLS (e.g. LDAP, UMC) no longer function. It is thus essential to verify the validity of the certificates and create new host certificates if necessary.

The following commands need to be performed on the UCS master.

univention-certificate can be used to check how long a certificate will remain valid:

univention-certificate dump -name ucs01.intra.example.org
[...]
Validity Not  Before:  Jun  19  10:40:13  2016  GMT
Not  After :  Jun  18  10:40:14  2018  GMT
[...]

When doing so, the FQDN of the computer name (computer name + domain) must be entered. A list of all available certificates can be generated with:

univention-certificate  list The certificates in a UCS domain usually have the same expiration period (default is 5 years from issuance onwards). If the root certificates expires, the certificate chain is considered invalid. Therefore it is necessary to renew all certificates, if the root certificate is about to expire. To create new certificates, proceed as follows:

Backup the old certificates:

cp  -a  /etc/univention/ssl  /etc/univention/ssl_$(date  --iso)

Renewal of the certificates

Renew the root certificate entering the contents of the /etc/univention/ssl/password file as the password: cd  /etc/univention/ssl/ucsCA
openssl x509 -in CAcert.pem -out NewCAcert.pem -days "$(ucr get ssl/default/days)" \
  -passin file:/etc/univention/ssl/password \
  -signkey private/CAkey.pem
mv NewCAcert.pem CAcert.pem

Attention: On very old UCS systems (before version 2.0) the folder "/etc/univention/ssl/ucsCA" was named "/etc/univention/ssl/udsCA"

Renewing all computer certificates:

eval "$(ucr shell)"
cd  /etc/univention/ssl
for i in *".$domainname"; do univention-certificate renew -name "$i.$domainname" -days "$(ucr get ssl/default/days)"; done

Copy the new certificates

Copy the new certificates to the other systems (each UCS/UCC system except UCS Backups - here using ucs03 as an example computer) eval "$(ucr shell)"
cd /etc/univention/ssl/
scp ucsCA/CAcert.pem root@ucs03:/etc/univention/ssl/ucsCA/
scp -r ucs03.$domainname root@ucs03/etc/univention/ssl/
scp -r ucs03.$domainname/* root@ucs03:/etc/univention/ssl/ucs03.$domainname/ The last step is not necessarily required on a UCS Backup computer because those systems copy the certificates automatically via cron.

The following command should be used to make the newly created certificate available to all users via the UCS master’s central administration websitecp  CAcert.pem  /var/www/ucs-root-ca.crtAfter the certificates have been renewd, the new information is not yet displayed in the Univention Management Console (UMC) and the corresponding monitoring check. It would only be updated during the next, regular check, as the cronjob set up for this purpose is only run once every day. To be able to verify the validity of the certificates immediately, the corresponding Univention Configuration Registry variables need to be evaluated. This can be done by running the following script/usr/sbin/univention-certificate-check-validityAll the services which use the SSL/TLS encryption need to be restarted. Alternatively, the system can be rebooted if it is not known exactly which services employ SSL/TLS.

SAML SSO

Since UCS 4.1 every UCS Master and UCS Backup is also a SAML Identity provider and every UCS system in the domain running the Univention Management Console (UMC) is a SAML Service provider. The SAML implementation for UCS uses also TLS encryption and a certificate for the DNS alias ucs-sso.$domainname - this needs to be replaced, too.

On every SAML Identity provider (UCS Master and all UCS Backups), execute the following:

eval "$(ucr shell domainname)"
cp "/etc/univention/ssl/ucs-sso.${domainname}/cert.pem" "/etc/simplesamlphp/ucs-sso.${domainname}-idp-certificate.crt"
cp "/etc/univention/ssl/ucs-sso.${domainname}/private.key" "/etc/simplesamlphp/ucs-sso.${domainname}-idp-certificate.key"
service univention-saml restart


On every SAML Service provider (every UCS system that is part of the domain), the new certificate needs to be evaluated:

eval "$(ucr shell ucs/server/sso/fqdn)"
rm /usr/share/univention-management-console/saml/idp/*.xml
ucr set umc/saml/idp-server="https://${ucs_server_sso_fqdn}/simplesamlphp/saml2/idp/metadata.php" || echo 'Failed!'
/etc/init.d/univention-management-console-web-server restart
univention-run-join-scripts --force --run-scripts 92univention-management-console-web-server.inst


Cyrus

Cyrus has been the default IMAP server until UCS 4.0-2. On sytems where the cyrus mail server is running, the cert.pem and private.key must also be copied to /var/lib/cyrus/

cp  /etc/univention/ssl/"$(hostname -f)"/cert.pem /var/lib/cyrus/ 
cp  /etc/univention/ssl/"$(hostname -f)"/private.key /var/lib/cyrus/

Afterwards the permissions (owner) of the new files must be adjusted:

AD Connection

If the AD connection is used, the certificates should be renewed as well.
cp /etc/univention/ssl/<FQDN of ad system>/{cert.pem,private.key} /var/www/univention-ad-connector/
chgrp www-data /var/www/univention-ad-connector/{cert.pem,private.key}

After this step, the new certificates can be downloaded from the UMC module of the AD Connection. Please follow the AD Connection documentation on how to update the certificates on the Windows system.

RADIUS

If the RADIUS App (freeradius) is used, the cert.pem and private.key must also be copied to /etc/freeradius/ssl/

cp  /etc/univention/ssl/"$(hostname -f)"/cert.pem /etc/freeradius/ssl/ 
cp  /etc/univention/ssl/"$(hostname -f)"/private.key /etc/freeradius/ssl/

Afterwards the permissions (owner) of the new files must be adjusted:

UCS 4

A system does not boot and seems to freeze at bootsplash.

Solution:

It is possbile to edit the boot options for the UCS system within GRUB. You can disable the splashscreen and raise the loglevel - by that you can see what happens when the system freezes:

1. Press button "e" in GRUB bootloader
2. Find the follwing line:
linux ... /vmlinuz-... ... ro quiet loglevel=0 rootdelay=5 splash3. Edit the line: Remove "quiet" and change "slpash" to "nosplash" - also set "loglevel" to 5
the line should now look like this:
linux ... /vmlinuz-... ... ro loglevel=5 nosplash4. Boot by pressing F10

These changes are temporaliy and are re-set at next boot. You can make that change permanent by booting into the working UCS system, logging in as root, and setting the following UCR variables:

ucr set grub/bootsplash=nosplash grub/quiet=no grub/loglevel=5

Advanced options:

The default text console only has 80 columns and 25 rows, so some important output might scroll of. You can configure a serial console to capture that output.

• Many USB serial converters don't work reliably.
• Many high-end servers support serial console re-direction over network, for example over IPMI. See How to do remote server administration over IPMI
• Many virtual machines provide virtual serial interfaces, which can also be used:

  For Qemu/KVM/Xen-HVM

  adding a <console/> or <serial/> element to the VM definition and use ttyS0

  Xen-PV

  Use hvc0 as the serial device name .

  The console can be accessed by running virsh console "$VM". As an alternative the output can be logged into a file on the host by using something like <serial type='file'><source path='/tmp/$VM.log' append='on'/></serial> in the VM definition. The virsh conole-command must be kept running as otherwise all output is lost.

1. Configuring the boot-loader GRUB to use a serial console is described in SDB 1367.
2. The Linux kernel can also be instructed to redirect its output to a serial console:
   ucr set grub/append='console=tty0 console=ttyS0,115200,8,N,1' This makes the serial console the default console (/dev/console), as ttyS0 is given last. By moving console=tty0 to the end, the VGA console will remain the main console.
3. Start a login getty on the serial console:
   

   Before UCS-4.2 with SysV-init

   sed -i -re '/^#?T0:/s/#//' /etc/inittab;telinit Q

   Since UCS-4.2 with systemd

   systemctl enable getty@ttyS0.service;systemctl start getty@ttyS0.service

UCS 4

Die Kommunikation zwischen den verschiedenen Rechnern einer UCS-Domäne erfolgt größtenteils TLS/SSL-verschlüsselt. Bei der TLS/SSL-Verschlüsselung werden ein Root-Zertifikat und für jeden Rechner ein Rechnerzertifikat benötigt. Das Root-Zertifikat hat nur einen bestimmten Gültigkeitszeitraum, ebenso wie die mit dem Root-Zertifikat erstellten Rechnerzertifikate. Ist dieser Zeitraum abgelaufen, funktionieren Dienste, die ihre Kommunikation mit TLS/SSL verschlüsseln (z.B. LDAP, UMC) nicht mehr. Es ist deshalb notwendig, die Gültigkeit der Zertifikate regelmäßig zu prüfen und ggf. neue Rechnerzertifikate zu erstellen.

Die folgende Befehle müssen auf einem UCS Master durchgeführt werden.

Die Überprüfung, wie lange ein Rechnerzertifikat noch gültig ist, kann mit univention-certificate erfolgen:

Hierbei ist immer der FQDN des Rechnernamens (Rechnername + Domain) anzugeben. Eine Liste aller verfügbaren Zertifikate erhält man mit

Normalerweise haben die Zertifikate aller Rechner einer UCS-Domäne das gleiche Gültigkeitsintervall (Standard sind 5 Jahre ab Erstellung). Wenn das Root-Zertifikat abläuft, wird auch die ganze Zertifikatskette als ungültig betrachtet. Daher müssen alle Zertifikate erneuert werden, sobald das Root-Zertifikat droht sein Gültigkeit zu verlieren. Um neue Zertifikate zu erstellen, ist folgendermaßen vorzugehen:

Sicherung der alten Zertifikate:

Erneuern der Zertifikate

Erneuerung des Root-Zertifikats, als Passwort ist der Inhalt der Datei /etc/univention/ssl/password anzugeben:

Achtung: Bei UCS Systemen in einer Version älter als 2.0 lautet das Verzeichnis "/etc/univention/ssl/udsCA" statt "/etc/univention/ssl/ucsCA".

Erneuerung aller Rechnerzertifikate:

Kopieren der Zertifikate

Kopieren der neuen Zertifikate auf die anderen Rechnersysteme (jedes UCS/UCC System, außer UCS Backups - hier am Beispielrechner ucs03):

Dieser Schritt ist auf UCS Backups nicht zwingend erforderlich, denn dort erfolgt das Kopieren auch automatisch per cron.

Um das neu erstellte Zertifikat allen Benutzern über die zentrale Verwaltungs-Webseite des UCS Masters zugänglich zu machen, kann der folgende Befehl verwendet werden:

Nach dem Aktualisieren der Zertifikate werden die neuen Informationen noch nicht in der Univention Management Console (UMC) und im entsprechenden Monitoring-Check angezeigt. 
Diese würden erst bei der nächsten regulären Prüfung aktualisiert, da der dafür angelegte Cronjob nur einmal am Tag ausgeführt wird.
Um die Gültigkeit der Zertifikate sofort prüfen zu können müssen die entsprechenden Univention Configuration Registry Variablen (ssl/validity/days) ausgewertet werden. Dies kann durch den Aufruf des folgenden Skriptes erfolgen:

Alle Dienste, die TLS/SSL-Verschlüsselung benutzen, müssen neu gestartet werden.
Alternativ kann ein Neustart des Systems durchgeführt werden, wenn nicht bekannt ist, welche Dienste mit TLS/SSL in Verbindung stehen.

SAML SSO

Seit UCS 4.1 sind UCS Master und alle UCS Backup auch SAML Identity provider und jedes UCS System in der Domäne, auf dem die Univention Management Console (UMC) läuft ist ein SAML Service provider. Die SAML Implementierung in UCS verwendet ebenfalls TLS-Verschlüsselung sowie ein Zertifikat, das auf den DNS Alias ucs-sso.$domainname ausgestellt ist - dieses muss ebenfalls ersetzt werden.

Auf jedem SAML Identity provider (UCS Master und alle UCS Backups) muss folgendes ausgeführt werden:

Auf jedem SAML Service provider (jedes UCS System, das Mitglied der Domäne ist) muss das neue Zertifikat eingerichtet werden:

Cyrus

Auf Rechnern, auf denen der cyrus-Mailserver läuft, sind zusätzlich die Dateien cert.pem und private.key nach /var/lib/cyrus/ zu kopieren:

Im Anschluß muß Besitzer- und Gruppezugehörigkeit der kopierten Dateien auf dem Mailserver angepasst werden

AD-Connector

Wird der UCS AD-Connector eingesetzt, sollten die Zertifikate auch auf dem AD-System aktualisiert werden.
Hierfür können die neuen Zertifikate für das AD-Connector System über die UMC zum Download bereitgestellt werden:

cp /etc/univention/ssl/<FQDN des AD-Systems>/{cert.pem,private.key} /var/www/univention-ad-connector/
chgrp www-data /var/www/univention-ad-connector/{cert.pem,private.key}

Anschließend können die Zertifikate wie in der AD-Connector Dokumentation beschrieben auf dem AD-System abgelegt werden.

RADIUS

Wird die RADIUS App (freeradius) eingesetzt, sind zusätzlich die Dateien cert.pem und private.key nach /etc/freeradius/ssl zu kopieren:

Im Anschluß muß Besitzer- und Gruppezugehörigkeit der kopierten Dateien auf dem Mailserver angepasst werden

Known issues

UCS-4.2 shipps with the newer QEMU version 2.8, which is currently incompatible with the QEMU version 1.1 previously used until UCS-4.1. Symptoms include

• guest operating systems gets stuck in a busy-loop using 100% virtual CPU time
• virtual machine reboots
• virtual machine does not reboot after shutdown request

This can happen after

• restoring the virtual machine from the suspend-to-disk state
• restoring the virtual machine from a snapshot taken from a running system
• live-migration from a UCS-4.1 (or older) system

Univention is working on this problem and will provide updated packages and information when the issue is fixed.

Mitigation

If possible, servers running the 'KVM virtualization server' app can simply stay on UCS 4.1 until Univention releases a fix for the UCS 4.2 QEMU version. All features will remain functional when controlling virtual machies via UVMM.

If the UCS servers running the'KVM virtualization server' app have to be updated to UCS 4.2, the issues can be mitigated by

• shutting down all virtual machines cleanly before performing the update
• upgrading the virtual machine hardware configuration
• creating and using snapshots only of cleanly shut down virtual machines
• migrating only shut down virtual machines between hosts using different UCS releases

All of these options must be performed before a virtual machine is migrated or the host system is updated. If this is not possible the virtual machine might crash, which can lead to data loss (only the state of the RAM, not the data on disk). After such a runtime crash the guest operating system usually performs a file system check to recover consistency. For snapshots this process can be repeated multiple times, but not for live migration and suspend, as the runtime state there is only available once.

Univention also has observed problems with virtual machines not rebooting properly: virtual machines running the Linux operating system display the message "reboot: Restarting system" last while systems running Microsoft Windows are stuck on the reboot screen. This can be mitigated by upgrading the virtual machine hardware configuration, which is described below.

How can I find virtual machines suspended to disk?

You can run the command "virsh list --all --with-managed-save".
As an alternative you can directly look at the file systeme level: for each VM there exists a file "/var/lib/libvirt/qemu/save/$VM.save".

How can I remove the managed save state of a virtual machine suspended to disk, if I prefer to start the VM in a crashed state instead of not starting it at all?

Run 'virsh managedsave-remove $VM"

How can I reset a virtual machine if it is not responding after restoring the running state?

Try "virsh restart $VM" to trigger a reboot. This will re-use the running QEMU process.
If that does not work, stop it with "virsh destroy $VM" and start it with "virsh start $VM" again. This will create a new QEMU process, which reload all files from disk.

What is a virtual machine hardware configuration and how can I upgrade it?

<domain type='kvm'>
  ...
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.8'>hvm</type>
    ...
  </os>
  ...
</domain>