Valid HTML 4.01 Transitional

Upgrading to OpenSuSE Tumbleweed

James F. Carter <jimc@jfcarter.net>, 2017-05-23

OpenSuSE Leap 42.1, which is what I'm running, just died, which is what prompts this upgrade. See SuSE's version lifetime page. 42.2 is expected to last until second quarter 2018, i.e. 1 year more, and I think it's been out about 6 months. 42.3 is expected to come out at the end of July 2017, i.e. in two months.

The Lifetime page says that a major release, i.e. the whole 42.x series, should last 3 years, but minor releases like 42.1 should appear annually and support (i.e. bugfixes and security) will continue for 6 months after the next version comes out. This schedule is very inconvenient for me. I'm going to try to switch to the Tumbleweed rolling distro.

Table of Contents

Introduction

Goals, Issues, and Actions

I always begin a project by stating its goals, issues likely to be encountered, and then the actions to be taken. I have a lot of experience upgrading SuSE distros from 6.4 to 13.1 and then Leap 42.1, but these have been fixed, non-rolling distros. Tumbleweed is going to be different in some ways. But I expect that a lot of steps for the fixed distros will carry over to Tumbleweed.

Goals

Issues

Actions

Here's an overview of installing a new distro:

Maintaining the Old Static Distro

Presently for the static distros I have this basic infrastructure:

Changes for Tumbleweed

For using Tumbleweed I want to continue as much as possible with these admin scripts and procedures. I envision these changes:

I need to create or modify these scripts and/or services:

Here's an important detail: what version number do I use for Tumbleweed? In some contexts SuSE uses tw, but my scripts in some cases assume a numerical version number. I'm going to use 99.8.

Infrastructure Setup

Following along in my writeup on Upgrading to OpenSuSE Leap 42.1:

The New Repo Definitions

I need a lot more repos for Tumbleweed than for v42.1. All will need new repo template files except SBS (SuSE Build Service) and CouchNet. Except for Couch-SuSE-tw, the local mirrors contain only installed packages.

As discussed previously, I don't want to locally mirror the entire Tumbleweed distro (40.2Gb with swarms of updates each day) plus PackMan; instead I want to keep mostly the packages I am actually using. However, I want to do network installations from a local mirror, and that means I need to put on the (local) net an authentic, untampered Tumbleweed snapshot DVD ISO image. [Done.]

I need to set up repo priorities. My goal is for the latest version of each package to be installed, whichever repo has it. But unfortunately Zypper has priorities backward. At least according to the SLES deployment guide dated 2016-03-14, if several repos have a package (referring to the basename), it is downloaded from the repo with the numerically lowest priority (in 1-200, default 99). If there is a choice of versions in repos of the same priority, the latest is preferred, but later versions in higher numbered repos are not considered. If I make the SuSE repo's priority numerically lower than the local mirror, I'll get the latest version, but for every package: the local mirror will never be used. Therefore I will need to identify and download new versions to the local mirror, but only for packages I actually use. This script is called updaterepo.

I've adopted these priorities (lower numbers are preferred) (repo aliases beginning with Couch are local):

In the local mirrors I want to limit the number of old versions kept. In the main OSS repo most packages have only one version, and when multiple they are close in time; the biggest time spread I saw was one month. A reasonable strategy would be to download every version but to delete local versions that are no longer on the master repo.

Designing the updaterepo script:

How is updaterepo going to obtain the rpm -qa output? This involves copying a file from each of the hosts to Diamond. I want it to run overnight, so it will have no human supervision and authority.

Creating the New Repos

Shell Variable Cruft Check

You can source the file /s1/SuSE/source.me to set useful shell variables for the distro root and the major scripts in distro maintenance. This file was reviewed for being up to date. It auto-sets the architecture (according to the machine it's run on) and the release, taking the lexically highest version in /s1/SuSE/SuSE-build/$ARCH/ , which will be the new version when you create the directory for it. Here is jimc's version of source.me.

Remove Old Distros

We need to keep files for v42.1 until all hosts have been upgraded, but 13.1 can definitely be tossed. There is nothing from before 13.1. All sub-repos already have only x86_64 (and noarch and setup). At home, before cleanup, the repo occupied 114 Gb (oink). After, 69Gb; tossed 85Gb.

At the same time, we need to get rid of the configuration files for old distro versions. These are in /home/post_jump/${RLSE} or on Mathnet, /h1/post_jump/${RLSE}. Being anally retentive I made a directory ./ancient and moved all the obsolete dirs into it. When deleting one of these directories it's a good idea to shred the non-public files. Here's how, illustrated for distro version 10.2:
find 10.2 -type f ! -perm -04 -ls |& less #Are you going to remove the right files?
find 10.2 -type f ! -perm -04 -print | xargs -n 25 shred -u -n 3
rm -r 10.2

Create Directories

Create all the directories for distro components. Owned by root:root, mode 755, except the update dir has to be owned by wwwrun. The actual content (metadata files) is provided later. These are illustrated for the new version, 99.8 and all paths are relative to $di (/s1/SuSE).

Download ISO Images

Web links:

To be totally straight arrow about your security, you need to follow this procedure for each downloaded image, here called $name.iso One of my downloaded files came from a Russian mirror site, and given the pervasive dirty tricks (particularly in I.T.), from Russia in the USA 2016 election, it would be a good idea to take security seriously.

A less high tech alternative is to do sha256sum $name.iso, then grep Tumbleweed $name.iso.sha256, and compare the sums by eyeball, 64 hex digits.

Populate the Distro Metadata

Formerly I mounted the fixed distro's ISO image on ./SuSE/x86_64/$version, but for Tumbleweed I need to populate it from the DVD (if I downloaded it, which I didn't) or from the SuSE repo itself. Metadata for the SuSE-build and CouchNet repos can be copied from the old versions. [Update: I still need to mount the DVD, and to download an up-to-date instance before upgrade campaigns or before installing Tumbleweed on a new machine.]

To populate the SuSE repo: (0.67Gb download) (The barely visible last argument is a dot, meaning to populate the current directory.)
cd ./SuSE/x86_64/99.8
rsync -a --no-r --dirs --log-format="%o %f" rsync://rsync.opensuse.org/opensuse-full/opensuse/tumbleweed/repo/oss/ .
rsync -a --log-format="%o %f" rsync://rsync.opensuse.org/opensuse-full/opensuse/tumbleweed/repo/oss/{boot,docu,media.1} .

To populate the non-OSS repo: (0.0002Gb download :-)
cd ./SuSE-noss/x86_64/99.8
rsync -a --no-r --dirs --log-format="%o %f" rsync://rsync.opensuse.org/opensuse-full/opensuse/tumbleweed/repo/non-oss/ .
rsync -a --log-format="%o %f" rsync://rsync.opensuse.org/opensuse-full/opensuse/tumbleweed/repo/non-oss/{boot,media.1} .

Package keys wanted: The first two come with the main distro DVD; the others go in the CouchNet or Mathnet repo.

These keys are obsolete:

How to identify a public key:

gpg gpg-pubkey-0cc9523f-4a9865cc.asc

It prints: pub 1024D/0CC9523F 2009-08-28 UCLA-Mathnet Distro Signing Key <distro@math.ucla.edu>

Check Update Download Script

For Tumbleweed the update repos and the main repos are updated the same way using the new script updaterepo. See the 42.1 transition writeup for how to update the script rsyncsuse if reverting to fixed distros.

We need to switch from weekly execution of $di/bin/rsyncsuse.sh to updaterepo. [Done.]

Virtual Machine

One of my virtual machines for development is called Oso. It is hosted on Diamond, the repo site and compute server, using KVM as the framework and qemu as the virtual executor. The guest connects itself to the host's network bridge (br0) and thinks that it is directly on the local network. Its disc is 17.2Gb (external). Deducting the swap partition and inode storage, it has 15.3Gb (total internal) of which 7.3Gb is unoccupied. This should be plenty to hold the RPM files that have to be downloaded from SuSE. So I'm going to develop it up the kazoo: it will be the first machine to migrate to Tumbleweed.

VM Specifications: Oso is defined in this file. Key parameters of Oso and Petra are:

An initial step is to junk old saved copies of Oso's disc and to make a new backup of its current state on v42.1. Command line: gzip -c disc1.raw > ./disc1.421final.gz Run time: 12 mins. Output file size: 5.7Gb.

Commands to use on the virtual machine with libvirt:

Lurking dragon: make sure on the VM that /etc/default/grub_installdevice says to install grub in the MBR. 99% of the time you want the device to be (hd0), not e.g. (hd0,1) which would install in the root partition's boot sector. To detect whether (hd0) is correct you could do:

grub2-probe -t drive /boot/.

Remove the partition ID, e.g. it might print (hd0,msdos2) and you ignore ,msdos2 keeping (hd0).

Initializing the Local Mirror Repo

Now transforming this package list into a script to download the actual packages to the CouchNet enterprise mirror. 10min flat to download; sheesh, that's fast! 2.46Gb actually downloaded. We have 50Mbit/s FIOS. Here's the script:

#!/bin/bash
# Downloads a list of packages.  Do "rpm -qa" to get the list.
# Expects to see names like pkgname-1.2-3.4.x86_64

function snarf () {
    local arch=${1##*.}
    echo $arch/$1.rpm
}

function myrsync () {
	# -a = preserve metadata, -R = preserve input path on output,
	# -no-{o,g} = files to be owned by local root:root, not whatever 
	# special user is on their server.  
    rsync -a --no-o --no-g -R --files-from - --log-format="%o %f" \
	rsync://rsync.opensuse.org/opensuse-full/opensuse/tumbleweed/repo/oss/suse/ \
	/s1/SuSE/SuSE/x86_64/99.8/suse/
}

cat <<EOF | (while read pkg junk ; do snarf $pkg ; done) | myrsync 
aaa_base-extras-13.2+git20170512.8fa87a3-1.1.x86_64
abiword-docs-3.0.2-1.2.noarch
etc. etc.
EOF

39 packages could not be downloaded. These are obsolete, except for one of them which appears to be no longer available.

7 packages were in SuSE Build Service (SBS) repositories and were downloaded into the local SBS repo. Two were in the non-OSS repo; a similar repo was created locally and the packages were downloaded to it.

The following 17 packages are probably from PackMan:

I'm going to defer the PackMan downloads, since most of these are undoubtedly 18 months back version, and to get current versions is going to involve either a lot of hand labor, or letting Zypper pick the version, which is much preferred. [Downloads accomplished using the new updaterepo script.]

Repo Maintenance Scripts

The next group of steps will be:

Initial Installation

Now I'm ready to install Tumbleweed for the first time. I'm going to want to upgrade v42.1 but to save the result before installing any CouchNet customizations, to know what customizations I'll actually need. I'm going to also do a completely default installation on bare (virtual) metal, to know what SuSE wants to give me when not influenced by existing package selection and configuration.

Unhacked Instance of Tumbleweed

To make progress I need an unhacked instance of Tumbleweed, to compare with the normal CouchNet v42.1 installation. It's going to go on Oso. Steps:

Checking Out Tumbleweed

Oso is on the net, and many items are working. First I ran checkout.sh just to see what survived. Discrepancies:

Only the dbus item is critical. Dbus seems to be working; the problem must be with the tester. I'll debug this stuff later.

I saved a copy of oso:/etc and its package list, rpm -qa.

Shutting down Oso and making a copy of this disc.

Default Installation

My next step will be to wipe Oso's disc and install on bare (virtual) metal. The result will be called oso-pure. This time around I'm going to accept its offer to put a btrfs filesystem on the root partition. [Update: reverted to ext4.] Later I plan to convert the old machines to btrfs. [Update: not going to happen.] In package selection I will use defaults as much as possible, to find out what they intend to give me, but I will use a XFCE desktop framework and will decline KDE and Gnome (as usual). AppArmor also will be bypassed, but the SuSEFirewall will stay (for now).

Package Selection

When rebooting Oso, remember that it still has the network installer disc in the virtual drive, and there will be a 30sec timeout before it does the default of booting from the hard disc. Soon this can be reverted.

Actually I faked myself out: Oso is supposed to be set up as a user workstation (rather than pure development), so it should have these patterns turned on: Office, Technical Writing, and Games. In couchnet.sel I don't select everything in these categories; the purpose is to see what they think a normal user workstation ought to have on it. Now I'm installing the omitted patterns. I've turned on keeppackages=1 and will retrieve missing packages into the local mirror repo. 2672 packages to download including texlive. Downloaded 1.3Gb, installed 2.8Gb, elapsed time 41min including 4min post-install activities.

For testing, I'm also creating a local account for myself (with SSH access).

I now have a set of files of package basenames (minus version and arch) on Claude (1967 packages) (Oso should come out similar), Oso pure installation (1859 packages), and Oso upgraded (8388 packages, oink). Plus Oso not upgraded but with the user packages (4531 packages). Command line to create these lists:

ssh oso rpm -qa | sed -e 's/-[^-]*-[^-]*$//' | sort -o bases.oso.pureuser

Package groups on Claude-42.1 and not on Oso-pureuser. Not every package, just the interesting ones.

Package groups on Oso-pure and not on Claude-42.1:

Notable daemons running on Oso-pure:

Packages are not radically different from what's on Claude now. Actually one of my leading issues is, do I get valuable security by running auditd, which I don't run now? There are, as usual, several interesting unfamiliar packages in the default load, which I will want to investigate later.

Toward a Working VM

For package selection, I'm going forward just copying couchnet.sel from v42.1. Of course screwups will be revealed and I'll deal with them when found. The goal in this step is to run post_jump on oso-pureuser and then to get it to pass the tests in checkout.sh. Real reconciliation between v42.1 config files and as-installed Tumbleweed config files will happen once that's done, so the edited v42.1 files will be functional in the Tumbleweed context.

Reconciling Configuration Files

Now that I have a working VM, it's time to get the configuration files into shape. The first is couchnet.sel, i.e. package selection. All references to v42.1 and v13.1 are to be tossed, except for historical notes. [Done.] I didn't do a package by package evaluation of which should be kept on the machine. Probably I should come back to do that, but there's no sign of radical reorganization, and I want to make progress so I can get my security patches installed.

After installing oso-pureuser I saved the pristine /etc. Comparing that with post_jump. Executing on Diamond.

diff -r -w /home/post_jump/99.8/etc /s1/scr/oso-setup-1706/oso-pureinst/etc >& oso.diff

There were 102 unequal files and 8689 lines of diff. Here are the highlights, omitting cases (like /etc/hosts) where the CouchNet version is obviously the better choice.

Template for an Upgrade

Reinstallation again. We're ready to re-test upgrading from v42.1 (first) and installing from scratch. This procedure (most of it) is going to become the template for upgrading the production machines.

Preparing to Upgrade

Upgrading the Target Machine

This was written from experience on Oso, but to make it generic I'm using $target for the target machine. Pre-set this variable.

Installing from Scratch on Oso

Overcoming Problems

Several problems were encountered during the various installations and upgrades, and their solutions turned out to be too big to include inline with the installation sections.

Customizing MDM (Mint Display Manager)

The display manager is the program that starts the X-Windows graphics server, puts up a window to solicit the user's loginID and password, and starts the user's session with his favorite desktop environment. It seems that on every operating system upgrade the previous display manager breaks and I have to find a new one. With v42.1 I was using lightdm, and a custom greeter using the Webkit engine, which basically means Javascript. It looked nice, but unfortunately it was unreliable, and after the upgrade to Tumbleweed, important symlinks set by update-alternatives were messed up, and I was unable to fix them by hand and get a working greeter.

For my new display manager I'm using MDM: Mint Display Manager. See this Wikipedia article about Linux Mint. It has a non-tree-form ancestor relation with Ubuntu and Debian; I am not tempted to investigate Mint to replace OpenSuSE. However, the requirements for MDM itself are not onerous; all the required libraries are clearly relevant to MDM's mission, and it doesn't drag in the whole Gnome infrastructure. It is themeable and the syntax and semantics of the theme file follow that for GDM (what version?) Basically you are arranging and specifying properties for GTK widgets (what version, 2 I think). Unfortunately Glade cannot handle the file format.

CouchNet Theme for MDM

Greeter design: Each of my machines has a background image that is used both for the greeter and for the webserver's front page. Generally photos like this have visual interest in the center, and therefore I try to move the greeter window into a corner, normally northeast. A black background for this window works out best when contrasted with the various colored images. These modifications are surprisingly hard for some greeters. As a fallback I have a design using XDM, which is functional but doesn't look completely professional.

The theme file is partially documented in this page from the GDM Reference Manual (what version?) Graphical Greeter Themes writeup. Surprisingly, a Google search does not reveal any other home for this page and it doesn't look like an official publication point for GDM. You will need to use your hacking ability to figure out some of the necessary properties and container classes.

The mdmsetup utility is mainly for picking a theme and setting most commonly used options like timed login. It isn't for editing the theme definition file.

The theme needs to be installed in /usr/share/mdm/themes/$ITS_NAME/ The files are:

Session Configuration for MDM

I have several issues which I'm tentatively blaming on MDM:

/etc/mdm/custom.conf and friends have not been hacked, and very likely this is the cause of many of the missing features. See /usr/share/mdm/defaults.conf for the pre-customization state. There are 68 default settings, 47 to "". Here is where it stands with these items:

No Shutdown Action

In /usr/share/mdm/defaults.conf, RebootCommand and friends are set to (e.g.) sbin/shutdown -r now "Rebooted via mdm.". While this is correct with SysVinit, Tumbleweed has systemd-sysvinit-233 which makes /sbin/shutdown a symlink to /usr/bin/systemctl, which may have responded to SysVinit options at one time, but does not do so now. I replaced the command line with /usr/bin/systemctl reboot --message="User reboot via MDM" and the shutdown or reboot now happens, including the message. Remember that HaltCommand should request poweroff, meaning to turn off power in the machine, versus halt, which means to halt (with power still on). Remember that /usr/share/mdm/distro.conf is the correct place for distro customization. Bug report here.

No Keyboard Grab for Password

There does not seem to be any configuration parameter affecting the keyboard grab. Bug report here.

Starts Asking for Password

That's not a bug, that's a feature! By default, the last user who logged in is preselected. However, suppose you're a different user; what's a good way to change to your loginID other than giving a garbage password? Hit the Start Over button. Except it does not start over. Bug report here.

No ssh-agent

Well, isn't that cute! For MDM we're using the generic PAM stack that comes with the package, and jimc's idea of setting up a full login session, like pam_ssh.so, is not happening. Fixed with a symlink to the full login stack.

No xkeyboard Setup

BaseXsession is set by default to /etc/mdm/Xsession. When I changed to my hacked /etc/X11/xdm/Xsession, all my normal session features returned, including configuring the keyboard.

Complaint about +extension XEVIE

defaults.conf adds this extension to the standard X-server command line, but /var/log/Xorg.0.log has a complaint that it is unsupported. It turns out that this is the X Event Interception Extension, for intercepting all keyboard and mouse activity, it is deprecated, and it was removed in X.Org-7.5 (we have 7.6.1). The distro should use distro.conf to make it vanish. Bug report here.

Is VNC Working?

No, and it's not going to, when MDM is involved. Unfortunately, XDMCP (and a lot of other stuff) has been decommitted in MDM (2013-09-25 by Clem). See the discussion below of VNC.

Diarrhea on the Log File

Error message:

GLib-CRITICAL: g_key_file_get_value: assertion 'key_file != NULL' failed

When MDM starts up it spews out 110 instances of this in syslog (including on all logged-in SSH sessions, due to the critical level). It also puts out two or three each time the greeter starts, i.e. when a user logs out and back in again.

The cure: touch /usr/share/mdm/distro.conf . Bug report here.

CPU Usage with KVM

It's pretty clear that when I upgrade a VM, even though the guest is completely idle, there is considerable CPU load on the host, at least twice what it was with v42.1 on the guest. For example, 2 snapshots:

I went through all the modules (device drivers) on the guest and unloaded them one by one. Bingo: when I unloaded uhci_hcd, the host's CPU dropped from 15-25% down to 1-2%.

On none of the virtual machines do I pass through USB devices from the host, so I'm blacklisting the driver. I would have expected that the right way to do so would be a blacklist command in /etc/modprobe.d/99-local.conf, but it was ineffective. install uhci_hcd /bin/true was also ineffective (and also deprecated). I ended up creating a systemd unit stomp-usb, which runs late, i.e. after graphical.target or multi-user.target, and which explicitly unloads the module. That took care of it.

ehci_hcd is also loaded. There is no evidence that it causes elevated CPU on the host, but per guilt by association I'm similarly blacklisting it.

The host's CPU load is inversely proportional to the intrinsic CPU power of the host. Which is the least on Jacinth, intermediate on Diamond, and greatest on Xena. The CPU load on these hosts (with and without uhci_hcd) is:

Host With uhci_hcd Without uhci_hcd
Jacinth 2.0% 1.6%
Diamond 15-25% 1-2%
Xena 1.3% 1.0-1.3%

On Jacinth with v42.1 and Orion with v42.1, and with uhci_hcd loaded on Orion, the CPU usage was 2.0%, i.e. I'm not seeing excessive CPU utilization. But when I unload uhci_hcd the CPU goes down by about 0.4%, so there is CPU utilization attributable to uhci_hcd, just not excessive. This host's processor is a AMD G-T40E @1.0GHz, while Diamond and Xena have Intel Core i7 and i5.

LDAP (slapd) was Hosed [Fixed]

This problem turned out to have multiple parts. My slapd is configured out of the database, the modern recommendation. /etc/systemd/system/ldap.service starts slapd with -F /var/lib/ldap/slapd.d which contains cn\=config.ldif and dependent objects (files). All configuration filenames below will be relative to this directory. Filenames must be quoted because they contain shell-active characters. I tried restoring everything from backups, but the operational instance and the backups shared the same problems (so I reverted to the operational instance).

back_hdb.la Not Found

The main database uses the HDB backend. The symptom was that both slapadd and slapd complained: lt_dlopenext failed: (back_hdb.la) file not found. The file exists: /usr/lib64/openldap/back_hdb.la from openldap2-2.4.45-24.1.x86_64. strace revealed that it looked for /usr/lib/openldap/modules/back_hdb.la followed by some generic and useless directories.

Growl! ./cn=config/cn=module{0}.ldif contains olcModulePath: /usr/lib/openldap/modules which was correct in v42.1 but no longer. I replaced it with /usr/lib64/openldap and on the v42.1 machines I made a symlink: ln -s ../lib/openldap/modules /usr/lib64/openldap, which will be overwritten when Tumbleweed is installed. This problem is fixed; now we proceed to the next one.

Not so fast! Zypper honors the symlink when installing, rather than overwriting it with a directory, so it's best to just not make it, and to do this fixup procedure after upgrading each directory server.

What a Mess!

First rule of troubleshooting: don't change anything not directly related to the current symptom being fixed. But /var/lib/ldap is such a mess, with the main database files strewn around. I created a new directory ./couchnet.db , mode 750 owned by ldap:ldap, and I moved all the database files into it, so now /var/lib/ldap contains just ./couchnet.db, ./slapd.d, and ./sync.d (for housekeeping to import changed flat files like /etc/passwd). And I edited ./cn=config/olcDatabase={1}hdb.ldif , olcDbDirectory: /var/lib/ldap/couchnet.d .

Configuration Error in Sync Overlay

Now, upon starting, slapd complains config error processing olcOverlay={0}syncprov,olcDatabase={1}hdb,cn=config. After I re-read my writeup on Setting Up LDAP plus section 18.3.3 of the OpenLDAP 2.4 Administrator's Guide (about multi-master replication), I had a brainwave: suppose openldap2-2.4.41 either has replication compiled into the program or knows where to find the module, but openldap2-2.4.45 needs an explicit olcModuleLoad: syncprov.la. (prepend {3} or whatever to make a unique list index.) This is in ./cn=config/cn=module{0}.ldif . Sure enough, when I loaded the module explicitly, slapd started working.

Checkout

I tried this command line:

ldapsearch -x -H ldap://xena.cft.ca.us -b uid=jimc,ou=People,dc=cft,dc=ca,dc=us -LLL

And it returned my People page, with or without -ZZ (use TLS), except omitting the password because I omitted authentication. A good sign. I have a utility ldapsync -v -n to sync flat files like /etc/passwd . It said that it wanted to modify the passwd and group rows for lxdm (yes, it's different in Tumbleweed) and add a row for mdm (yes, it's new in my Tumbleweed package selection), but all the other data is identical. So LDAP is entirely operational.

Supporting VNC

A VNC (Virtual Network Computing) viewer and server use the RFB (Remote Frame Buffer) protocol to transfer graphical data from the server to the viewer, where the user can view it, and to transfer keyboard and pointer (mouse) events from the viewer to the server. On Unix the server imitates a X-Windows server by making a framebuffer available to server-side programs. This can be the physical framebuffer or a virtual one. In either case the server-side programs use normal X-Windows libraries to draw on the framebuffer, the same as they would for a user physically present at the physical display. If the server is configured to allow multiple connections to the same framebuffer, e.g. if a user is logged in to the physical display and a VNC connection to it is allowed at the same time, authentication and access control are important. But if sharing the framebuffer is forbidden, the normal paradigm is that the server's display manager puts a greeter on the framebuffer, which takes care of authentication and authorization, same as on the physical display. Then the user's preferred desktop environment is started. The RFB protocol is not intrinsically encrypted, and in most cases a separate secure tunnel is used for the data transfer.

In my environment, VNC is used for developing and testing the display manager and the desktop environment sessions; it is rare for a user to have a desktop environment on the viewer host and to do real work in a separate desktop environment on the server. But rarely does not mean never. Even so, an important goal is that VNC support should be unobtrusive and light on resources. The socket activation paradigm of systemd is ideal: systemd on the server listens for a connection, then starts the VNC server, which exits when the session is over. Xinetd is an obsolete alternative. It's overkill for me to start a VNC server at boot time.

The default connect port for VNC is 5900/tcp, corresponding to display :0. However, the qemu virtual machine emulator uses 5900 by default to send out the VM's physical display over VNC, so I plan to use 5901 to serve a virtual framebuffer on the host. The VNC server can also serve to a (web?) browser on 5800, or can call back to the viewer on 5500, but I don't use either of these features.

The normal paradigm is that the Xvnc server (package xorg-x11-Xvnc-1.8.0) contacts a XDMCP-capable display manager on localhost via port 177/udp. They negotiate a session cookie, and the display manager puts a greeter window on the virtual framebuffer, and after authentication spawns the user's session. XDM, GDM and LightDM are XDMCP-capable. But unfortunately XDMCP (and a lot of other stuff) has been decommitted in MDM (2013-09-25 by Clem).

So how am I going to get a greeter going?

How's this for a kludge: MDM will constitutively run what it thinks is an X-server on display #1, but it is really a script which waits forever. The [daemon] MdmXserverTimeout parameter will have to be set to the duration of forever. Systemd will wait for a connection to the VNC port. When a connection is made, the socket (on stdin) will magically be transferred to the waiting script and it will be released and will run Xvnc.

This is going to take some work to set up, so I'm going to defer it until it's actually needed. PROBLEM

What To Do About MythTV

Until recently, Iris' main role was to run MythTV to record TV shows. However, the source for my wife's preferred shows has dried up, and most likely we will rely on Internet streaming from now on. This section probably ought to be moved to a separate document, but for now it stays as part of the upgrade history. We have been using MythTV and like it, but PHP has grown to PHP7, whereas MythTV still uses PHP5, and installing it makes a mess.

Other packages also use PHP5 and not PHP7: Roundcubemail and ownCloud. The solution was to evict the php7 package, zypper addlock php7, and then reinstall php5 and the needed extension modules.

Goals for the home theater machine and software:

What's currently available? Here are some recent 10 best lists.

Diversion for CouchNet System Administration

I had to take some time out to deal with a few problems on the non-upgraded machines on CouchNet.

Upgrading Production Machines

Order of Upgrades

It looks like Oso is pretty close to ready, and it's time to put Tumbleweed into production, Here's the order for doing the upgrades:

Upgrading Petra

I repeated the upgrade process on Petra, but not taking detailed notes. Since I had improved the procedure from experience with Oso, it went smoothly on Petra, except for coddling needed for the bizarre networking.

Cloning Claude into Orion

Since Claude is mission critical, I wanted to clone it, upgrade the clone, and then atomically replace the old Claude with the clone. I use the name Orion for machines that I'm installing, before an atomic replacement like this. The cloning process is sufficiently complex, and sufficiently useful in the future, that I wanted to make a record of the steps.

Upgrading Claude (Continued)

Problems encountered when upgrading Claude. Most but not all are related to MDM customization.

Upgrading Aurora

Following along in Template for an Upgrade.

Upgrading Xena

To discover hidden problems, I need to start eating my own dogfood as soon as possible, hence Xena (my laptop) is the next on the upgrade schedule.

Upgrade procedure: Following along in Template for an Upgrade.

Once the basic problems were dealt with, I found these issues in the user session. See Session Configuration for MDM for fixes (for most of them).

Upgrading Kermit

Kermit (see hardware review) is an AMD E-350 which has bounced between several roles but is now doing audio playback. The only challenge in this upgrade may be the sound. Following along in the upgrade instructions:

Upgrading Iris

Iris is the home theater PC, running MythTV and also doing audio playback. Following along in the upgrade instructions:

Upgrading Jacinth

I had intended to upgrade Diamond after Iris and before Jacinth, but Microsoft Windows 10 (on the VM Baobei hosted on Diamond) self-destructed just before I started the upgrade. Obviously monkeying with Diamond's KVM will not affect problems on Baobei, but it's prudent to not touch Diamond until Baobei is dealt with.

Steps in upgrading Jacinth:

Upgrading Diamond

The Windows VM Baobei has been reinstalled and appears to be working normally, and the guest is known to work on a host with Tumbleweed, so I'm ready to upgrade Diamond fromm v42.1 to Tumbleweed. Preparation steps:

Doing the upgrade:

Updating Diamond is now finished, and this is the last host that needs to be updated.

Remaining Problems

Fixed Problems