Setting up the Blackbird ? GNUcode.org

Setting up the Blackbird

by Joshua Branson ? October 27, 2025

Let's install Devuan

I've been distro hopping and trying to find a good Linux distribution that works with power9 computers. I've tried, Debian (worked awesomely), Fedora (mostly worked), Chimera Linux (mostly worked), Alpine (could not get X or wayland to start), OpenBSD (could not get X to start, probably operator error), and Devuan. We settled on Devuan with KDE. Devuan is Debian without systemd and non-woke. They don't care what your political persuasion is. If you can code and would like to help, they let you. That's pretty awesome in our book. Let's not let politics ruin free software projects.

So, we have 4 drives that are each 1 TiB, and I wanted to set up raid1 on those drives. I wanted to use bcachefs, but the bcachefs people in #bcache said we should be using Linux version 6.16. Unfortunatly, Devuan uses Linux 6.12. I decided to install Devuan on one of the drives using btrfs.

joshua@blackbird-devuan.local:~/ $ lsblk

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 931.5G  0 disk 
??????sda1   8:1    0     7M  0 part 
??????sda2   8:2    0   977M  0 part 
??????sda3   8:3    0 930.6G  0 part 
sdb      8:16   0 931.5G  0 disk 
??????sdb1   8:17   0     7M  0 part 
??????sdb2   8:18   0   977M  0 part /boot
??????sdb3   8:19   0 930.6G  0 part /
sdc      8:32   0 931.5G  0 disk 
??????sdc1   8:33   0 931.5G  0 part 
sdd      8:48   0 894.3G  0 disk 
??????sdd1   8:49   0   1.9G  0 part 
??????sdd2   8:50   0 892.4G  0 part

I tried setting up Devuan with raid1 on / with 4 drives. That did not work, possibly because, when I manually set up the partitions, I did not create a powerpc PReP partition, which is where grub in installed. Then I tried setting up Devuan with raid1 with / and /boot (without a PReP partition), which also didn't work.

Then I gave up and let Devuan set up the partitions for me on one drive, which finally booted. Then I found out that you can add a btrfs device to an existing mount point. Let's give that a try. The cool guys in irc recommended I take a look at this website, which let me realize that btrfs supports raid1c4, which copies each file to each device. Pretty cool!

root@blackbird-devuan.local:~/ # btrfs device add -f /dev/sdb3 /  # 930 GiB
root@blackbird-devuan.local:~/ # btrfs device add -f /dev/sdc3 /  # 930 GiB
root@blackbird-devuan.local:~/ # btrfs device add -f /dev/sdd1 /  # 930 GiB
root@blackbird-devuan.local:~/ # btrfs balance start -dconvert=raid1c4 -mconvert=raid1c4 /

This command seems to verify that we have raid1c4 (RAID1 with 1 copy of each file on each disk).

root@blackbird-devuan.local:~/ # btrfs device usage /

/dev/sdc3, ID: 1
   Device size:           893.30GiB
   Device slack:              0.00B
   Data,RAID1C4:           11.00GiB
   Metadata,RAID1C4:        1.00GiB
   System,RAID1C4:         32.00MiB
   Unallocated:           881.27GiB

/dev/sda1, ID: 2
   Device size:           931.51GiB
   Device slack:              0.00B
   Data,RAID1C4:           11.00GiB
   Metadata,RAID1C4:        1.00GiB
   System,RAID1C4:         32.00MiB
   Unallocated:           919.48GiB

/dev/sdd1, ID: 3
   Device size:           931.51GiB
   Device slack:              0.00B
   Data,RAID1C4:           11.00GiB
   Metadata,RAID1C4:        1.00GiB
   System,RAID1C4:         32.00MiB
   Unallocated:           919.48GiB

/dev/sdb1, ID: 4
   Device size:           931.51GiB
   Device slack:              0.00B
   Data,RAID1C4:           11.00GiB
   Metadata,RAID1C4:        1.00GiB
   System,RAID1C4:         32.00MiB
   Unallocated:           919.48GiB

Yes, I am aware that one of the drives is slightly smaller that the other drives. The slightly smaller drive is apparently military grade/CIA/space level good. Apparently the data on that drive should last for 100+ years. ????

We originally wanted to try a btrfs raid1c4 with 4 drives. We got it working, but we tried simulating a 1 drive failure, via pulling the power on one drive. We could not get Devuan to boot, which means I'd need to follow something like this guide to fix the raid. We'd rather have a raid option that lets you boot, even if one drive fails. Apparently btrfs supports this via the degraded mount option. So how does one use the degraded mount option?

Let's see if I can boot with when one hard drive is not connected from power. I'll add the "degraded" option to fstab. We'll see if this works.

root@blackbird-devuan.local:/ # cat /etc/fstab | grep degraded

UUID=93a272dd-1d79-494b-9e98-ef29f20a34b0 /               btrfs   defaults,degraded,subvol=@rootfs 0       0

Well that makes the boot fail. I am stuck in the initramfs.

This guide may be helpful: https://christitus.com/btrfs-guide/.

I tried adding the degraded mount option to grub, and that still didn't work. :(

root@blackbird-devuan.local:/ # cat /etc/grub.d/10_linux | grep degraded
root@blackbird-devuan.local:/ # update-grub

GRUB_CMDLINE_LINUX="rootflags\"=subvol=${rootsubvol} degraded\" ${GRUB_CMDLINE_LINUX}"

an xfs raid0

After fiddling with gparted, here is what our blackbird currently looks like:

joshua@blackbird-devuan.local:/ $ lsblk

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 931.5G  0 disk 
??????sda1   8:1    0 931.5G  0 part 
sdb      8:16   0 931.5G  0 disk 
??????sdb1   8:17   0 931.5G  0 part 
sdc      8:32   0 931.5G  0 disk 
??????sdc1   8:33   0 931.5G  0 part 
sdd      8:48   0 894.3G  0 disk 
??????sdd1   8:49   0     7M  0 part 
??????sdd2   8:50   0   977M  0 part /boot
??????sdd3   8:51   0 893.3G  0 part /

We currently have three extra 1 TB drives that we are not using. Why not set up a raid0 array with those 3 drives ? We could have a bitcoin node!

root@blackbird-devuan.local:~/ # mdadm --create --verbose /dev/md0 \
      --level=0 --raid-devices=3 \
      /dev/sdb1 /dev/sdc1 /dev/sdd1
root@blackbird-devuan.local:~/ # echo now I will format an xfs filesystems
root@blackbird-devuan.local:~/ # mkfs.xfs /dev/md0
root@blackbird-devuan.local:~/ # mount /dev/md0 /mnt

mdadm: chunk size defaults to 512K
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
now I will format an xfs filesystems
log stripe unit (524288 bytes) is too large (maximum is 256KiB)
log stripe unit adjusted to 32KiB
meta-data=/dev/md0               isize=512    agcount=32, agsize=22889728 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=1
         =                       reflink=1    bigtime=1 inobtcount=1 nrext64=1
         =                       exchange=0   metadir=0
data     =                       bsize=4096   blocks=732471296, imaxpct=5
         =                       sunit=128    swidth=384 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1, parent=0
log      =internal log           bsize=4096   blocks=357656, version=2
         =                       sectsz=512   sunit=8 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
         =                       rgcount=0    rgsize=0 extents
Discarding blocks...Done.

Now, let's see how large the xfs filesystem is, eh?

joshua@blackbird-devuan.local:~/ $ df -h | grep /mnt

/dev/md0        2.8T   54G  2.7T   2% /mnt

2.7 TB isn't all that bad! I know that having raid0 may make some people a little anxious: you could lose data! That's true, but I am only planning on putting the bitcoin blockchain on /dev/md0. If we lose data or a drive. It's not the end of the world. There are hundreds (if not thousands) of other bitcoin miners out there backing up the blockchain.

Let's get the UUID of /dev/md0 eh ?

root@blackbird-devuan.local:~/ # blkid | grep /dev/md0 | \
    awk '{ print $1 " " $2 }'

/dev/md0: UUID="2bfcd0f3-14bc-4295-b80e-d9d9a9421b75"

Now let's make optionally mount the xfs raid0 drive.

joshua@blackbird-devuan.local:~/ $ cat /etc/fstab | grep xfs

# make mounting the xfs raid of 3 drives optional
UUID=2bfcd0f3-14bc-4295-b80e-d9d9a9421b75 /home/jeff/.bitcoin xfs defaults,nofail 0 0

Getting the speakers to work

While Devuan booted just fine, I could not get the speakers to work. I knew that it could work, because this command worked: speaker-test -c 2 -t wav. I got some great help from irc that recommended this forum post. Essentially, let's install and set up pipewire.

root@blackbird-devuan.local:~/bin/ # apt install pipewire-audio
root@blackbird-devuan.local:~/bin/ # cat start-wireplumber

#!/usr/bin/env bash

wireplumber &

pipewire &

pipewire-pulse &

In KDE's session I added start-wireplumber to the auto-started applications upon login. A simple reboot later, and sound worked! FYI, sudo apt install shortwave. Then listen to WKHR; it's a great station!

Upgrading RAM

We also wanted to upgrade the RAM in the blackbird. It currently has 32 GB of RAM, but the motherboard supports 256GB (2x128GB). Two 128 GB DDR4, ECC, RDIMM, 288 pin RAM. I found prices in a large margin. It looks like chips costs between $500-$1600.

  • $482 for 256 GB

https://atechmemory.com/products/256gb-2x128gb-ddr4-2666-rdimm-pc4-21300r-quad-rank-x4-server-kit?pr_prod_strat=e5_desc&pr_rec_id=971e53647&pr_rec_pid=7224623497259&pr_ref_pid=7224649547819&pr_seq=uniform

Fun fact, we actually took out the AMD GPU on the blackbird. The onboard ast is actually good enough for us. 1440p video from youtube works just fine, but if you want to get an amdgpu to work, read on.

Using an amdGPU

Let's tweak apt's sources and add 'non-free-firmware'. This debian wiki page was super helpful.

joshua@talos-devuan.local:~/ $ cat /etc/apt/sources.list

deb http://deb.devuan.org/merged excalibur main non-free-firmware

deb-src http://deb.devuan.org/merged excalibur main non-free-firmware

deb http://deb.devuan.org/merged excalibur-security main non-free-firmware
deb-src http://deb.devuan.org/merged excalibur-security main non-free-firmware

# excalibur-updates, to get updates before a point release is made;
# see https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_updates_and_backports
deb http://deb.devuan.org/merged excalibur-updates main non-free-firmware
deb-src http://deb.devuan.org/merged excalibur-updates main non-free-firmware
root@talos-devuan.local:~/ # apt update
root@talos-devuan.local:~/ # apt-get install firmware-amd-graphics libgl1-mesa-dri \
        libglx-mesa0 mesa-vulkan-drivers xserver-xorg-video-all

let's install guix

Debian is pretty fantastic, but I am a huge fan of guix. Why not use both? Here's how you set up guix.

joshua@blackbird-devaun.local:~/ $ sudo apt install nscd
joshua@blackbird-devaun.local:~/ $ sudo service nscd start
joshua@blackbird-devaun.local:~/ $ sudo apt install daemonize
joshua@blackbird-devaun.local:~/ $ wget https://guix.gnu.org/install.sh  # guix's installer
joshua@blackbird-devaun.local:~/ $ sudo ./install.sh
joshua@blackbird-devaun.local:~/ $ guix pull
joshua@blackbird-devaun.local:~/ $ sudo -i guix pull

Let's make sure that we run guix pull daily. We can edit the user's and root's crontab with:

joshua@blackbird-devuan.local:~/ $ crontab -e
joshua@blackbird-devuan.local:~/ $ sudo crontab -u root -e

Here's what those crontabs look like:

joshua@blackbird-devuan.local:~/ $ crontab -l | grep daily

@daily	guix pull
root@blackbird-devuan.local:~/ # crontab -u root -l | grep daily

@daily guix pull

setting up bitcoin knots

It's fairly easy to install bitcoin knots on devuan linux. Just download the binary. Hopefully guix will package it soon. With a simple script I am autostarting bitcoind at login:

joshua@blackbird-devuan.local:~/ $ cat ~/.local/bin/start-bitcoind

#!/bin/sh

# if the xfs raid0 is mounted at ~/.bitcoin, then 
# start bitcoind
if mount | grep /home/jeff/.bitcoin; then
    bitcoind
else
    echo "We cannot start bitcoin"
fi

Setting up FreeCAD with FEM

My client is a rather talented engineer, who is familiar with solidworks, which is a proprietary modeling software. He's interested in trying outreeCAD, which now supports a finite element analysis. It can't be that hard to set it up right?

  root@blackbird-devuan.local:~/ # aptitude install netgen gmsh
  root@blackbird-devuan.local:~/ # aptitude install libx11-dev libx11-xcb-dev \
           libxmu-dev libxi-dev \
           libglu1-mesa-dev libglut-dev \
           psutils gv gnuplot tetgen
  root@blackbird-devuan.local:~/ # cd Downloads; wget https://www.dhondt.de/cgx_2.23.all.tar.bz2
root@blackbird-devuan.local:~/ # bzip2 -d cgx_2.23.all.tar.bz2
  root@blackbird-devuan.local:~/ # tar -xvf cgx_2.23.all.tar
  root@blackbird-devuan.local:~/ # cp CalculiX /usr/local
  root@blackbird-devuan.local:~/ # cd /usr/local/CalculiX/src/
  root@blackbird-devuan.local:~/ # # I made some tiny edits to the Makefile
  root@blackbird-devuan.local:~/ # make
  root@blackbird-devuan.local:~/ # cp ./cgx ../../../bi
  root@blackbird-devuan.local:~/ # chmod ao+rx /usr/local/bin/cgx

Here is some installation details from the install guide.

You might search for this files with for example "locate libGL" or
"find /usr/* -name libGL* -print"

If you find only libMesaGL and libMesaGLU instead of libGL and libGLU
you have to create a link or a copy with the names libGL and libGLU
but ceep the extension .a or .so.

This files should be located in:
/usr/lib64/

otherwhise you have to modify the file:
/usr/local/CalculiX/cgx_2.23/src/Makefile
root@blackbird-devuan.local:~/ # find /usr/* -name libGL* -print

/usr/lib/powerpc64le-linux-gnu/libGLdispatch.so.0.0.0
/usr/lib/powerpc64le-linux-gnu/libGLdispatch.so.0
/usr/lib/powerpc64le-linux-gnu/libGLX_mesa.so.0.0.0
/usr/lib/powerpc64le-linux-gnu/libGLX_indirect.so.0
/usr/lib/powerpc64le-linux-gnu/libGLX_mesa.so.0 /m
/usr/lib/powerpc64le-linux-gnu/libGLX.so.0.0.0
/usr/lib/powerpc64le-linux-gnu/libGLX.so.0
/usr/lib/powerpc64le-linux-gnu/libGLESv2.so.2.1.0
/usr/lib/powerpc64le-linux-gnu/libGLESv2.so.2
/usr/lib/powerpc64le-linux-gnu/libGL.so.1.7.0
/usr/lib/powerpc64le-linux-gnu/libGL.so.1
/usr/lib/powerpc64le-linux-gnu/libGLU.so.1.3.1
/usr/lib/powerpc64le-linux-gnu/libGLU.so.1
/usr/lib/powerpc64le-linux-gnu/libGLEW.so.2.2.0
/usr/lib/powerpc64le-linux-gnu/libGLEW.so.2.2
/usr/lib/powerpc64le-linux-gnu/libGLX.so
/usr/lib/powerpc64le-linux-gnu/libGL.so
/usr/lib/powerpc64le-linux-gnu/libGLU.a
/usr/lib/powerpc64le-linux-gnu/libGLU.so
/usr/lib/chromium/libGLESv2.so

With a simple make and cp cgx to /usr/local/bin/cgx, I could test the program worked via:

cgx -b dummy.fbd