Tag: linux

Categories
Linux

Building package from source:Cannot open POTFILES.in.temp for writing

I recently ran to an error when building a .deb package from source (on Ubuntu). I found a few people asking for help with the error message over the year, but I didn't find anyone offering an answer.

~/src/collectd-5.12.0$ debuild -us -uc -i -I
dpkg-buildpackage -us -uc -ui -i -I
dpkg-buildpackage: info: source package collectd
dpkg-buildpackage: info: source version 5.12.0-6
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Bernd Zeimetz bzed@debian.org
dpkg-source -i -I --before-build .
dpkg-buildpackage: info: host architecture amd64
fakeroot debian/rules clean
dh_testdir
dh_testroot
rm -f build-stamp
[ ! -f Makefile ] || /usr/bin/make distclean
rm -f debian/README.Debian.plugins
rm -f src/.1 src/.5
rm -rf debian/pkgconfig
dh_autoreconf_clean
dh_clean
debconf-updatepo
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
print() on closed filehandle OUT at /usr/share/intltool-debian/intltool-extract line 942.
Cannot open POTFILES.in.temp for writing at /usr/share/intltool-debian/intltool-update line 615.
make: *** [debian/rules:271: clean] Error 1
dpkg-buildpackage: error: fakeroot debian/rules clean subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui -i -I failed

In my case, the error occurred because I had downloaded the source package as root, so this caused permissions issues while building the package as a user.

$ sudo apt source collectd

If your source is owned by root as mine was, the solution is to change the ownership of the files to your user.

$ sudo chown <YOUR USERNAME>.<YOUR USERNAME> -R ~/src/collectd-5.12.0
Categories
Linux

Raspberry Pi camera & Shinobi CCTV

Raspberry Pis make for really good HD CCTV cameras as they're cheap, small and have wifi in the newer iterations. There seem to be two good CCTV recording packages at the moment - Shinobi & MotionEyes. It took some effort to get Shinobi to communicate with the camera on my Raspberry Pi, so this post records what I had to do, for posterity.

If you search on the internet about RaspberryPi cameras and Shinobi you find a bunch of posts talking about setting up an RTSP stream for Shinobi to connect to. And on the Shinobi discord, a developer pointed to https://gitlab.com/Shinobi-Systems/shinobi-ip-camera. None of these instructions worked - I think they may have been for an older version of Raspbian. So firstly lets go through the software versions and hardware versions that I'm dealing with.

The hardware is a Raspberry Pi 3, and a Raspberry Pi ZeroW. Both of these have built in wifi which makes things easier. With these I'm using official Raspberry Pi cameras. I'm using the latest version of Raspbian at the time of writing - Raspbian GNU/Linux 10, and Shinobi ocean-1.

What eventually worked for me was adapted from a couple of web pages, particularly https://gist.github.com/neilyoung/8216c6cf0c7b69e25a152fde1c022a5d, and involved gst-rpicamsrc and gst-rtsp-server which contained the test-launch programme.

  1. Download and build https://github.com/thaytan/gst-rpicamsrc. In the future this will be part of gstreamer (from version 1.18) so you won't need to do this step any more.
  2. Find out your gstreamer (gst) version e.g. with apt search libgstreamer. At the time of writing this is libgstreamer1.0-0/stable 1.14.4-1 armhf. You then need to download the version of gst-rtsp-server that matches your libgstreamer1.0 version. So I downloaded gst-rtsp-server-1.14.4 from https://github.com/GStreamer/gst-rtsp-server and build that.
  3. sudo apt install gstreamer1.0-plugins-ugly gstreamer1.0-plugins-bad gstreamer1.0-plugins-rtp
  4. gst-rtsp-server provides a utility called test-launch. It sets up an rtsp server. I manually ran this with test-launch ( rpicamsrc bitrate=8000000 awb-mode=auto preview=false rotation=180 ! video/x-h264, width=960, height=540, framerate=10/1 ! h264parse ! rtph264pay name=pay0 pt=96 ). This command had a resolution for a v1 camera. If you use a v2 camera you will want to choose a different resolution. I found this page to be useful for understanding the modes available.
  5. I then was able to configure Shinobi to connect to rtsp://CAMERA_IP:8554/test.
  6. The final setup was to install test-launch as a permanent background task in systemd. Put a file with the following contents into /etc/systemd/system called e.g. cam-stream.service then run systemctl start cam-stream && systemctl enable cam-stream.

[Unit]
Description=auto start cam stream
After=multi-user.target
[Service]
Type=simple
ExecStart=/home/xxx/bin/test-launch ( rpicamsrc bitrate=8000000 awb-mode=auto preview=false rotation=180 ! video/x-h264, width=960, height=540, framerate=10/1 ! h264parse ! rtph264pay name=pay0 pt=96
User=xxx
WorkingDirectory=/home/xxx/shinobi
Restart=on-failure
[Install]
WantedBy=multi-user.target

Categories
Linux

Mitigating bitrot and corruption in a linux md raid-1 array

A while ago I had the misfortune of running into data corruption issues with two different motherboards both of which caused corruption in my mdadm raid-1 array. This is the overview of how I identified corrupt files and healed the array, which I hope may provide a useful signpost to anyone else who runs into similar issues. At the end of this article I link to another post where I provide more detail about the problems with a particular Gigabyte motherboard.

ASUS P5QL-Pro

The first, an ASUS P5QL-Pro would occasionally misread bytes 14 or 15 (mod 16). It had been doing this for several months before I gained a full understanding of what was happening. I experienced occasional processes crashing for no clear reason, a downloaded iso which,when burnt to a USB stick and booted, was corrupt. I had also noticed around the same time, that transmission would occasionally state that already downloaded files were corrupt, so I would have to recheck them and download any corrupt parts. I blamed transmission (the torrent client I had used) for having buggy torrent code. Eventually I noticed that if I copied a large file, its md5sum wouldn't be the same as the original. I was able to simplify the issue in the end merely by running md5sum twice on the same 64GB file and observing different checksums each time.

After trying various combinations of hard discs, sata cables, sata ports and filesystems, I ruled out the possibility of a failing hard disc, a bad sata cable, a damaged sata port or a Linux filesystem bug. I also used the most recent version of md5sum and sha256sum, ran memtest86, used a sata card instead of the onboard sata ports, and reproduced the issue after booting from a usb stick. This eliminated software issues in md5sum, bad memory, a bad sata controller chip and a bad Linux installation. I suspected that the motherboard might be at fault, so I swapped everything over to another motherboard. This solved the immediate problem but..

Gigabyte GA-P35-DS4

After swapping everything over to the Gigabyte board, it would no longer boot! Grub failed to load the raid partition. After some investigation it turned out that the BIOS was confiscating part of the disc in order to store a backup of itself in a "Host Protected" Area and there was no option to disable this feature in the BIOS! After removing the HPA and changing the SATA ports the affected discs were connected to, I was able to boot from the partition, but a chunk of data on one half of the raid-1 array had been overwritten. Raid-1 has two copies of data, so losing one of them might not be fatal.

How bad was it?

A quick google didn't reveal many people who'd recovered from incidents like this but there didn't seem to be much support within the mdadm toolset for raid-1 recovery, and there didn't seem to be many pre-existing tools to help.

I needed to catalogue the extent of the damage, so I wrote a programme to compare the two constituent partitions that made up the raid array and printed a message when they differ. After identifying the sectors that differed, it was clear that it was just a few megabytes at the end of 1 disc.

It was then a simple matter to replace the corrupted data with data from the good disc, and the md array would load properly!

The next challenge was to prevent this happening again, and the simplest method ended up being changing the order that my disc drives were connected to the sata controllers, so that the bios "stole" some of a different disc for its bios backup. Have a look at the following post if you want a lot more detail about Gigabyte's unfortunate bios.

Warning about large hard discs, GPT, and Gigabyte Motherboards such as GA-P35-DS4