Monitoring several log files in one terminal window using screen

When debugging website problems I often run tail on /var/log/apache2/error.log or on /var/log/apache2/access.log. Today I needed to monitor both at the same time. One solution is to open two terminal windows, however this uses up valuable entries in the alt-tab list. A better solution for this purpose was to create two windows with the screen utility.

There are 3 key commands to use:

  • "Ctrl-a |" which splits the screen into two windows
  • "Ctrl-a <tab>" which sends focus to the next window
  • "Ctrl-a c" which opens a new shell in the current window

So after running screen, I ran
tail -f /var/log/apache2/error.log
I then pressed "Ctrl-a" followed by "|" (after releasing the control key) to split the window, "Ctrl-a" followed by tab to switch focus into the window I just created, "Ctrl-a" followed by "c" to start a new shell, and finally:
tail -f /var/log/apache2/access.log

Split screen monitoring

Posted in Linux | Leave a comment

Steam Hardware Survey: All Linux distributions increase in popularity

Valve's Hardware Survey for July 2013 shows an increase in usage for every recorded Linux distribution, with each distribution seeing increases between 7-22%. As every single distribution seems to be increasing in popularity, I interpret this to mean that as time goes on more people are trying gaming on Linux and they are staying with Linux.

Distribution Proportion of users Increase as proportion of total users Relative increase
Ubuntu 13.04 64 bit 0.43% +0.03% +7%
Ubuntu 12.04.2 LTS 64 bit 0.20% +0.02% +10%
Ubuntu 13.04 0.13% +0.01% +8%
Ubuntu 12.04.2 LTS 0.11% +0.01% +9%
Linux 64 bit 0.10% +0.01% +10%
Linux Mint 15 Olivia 64 bit 0.09% +0.02% +22%
Total 1.06% +0.10% +9%

Note that due to the precision of data that is provided there are likely to be rounding issues, so the relative result in the right hand column could be +/- a few percent.

Posted in Linux | 1 Comment

dkms.conf: Error! No 'BUILT_MODULE_NAME' directive specified for record #0.

I recently had an error crop up with an installation of Linux Mint Debian Edition. When upgrading a kernel, or adding/removing certain software packages, or typing "dkms autoinstall" the following error appeared:

dkms.conf: Error! No 'BUILT_MODULE_NAME' directive specified for record #0.
Error! Bad conf file.
File: 
does not represent a valid dkms.conf file.

The error message seemed to be complaining about a file called dkms.conf. Searching for files with that name led me to a file which clearly didn't have a BUILT_MODULE_NAME directive:
/var/lib/dkms/ndiswrapper/1.57/build/dkms.conf

$ cat /var/lib/dkms/ndiswrapper/1.57/build/dkms.conf
 PACKAGE_NAME="ndiswrapper"
 PACKAGE_VERSION="1.57"
 DEST_MODULE_LOCATION[0]="/updates"
 AUTOINSTALL="yes"

This file came from the package ndiswrapper-dkms, so purging the package cleared out the problem and allowed modules to be built once again.

dpkg --purge ndiswrapper-dkms

Posted in Linux | Leave a comment

Dealing with website forms that disable pasting

Occasionally I run across a website that thinks its a good idea to arbitrarily restrict fundamental functions of my browser. Recently I signed a petition which asked for my email address twice. Obviously I'm not going to type it twice, I'm going to type it once, copy it, then paste it into the second box. Except nothing happens. Have I hit the wrong key? Try again. Nope still doesn't work. Frustration mounts.

I took a look at the code to see how they did it by right-clicking in the offending text box and selecting "inspect element". some bright spark used on_paste="return false"
It's easy to see that they have implemented this anti-feature with the piece of code that says:
onpaste="return false"

One of the great features about the Chrome/Chromium browser is that you can directly edit the page via Inspect Element and instantly see the results. So after double-clicking on the onpaste element, and deleting it - I was able to paste my email address into the form!

Posted in Uncategorized | 3 Comments

Improving the fonts in Linux Mint Debian Edition with Infinality

It turns out that the fonts in LMDE look a bit different to those in say Ubuntu. This is partly because of Ubuntu's theme, and partly because Debian's font rendering is a little dated. There is a guy over at http://www.infinality.net/blog/ who has made some patches to the font renderer (freetype and fontconfig) which haven't yet been picked up by Debian. I installed these patches and it made quite a difference to some text in Firefox.

some text from firefox before and after installing infinality

Notice how the text in the upper half has the C and u in "Currently" touching, and each letter is quite skinny. While in the lower half of the image which I took after installing infinalty, the letters are better spaced, and a bit denser.

This is how I did it:

sudo apt-get install devscripts quilt
cd /tmp
wget https://github.com/chenxiaolong/Debian-Packages/archive/master.zip
unzip master.zip
cd Debian-Packages-master
cd fontconfig-infinality
./build.sh
cd ../freetype-infinality
./build.sh
cd ..
sudo dpkg -i *.deb

Posted in Linux | Leave a comment

Copying Thunderbird settings from Windows to Linux (or vice versa)

One of the benefits of cross-platform software such as the Thunderbird mail client is that its often quite easy to transfer data from one platform to another. The data that Thunderbird creates (that contains your account settings, downloaded emails, mail filters etc.) seems to be identical whichever platform you run it on.

Under Linux, the data files live in "~/.thunderbird/". Under Windows 7, I found my settings in "C:\Users\<username>\AppData\Roaming\Thunderbird\". I suspect this location might move around a bit from one version of Windows to another. Searching for profiles.ini might be the best bet if you're trying to locate an existing installation.

So, once I had located the data files, all I had to do was mount the windows partition under Linux, and copy "profiles.ini" and the "Profiles" directory from the Windows location to "~/.thunderbird/".

You could also use this method to transfer your Thunderbird settings from one computer to another.

Posted in Linux | 1 Comment

Installing Steam on 64-bit Linux Mint Debian Edition

One of my motivations for installing Linux Mint Debian Edition was to see for myself the efforts Valve have been making over the last few months, porting most of their games and their Steam platform to Linux.

It turns out that LMDE was not the best choice for this as the version of Linux that Valve primarily supports is Ubuntu, and there are a few differences between Debian and Ubuntu distributions - even though Ubuntu is more or less based on Debian.

One of the consequences is that the version of Steam distributed by Valve doesn't actually install on LMDE. Fortunately nearly all of the work to resolve this has been done already over here http://www.sarplot.org/howto/45/Steam_on_64bit_Debian_Wheezy_70. If you don't have an ATI graphics card those instructions might be all you need. If you do have an ATI card (like me) then when you try to launch steam, you may see the following error:

Error: You are missing the following 32-bit libraries, and Steam may not run:
libGL.so.1

One final step is needed, to have steam launching correctly:

sudo apt-get install libgl1-fglrx-glx:i386

Posted in Linux | 6 Comments

Removing Linux Mint Debian Edition's Chromium search page

When you install Chromium in the latest version of Linux Mint Debian Edition (LMDE), typing anything in the search/address bar takes you to an awful custom search engine that wastes loads of space and hurts my eyes.

The awful search page that Mint installsSo, how do we get rid of it? We go to the chromium settings via the triple line menu, then we choose Manage Search Engines, and observe that there are two "google" search engines. One has a url similar to google.com/cse (for custom search engine) and is marked as the default. All we do is set one of the other search engines as default, then click the X to delete Mint's one.

Removing Mint's Default Search Engine

Posted in Linux | Leave a comment

Installing Linux Mint Debian Edition via a USB stick

I installed 64-bit Mint Debian Edition today (LMDE) but I had a problem getting the image onto the USB stick. I used LinuxLive (LiLi) Usb Creator version 2.8.22 released June 1st 2013, however the default settings resulted in a usb stick that wouldn't boot properly.

The symptom was a kernel panic due to the init process being terminated. It wasn't very easy to see what the cause was, as after the panic the kernel changed the graphics mode back to a text mode which got rid of the error messages. It turned out that after mounting the initrd, it had a problem finding /boot/casper.

This problem occurred because LiLi didn't recognise the iso file, and guessed that it should use its settings for 64-bit Linux Mint Olivia. The fix was to manually select the settings for the most recent version of 64-bit Mint Debian Edition that LiLi knew about.

Settings to choose in LiLi

I was then able to boot into live mode and install to disc from there. I was slightly surprised that there was no direct install link from the boot menu but you can't have everything!

Posted in Linux | 2 Comments

Compiling Octave from source on Linux Mint Nadia (Ubuntu 12.10)

I recently needed to build GNU Octave from source, as the version available in the Ubuntu repositories (3.6.2) had a crash that reliably triggered when I attempted to run a matlab script from a lecture I had been watching (a Stanford lecture series called The Fourier Transforms and its Applications).

error: source: error sourcing file `/home/frankster/Reference/TheFourierTransform/materials/lsoftaee261/sinesum2.m'
*** glibc detected *** octave: corrupted double-linked list: 0x00000000020d6380 ***

I pulled the mercurial repository containing the source, and followed the build instructions in etc/HACKING which said to run the ./bootstrap script. Unfortunately, when I ran this script I got the error:

~/octave$ ./bootstrap
./bootstrap: one of these is required: glibtoolize libtoolize

It wasn't immediately clear to me what this error meant, and I couldn't find any answers with a quick search. After a little bit of trial and error, I realised that there might be some build pre-requisites that were not installed. Fortunately on debian-based distributions its incredibly easy to sort this out (for packaged software at least):

sudo apt-get build-dep octave

Development versions of octave have a new QT interface. In order to get this to compile I also installed some qt packages:

sudo apt-get install libqt4-core libqt4-gui libqt4-network libqt4-dev

Posted in Linux | Leave a comment

grub-probe: error: not a directory. (grub 1.99 in Ubuntu Linux 12.04)

I upgraded some packages earlier (including a kernel) and received an odd error from grub (version 1.99-21ubuntu3.9)

$ sudo grub-install --recheck /dev/sda
 /usr/sbin/grub-probe: error: not a directory.
 Auto-detection of a filesystem of /dev/sdb1 failed.
 Try with --recheck.
 If the problem persists please report this together with the output of "/usr/sbin/grub-probe --device-map="/boot/grub/device.map" --target=fs -v /boot/grub" to <bug-grub@gnu.org>

I couldn't complete the configuration of the updates that I had downloaded, and worse the machine was probably now unbootable. Poking around a bit, I find that the error reported by grub-probe could be reproduced with:

$ sudo grub-probe -d /dev/sdb1 --target=fs_uuid
grub-probe: error: not a directory.

I didn't know where to go from here so the first thing I tried was to purge grub and reinstall it.

sudo apt-get purge grub*

sudo apt-get install grub-pc

Didn't make any difference, so the next thing I tried was to downgrade to an earlier version of grub (1.99-21ubuntu3).

sudo apt-get purge grub*

sudo apt-get install grub-pc=1.99-21ubuntu3 grub2-common=1.99-21ubuntu3 grub-pc-bin=1.99-21ubuntu3 grub-common=1.99-21ubuntu3

Unfortunately this made no difference either. So the next thing I tried was to install the version of grub from Ubuntu 12.10. There were some dependency conflicts which I had to manually fix as you can see here:

sudo apt-get remove apport-hooks-medibuntu apport-gtk apport

sudo dpkg -i liblzma5_5.1.1alpha+20120614-1_amd64.deb grub2-common_2.00-7ubuntu11_amd64.deb  grub-common_2.00-7ubuntu11_amd64.deb  grub-gfxpayload-lists_0.6_amd64.deb  grub-pc_2.00-7ubuntu11_amd64.deb  grub-pc-bin_2.00-7ubuntu11_amd64.deb

Finally, grub installed ok! Crisis averted.

Posted in Linux | Leave a comment

Strip audio from a video in Ubuntu Linux

Lets say you have a video you recorded on your mobile phone, and you want to send it to someone but there is a noisy audio track which is irrelevant to the subject. Stripping out the audio track is fairly simple using the libav-tools package.

sudo apt-get install libav-tools

avconv -i INPUT.3gp -an -c:v copy OUTPUT.3gp

"-i INPUT.3gp" specifies the input file

"-an" specifies that there will be no audio track in the output file (or -vn would have no video track). Note that if "-an" appeared before "-i INPUT.3gp" we would copy no audio track from the input file (but there would still be an audio track in the output file, albeit empty possibly).

"-c:v copy" specifies the codec we use to encode the video track in the output file. Here we specify copy which means we don't transcode it. So rather than rendering the input video track to a buffer, then re-encoding it we are just copying it directly from the input file to the output file which means we don't lose any quality (whereas if we re-encoded it with a lossy codec then we would have a lower quality output file). Note that if we specified "-c:v <CODEC>" before the "-i INPUT.3gp" we would be specifying the codec used to decode the input file, rather than encode the output file.

"OUTPUT.3gp" specifies the output file. Obviously.

Posted in Uncategorized | Leave a comment

How NOT to calculate daily compound interest rate

I wanted to work out how much interest I would gain by leaving some money in a certain account for a few days. I knew I could derive this myself using logarithms but I was too lazy, so I searched google and clicked on the top result which turned out to be completely incorrect.

The author David Ingram recommends calculating a weekly interest rate from an annual interest rate by dividing by 12 months then 4 weeks per month which is just plain stupid, as the correct answer might be to divide by 365 (or 366!) days, then multiply by 7 days per week.

David Ingram fails to consider this distinction again and compounds his error when he suggests calculating your daily interest rate simply by dividing the incorrect weekly rate by 7. His method of calculation causes an error of 8% which is pretty significant!

This is of course assuming that we are talking about simple rather than compound interest, although the author makes no mention of this distinction. Certainly in the UK its rare to see simple interest used in financial products.

So if we're actually interested in the compound daily interest rate that is equivalent to a particular annual interest rate, this is what we need:

Daily Interest Rate = e^ {log (Annual Interest Rate) / 365}

Posted in Uncategorized | 2 Comments

ZoneMinder: Bareword "ZM_PATH_LOGS" not allowed while "strict subs" in use

Since a recent burglary I have been using ZoneMinder to provide CCTV security for my house on my Ubuntu 12.04 machine. I noticed earlier today that zoneminder had stopped running over the weekend and when I tried to start the service again, I got the following error:

Starting ZoneMinder: Bareword "ZM_PATH_LOGS" not allowed while "strict subs" in use at /usr/share/perl5/ZoneMinder/Logger.pm line 153.
BEGIN not safe after errors--compilation aborted at /usr/share/perl5/ZoneMinder/Logger.pm line 168.
Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 34.
BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 34.
Compilation failed in require at /usr/bin/zmpkg.pl line 37.
BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 37.
failure

Looking through the ZoneMinder code I discovered that the error indicated that a configuration variable somehow wasn't set. My first thought was that perhaps a new configuration variable had been added in an upgrade but hadn't been automatically added to the configuration file. So I added a line containing ZM_PATH_LOGS to /etc/zm/zm.conf but now when I tried to start zoneminder, I received a similar (but different error):

Starting ZoneMinder: Bareword "ZM_USE_DEEP_STORAGE" not allowed while "strict subs" in use at /usr/share/perl5/ZoneMinder/General.pm line 207.
BEGIN not safe after errors--compilation aborted at /usr/share/perl5/ZoneMinder/General.pm line 226.
Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 35.
BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 35.
Compilation failed in require at /usr/bin/zmpkg.pl line 37.
BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 37.
failure

I added a line for this then another similar (but different) error appeared! Clearly this approach isn't correct. So I logged into mysql and had a poke around:

mysql -u XXX -p
mysql> show databases;
mysql> use zm;
mysql> show tables;
+----------------+
| Tables_in_zm   |
+----------------+
| Config         |
| ControlPresets |
| Controls       |
| Devices        |
| Events         |
| Filters        |
| Frames         |
| Groups         |
| Logs           |
| MonitorPresets |
| Monitors       |
| States         |
| Stats          |
| TriggersX10    |
| Users          |
| ZonePresets    |
| Zones          |
+----------------+
17 rows in set (0.00 sec)

mysql> select * from Config;

0 rows in set (0.00 sec)

Like in iPhone adverts, the above sequence is shortened. But the surprising result appears at the end: there are no lines in the Config table! This must be the culprit preventing ZoneMinder from starting.

Its not clear how this config table has been reset, as I don't recall updating the ZoneMinder packages recently, so I don't think its a botched database upgrade. I have restarted the machine that it runs on a couple of times over the weekend so perhaps the database got corrupted at shutdown, though I can't find any mysql logs complaining about this. In conclusion, I don't know how it happened! But if it happens again all the info is here.

To resolve this, I download the ubuntu package for zoneminder and had a look in the postinst install script to see how it initialised the database. It turns out there is an sql file at /usr/share/zoneminder/db/zm_create.sql which is applied at package installation. I edited this file and removed all lines that didn't affect the Config table and applied it manually.

Finally zoneminder started up (albeit without my configurations)! After reconfiguring zoneminder, I noticed in ZoneMinder's log database the following entry:

Config mismatch, expected 218 items, read 0. Try running 'zmupdate.pl -f' to reload config.

If this happens again, perhaps 'zmupdate.pl-f' is sufficient to fix it, instead of error-prone sql hacking. (update: according to commenters this does not work!)

And the moral of the story is: always backup your databases!

Posted in Linux | 25 Comments

Acquiring hxcomp.exe without having Visual Studio installed

I have been using the excellent IDA Free to reverse engineer the driver for Akai's MPD16 as Akai have declined to support Windows 7 and it seems a shame for perfectly good hardware to be condemned to landfills prematurely.

While I was doing this I ran across a Zynamics script which purports to import function details from MSDN documentation into IDA. To cut a long story short you need hxcomp.exe which as you can read about here comes from the Visual Studio SDK. Unfortunately for me, the SDK installer refuses to install unless you have Visual Studio (non-express) installed. Not to worry, its possible to unpack the files from the installer manually with the help of some open source software.

I used Visual Studio 2005 SDK Version 4.0 which arrived in a file called VsSDKFebruary2007.exe. I expect it will work in more or less the same way with other SDK versions as well.

Use 7zip or similar to unpack VsSDKFebrurary2007.exe. There are several files inside but you only need vssdk.msi.

7zip isn't good enough at seeing inside vssdk.msi to get at hxcomp.exe so I found lessmsi which did the job nicely.

Fire up lessmsi and open vssdk.msi. Sort by Directory then scroll down until you find SourceDir\PFiles\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\Archive\Help. Select everything in that directory then click the extract button in the bottom right.

Now you have the files you need, you must manually install them. Note that there is an error in the description provided there and he accidentally pastes the contents of uninstall.bat twice so you don't have the correct install.cmd. What you actually need in install.cmd as far as I can tell is:

@echo off
set HXCOMP_DIR=C:\Program Files\HxComp
set PATH=%PATH%;"%HXCOMP_DIR%"
regedit /s install.reg
hxreg -n Hx -i Resources -l 1033 -s Resources.HxS

If you are after the Zynamics IDA script then you should now be able to decompile the help files in your sdk with a command like that described in the comments here.

for /R %x in (*.hxs) do hxcomp -u "%x"
Posted in Uncategorized | 1 Comment