How NOT to calculate daily compound interest rate


Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/francisfisher/francisfisher.me.uk/problem/wp-content/plugins/latex/latex.php on line 93

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/francisfisher/francisfisher.me.uk/problem/wp-content/plugins/latex/latex.php on line 94

I wanted to work out how much interest I would gain by leaving some money in a certain account for a few days. I knew I could derive this myself using logarithms but I was too lazy, so I searched google and clicked on the top result which turned out to be completely incorrect.

The author David Ingram recommends calculating a weekly interest rate from an annual interest rate by dividing by 12 months then 4 weeks per month which is just plain stupid, as the correct answer might be to divide by 365 (or 366!) days, then multiply by 7 days per week.

David Ingram fails to consider this distinction again and compounds his error when he suggests calculating your daily interest rate simply by dividing the incorrect weekly rate by 7. His method of calculation causes an error of 8% which is pretty significant!

This is of course assuming that we are talking about simple rather than compound interest, although the author makes no mention of this distinction. Certainly in the UK its rare to see simple interest used in financial products.

So if we're actually interested in the compound daily interest rate that is equivalent to a particular annual interest rate, this is what we need:

Daily Interest Rate = e^ {log (Annual Interest Rate) / 365}

Posted in Uncategorized | 2 Comments

ZoneMinder: Bareword "ZM_PATH_LOGS" not allowed while "strict subs" in use

Since a recent burglary I have been using ZoneMinder to provide CCTV security for my house on my Ubuntu 12.04 machine. I noticed earlier today that zoneminder had stopped running over the weekend and when I tried to start the service again, I got the following error:

Starting ZoneMinder: Bareword "ZM_PATH_LOGS" not allowed while "strict subs" in use at /usr/share/perl5/ZoneMinder/Logger.pm line 153.
BEGIN not safe after errors--compilation aborted at /usr/share/perl5/ZoneMinder/Logger.pm line 168.
Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 34.
BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 34.
Compilation failed in require at /usr/bin/zmpkg.pl line 37.
BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 37.
failure

Looking through the ZoneMinder code I discovered that the error indicated that a configuration variable somehow wasn't set. My first thought was that perhaps a new configuration variable had been added in an upgrade but hadn't been automatically added to the configuration file. So I added a line containing ZM_PATH_LOGS to /etc/zm/zm.conf but now when I tried to start zoneminder, I received a similar (but different error):

Starting ZoneMinder: Bareword "ZM_USE_DEEP_STORAGE" not allowed while "strict subs" in use at /usr/share/perl5/ZoneMinder/General.pm line 207.
BEGIN not safe after errors--compilation aborted at /usr/share/perl5/ZoneMinder/General.pm line 226.
Compilation failed in require at /usr/share/perl5/ZoneMinder.pm line 35.
BEGIN failed--compilation aborted at /usr/share/perl5/ZoneMinder.pm line 35.
Compilation failed in require at /usr/bin/zmpkg.pl line 37.
BEGIN failed--compilation aborted at /usr/bin/zmpkg.pl line 37.
failure

I added a line for this then another similar (but different) error appeared! Clearly this approach isn't correct. So I logged into mysql and had a poke around:

mysql -u XXX -p
mysql> show databases;
mysql> use zm;
mysql> show tables;
+----------------+
| Tables_in_zm   |
+----------------+
| Config         |
| ControlPresets |
| Controls       |
| Devices        |
| Events         |
| Filters        |
| Frames         |
| Groups         |
| Logs           |
| MonitorPresets |
| Monitors       |
| States         |
| Stats          |
| TriggersX10    |
| Users          |
| ZonePresets    |
| Zones          |
+----------------+
17 rows in set (0.00 sec)

mysql> select * from Config;

0 rows in set (0.00 sec)

Like in iPhone adverts, the above sequence is shortened. But the surprising result appears at the end: there are no lines in the Config table! This must be the culprit preventing ZoneMinder from starting.

Its not clear how this config table has been reset, as I don't recall updating the ZoneMinder packages recently, so I don't think its a botched database upgrade. I have restarted the machine that it runs on a couple of times over the weekend so perhaps the database got corrupted at shutdown, though I can't find any mysql logs complaining about this. In conclusion, I don't know how it happened! But if it happens again all the info is here.

To resolve this, I download the ubuntu package for zoneminder and had a look in the postinst install script to see how it initialised the database. It turns out there is an sql file at /usr/share/zoneminder/db/zm_create.sql which is applied at package installation. I edited this file and removed all lines that didn't affect the Config table and applied it manually.

Finally zoneminder started up (albeit without my configurations)! After reconfiguring zoneminder, I noticed in ZoneMinder's log database the following entry:

Config mismatch, expected 218 items, read 0. Try running 'zmupdate.pl -f' to reload config.

If this happens again, perhaps 'zmupdate.pl -f' is sufficient to fix it, instead of error-prone sql hacking. (update: according to commenters this does not work!)

And the moral of the story is: always backup your databases!

Posted in Linux | 28 Comments

Acquiring hxcomp.exe without having Visual Studio installed

I have been using the excellent IDA Free to reverse engineer the driver for Akai's MPD16 as Akai have declined to support Windows 7 and it seems a shame for perfectly good hardware to be condemned to landfills prematurely.

While I was doing this I ran across a Zynamics script which purports to import function details from MSDN documentation into IDA. To cut a long story short you need hxcomp.exe which as you can read about here comes from the Visual Studio SDK. Unfortunately for me, the SDK installer refuses to install unless you have Visual Studio (non-express) installed. Not to worry, its possible to unpack the files from the installer manually with the help of some open source software.

I used Visual Studio 2005 SDK Version 4.0 which arrived in a file called VsSDKFebruary2007.exe. I expect it will work in more or less the same way with other SDK versions as well.

Use 7zip or similar to unpack VsSDKFebrurary2007.exe. There are several files inside but you only need vssdk.msi.

7zip isn't good enough at seeing inside vssdk.msi to get at hxcomp.exe so I found lessmsi which did the job nicely.

Fire up lessmsi and open vssdk.msi. Sort by Directory then scroll down until you find SourceDir\PFiles\Visual Studio 2005 SDK\2007.02\VisualStudioIntegration\Archive\Help. Select everything in that directory then click the extract button in the bottom right.

Now you have the files you need, you must manually install them. Note that there is an error in the description provided there and he accidentally pastes the contents of uninstall.bat twice so you don't have the correct install.cmd. What you actually need in install.cmd as far as I can tell is:

@echo off
set HXCOMP_DIR=C:\Program Files\HxComp
set PATH=%PATH%;"%HXCOMP_DIR%"
regedit /s install.reg
hxreg -n Hx -i Resources -l 1033 -s Resources.HxS

If you are after the Zynamics IDA script then you should now be able to decompile the help files in your sdk with a command like that described in the comments here.

for /R %x in (*.hxs) do hxcomp -u "%x"
Posted in Uncategorized | 2 Comments

BT Infinity and OpenWRT

So my girlfriend can't get reliable wifi upstairs on her iphone, maybe the bed's metal frame is acting as a faraday cage, maybe the BT HomeHub 3 router is not very good. Regardless of who's to blame for the wifi issues, you can't change DNS servers ostensibly for security reasons but as likely to be related to BT's commercial decision to redirect failed DNS queries to advertising web pages. (Really irritating when you are trying to find out if a host is up by pinging it, and the ping succeeds when the host is actually down because you mistyped the hostname). Anoyingly if the broadband connection goes down, the router starts redirecting all web requests to itself at the address bthomehub.home which causes certificate errors in thunderbird and is generally a pain in the arse.

So BT can't be trusted to provide a correct DNS service and coupled with some moaning about iphone wifi receiption, its clearly time to ditch the HomeHub 3 and get a proper router. Given the ddwrt GPL violations, OpenWRT is the obvious choice of OS.

Although OpenWRT can be put on a HomeHub 3, the current procedure involves a soldering iron. Therefore I bought myself a TP-Link WR1043ND which is supported by OpenWRT and has a gigabit switch - not bad for £50. Upgrading the firmware was a doddle, simply a matter of downloading the latest firmware for my device (http://downloads.openwrt.org/backfire/10.03.1/ar71xx/openwrt-ar71xx-tl-wr1043nd-v1-squashfs-factory.bin at time of writing) and using the router's built in firmware upgrade page! Very smooth.

Its all fairly straightforward to configure via the web interface (and in fact it comes with a pretty much decent configuration out of the box). I connected the bt infinity modem via the WAN port, and set up the PPPoE connection using the password from my homehub 3 router (bthomehub@btbroadband.com). There is no password listed in the homehub 3 configuration pages but it turns out I had to set up a password of "BT" before the connection succeeded. This was the only non-obvious thing I had to do to get everything working.

Posted in Uncategorized | 6 Comments

Secure MythWeb with Mythbuntu 12.04

When you install the MythWeb module, the default security settings in Mythbuntu allow password-less access from anywhere. If you wish to be able to schedule (or watch) recordings when you are away from home then you may want to allow users on your local network to access the screens without a password, while requiring a password for connections from outside your network.

Tell Apache to require authentication for MythWeb

Open /etc/apache2/sites-enabled/mythweb.conf (remembering to gain root access) and find the commented out section that looks like this:

    ############################################################################
    # I *strongly* urge you to turn on authentication for MythWeb.  It is disabled
    # by default because it requires you to set up your own password file.  Please
    # see the man page for htdigest and then configure the following four directives
    # to suit your authentication needs.
    #
    #    AuthType           Digest
    #    AuthName           "MythTV"
    #    AuthUserFile       /var/www/htdigest
    #    Require            valid-user
    #    BrowserMatch       "MSIE"      AuthDigestEnableQueryStringHack=On
    #    Order              allow,deny

I am not convinced /var/www is a safe place to keep your htdigest file, so I changed that line to store it in /etc/mythtv/htdigest. I added an "Allow from 192.168.0." line to allow users with an ip address starting 192.168.0 to connect without a password, then a "Satisfy any" line to state that the page can only be served to an authenticated valid-user (from the Require line), or on the local network (from the Allow from line). This is what my final section looked like.

        AuthType           Digest
        AuthName           "MythTV"
    #    AuthUserFile       /var/www/htdigest
        AuthUserFile       /etc/mythtv/htdigest
        Require            valid-user
        BrowserMatch       "MSIE"      AuthDigestEnableQueryStringHack=On
        Order              allow,deny
        Allow from 192.168.0.
        Satisfy            any

Enable the Apache module which implements the Digest authentication method

You now must enable apache's auth_digest module using the a2enmod command.

sudo a2enmod auth_digest

If you forget to do this you will see errors like this in your apache error log when you attempt to connect from a non-local address:

[Mon Jul 09 23:27:21 2012] [crit] [client 69.163.149.163] configuration error:  couldn't check user.  Check your authn provider!: /mythweb

Choose a username and password for remote access

Assuming you used the same values for AuthUserFile and AuthName as I did) then you can add the first user to the password file:

sudo htdigest -c /etc/mythtv/htdigest MythTV username_to_add

The -c parameter creates a blank file so don't use it for any additional users you add. If you don't have htdigest installed, you may need to install the apache2-utils package (sudo apt-get install apache2-utils).

Ensure that only the apache server can read the password file:

sudo chown www-data /etc/mythtv/htdigest
sudo chmod 640 /etc/mythtv/htdigest

Tell Apache about the configuration changes

sudo service apache2 reload

It should now be safe to expose MythWeb to the outside world!

 

Posted in Linux | 12 Comments

Shuttle XS35GS V3 and Mythbuntu 12.04

I bought the recently released Shuttle XS35GS V3 (also called XS35GTA V3) to use as a mythtv frontend. As usual, nothing works correctly first time round and there were a few things I needed to set up manually to get it working correctly.

1) I had problems with the onboard wifi - it would apparently connect then as soon as you tried to use the connection various errors messages would appear in the debug log.
Jul  5 10:37:18 bedroom kernel: [  157.712828] rtl8192c_common:rtl92c_firmware_selfreset(): 8051 reset fail.
Jul  5 10:37:23 bedroom kernel: [  162.696823] rtl8192ce:_rtl92ce_llt_write():<0-0> Failed to polling write LLT done at address 0!
Jul  5 10:37:23 bedroom kernel: [  162.696833] rtl8192ce:rtl92ce_hw_init():<0-0> Init MAC failed

So I disabled the wifi in the bios and used a USB wifi stick for the time being. I may end up installing a cable if wifi does not give adequate streaming performance for mythtv.

2) I connected the shuttle via HDMI cable to the TV, unfortunately no sound was transmitted initially. The shuttle model I chose has an ATI chipset, so I first tried installing ATI binary drivers:
sudo apt-get install fglrx-updates
After that I listed the drivers and played test sounds on each device with VLC until I found which one worked.
frankster@bedroom:~$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Intel [HDA Intel], device 0: STAC92xx Analog [STAC92xx Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0

VLC confirmed that the final device "HD-Audio Generic" is the correct one so the next step is to apply that setting inside mythtv. Unforunately the list inside mythtv is different to what I saw in the output of "aplay -l", but I worked out that mythtv describes it thus:
hdmi:CARD=Generic,DEV=0

3) Using the latest version of fglrx-updates (2:8.960-0ubuntu1.1) I found that switching the TV (connected via HDMI) off then on again caused the sound to stop working, requiring a system reboot. I downloaded the drivers from the ATI website and the issue was solved. The latest version at the time I downloaded was catalyst 12.6, which generated packages with a version of 8.980 (in contrast to the Ubuntu packages which are version 8.960).

4) The bios version that came with my device (1.00) had two problems. When requesting a restart, the machine would instead shutdown. Secondly the Ubuntu 64-bit installer didn't believe that 64-bit could be installed so I had to install 32-bit. (I'm about 90% certain that you should be able to install 64-bit on the Atom D2700).

I installed the latest bios driver (1.05) and this resolved the reboot issue. I have not attempted to reinstall since, so I don't know about the 64-bit installer issue.

Updated 19th August 2012

Posted in Linux | 7 Comments

Kworld PC160-2T remote control on Ubuntu Linux 11.10

I recently bought a dual tuner card for my Mythtv box (and I discussed getting it working with mythtv here). The good news is that the IR receiver on the card is detected by the driver, however the bad news is that the remote does not work out of the box with Ubuntu 11.10 because Ubuntu doesn't have a keymap for it. You can see in the dmesg output that it loads the default rc-empty keymap:

[ 31.058541] Registered IR keymap rc-empty
[ 31.058604] input: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1e.0/0000:05:01.2/usb3/3-1/rc/rc0/input4
[ 31.058713] rc0: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1e.0/0000:05:01.2/usb3/3-1/rc/rc0
[ 31.058716] dvb-usb: schedule remote query interval to 500 msecs.

So I sorted it out by creating my own keymap - its still a bit of a work in progress as I have no idea what functions some of the pictures on the buttons represent! To create the keymap, I used the programme evtest to capture the scancodes generated by the remote. I then added the scancode to a file and mapped it to a sensible looking event which I chose from this list (see also this one). I saved my keymap in the /etc/rc_keymaps directory and inserted it (I first closed mythtv and switched out of X to a terminal with ctrl-alt-F2 to ensure that no application had the event device open).

frankster@mythtv:/etc/rc_keymaps$ sudo ir-keytable -c -p NEC -w kworld_pc160-2t
Read kworld_pc160-2t table
Old keytable cleared
Wrote 30 keycode(s) to driver
Protocols changed to NEC

I then verified my keymap was working with ir-keytable. Now we receive as well as the scancode, a key up and down event every time a button is pressed.

frankster@mythtv:/etc/rc_keymaps$ sudo ir-keytable -t
Testing events. Please, press CTRL-C to abort.
1329667198.608627: event MSC: scancode = 866b00
1329667198.608631: event key down: KEY_NUMERIC_1 (0x0201)
1329667198.608632: event sync
1329667198.859974: event key up: KEY_NUMERIC_1 (0x0201)
1329667198.859975: event sync

Its all very well being able to manually install the keymap, but we need to make this a bit more permanent. I am currently preparing a kernel patch so that this remote control can be handled automatically, but for now we can achieve the same effect in userspace. I added to /etc/rc_maps.cfg the following line:

af9015 * /etc/rc_keymaps/kworld_pc160-2t

ir-keytable could then load the appropriate keytable.

sudo ir-keytable -a /etc/rc_maps.cfg 

This is the keymap I am currently using, I expect I will do some more tweaking as time goes on.

# table kworld_pc160-2t, type: NEC

0x866b1d KEY_POWER
#0x866b0x kworld log button - not sure what this does
#0x866b16 button between kworld button and power button

0x866b0a KEY_NUMERIC_0
0x866b00 KEY_NUMERIC_1
0x866b01 KEY_NUMERIC_2
0x866b02 KEY_NUMERIC_3
0x866b03 KEY_NUMERIC_4
0x866b04 KEY_NUMERIC_5
0x866b05 KEY_NUMERIC_6
0x866b06 KEY_NUMERIC_7
0x866b07 KEY_NUMERIC_8
0x866b08 KEY_NUMERIC_9

#0x866b09 some kind of reload symbol
0x866b14 KEY_MUTE

#0x866b1e small rectangle and big rectangle
#0x866b17 fullscreen?
0x866b1f KEY_HOME
0x866b0e KEY_BACK

0x866b20 KEY_UP
0x866b21 KEY_DOWN
0x866b42 KEY_LEFT
0x866b43 KEY_RIGHT
0x866b0b KEY_ENTER

0x866b10 KEY_CHANNELUP
0x866b11 KEY_CHANNELDOWN
0x866b12 KEY_VOLUMEDOWN
0x866b13 KEY_VOLUMEUP

#0x866b19 scheduled recording?
0x866b1b KEY_RECORD
0x866b4b KEY_EPG

0x866b1a KEY_STOP
0x866b44 KEY_PLAYPAUSE
0x866b40 KEY_BACK
0x866b41 KEY_FORWARD

#0x866b22 Tx button

#0x866b15 looks like some kind of transmit or infrared symbol
#0x866b0f widescreen?
#0x866b1c screenshot?
0x866b4a KEY_SLEEP
Posted in Linux | 4 Comments

Using the KWorld PC160-2T with Mythtv on Ubuntu Linux 11.10

The PCI card is internally implemented as a USB interface connected to the two tuners over USB which is interesting to say the least. Presumably its cheaper to put an extra USB chip on a PCI card than to make a new design specifically for PCI.

The first problem I had was that there was no firmware. I spotted this in the output from dmesg:

[ 13.980942] dvb-usb: found a 'KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T)' in cold state, will try to load a firmware
[ 15.147767] dvb-usb: did not find the firmware file. (dvb-usb-af9015.fw) Please see linux/Documentation/dvb/ for more details on firmware-problems. (-2)
[ 15.147781] dvb_usb_af9015: probe of 3-1:1.0 failed with error -2

Although the documentation describes a method to obtain the firmware by running a script, it turns out that Ubuntu already has the firmware packaged. So I just needed to install linux-firmware-nonfree.

sudo apt-get install linux-firmware-nonfree

Then I needed reinsert the modules so that the firmware could get downloaded:

sudo rmmod dvb_usb_af9015
sudo rmmod dvb-usb
modprobe
modprobe dvb-usb
sudo modprobe dvb-usb
sudo modprobe dvb_usb_af9015

Next time I ran dmesg, there were some much more promising lines:

[   29.822162] dvb-usb: found a 'KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T)' in cold state, will try to load a firmware
[   29.967469] dvb-usb: downloading firmware from file 'dvb-usb-af9015.fw'
[   30.037577] dvb-usb: found a 'KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T)' in warm state.
[   30.037783] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[   30.038103] DVB: registering new adapter (KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T))
[   30.042065] af9013: firmware version:4.95.0.0
[   30.046814] DVB: registering adapter 0 frontend 0 (Afatech AF9013 DVB-T)...
[   30.048791] tda18271 0-00c0: creating new instance
[   30.054815] TDA18271HD/C2 detected @ 0-00c0
[   30.324093] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.
[   30.324505] DVB: registering new adapter (KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T))
[   30.596052] ADDRCONF(NETDEV_UP): wlan0: link is not ready
[   31.040928] af9013: found a 'Afatech AF9013 DVB-T' in warm state.
[   31.044418] af9013: firmware version:4.95.0.0
[   31.058424] DVB: registering adapter 1 frontend 0 (Afatech AF9013 DVB-T)...
[   31.058533] tda18271 0-00c0: attaching existing instance
[   31.058541] Registered IR keymap rc-empty
[   31.058604] input: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1e.0/0000:05:01.2/usb3/3-1/rc/rc0/input4
[   31.058713] rc0: IR-receiver inside an USB DVB receiver as /devices/pci0000:00/0000:00:1e.0/0000:05:01.2/usb3/3-1/rc/rc0
[   31.058716] dvb-usb: schedule remote query interval to 500 msecs.
[   31.058720] dvb-usb: KWorld PlusTV Dual DVB-T PCI (DVB-T PC160-2T) successfully initialized and connected.
[   31.076254] usbcore: registered new interface driver dvb_usb_af9015

I found http://parker1.co.uk/mythtv_dvb.php to have some useful information - it shows a method of verifying that the card is working by taking mythtv out of the equation.

Now that I knew that the card was working, I configured it in mythtv. As there are two tuners I configured both in mythtv, making sure to increase the setting allowing multiple streams to be recorded from the same multiplex. (This is due to the way dvb-t/freeview works - several stations are transmitted on the same channel). My biggest worry when I ordered the card was that it would not have this funtionality and that I would never be able to watch or record more than 2 channels simultaneously, but the following line in the output of dmesg reassured me:

[   30.324093] dvb-usb: will pass the complete MPEG2 transport stream to the software demuxer.

I was able to watch some TV on mythtv and caught the end of Gypsy Wedding...probably the only reality TV show that I can tolerate! I then set up some recordings to take place over the next few days unfortunately none of these worked. It turns out that there is currently a bug in the 3.0 and 3.2 kernels' drivers that handles the chip - the driver doesn't like switching between the two tuners. A symptom of the bug is i2c read errors such as this:

[   38.379309] af9013: I2C read failed reg:d417

This bug causes both tuners to eventually stop working. I worked around this by disabling the second tuner in mythtv, and mythtv has been stable since. I will solve this later, but for now I need to get the remote working!

Posted in Linux | 14 Comments

undefined reference to `ceil' (or pow or floor or ...) in ubuntu 11.10

There has been a change to the build tools in ubuntu 11.10 - The "--as-needed" flag is now passed to the linker. This primarily has implications for dynamic library linking but it also affects the order that libraries appear on the command line even for static linking.

A simple example of the problem:

frankster@frankie-laptop:~> cat testm.c
#include <math.h>

int main(int argc, char*argv[]){
double x;
double return_val = ceil(x);
return 0;
}
frankster@frankie-laptop:~> gcc -lm testm.c
/tmp/ccNkQLo6.o: In function `main':
testm.c:(.text+0x15): undefined reference to `ceil'
collect2: ld returned 1 exit status>

A quick fix solution could be to pass the "-Wl,--no-as-needed" option to gcc which turns off the option in ld. However the real problem in the example I provided is the order of -lm and testm.c. The following builds successfully:

gcc testm.c -lm

More info at http://www.gentoo.org/proj/en/qa/asneeded.xml and https://wiki.ubuntu.com/OneiricOcelot/ReleaseNotes?action=show&redirect=OneiricOcelot%2FTechnicalOverview#GCC_4.6_Toolchain

Posted in Linux | 7 Comments

EMU Legacy archive

Since Creative have taken the EMU Legacy pages offline, its not longer possible to get hold of for example OS updates for various devices such as the Proteus 2000, MP7, EMU Ultra etc. I am making available as much of the old material as possible here. Let me know in the comments if there's anything that's missing. Don't blame me if anything doesn't work (but please do let me know)!

(Edit: More recently some of these files have become available again  at http://www.creative.com/emu/support/downloads/legacy/ so its worth having a look there as well.)

Proteus

pcp2xr.zip
pcp3xr.zip
pcp1.zip
pcp2.zip
pcp1xr.zip
pcp3.zip

Proteus 1000

This should be the OS update you need if you have any of the following:

  • Proteus 1000
  • XL-1 (not the turbo version)
  • Mo'Phatt
  • B-3
  • Planet Earth

p1k226.zip

Proteus 2000

The Proteus 2000 family contains the following:

  • Proteus 2000
  • Virtuoso 2000
  • Turbo Phatt
  • Orbit-3
  • Vintage Pro
  • XL-1 (turbo version)

E-Loader_1.1_Install.zip
Proteus2KOp-E.pdf
Proteus Family SysEx 2.2.pdf
p2k226.zip
E-LoaderOperationManual.pdf
E-Loader_1.1_Install.sit

Proteus 2500

E-Loader_1.1_Install.zip
E-LoaderOperationManual.pdf
patterns2500.zip
Command200.zip

Ultra

EOS28CL.EXE
eos40manual.zip
462prep.zip
EOS410CL.EXE

MP7

MP7Op-F.pdf
MpToMp7SMFBanks.zip
E-Loader_1.1_Install.zip
Proteus Family SysEx 2.2.pdf
command200.zip
MP7Patterns.zip
CSprepreadme.txt
MpToMp7SMFBanks.zip
E-Loader_1.1_Install.zip
command200.zip
MP7Patterns.zip
MpToMp7MIDBanks.zip
E-LoaderOperationManual.pdf
E-Loader_1.1_Install.sit
Cmd_Sta_2.0_Addendum.pdf
MpToMp7MIDBanks.zip
E-LoaderOperationManual.pdf
E-Loader_1.1_Install.sit
commandprep10.dli

EOS

EOS28CL.EXE
EOS_FAT_Addend.pdf
eos28cl.sit
eos40manual.zip
EOS410CL.EXE
EOS4-0.pdf
EOS470.zip
EOSV461.EXE
eos_v461.sit
eos410cl.sit
EOS460Addend.pdf
EOS_V461.EXE
EOS_V460.EXE
EOS410UL.EXE

Misc

ORCH2_drummap.txt
XLEAD_drummap.txt
AOrch_drummap.txt
World_drummap.txt
PHATT_drummap.txt
TeCnO_drummap.txt
P123_drummap.txt
ZR_drummap.txt

Manuals

Audity 2000: Audity_2000.pdf
B3: B3_Manual.pdf
Mo'Phatt: MOPhattRevB.pdf
Orbit 3: E-Mu_Orbit-3_Manual.pdf
Planet Phatt: E-Mu_PlanetPhatt_Manual.pdf
Proteus 1000: E-Mu_Proteus1000_Manual.pdf
Proteus 2000: Proteus2KOp-E.pdf
Xtreme Lead-1: E-Mu_XL-1_Manual.pdf
Xtreme Lead-1 Turbo: XL1TurboOp-A.pdf
Vintage Pro: E-Mu_VintagePro_Manual.pdf
Virtuoso 2000: Virtuoso 2000.pdf

Proteus 2000 Family SysEx specification: Proteus Family SysEx 2.2.pdf
There is another site archiving the old emu files, the Emu Legacy Wiki, and the Proteus 2000 family yahoo group. Some manuals available here.

Posted in Music | 154 Comments

Yamaha FS1R tutorial

I found this old tutorial for the Yamaha FS1R fm synthesiser in the wayback machine and have reproduced it here for posterity.

Beginners Guide To The Yamaha FS1R

© 2000 R.C.Strange
written by R.C.Strange
email: rcstrange@hotmail.com
website: www.geocities.com/rcstrange2000

This Word document contains my understanding of the FM/FS synthesis used by the FS1R. I stress my understanding, as everything in here I have gleaned from the user manual, front panel and a couple of bits of demo software and have probably misunderstood some of it. If I’ve got anything wrong or missed anything important out, please let me know.

Disclaimer

I have no link with Yamaha whatsoever, other than the fact that I have bought one of their synths.

The point in writing this is to try to get a decent understanding of how the FS1R synthesises sound, to allow easier and better programming of sounds.

I start with the bleeding obvious but no doubt get into some complicated stuff pretty quickly. I’ve tried to write it assuming that whoever’s reading it knows very little about the subject, but wants to know about it. The reason for this is that we’ve all been there and there’s nothing worse than someone hiding behind a load of jargon. Jargon is often a sign that the person using it doesn’t understand what they’re on about either.

I’ve tried to put in a few practical examples along the way, as this not only makes it easier to understand, it makes it easier to explain too. So, I’d recommend reading this in front of your FS1R (if you have one). Also, don’t expect to follow everything or understand it immediately – it’s taken me months to get this far.

Although this information is copyrighted, please feel free to distribute this information as much as you wish if you are an individual – just as long as my name, email and website are at the top. Any companies or organisations wishing to use all or any part of this text are strictly prohibited from doing so.

Basics

The FS1R is a synthesiser. That means it makes up its sounds from scratch, unlike a sampler, which (fundamentally) plays back a digital recording (like a tape recorder), or many other keyboards which, although called synths, are actually a combination of samples and synthesis.

The type of synthesis used is based on Frequency Modulation (FM). Basically, it produces a voltage which varies with time (ie. it goes up and down). When you put this voltage into your amplifier, it gets converted to sound by the loud speakers moving forwards and backwards. All sound is made by this principle – something vibrates which pushes air about. Therefore, (in theory at least), any sound can be replicated as long as you push the air about in the correct way.

Another thing to know is that sound is made up of lots of different sine waves. If you had all the sine waves that the human ear can hear, all going at the same time, it’d be possible (in theory) to recreate any sound just by changing the volume of each sine wave up and down by the appropriate amount.

FM synthesis tries to recreate this by producing the sine waves required to create a sound. It’d be difficult to produce all sine waves that a human ear can hear, so instead it tries to get round it, getting close to the sound.

 

FM Synthesis

 

To start with, here’s the minimalist approach.

The FS1R is capable of producing a sine wave at a defined frequency. The frequency can be changed in many ways, but most important is that a keyboard attached via the MIDI port can do it by pressing different keys (ie. it can be ‘played’).

This produces a very simple ‘whistle’ sound – not too good if you want to recreate a piano or produce some weird, spacey effects.

So, you add another sine wave. This is a bit better but still quite simple. To get a more complicated sound, you could try using the first one to change the pitch of the second. This is called Frequency Modulation and starts to produce more complicated sounds.

The FS1R is capable of producing 8 of these sine waves (using 8 Oscillators), which can be combined in one of 88 fixed combinations (called Algorithms).

Each Oscillator can have its characteristics changed in various ways. For example, the volume can be changed over time, using the Envelope Generator (abbreviated to EG). This can fade in an Oscillator, hold it at a certain volume, then fade it out again after the key is released. The pitch of the Oscillator can also be changed in much the same way using the Pitch Envelope Generator (PEG).

There are also two Low Frequency Oscillators (LFOs). These are similar to the sound-producing Oscillators, but operate at a lower frequency (hence the name). These can also be used to vary the amplitude and frequency of the Oscillators to produce ‘tremolo’ and ‘siren’ effects. The amount of LFO applied to the Oscillators is often set up to be controlled via MIDI using the modulation wheel, so the further the wheel is pushed, the more the LFO varies the amplitude/pitch.

The output of the Algorithm can then be input to a Filter, which alters the treble and bass characteristics of the sound. The Filter can also be varied using the LFOs and via MIDI controllers.

On most older FM synths, oscillators could only generate sine waves. The FS1R is capable of producing these and other types of waves.

Noise Noise Noise

Many sounds have a ‘noise’ element associated with it, which is made up of lots of random frequencies. For example, a flute sound has the pitch of the note being played but also the sound of the air being blown over the mouth-piece. To help make these types of sound more realistic, each Oscillator also has a Noise-Generator associated with it. The Oscillator/Noise go to make up what Yamaha call an Operator. Yamaha call the Oscillator a Voiced Operator and the Noise Generator an Unvoiced Operator.

The grouping of Operators (i.e. the Algorithm) and the way their pitch and volume are varied is called a ‘Voice’. Up to 4 Voices can be combined to make up a ‘Performance’. Each Voice in a Performance can be input to the effects sections of the FS1R to give the sound a bit more depth and variation.

Note that only the Voiced Operators are linked together in an Algorithm. The Unvoiced Operator outputs are just added together like inputs to a mixing desk (i.e. exactly the same as Algorithm 1 on the FS1R).

Formants

The FM method described above is the way that many synths (such as the DX7) generate their sound. What makes the FS1R different is the inclusion of Formant Operators.

A Formant is basically a band pass filter, of which the centre-frequency, amplification and width can be varied with time. When several of these Formants are used within a sound, it can start to sound like a human voice.

Now, this is where the information from Yamaha starts to get woolly. Exactly how the Formant interacts with the Voice/Noise Operators hasn’t really been described by Yamaha in their manual. I know it’s a bit technical, but if you want to be able to create your own sounds from the lowest level up, you need to know this information. Yes, it’s possible to experiment and come up with some noises, but this is just guess work. A full explanation of this part of the synth wouldn’t go amiss. The user guide details each parameter that can be modified and what settings it can take, but without a bit more explanation about how the parameters interact, they end up just stating the obvious. Take the ‘form’ parameter under the ‘voice/operator/osc’ menu. The settings for this can be sine, odd1, odd2, res1, res2 or formant. OK, so the first ones are standard waveforms that will be described in most books on synthesisers. The last one, though, is particular to the FS1R. It seems to be fundamental to the Formant Synthesis part of the synth, so an explanation of what it is, what it interacts with, and what interacts with it wouldn’t go amiss. Just because the bods at Yamaha know this stuff doesn’t mean the rest of us do. If they want us to go out and use their synths, they should make this information freely available.

Try It Out

Right – try this out. Initialise a Performance on the FS1R and set it up to only consist of Part 1. Initialise the Voice associated with this Part. Don’t worry about losing the Performance and Voice – as long as you don’t store them, they will be restored when you select a new Performance (you could always do a MIDI dump to backup the FS1R if you’re worried).

Playing this Performance should now produce a sine wave. This is because the initialised Voice has only one Voiced Operator active which is set up to produce (surprise, surprise), a sine wave. The Algorithm used is number 1, which just mixes the outputs of all Operators together – no Frequency Modulation takes place.

Try adding another Voiced Operator to fatten the sound up a bit. To do this, select Voiced Operator 2 and turn its ‘output’ parameter up. Set its frequency to be the twice the first Operator. You should now be able to hear 2 sine waves. Adding more Voiced Operators will fatten up the sound.

Next, let’s make some noise. Turn off all the Voiced Operators by setting their ‘output’ parameters to 0. Then select Unvoiced Operator 1 and turn its ‘output’ parameter up. This should now produce a sound like a radio does when it’s not tuned in to a station. This is caused by lots of sine waves being generated at lots of differing frequencies.

By altering the ‘frequency’, ‘bandwidth’ and ‘skirt’ parameters, the characteristics of the noise can be altered. Changing the ‘resonance’ parameter will exaggerate the centre frequency. Increasing the resonance and decreasing the bandwidth will make the range of frequencies produced less and less until, at its maximum value, it will filter through just one frequency, so a sine wave is produced (useful if you run out of Voiced Operators).

This is all very good, but what about these bleeding formants?

Well… it seems that the Voiced Operators act as formants when the ‘wave’ parameter is set to ‘frmt’. As the Voiced/Unvoiced Operators form a ‘formant pair’, the Unvoiced Operator gets ‘formanted’ by the Voiced one.

If the Operators have their ‘fseq switch’ set to on, they’ll formant according the current fseq set in the ‘performance/common/fseq’ menu (as long as the fseq is assigned to part 1 of the performance in this menu).

There’s also an ‘fseq track’ number parameter that determines the fseq track to use for each Operator. Apparently, fseqs are made up of 8 tracks (like an 8-track tape recorder). Each track contains information on how the frequency, volume and width for one formant changes when the fseq is played. Each Operator then has a track assigned to it. Usually, track 1 is assigned to Operator 1, track 2 to Operator 2, etc, but this doesn’t have to be the case. All Operators could have track 1 assigned to them, or Operator 8 could have track 8 assigned, Operator 2 track 7, etc. Each fseq track can be assigned to more than one Operator, but each Operator can only have one track assigned to it.

Try setting all Voiced Operator volumes up, set them to be ‘fmnt’s using the ‘Form’ parameter and set their ‘Fseq Switch’ to ‘on’. Turn all Unvoiced Operator volumes up, set their ‘FreqMode’ to ‘linkF0’ and also set their ‘Fseq Switch’ to ‘on’. Make sure Algorithm 1 is selected.

This should now let you hear all fseqs when they are played. Some sound better than others. I think this is because some fseqs need to be used with particular types of sounds (eg. Drums need more noise, vocal effects need more FM).

Algorithm

At last. You’ve set up a Voice from scratch to produce an fseq. If you’ve followed what was described in the previous section, you’ll have 8 Unvoiced Operators being formanted by the 8 Voiced ones, the outputs of which are all mixed together to produce the sound you hear.

Remember, though, the FS1R is capable of FM synthesis – i.e. one Operator modulating another. The above example used Algorithm 1, which just mixes the outputs of all the Operators together. Instead, a different Algorithm could be used. Number 6, for example, modulates Operator 2 using Operator 1, and modulates Operator 4 using Operator 3. So, Operators 2 and 4 could be used to formant Operators 1 and 3.

Let’s try this out. Re-initialise the current Performance and Voice and select Algorithm 6. Now, set the ‘output’ parameters for Voiced Operators 1..4 to their maximum value. As all Operators are producing sine waves, a basic sort of sound should come out which probably sounds much like an old 60’s organ. Now, try setting the wave forms of Operators 2 and 4 to ‘frmt’ (using the ‘form’ parameter in ‘Voice/operator/Osc’) and turn their ‘fseq switch’ parameters to ‘on’. You will also need to make sure the ‘Performance/common/fseq’ parameter is set to Part 1, as before, and that the selected Fseq is valid,e.g. ‘Pre>01’ (my FS1R defaults to an internal Fseq, of which I have none. This means no Fseq is played).

This produces some weird sounds, some of which nearly sound like voices. Some don’t produce sound at all (eg. The drum loops) – typically these are the ones that need some noise in order to sound properly.

Formant Sequences (Fseqs)

Up until now, the fseqs have probably been playing from start to finish (the FS1R seems to change the ‘start’ and ‘end’ parameters of the fseq itself sometimes). Found any that you’d ever seriously use in a recording? Me neither. However, with a bit of messing around, it’s possible to change all that.

Fseqs can be played in a number of different ways. The most obvious is starting at the beginning and running until the end. The point at which to start and end the fseq can be altered and fseqs can also be looped, much like looping a sample. If the start point is set to be higher than the end point, the fseq will play backwards. This gives the opportunity to create all sorts of weird sounds, e.g. playing a short part of a sequence, looping it or reversing it.

The speed of an fseq can be changed, so slowing one right down when applied to a synth pad sound may give it that ‘spacey’ quality where it’s continuously changing. This can be used in conjunction with the LFOs and EGs to produce some great pad sounds. Add a few effects to this and you’ll think you’ve been transported onto the 2001 film set.

The speed can also be synced to a MIDI clock – turn the speed down past 0% to do this - then start your sequencer and play a few notes. If the selected fseq has a rhythm to it, it should be in time with the sequencer.

All the above is done in ‘fseq’ mode. There is also a ‘scratch’ mode, whereby the fseq can be played by moving the ‘Format’ controller on the front panel. Rotate it one way and it plays forwards as you rotate it, rotate it the other way and it’ll play backwards, a bit like pushing a record round on a record-deck (hence the ‘scratch’ name).

Changing Formants – Do It Yourself

It probably hasn’t escaped your attention that there is a ‘Formant’ knob stuck on the front of the FS1R. Whilst this can be configured to control many parameters, a lot of the preset Performances use it to control the Formant Operators.

Basically, it allows you to change 5 of the ‘Formant’ parameters at the same time using one controller – especially useful for live work. For example, the centre frequency of Voiced Operators 1..5 can all be assigned to the one controller, or width, centre frequency and volume for Unvoiced Operator 1, and frequency and width for Voiced Operator 1, etc, etc.

How much effect turning the knob has can also be set up – positive values increase the selected Formant parameters when the knob is turned clockwise, negative values increase the parameters when it is turned anti-clockwise.

This can be used on its own to control the sound (for yet more slowly varying pad sounds, this time with you varying the formants rather than the fseq), or in conjunction with the fseqs.

Goodbye

Now you’re on your own. If you followed it this far, you know everything you need in order to create sounds that are really different to the usual ones you get in most synths.

Those of you that gave up and sold your FS1R – you mugs! You don’t know what you’re missing.

Posted in Music | 4 Comments

M-Audio Delta 1010 sound stops working for some applications on Windows 7 x86

I've had occasional audio playback problems. My symptoms are that websites and videos will suddenly stop making any sound in firefox while I can still play tunes in vlc. Restarting firefox makes no difference. If I fire up other applications, they stop making sound as well.

It turns out that there's a problem with the version 5076 drivers for the M-Audio Delta 1010 and 1010LT on 64 bit Windows 7. It can be solved by restarting the windows sound service.

Posted in Music | 2 Comments

Deciphering the TC Electronic M350 SysEx format

I recently acquired a TC Electronic M350 effects unit. One if its features is midi connectivity, which allows you to backup or edit the presets on a PC - although the interface exposes every setting so there's not much reason to edit them on the PC.

Unfortunately the librarian/editing software that TC electronic make available (called Vyzor but actually a customised version of a librian product called Uniquest) doesn't work very well on Windows 7 64 bit.

They barely mention their sysex support in the manual and in fact what little they do mention, they get wrong! (The correct procedure to manually initiate a bulk preset dump is to hold the preset on/off key down for approximately 2.5 seconds, scroll up until the display says bd for bulk dump mode, and then press the preset on/off button again).

Anyway I could find no information on the internet concerning the sysex specification of the device so it was clear I needed to reverse engineer the M350's sysex messages and implement a driver for it in JSynthLib which is a librarian/editor that actually works on Windows 7 (and also Macs and Linux as it is cross-platform).

I have made the following notes which I am leaving up here in case they are useful to anybody in the future.

TC Electronic M350 system exclusive spec for firmware version 1.3:

Device Inquiry f0 7E 7F 06 01 F7 elicits a reponse F0 7E 7F 06 02 00 20 1F 58 00 00 00 00 00 01 03 F7. Note that prior to upgrading the firmware, it responded to the Device Inquiry thusly: F0 7E 7F 06 02 00 20 1F 57 00 00 00 00 00 01 01 F7. Note that not only does the software version increase form 1.1 to 1.3, the "device number" changes from 57 to 58, perhaps indicating that the sysex implementation has changed somewhat between versions.

The basic sysex format is:

F0 00 20 1F 00 58 mm <data> F7

mm is the message type and there seem to be the following message recognised:

20: patch data (bi-directional)
22: value change (bi-directional)
45: request patch (to device only)
47: request parameter value (to device only)

message type 20 patch data

F0 00 20 1F 00 58 20 pp uu aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa bb bb cc dd ee ff gg hh ii jj kk ll F7
pp = patch number, (00 = edit buffer)
uu: unknown always 00(maybe bank #? but unused as far as I can see)
aa 20 chars patch name
bb bb 2 bytes - tap (milliseconds, LSB first)
cc input gain
dd mix ratio
ee effect bal
ff delay effects
gg delay/timing
hh feedback/depth
ii reverb type
pre delay
jj decay time
kk colour filter
ll checksum - this is calculated simply by summing all the data bytes (i.e. not the header bytes and not the checksum/tail bytes)

message type 22 real time parameter changes sent by editor software or device (seems to be a duplicate of the normal midi controller messages)

f0 00 20 1F 00 58 22 01 xx yy yy F7

yy = value (second yy is 0 apart form for 0d tap in ms)
xx = parameter number:
00 = input gain
01 = mix ratio
02 = effect bal
03 = delay effects ( yy = 00 - 0f )
04 = delay timing
05 = feedback depth
06 = reverb type (yy = 00 - 0f)
07 = predelay
08 = delay time
09 = colour filter
0a = bypass toggle (yy = 00 / 7f)
0b = dual input mode toggle ??????? pure guess
0c = digital input toggle (yy = 00 / 7f)
0d = tap in ms ( 2 bytes, LSB first )
0f = toggles front panel lock - 40+ makes it remote controlled, less than that makes it editable
10 = delay effects on/off toggle?
11 = reverb effects on/off toggle?

Note that parameters 0a/0b/0c/0f/10/11 can be controlled via the software, but are not sent or stored in the patches.

message type 45 request patch
F0 00 20 1F 00 58 45 pp bb F7

pp is patch number

bb is always 00 but I speculate this could be bank number, possibly in future versions of this device.
message type 47 request parameter value
F0 00 20 1F 00 58 47 xx F7
xx = parameter number as listed for message 22

Posted in Music | 15 Comments

Running a python script that requires libgsf on Ubuntu 11.04

I was exploring a project concerned with reverse engineering some file formats (https://gitorious.org/re-lab/tools) and I needed to run a python script. This python script requires libgsf (the gnome structured file library: http://projects.gnome.org/libgsf/ ), however the debian/ubuntu packages for this aren't built with python support!

This led to errors such as:

frankster@linux-virtualbox:~/relab/tools/oletoy$ ./view.py
Traceback (most recent call last):
  File "./view.py", line 23, in <module>
    import Doc, cmd
  File "/home/frankster/relab/tools/oletoy/Doc.py", line 19, in <module>
    import ole,mf,svm,cdr,clp,rx2
  File "/home/frankster/relab/tools/oletoy/ole.py", line 23, in <module>
    import vsd, xls, ppt, vba, doc
  File "/home/frankster/relab/tools/oletoy/vsd.py", line 19, in <module>
    import gtk,gsf
ImportError: No module named gsf

Getting libgsf compiled was a bit of a pain so here are some tips that may be helpful. I initially attempted to modify the ubuntu package and simply enable the configuration switch for python support. However there are some problems with the paths for python includes in the debian build scripts that I didn't get to the bottom of. So instead of building a custom ubuntu package, I uninstalled the official libraries and built my own version. I was still able to use the debian package build infrastructure to automatically fetch the source code and the build-dependencies which made things a little easier!

sudo apt-get source libgsf
sudo apt-get install python-dev
sudo apt-get build-dep libgsf
cd libgsf-1.14.21
./configure --with-python
make
sudo make install

Check that you had all required python development libraries installed during the configure stage by looking for the following lines towards the end:

checking for python... /usr/bin/python
checking for python version... 2.7
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.7/dist-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.7/dist-packages
checking for headers required to compile python extensions... found
checking for PYGTK... yes
checking for pygtk-codegen-2.0... /usr/bin/pygtk-codegen-2.0
yes

If you only saw some of these lines, then the python component won't get built!

Once you have got this far, your python script should now detect the gsf module. As your manual install of libgsf will have gone into the /usr/local/ tree, your python script may not be able to locate the library it needs to link to. If this happens you will see an error like the following:

Traceback (most recent call last):
  File "./view.py", line 23, in <module>
    import Doc, cmd
  File "/home/frankster/relab/tools/oletoy/Doc.py", line 19, in <module>
    import ole,mf,svm,cdr,clp,rx2
  File "/home/frankster/relab/tools/oletoy/ole.py", line 23, in <module>
    import vsd, xls, ppt, vba, doc
  File "/home/frankster/relab/tools/oletoy/vsd.py", line 19, in <module>
    import gtk,gsf
  File "/usr/local/lib/python2.7/dist-packages/gsf/__init__.py", line 8, in <module>
    from _gsf import *
ImportError: libgsf-1.so.114: cannot open shared object file: No such file or directory

If this happens you can solve it by telling the linker where it needs to look to find the library:

$ LD_LIBRARY_PATH=/usr/local/lib ./view.py

You can of course make this permanent in your .profile or .bashrc or whatever.

Posted in Linux | 2 Comments

Increasing the size of the main linux partition (Ubuntu 11.04)

I have an installation of Ubuntu 11.04 running as a virtual machine under VirtualBox. It recently ran out of space, so I needed to expand the partition.

First I made a new partition that was larger than my existing partition (i.e. I created a new disc image in VirtualBox).

I then added the Ubuntu 11.04 DVD image to my VM as a new DVD device. I also needed to remove the VBoxGuestAdditions.iso, as it turns out that the VirtualBox bios only lets you boot off the first CD/DVD it finds.

That done, I fired up my VM and pressed F12 to get into the boot device menu, then chose to boot from CD. When this booted up I chose the option to run the live cd rather than install Ubuntu.

Once I hit the gnome2 desktop, I fired up a terminal and ran

$ sudo gparted

gparted showed two discs - sda and sdb. sda1 was my linux partition, and there was an extended partition containing a swap partition on sda5. sdb was empty as expected. I performed the following actions:

  • select and copy the sda1 partition.
  • switch to sdb and create a partition table.
  • paste the sda1 partition into sdb, and change its size so it fills the entire disc minus 500MB or whatever you want for the swap partition.
  • create an extended partition on sdb
  • create a new swap partition inside the extended partition.
  • I set the partition flag bootable on sdb1 although I don't think this was necessary.

Having done this I now needed to install grub. To get grub to work, I had to chroot into the new partition before installing it.

$ sudo mkdir /mnt/sdb1
$ sudo mount /dev/sdb1 /mnt/sdb1 -t ext2
$ sudo mount --bind /dev /mnt/sdb1/dev
$ sudo mount --bind /proc /mnt/sdb1/proc
$ sudo mount --bind /sys /mnt/sdb1/sys
$ sudo chroot /mnt/sdb1
$ sudo grub-install
$ sudo update-grub

At this stage I shutdown, and removed the old linux partition from my VM. When I started up again, it succesfully booted into my VM, from where I am writing this article!

Posted in Linux | Leave a comment