Making a Gentoo LiveCD using Linux-Live! scripts

Preamble

For various reasons, I maintain my own name server. Over the years, the name server has died repeatedly, often because of a faulty harddrive. This is a major pita, since its such a pain to go all the way over to the server hall (on the other side of Stockholm). It's also a pain for my former collegues since someone has to stay at work in the evening to let me in to my co-lo. Finally, the pain is doubled since I need to take the machine back home to get it fixed up, then I have to return it when its fixed.

Just before Christmas 2005ce, the hard disk started behaving irregularly, dumping lots of icky lines in /var/log/messages. Thats when I decided to create my own livecd, which would (hopefully!) replace the OS which boots from the hard disk. This operating system would have just the most basic of tools, but would be a 'complete' system. In other words I didnt want to cut corners when it came to the vital tools (lsof, screen, vim, nmap, telnet, et cetera) but I really didn't want the overhead of apache, X11 and its minions, databases, java, et cetera and so forth.

I sat down for a few weeks, stealing an hour here and an evening there, and built the system. While building it, I wrote some helper scripts and stumbled upon a relatively scalable project method for building new livecds by using the base system.

What does this document cover?

This documentation will help you build just that- a base Gentoo livecd operating system. For those of you aquainted with Gentoo, its pretty much what you have after following the Gentoo documentation handbook's installation guide, plus

  1. the necessary modifications necessary to get the system to boot off a livecd, and
  2. a few shell functions and scripts which enable you to save and retrieve stuff from an usb stick.

If you follow the steps below, you'll end up with a vanilla installation of Gentoo, on which you can install... anything you want. Want your own firewall livecd? Or a webserver serving resources from a smbfs mount? Or go heavyweight and build your own alternative to Knoppix or slax? Like they say, the possibilities are endless.

Some of you might be thinking: whats the difference from Slax? Well, Slax is great- I actually started with Slax's FRODO distribution for my name server, but me being a Gentoo guy... well, I just couldn't get comfortable with the fact that someone else had compiled the software binaries. Its really a silly reason, but there you go. I also realized that four years of Gentoo had set its imprint upon me. I was just plain faster at administrating it.

Its all about choice isn't it? Thats what's great about our big happy GNU/Linux family.

Anyway...

One thing I learned from my Slax experience- whatever you do, use the Linux-Live! scripts. These scripts enable an administrator to create a iso9660 image file from an existing reference Operating System. Quick and painless. Just what I need!

The flexibility of Gentoo and the simplicity of the Linux-Live! scripts together turn the pain of creating a new livecd into a pleasure.

Finally- I don't claim in any way that I did all the research behind this livecd myself. Like someone said; "I stand upon the shoulders of giants"; most of the work has already been done by others- I've referred to the main authors below, under "References". Any mistakes, of course, are my own :)

Nuts & Bolts

Important!!! Please read the ENTIRE how-to before commensing with your installation. I mean it!

Preparation

First off, to get this to work, you'll be needing a functional GNU/Linux box, where we will create the CD operating system. It should if possible be a relatively fast machine, since we'll be doing a lot of compiling. We'll need at least four Gig free space on your harddisk. If you want to have several versions of the target OS side-by-side, you'll need more. I'll be using Gentoo linux, since this is the OS I'm most comfortable with.

Another advantage of Gentoo is that since you pretty much build it from scratch, you know exactly what is on the livecd. You won't have lots of 'other programs' which you will never use, which will fill out the CD, making seektime for files/binaries longer.

Anyway, I'll also be assuming that you've got some knowledge of GNU/Linux. Im not just calling it GNU/Linux to be nitpicky- I'll be assuming that you have some knowledge on both the GNUserland tools and the Linux kernel. Though it isn't strictly necessary to know Gentoo, it will make this howto a lot easier to understand, since I'll be using Gentoo terminology.

Some theory

Im not an expert on this, so please bear with me. What we're going to do is to:

  1. create an OS which will be compressed and saved on the CD, to be uncompressed and mounted onto the base filesystem at boot time. We'll call this the target OS.
  2. create a kernel adapted for this.
  3. modify the target OS boot runlevel scripts a bit, to reflect its livecd-ness.
  4. add services to the target OS.
  5. finally, wrap it all up into a tight little ball, and create a livecd iso file.
  6. burn it.

Terminology:

Ok, on to preparing the workspace.

Create a workspace

We'll need a place to work. This is where we'll download stuff, create the first livecd project and its successors. I decided to put my livecd project here: /home/livecd. Remember to put it somewhere where space isn't really an issue. This is because we'll be installing at least one complete gentoo system under this directory. Which means many, many (many!) files and space for compilation runtime data.

Below are a bunch of variables. They're useful in the documentation so that I dont have to refer to the directory where you downloaded resource XYZ by saying "The directory where you downloaded resource XYZ". Instead I just have to call it by its variable name.

machine # export NAME=bare
machine # export VERSION=0.1
machine # export PROJECT_BASE=/home/livecd/
machine # export DOWNLOADS=${PROJECT_BASE}/downloads
machine # export WORKSPACE=${PROJECT_BASE}/projects/version-${VERSION}
machine # export PROOT=${WORKSPACE}/root
machine #
machine # mkdir -p ${PROJECT_BASE}/{bin,downloads,notes,projects}
machine # mkdir -p ${PROJECT_BASE}/projects/version_${NAME}-${VERSION}/{dmz,root,usb}
machine # cd ${PROJECT_BASE}

A quick description of the directories in the workspace:

Directory Description
bin/ Helper scripts
notes/ all notes here. Like a file mapping version# with changes between versions.
downloads/ downloaded files here
projects/ This directory will contain the different generations of the target OS
projects/version-bare-0.1/ The target OS workspace
projects/version-bare-0.1/dmz/ When rolling your ISO, put stuff here which you don't need on the target OS (like portage, kernel sources, etc)
projects/version-bare-0.1/usb/ The usb stick fs backup here
projects/version-bare-0.1/root/The target OS root directory here

I suggest you write the variables above into your shell manually (or cut and paste) the first time, but after that, it gets a little tedious. So I've written a little script which does it for you, which you can get here. In the environment script theres a bunch of functions which we'll be using in this documentation. Also, I've added some useful navigational aliases. Download and enjoy. Oh- and by the way, when you want to use it- source the script. Don't run it. The script prints some stuff to the screen and defines a bunch of things, but otherwise it does nothing.

machine # # sourcing the script
machine # cd /some/where
machine # wget http://blahonga.yanson.org/howtos/livecd/tools/init_livecd.env.gz
machine # gunzip init_livecd.env.gz
machine # source init_livecd.env # and follow instructions.

Initial Installation

Download Linux-live scripts

I got version 5.1.8 from http://www.linux-live.org. Untar it into the target OS /tmp directory.

machine # cd ${DOWNLOADS}
machine # wget http://www.linux-live.org/dl/linux-live-5.1.8.tar.gz
machine # tar zxvf linux-live-5.1.8.tar.gz -C ${PROOT}/tmp

Download and install Gentoo base

This is quite straight forward, lets just get them. I chose the mirror over at pudas, maybe you will want to get them from somewhere closer to your current location.

machine # cd ${DOWNLOADS}
machine # wget http://mirror.pudas.net/gentoo/releases/x86/2005.1/stages/x86/stage3-x86-2005.1.tar.bz2
machine # wget http://mirror.pudas.net/gentoo/snapshots/portage-latest.tar.bz2
machine # tar xjvpf stage3-x86-2005.1.tar.bz2 -C ${PROOT}
machine # tar xvjf portage-latest.tar.bz2 -C ${PROOT}/usr

Configure make.conf

So, lets set make.conf. I decided to set the CHOST for i386, but the CFLAGS to contain the i686- how does this make sense? Well, the -mcpu flag shouldn't optimize things so hard that you cant run the compiled code on older architectures. In theory. I haven't got a 586 to play with so Im not sure if this really truly is the case. Just don't set -march to anything, since that will most certainly mess things up if you boot the CD on some older machine.

Aside from that, the most important thing, I suppose, is to compile the sources optimized for small binaries, thus the -Os. I've got a hyperthreading processor, so I specified the -j3. If you have a normal processor, go for -j2. Oh and the -pipe argument tells the compiler to use pipes instead of temporary files when sending data between individual compilations.

edit make.conf
machine # bk ${PROOT}/etc/make.conf machine # cat > ${PROOT}/etc/make.conf <<EOF CHOST="i386-pc-linux-gnu" CFLAGS="-Os -mcpu=i686 -pipe" CXXFLAGS="${CFLAGS}" MAKEOPTS="-j3" EOF

A note on the function bk. This shell function is defined, like the other shell functions mentioned in this how-to, in the init file init_livecd.env. Given a filename argument, this function will back the file up with a reasonable backup filename (includes the mtime of the file) and leave the original alone. For example:

backup function bk in action
machine # $ ls -l foobar.txt 
-rw-r--r--  1 fimblo users 269 2006-03-16 07:18 foobar.txt
machine # $ bk foobar.txt 
machine # $ ls -l foobar.txt*
-rw-r--r--  1 fimblo users 269 2006-03-16 07:18 foobar.txt
-rw-r--r--  1 fimblo users 269 2006-03-29 23:05 foobar.txt_orig_20060316
machine # $ 

Copy DNS info

We're going to want to be able to access the internet by using domain names even after chrooting, so lets copy this file to the target OS.

machine # cp -L /etc/resolv.conf ${PROOT}/etc

Chroot into the workspace

Ok, we're ready to chroot into our target operating system! Just cut and paste the following commands and you're in :-)

machine # mount -t proc none ${PROOT}/proc
machine # chroot ${PROOT} /bin/bash
machine # env-update
machine # source /etc/profile
machine # export PS1="live # "
live #
Your prompt should now look a little different which will help you remember that you've chrooted.

Note on this documentation- from now, the prompt will specify if we're in a chroot or not. If it says 'machine #' we're not in chroot, and if it says 'live #' we are.

Update your portage tree

Sync your portage tree with a rsync server.

live # emerge --sync

Redefine the bk function in the chroot

Since we chrooted, we'll need to redefine the bk function.

backup function bk defined
live # bk() { 
>   local s=${1}
>   local d=${s}_orig_$(\ls -la --time-style="+%Y%m%d" ${s} | \awk '{print $6}')
>   \cp ${s} ${d}
> }

Configure your use flags

Since the system is supposed to be as small as possible, I did a lot of removing, and just a few additions. Obviously, add and remove as much as you like- I added maildir support since I suspect I might be installing postfix in the future. Also- don't forget to add the livecd use flag.

live # bk /etc/make.conf
live # cat >> /etc/make.conf <<EOF
     # Remove these use flags
     USE="-ipv6 -cups -postgres -slang -X -alsa -acpi -apm -oss   \
             -pcmcia -qt -gtk -gtk2 -kde -gnome -samba -doc -xmms \
             -arts -foomaticdb -emboss -fortran -avi -eds -gif -gpm \
             -gstreamer -imlib -jpeg -libwww -mad -mikmod -motif \
             -mp3 -mpeg -nls -ogg -oggvorbis -opengl -pdflib -png \
             -quicktime -sdl -truetype -truetype-fonts -type1-fonts \
             -vorbis -xv -java"

     # Add these use flags
     USE="${USE} idn maildir usb livecd"
EOF

Set glibc locales

I decided to specify the glibc locales on the machine, since it would (hopefully) minimize the number of installed locales on the machine. I wasn't really sure if this would actually work, but it was worth a try. If anyone reads this who knows more on the subject, please drop me a line!

live # mkdir -p /etc/portage
live # echo "sys-libs/glibc userlocales" >> /etc/portage/package.use
live # bk /etc/locales.build
live # cat >> /etc/locales.build <<EOF
en_US/ISO-8859-1
en_US.UTF-8/UTF-8
EOF

Set the timezone

I live in Stockholm, so I set my timezone to... Stockholm.

live # cp /usr/share/zoneinfo/Europe/Stockholm /etc/localtime

Emerge and modify kernel

Besides the regular recommendations from Gentoo, you'll be needing squashfs compiled as a module. Also, save the kernel image in the boot directory as vmlinuz. Finally, write down the kernel name so we can refer to it later. In my case, I got '2.6.14-gentoo-r5' (we can't use uname -r, since we remounted proc... remember?).

live # emerge gentoo-sources
live # cd /usr/src/linux
live # make menuconfig      # see notes below
live # make
live # make modules_install
live # make install
live # cp /usr/src/linux/arch/i386/boot/bzImage /boot/vmlinuz
live # cp .config /boot/vmlinuz_config

Notes on compiling a kernel:

Another note: If you don't know the version of the kernel you've got, take a look at the top of the Makefile in /usr/src/linux. It will look something like this:

live # head -n 5 /usr/src/linux/Makefile 
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 14
EXTRAVERSION = -gentoo-r5
NAME=Affluent Albatross

This shows that the version of the kernel I have is 2.6.14-gentoo-r5. It's project name, curiously, is Affluent Albatross :)

Now that we have a kernel, we'll need to emerge unionfs to get the unionfs module. Note: I know that we're not supposed to use the ACCEPT_KEYWORDS envvar, but I couldnt be bothered to look up how to use the /etc/portage/packages.keywords properly...

live # ACCEPT_KEYWORDS="~x86" emerge unionfs

Copying kernel modules to livecd bootstrap fs

When we boot the CD, we want the kernel (which resides in the boot filesystem) to be able to access the squashfs and unionfs modules. So lets copy them to the linux-live directory so it can be bundled by the scripts later on. Remember I told you to write down your kernel name previously? Well, we'll be using it now. Oh, and we need to gzip them as well.

live # LINUX_KERNEL="2.6.14-gentoo-r5" # use your kernel name!
live # LIVE_LIBDIR="/tmp/linux-live-5.1.8/initrd/kernel-modules/${LINUX_KERNEL}"
live # mkdir -p ${LIVE_LIBDIR}
live # for module in squashfs.ko unionfs.ko ; do
>   cp $(find /lib/modules/${LINUX_KERNEL} -name ${module}) ${LIVE_LIBDIR}
>   gzip ${LIVE_LIBDIR}/${module}
> done

File system table

At boot time, the linux-live scripts add all detected disks (hd*) to fstab. So you only need to have one line in fstab; namely the proc filesystem. I also added usbfs, since it will be useful for the testing phase later on.

live # bk /etc/fstab
live # cat > /etc/fstab <<EOF
proc   /proc           proc    defaults   0 0
none   /proc/bus/usb   usbfs   defaults   0 0
EOF

Adapting base install to our needs

Adapting boot runlevel configs (regular stuff)

We'll also want to do the usual configuring of the configuration files. My machine is called helmer, and belongs to the domain yanson.org, so here goes:

live # cd /etc/conf.d

live # bk hostname
live # echo HOSTNAME=\"helmer\" > hostname

live # bk domainname
live # cat > domainname <<EOF
OVERRIDE=1
DNSDOMAIN="yanson.org"
EOF
live # rc-update add domainname default

And now for network configuration. At home I use the a class A address range defined in RFC 1918. The real destination of the target OS is a server hall where I use a real (public) internet protocol address. Therefore I decided to set my public ip on interface eth0, and my private one on an aliased interface eth0:1.

I set the default gateway to my gateway ip in the network where I will place the nameserver. Why I won't set the gateway for my home with some smart script? Well, when I start the livecd in a machine from home, it will be for testing, and so I won't need to send packages beyond the scope of my LAN. So no need to set a backup default gateway with some nifty script.

live # bk /etc/conf.d/net
live # cat > /etc/conf.d/net <<EOF
config_eth0=( 
  "194.68.48.117/24 " 
  "10.0.0.30/8" 
)
routes_eth0=( "default gw 194.68.48.4" )
EOF
live # rc-update add net.eth0 default

live # bk /etc/hosts
live # cat > /etc/hosts <<EOF
127.0.0.1       localhost helmer helmer.yanson.org
EOF

Users

K, now let's add users and set passwords. First set the root password. I normally don't let root log in via ssh, so lets add another user, as well as a password for her.

Don't forget to add your user to the group 'wheel', so that you can access root via the command 'sudo'.

live # # Set root password
live # passwd 

live # # add regular user
live # useradd -g users -G wheel -d /home/USERNAME -m USERNAME
live # passwd USERNAME

Serial port login

Since my machine won't have a monitor, I want to be able to log in without a) having a spare monitor in my back pocket, or b) turning off the machine and moving it to the closest monitor.

Most modern computers still have serial ports (ahem. my main laptop doesn't...). You might want to activate the possibility of plugging your laptop to the machine using a serial cable. With my last laptop, and my last job, this was the definitive way of accessing a Cisco router if you didn't or couldn't connect via network.

live # bk /etc/securetty
live # echo "tts/0" >> /etc/securetty

Just remember to emerge a simple serial port terminal program onto your laptop. I used to use minicom, but maybe there are better apps out there nowadays...

Edit rc.conf and its friends

Now you'll want to configure some system settings in the files below. In my case I edited rc.conf, clock, and keymaps. But depending on your preferences you might want to play with keymaps as well.

I added /etc/conf.d/rc here, even if I didnt fiddle with it (this time). Im thinking of the variable RC_FORCE_AUTO, which is set to 'no' by default. Im guessing that the boot-up process might go more smoothely if its set to yes. But again- I didnt fiddle with that file.

Install system tools

K, lets just zip through this, Im going to install some system utilities.

live # emerge syslog-ng logrotate vixie-cron at ntp 
live # rc-update add syslog-ng default
live # rc-update add vixie-cron default
live # rc-update add at default
live # rc-update add ntp-client default
live # rc-update add ntpd default

live # emerge coldplug hotplug
live # rc-update add coldplug boot
live # rc-update add hotplug boot  # see note(1) below

live # emerge pciutils slocate app-admin/sudo
live # emerge cdrtools             # see note(2) below

live # # woah! lets not forget adding sshd to default runlevel!
live # rc-update add sshd default
1) not strictly necessary, since it does nothing. I just get nervous without it in my runlevels..
2) we need cdrtools for the fine mkisofs utility.

Configure some services

sshd

I normally tighten sshd's security settings a bit.

live # cat > /tmp/mydiff <<EOF
37c37
< #PermitRootLogin yes
---
> PermitRootLogin no
105a106,107
> AllowUsers myNonRootUser
> 
#
EOF
live # patch /etc/sshd/sshd_config /tmp/mydiff
Replace the string "myNonRootUser" with your the user you created previously.

Also note that you should only run protocol 2, protocol 1 is proven breakable. This should be set by default nowadays:

live # grep Protocol /etc/sshd/sshd_config
Protocol 2

syslog-ng

I used the default values of syslog-ng, but you might want to consider sending all log events to another machine, which has a physical harddrive to save the logs in.

logrotate.conf

Since I didn't use networking as an alternative in syslog-ng, Im facing a potential full ramdisk if the logs get too full. This, in tech-lingo, is called a "bad thing". So I edited the logrotate conf so that it cycles the logs every two weeks instead of the default of four weeks.

live # bk /etc/logrotate.conf
live # cat > /etc/logrotate.conf<<EOF
weekly
rotate 2
create
compress
include /etc/logrotate.d
notifempty
nomail
noolddir
/var/log/wtmp {
    weekly
    create 0664 root utmp
    rotate 2
}
EOF

As an afterthought, I realize that it would be wiser to tell logrotate to rotate logs when a file has reached a specific size. This way we won't be caught with our pants down half-way through a rotation period with very large logs and a full filesystem...

ntp configs

Since this machine will contain a server of some sort (at least for me) I need reliable timestamps on the logs. And I want ssh to act sanely. So we'll be needing the ntp services.

There are two files to mess with here: /etc/ntp.conf and /etc/conf.d/ntp-client. Hmm. just fyi, ntp-client sets the date at boot time, once- even if your computer's local time is completely off the mark. ntpd in turn sees to that the time on your system doesn't drift, but if the local time is very different from the upstream servers, it wont start. So we need both.

Config file for ntp-client
live # bk /etc/conf.d/ntp-client
live # cat > /etc/conf.d/ntp-client <<EOF
NTPCLIENT_CMD="ntpdate"
NTPCLIENT_OPTS=" -b -u your.favourite.ntp.server"
NTPCLIENT_TIMEOUT=30
EOF

Config file for ntpd
live # bk /etc/ntp.conf
live # cat > /etc/ntp.conf <<EOF
server your.favourite.ntp.server prefer
server pool.ntp.org
driftfile /var/lib/ntp/ntp.drift
restrict default nomodify nopeer
restrict 127.0.0.1
EOF

Note: remember that if you use dhcp to get an address, that dhcp will wipe your ntp.conf file unless you tell it not to in its configuration file. (I think it was the -N flag). Check the man page or the gentoo wiki for details.

Prettify /etc/issue

For some reason I cant be bothered to figure out, some boot script finds out that we're running on a CD and replaces the /etc/issue file with some warnings. This error message is harmless since Im running on a livecd, but it doesn't look nice, so I added some code to /etc/conf.d/local.start which replaces the contents of /etc/issue

live # cat > /etc/conf.d/local.start <<EOF
# Override the warning text
cat > /etc/issue <<END_OF_MESSAGE
This is my livecd server.
This is \n.\O (\s \m \r) \t
END_OF_MESSAGE
EOF

Securing things a bit more

Ok, so now lets secure the server a wee bit more. With our current configuration, if some malicious script-kiddie manages to get shell access on our box via, say a httpd or ntp external exploit, they can try to take a shot at guessing the root password using su. If they manage, they can run /bin/su - to get root access. Ouch.

Enter sudo. If we let our regular user use sudo to access a root shell, and remove the setuid bit on /bin/su, we'll remove the risk of the above happening. This since we set tight rules on who can sudo.

live # # Add your regular user to group wheel
live # groups=$(echo "$(groups) wheel" | tr ' ' "\n" | sort | uniq )
live # usermod -G $(echo ${groups} | tr ' ' ',') username
live # 
live # # Make sure that /etc/sudoers allows wheel to sudo
live # cd /etc
live # chmod 0770 sudoers
live # bk sudoers
live # cat > sudoers <<END_OF_MESSAGE
live # Defaults env_reset
live # root     ALL=(ALL) ALL
live # %wheel   ALL=(ALL) ALL
live # EOF
live # chmod 0440 sudoers
live # 
live # # Remove suid bit from /bin/su
live # chmod u-s /bin/su
Replace 'username' with your regular user's username.

Persistent data on USB stick

Why a USB stick?

Now, assuming that we fix all the problems and burn the CD, boot from it and it works, how to we make changes to the CD operating system persistent once we're running it live? After all, rebooting will wipe changes right?

There's lots of ways of doing this- via network, floppy, hard disk, etc. I went for the USB stick. Why?

Finally, what made me decide in favour of the USB stick is that for the amount of data you can store its very cheap. You can also buy it in almost any store nowadays, so its easy to replace.

Persistancy tools

What we'll need now is a set of functions which will enable us to easily copy changed files to the usb-stick. Likewise, we'll need tools which will let us copy the changed files back to the original location at boottime. We'll need a place to put the functions, and all the edited files as well. Hmm, what else do we need? Perhaps some defaults, if the USB stick isn't inserted at boot time. And finally, to make this a painless process, and an editable one as well, we'll stick all the functions and variables onto the usb-stick itself.

Create a mount point, fill it with defaults
live # mkdir /home/usb
live # cd /tmp
live # wget http://blahonga.yanson.org/howtos/livecd/tools/usb_default.tar.gz
live # tar zxvf usb_default.tar.gz
live # mv usb_default/* /home/usb
live # rm -r usb_default

What did we just fill the directory /home/usb with? Theres three directories in the tarball:

Obviously, if the USB stick hasn't been inserted theres no point in saving stuff in the directory saved_states, but I decided to create this directory anyway so that the bootup of the system goes smoothly.

Now its time to format the usb stick. Insert your USB stick into your build machine. I'll just assume that the stick ends up at /dev/sda1. Yours might end up someplace else- in that case run dmesg to see where it is.

Run dmesg to make sure that the stick has settled. On my machine, looks like this (truncated output):

dmesg output
live # dmesg
------- snip -------
scsi0 : SCSI emulation for USB Mass Storage devices
usbcore: registered new driver usb-storage
USB Mass Storage support registered.
usb-storage: device found at 2
usb-storage: waiting for device to settle before scanning
------- snip -------
EXT3-fs: mounted filesystem with ordered data mode.
  Vendor: SanDisk   Model: Cruzer Micro      Rev: 0.2 
  Type:   Direct-Access                      ANSI SCSI revision: 02
SCSI device sda: 250879 512-byte hdwr sectors (128 MB)
sda: Write Protect is off
sda: Mode Sense: 03 00 00 00
sda: assuming drive cache: write through
SCSI device sda: 250879 512-byte hdwr sectors (128 MB)
sda: Write Protect is off
sda: Mode Sense: 03 00 00 00
sda: assuming drive cache: write through
 sda: sda1
Attached scsi removable disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0,  type 0
usb-storage: device scan complete
------- snip -------

Once the stick has settled, we can create a partition on it. Fire up cfdisk (or fdisk, or if you're a masochist, sfdisk) and create a partition on the usb disk. My disk looks like this:

cfdisk /dev/sda
                                  cfdisk 2.12i

                              Disk Drive: /dev/sda
                         Size: 128450048 bytes, 128 MB
               Heads: 8   Sectors per Track: 32   Cylinders: 979

    Name        Flags      Part Type  FS Type          [Label]        Size (MB)
 ------------------------------------------------------------------------------
sda1 Primary Linux ext3 [usb_save] 128.32
[Bootable] [ Delete ] [ Help ] [Maximize] [ Print ] [ Quit ] [ Type ] [ Units ] [ Write ]

As you can see I have one partition on my usb stick, and its some 128M big. Ok, on to formatting the partition, and filling it with data. I chose the ext3 filesystem for its journalling capabilities.

Format, create mount point, and add data.
live # mke2fs -j /dev/sda1
live # mount /dev/sda1 /home/usb
live # cd /tmp
live # wget http://blahonga.yanson.org/howtos/livecd/tools/usb_stick.tar.gz
live # tar zxvf usb_stick.tar.gz
live # mv usb_stick/* /home/usb
live # rm -r usb_stick

Btw, if you're wondering why I placed the mount point in /home as opposed to /mnt, its because the linux-live scripts rolls the contents of /home into the livecd, but not the contents of /mnt. I could have edited the linux-live scripts, but then I'd have to maintain that for future versions... I can't be bothered.

cd over to /home/usb and take a look around. There are three important files:

Filename Description
/home/usb/etc/usb.env Environment script. Holds important variables.
/home/usb/etc/usb.func Shell functions for saving and restoring files to/from the usb stick.
/home/usb/etc/usb.bootscript Script to run at boot time.

So, to summarize, there are two copies of each script- one inside the unmounted directory /home/usb and the other in the usb stick, mounted onto the same. If you add new scripts or modify the behaviour of the current script-set, remember to add it to the CD if you need it to work even if the USB stick is not inserted.

Sourcing the usb functions at login

You'll want to source the functions when you log in as root so that you can easily add and remove stuff from the usb-stick.

Editing the .bashrc file
live # cat >> /root/.bashrc < EOF

# Source livecd functions
source /home/usb/etc/usb.env
source /home/usb/etc/usb.func

echo
echo "Enter <help_livecd> for help on livecd functions."
EOF
live # 

In the next section we'll be looking (among other things) at how these scripts are called.

Boot/shutdown details

Adding usb stick runlevel script

The scripts we added in the previous section are nice but completely worthless if we don't call them at boot time. So lets add one boot runlevel script and two configs for boot and shutdown to our target OS.

Filename Description
/etc/init.d/usb_stick Start/stop script
/etc/conf.d/usb_stick.start What usb_stick script should do at boot
/etc/conf.d/usb_stick.stop What it should do at shutdown

So lets download the scripts and put them into place:

Getting and installing usb boot scripts
live # cd /tmp
live # wget http://blahonga.yanson.org/howtos/livecd/tools/bootscripts.tar.gz
live # tar zxvf bootscripts.tar.gz -C /
live # rc-update add usb_stick boot

Adapting the existing boot runlevel scripts for livecd

Since we'll be booting off a CD, we really don't want the OS to do root filesystem checking and such during bootstrap. So now we're going to remove this part of the bootup process.

live # rc-update del checkroot boot
live # rc-update del checkfs boot
live # 
live # bk /etc/init.d/checkfs
live # bk /etc/init.d/checkroot
live # rm /etc/init.d/{checkfs,checkroot}
live # 
live # # remove checkroot and checkfs from DEPEND block
live # # empty DEPEND blocks are ok.
live # cd /etc
live # list=$(grep -rlE 'checkroot|checkfs' *)
live # for n in ${list} ; do
> bk ${n}
> done
live # vi ${list}
live # 
live # # remove checkroot and checkfs from CRITICAL_SERVICES (around line 128)
live # bk /sbin/rc
live # vi /sbin/rc

Finally, we'll need to fix some boot runlevel script symlinks. You might not have the same problem I had, but I noticed that the symbolic links didnt point at the correct place in the filesystem, that is, /etc/init.d/. If you run into the same problem, run this code:

live # cd /etc/runlevels/boot
live # for n in * ;  do
> rc-update del ${n} boot
> rc-update add ${n} boot
> done

The halting problem (hey Mr Turing!)

Sorry to let you down folks, this hasn't got anything whatsoever to do with Alan Turing's Halting Problem. Aw, shucks...

Anyway, the problem we have now is that if we create the iso, burn the CD and boot off it, we'll notice that most things work quite well (hopefully!). The thing is, the moment you try to turn off the computer with any of the usual commands halt, shutdown, reboot, and poweroff, you'll notice something. The machine will happily start turning off services, and at the very end, INIT goes veggie- it writes one last thing to tty: INIT: no more processes left in this runlevel. Afterwards it just sits there in contentment, forever.

Needless to say this was a major pita, especially for me since my livecd will be running in a colo, where I can't rush over and push the 'off' or 'reset' button, but I fixed it like this:

  1. Replace the original file /etc/init.d/halt.sh with this:

    live # bk /etc/init.d/halt.sh
    live # cat > /etc/init.d/halt.sh <<EOF
    #!/bin/sh
    
    echo "Starting (modified) Slax live-cd halt procedure."
    /etc/rc.d/rc.6
    EOF
    

    Note that the script above just runs another script, namely /etc/rc.d/rc.6, which is a modified version of the Slax livecd halt script.

  2. Now lets create the script which the above script pointed at. Its called /etc/rc.d/rc.6. Note: I kept the file location and filename of the original from the slax distro so that I get forcibly reminded that this is an alien part of my otherwise pure gentoo OS. A pretty crude method to be sure, but it works for me :-).

    live # mkdir /etc/rc.d
    live # cd /etc/rc.d
    live # wget http://blahonga.yanson.org/howtos/livecd/tools/rc.6.gz
    live # gunzip rc.6.gz
    live # chmod 755 rc.6
    

Why all this hassle? Well, initially I just went with the default Gentoo /etc/init.d/halt.sh, which I thought should do to job. Assuming that you add cdroot to the kernel command at boot time, the script should take you to a poweroff (or reboot, depending on your command).

But the thing is, since I decided to use the linux-live scripts, the system is kick-started using a slax base filesystem (which in turn is based on slackware). Which means that all the apps (above all, busybox) are located in slightly different places, resulting in broken ${PATH}s. Also, there seem to be general inconsistencies between the slax and gentoo way of booting (and thus shutting down) a livecd.

As a consequence of all this, I just couldn't restart the stupid computer. After lots of websearching, testing on my part, and unrebootable compact discs, this was the simplest result I could get. Ugly, but it works.

isolinux woes

Ok, now I can get past INIT, but I noticed that when the reins get passed to BIOS, that it somehow couldn't start up the system. The kernel never gets loaded. As a matter of fact, it seemed to have hung at the bootloader. *grumble*.

Everything pointed at the isolinux bootloader. Isolinux is a mini bootloader adapted for iso9660 filesystems- a small replacement for grub or lilo (or silo!). Go look for it- some of you will find that you have two copies of isolinux in your linux-live scripts directory. This confused me for some time but I finally took the time to read the scripts around them, and I saw that the one ending with bi_ was a backup of isolinux.bin, necessary for restoring isolinux.bin after making a CD (it gets changed).

Anyway, my current situation was that I could not reboot. I searched the web a bit, and found a solution- replace isolinux.bin with a newer version.

I got mine from kernel.org, but discovered afterwards that it was in portage. *bonk on head* Always check portage before going out and downloading yourself :)

machine # # please note! OUTSIDE OF CHROOT
machine # emerge syslinux
machine # cp /usr/lib/syslinux/isolinux.bin ${PROOT}/tmp/linux-live-5.1.8/cd-root/boot/isolinux.bin
machine # cp /usr/lib/syslinux/isolinux.bin ${PROOT}/tmp/linux-live-5.1.8/cd-root/boot/isolinux.bi_

I did the emerge outside of my chroot since I don't want all the clutter of the accompanying files on the target OS.

Assembling the livecd

Alright! Assuming that you followed all my instructions and that I remembered all the steps, you should be ready to assemble the livecd!

I've written a short shell function which does some basic checking, and if the tests succeed, it moves the larger unnecessary directories to our dmz directory. The 'unnecessary' directories are the kernel source directory, and the portage directory and db. These aren't needed at runtime. The function is included in the init_livecd.env script specified at the beginning of this document. The bash function is called prepare_for_linuxlive_scripts. Oh, and btw- this script should be called from outside the chroot.

machine # prepare_for_linuxlive_scripts 
Checking if linux-live scripts are installed...ok.
Checking for mkisofs...ok.
Checking if livecd USE flag is in make.conf...ok, found it.
Are all references of checkroot and checkfs removed from init.d and rc? yes.
Checking if checkroot and checkfs are removed from boot runlevel...ok.
Checking if kernel is in boot partition...ok.
Kernel version: 2.6.14-gentoo-r5
Checking kernel module squashfs.ko.gz... ok, found it.
Checking kernel module unionfs.ko.gz... ok, found it.
All ok.

Setting name and version number into file /etc/livecd_version
Cleaning up unnecessary files
Moving kernel and portage to dmz...
To move them back, use the dmz2root function.
machine # 

Once you get the ok from the script, you should be ready to roll your first iso!

Make your iso
live #  cd /tmp/linux-live-5.1.8
live #  ./runme.sh 
Changing current directory to /tmp/linux-live-5.1.8
Linux Live scripts were installed successfuly in /
Creating LiveCD from your Linux
some debug information can be found in /tmp/linux-live-debug.log
copying cd-root to /tmp/live_data_8259, using kernel from /boot/vmlinuz
Using kernel modules from /lib/modules/2.6.14-gentoo-r5
creating initrd image...
creating compressed images...
base/bin.mo
base/etc.mo
base/home.mo
base/lib.mo
base/opt.mo
base/root.mo
base/usr.mo
base/sbin.mo
base/var.mo
creating LiveCD ISO image...
mkisofs 2.01 (i686-pc-linux-gnu)
Scanning .
Scanning ./base
Scanning ./boot
Scanning ./boot/DOS
Scanning ./tools
Scanning ./tools/DOS
Scanning ./modules
Scanning ./rootcopy
Scanning ./optional
Writing:   Initial Padblock                        Start Block 0
Done with: Initial Padblock                        Block(s)    16
Writing:   Primary Volume Descriptor               Start Block 16
Done with: Primary Volume Descriptor               Block(s)    1
Writing:   Eltorito Volume Descriptor              Start Block 17
Size of boot image is 4 sectors -> No emulation
Done with: Eltorito Volume Descriptor              Block(s)    1
Writing:   Joliet Volume Descriptor                Start Block 18
Done with: Joliet Volume Descriptor                Block(s)    1
Writing:   End Volume Descriptor                   Start Block 19
Done with: End Volume Descriptor                   Block(s)    1
Writing:   Version block                           Start Block 20
Done with: Version block                           Block(s)    1
Writing:   Path table                              Start Block 21
Done with: Path table                              Block(s)    4
Writing:   Joliet path table                       Start Block 25
Done with: Joliet path table                       Block(s)    4
Writing:   Directory tree                          Start Block 29
Done with: Directory tree                          Block(s)    10
Writing:   Joliet directory tree                   Start Block 39
Done with: Joliet directory tree                   Block(s)    9
Writing:   Directory tree cleanup                  Start Block 48
Done with: Directory tree cleanup                  Block(s)    0
Writing:   Extension record                        Start Block 48
Done with: Extension record                        Block(s)    1
Writing:   The File(s)                             Start Block 49
  7.89% done, estimate finish Mon Mar 13 10:50:36 2006
 15.79% done, estimate finish Mon Mar 13 10:50:42 2006
 23.66% done, estimate finish Mon Mar 13 10:50:40 2006
 31.56% done, estimate finish Mon Mar 13 10:50:39 2006
 39.43% done, estimate finish Mon Mar 13 10:50:38 2006
 47.33% done, estimate finish Mon Mar 13 10:50:38 2006
 55.21% done, estimate finish Mon Mar 13 10:50:37 2006
 63.10% done, estimate finish Mon Mar 13 10:50:37 2006
 70.98% done, estimate finish Mon Mar 13 10:50:37 2006
 78.88% done, estimate finish Mon Mar 13 10:50:38 2006
 86.75% done, estimate finish Mon Mar 13 10:50:38 2006
 94.65% done, estimate finish Mon Mar 13 10:50:38 2006
Total translation table size: 2048
Total rockridge attributes bytes: 5590
Total directory bytes: 16384
Path table size(bytes): 120
Done with: The File(s)                             Block(s)    63207
Writing:   Ending Padblock                         Start Block 63256
Done with: Ending Padblock                         Block(s)    150
Max brk space used 2e000
63406 extents written (123 MB)
Your ISO is created in /tmp/livecd.iso
live #

As you can see from the log, the image is 123M big (the original, uncompressed filesystem is 995M excluding /proc. FYI, if you don't remove the kernel and portage tree, the total size is 290M. It might not sound like much but when unpacked, 290M is approximately 1.8G.

The entire operation took under three minutes. Hats off to the linux-live script hackers!

Testing the livecd

CDs are cheap, but in the process of building this livecd I would have used some forty CDs if it hadn't been for qemu, and later on vmplayer. Both are x86 emulators, qemu being a F/OSS project, and vmplayer being a free but proprietary emulator. Vmplayer is quite a bit faster than qemu, but qemu is simpler to use if you like command line interfaces and a no-nonsense interface. I'll leave the decision to you.

Start the livecd in your favourite virtual environment, and test away. Remember that the USB support on the target OS is limited, since the usb devices are taken by the host system. To test this you'll just have to burn an actual CD and boot off it.

I use qemu to test all basic stuff, like if it boots, reboots, if all the basic services have started, if I've spelled something wrong in some script, etc and so forth. If something doesn't work as you think it should, just edit the target OS in the chrooted environment. I normally had the chrooted environment in one window, and unchrooted one in another, and qemu in a third window. My normal testing pattern:

  1. run prepare_for_linuxlive_scripts to see if all is ok.
  2. make the iso (in chroot)
  3. boot the iso in qemu
  4. test, and discover a bug
  5. run the shell function dmz2root to restore kernel and portage to target OS
  6. edit target OS (in chroot)
  7. increment the version number in the file init_livecd.env
  8. back to step one.

Once the system seems sane in the virtual environment, you can burn a real, physical CD off the iso.

By the way, the version number can be found on the target OS in the file /etc/livecd_version. Nice to have if you're like me and you can't be bothered to mark the CDs you burn... and you eventually have a pile of unmarked CDs on your physical desktop and you're wondering which one was the CD you burned last...

Anyway, things you might want to test:

Lastly...

Updating the slocate database

Before you create your final iso of the livecd, remember to update the slocate database. What I did was boot off the penultimate iso in vmplayer (not qemu, since I couldn't be bothered to fix the network connection in it... vmplayer was easier) , and run /etc/cron.daily/slocate. Then I took the slocate.db file and scp'd it to the same directory in my target OS.

Removing unnecessary files

The shell function prepare_for_linuxlive_scripts will, when called, move the kernel and portage directories out of the target OS directory space. You might want to look around to see if there are any other files (or entire directories) you want to move out of the target OS. Size matters here, since the smaller the iso, the faster unionfs/squashfs can seek and find files in the compressed file system. I seriously considered moving all the man pages out, as well as the documentation in /usr/share. In the end I didn't do this, since I felt that the trade-off wasn't worth it. When I want documentation, I want it now. I don't want to have to log into another machine, or search the web for it.

At this point, you should have a complete install of a gentoo-ized livecd, which can be used as a base for further fiddling.

Final notes

Ok, so now you have a livecd. Now what?

I wanted a livecd based name server, so I proceeded to copy the entire project tree to a new directory, gave it a new project name and reset the version number to 0 in init_livecd.env.

Inside this new target OS, I removed ssmtp, installed postfix, a chrooted named, added big brother for monitoring, and added some simple iptable rules. I also removed all the backup files I created during the creation of the first base livecd (the ones created by my shell function bk).

I also made logrotate rotate the logs more frequently than the default, and it emails me the logs instead of deleting them. I wrote a hack to monitor all logins into the system, and added a simple rootkit checker just to be on the safe side.

Afterwards, I did some testing and before long it was done. Today, its happily running at my colo. Nice!

Finally- if you find any errors in this documentation, or if you have some refinements- dont hesitate to email me at m at yanson dot org. Make sure you add the string "livecd howto" in the subject line somewhere so my filters can put it in the right directory.

If you have any questions, please check the various forums first before emailing me. The gentoo forums, the gentoo wiki and the linux-live forums are all excellent resources. If you're really stuck, you can email me. Just don't expect a prompt answer, and if the question is very newbiesque (i.e. 'What is a filesystem?' or 'How do I burn a CD?'), I'll probably just send a link to tldp.

References

About me

My name is Mattias Jansson. I'm half-Japanese/half-Swedish, and live in Stockholm. My days are spent in disguise at a large Swedish bank where I administrate an interest rate derivative system, which runs on Solaris and Wintendo machines. In the evenings I throw off my disguise and play with GNU/Linux.

I've taught internet routing protocols at KTH (Royal Institute of Technology, Stockholm) in a previous life, and enjoy talking network protocols and operating systems the way ordinary mortals like playing FPSs or MMORPGs.

My favourite programming languages are currently perl and the bourne shell descendents (bash and ksh primarily), but I'd say I'm quite proficient at building multithreaded, networked java server applications. I like emacs-lisp, tho I never have time to learn it properly. I write C hacks when I have to. I avoid C++ at all costs, unless its a part of my job (its a part of my job).

My favourite non-computer interests are post-revolutionary European History and thus by implication political science. I like dabbling in metaphysics, with a special focus on the mind-body problem.

Link to my blog: http://blahonga.yanson.org.