For various reasons, I maintain my own name server. Over the years, the name server has died repeatedly, often because of a faulty harddrive. This is a major pita, since its such a pain to go all the way over to the server hall (on the other side of Stockholm). It's also a pain for my former collegues since someone has to stay at work in the evening to let me in to my co-lo. Finally, the pain is doubled since I need to take the machine back home to get it fixed up, then I have to return it when its fixed.
Just before Christmas 2005ce, the hard disk started
behaving irregularly, dumping lots of icky lines in
/var/log/messages. Thats when I decided to create
my own livecd, which would (hopefully!) replace the OS which
boots from the hard disk. This operating system would have just
the most basic of tools, but would be a 'complete' system. In
other words I didnt want to cut corners when it came to the
vital tools (lsof, screen, vim, nmap, telnet, et cetera) but I
really didn't want the overhead of apache, X11 and its
minions, databases, java, et cetera and so forth.
I sat down for a few weeks, stealing an hour here and an evening there, and built the system. While building it, I wrote some helper scripts and stumbled upon a relatively scalable project method for building new livecds by using the base system.
This documentation will help you build just that- a base Gentoo livecd operating system. For those of you aquainted with Gentoo, its pretty much what you have after following the Gentoo documentation handbook's installation guide, plus
If you follow the steps below, you'll end up with a vanilla installation of Gentoo, on which you can install... anything you want. Want your own firewall livecd? Or a webserver serving resources from a smbfs mount? Or go heavyweight and build your own alternative to Knoppix or slax? Like they say, the possibilities are endless.
Some of you might be thinking: whats the difference from Slax? Well, Slax is great- I actually started with Slax's FRODO distribution for my name server, but me being a Gentoo guy... well, I just couldn't get comfortable with the fact that someone else had compiled the software binaries. Its really a silly reason, but there you go. I also realized that four years of Gentoo had set its imprint upon me. I was just plain faster at administrating it.
Its all about choice isn't it? Thats what's great about our big happy GNU/Linux family.
Anyway...
One thing I learned from my Slax experience- whatever you do, use the Linux-Live! scripts. These scripts enable an administrator to create a iso9660 image file from an existing reference Operating System. Quick and painless. Just what I need!
The flexibility of Gentoo and the simplicity of the Linux-Live! scripts together turn the pain of creating a new livecd into a pleasure.
Finally- I don't claim in any way that I did all the research behind this livecd myself. Like someone said; "I stand upon the shoulders of giants"; most of the work has already been done by others- I've referred to the main authors below, under "References". Any mistakes, of course, are my own :)
Important!!! Please read the ENTIRE how-to before commensing with your installation. I mean it!
First off, to get this to work, you'll be needing a functional GNU/Linux box, where we will create the CD operating system. It should if possible be a relatively fast machine, since we'll be doing a lot of compiling. We'll need at least four Gig free space on your harddisk. If you want to have several versions of the target OS side-by-side, you'll need more. I'll be using Gentoo linux, since this is the OS I'm most comfortable with.
Another advantage of Gentoo is that since you pretty much build it from scratch, you know exactly what is on the livecd. You won't have lots of 'other programs' which you will never use, which will fill out the CD, making seektime for files/binaries longer.
Anyway, I'll also be assuming that you've got some knowledge of GNU/Linux. Im not just calling it GNU/Linux to be nitpicky- I'll be assuming that you have some knowledge on both the GNUserland tools and the Linux kernel. Though it isn't strictly necessary to know Gentoo, it will make this howto a lot easier to understand, since I'll be using Gentoo terminology.
Im not an expert on this, so please bear with me. What we're going to do is to:
Terminology:
Ok, on to preparing the workspace.
We'll need a place to work. This is where we'll download
stuff, create the first livecd project and its successors. I
decided to put my livecd project here:
/home/livecd. Remember to put it somewhere where
space isn't really an issue. This is because we'll be
installing at least one complete gentoo system under this
directory. Which means many, many (many!) files and space for
compilation runtime data.
Below are a bunch of variables. They're useful in the documentation so that I dont have to refer to the directory where you downloaded resource XYZ by saying "The directory where you downloaded resource XYZ". Instead I just have to call it by its variable name.
machine # export NAME=bare
machine # export VERSION=0.1
machine # export PROJECT_BASE=/home/livecd/
machine # export DOWNLOADS=${PROJECT_BASE}/downloads
machine # export WORKSPACE=${PROJECT_BASE}/projects/version-${VERSION}
machine # export PROOT=${WORKSPACE}/root
machine #
machine # mkdir -p ${PROJECT_BASE}/{bin,downloads,notes,projects}
machine # mkdir -p ${PROJECT_BASE}/projects/version_${NAME}-${VERSION}/{dmz,root,usb}
machine # cd ${PROJECT_BASE}
A quick description of the directories in the workspace:
| Directory | Description |
bin/ | Helper scripts |
notes/ | all notes here. Like a file mapping version# with changes between versions. |
downloads/ | downloaded files here |
projects/ | This directory will contain the different generations of the target OS |
projects/version-bare-0.1/ | The target OS workspace |
projects/version-bare-0.1/dmz/ | When rolling your ISO, put stuff here which you don't need on the target OS (like portage, kernel sources, etc) |
projects/version-bare-0.1/usb/ | The usb stick fs backup here |
projects/version-bare-0.1/root/ | The target OS root directory here |
I suggest you write the variables above into your shell manually (or cut and paste) the first time, but after that, it gets a little tedious. So I've written a little script which does it for you, which you can get here. In the environment script theres a bunch of functions which we'll be using in this documentation. Also, I've added some useful navigational aliases. Download and enjoy. Oh- and by the way, when you want to use it- source the script. Don't run it. The script prints some stuff to the screen and defines a bunch of things, but otherwise it does nothing.
machine # # sourcing the script machine # cd /some/where machine # wget http://blahonga.yanson.org/howtos/livecd/tools/init_livecd.env.gz machine # gunzip init_livecd.env.gz machine # source init_livecd.env # and follow instructions.
I got version 5.1.8 from http://www.linux-live.org. Untar it into the target OS /tmp directory.
machine # cd ${DOWNLOADS}
machine # wget http://www.linux-live.org/dl/linux-live-5.1.8.tar.gz
machine # tar zxvf linux-live-5.1.8.tar.gz -C ${PROOT}/tmp
This is quite straight forward, lets just get them. I chose the mirror over at pudas, maybe you will want to get them from somewhere closer to your current location.
machine # cd ${DOWNLOADS}
machine # wget http://mirror.pudas.net/gentoo/releases/x86/2005.1/stages/x86/stage3-x86-2005.1.tar.bz2
machine # wget http://mirror.pudas.net/gentoo/snapshots/portage-latest.tar.bz2
machine # tar xjvpf stage3-x86-2005.1.tar.bz2 -C ${PROOT}
machine # tar xvjf portage-latest.tar.bz2 -C ${PROOT}/usr
So, lets set make.conf. I decided to set the CHOST for i386, but the CFLAGS to contain the i686- how does this make sense? Well, the -mcpu flag shouldn't optimize things so hard that you cant run the compiled code on older architectures. In theory. I haven't got a 586 to play with so Im not sure if this really truly is the case. Just don't set -march to anything, since that will most certainly mess things up if you boot the CD on some older machine.
Aside from that, the most important thing, I suppose, is to compile the sources optimized for small binaries, thus the -Os. I've got a hyperthreading processor, so I specified the -j3. If you have a normal processor, go for -j2. Oh and the -pipe argument tells the compiler to use pipes instead of temporary files when sending data between individual compilations.
A note on the function bk. This shell function is defined, like the other shell functions mentioned in this how-to, in the init file init_livecd.env. Given a filename argument, this function will back the file up with a reasonable backup filename (includes the mtime of the file) and leave the original alone. For example:
machine # $ ls -l foobar.txt -rw-r--r-- 1 fimblo users 269 2006-03-16 07:18 foobar.txt machine # $ bk foobar.txt machine # $ ls -l foobar.txt* -rw-r--r-- 1 fimblo users 269 2006-03-16 07:18 foobar.txt -rw-r--r-- 1 fimblo users 269 2006-03-29 23:05 foobar.txt_orig_20060316 machine # $
We're going to want to be able to access the internet by using domain names even after chrooting, so lets copy this file to the target OS.
machine # cp -L /etc/resolv.conf ${PROOT}/etc
Ok, we're ready to chroot into our target operating system! Just cut and paste the following commands and you're in :-)
machine # mount -t proc none ${PROOT}/proc
machine # chroot ${PROOT} /bin/bash
machine # env-update
machine # source /etc/profile
machine # export PS1="live # "
live #
Note on this documentation- from now, the prompt will specify if we're in a chroot or not. If it says 'machine #' we're not in chroot, and if it says 'live #' we are.
Sync your portage tree with a rsync server.
live # emerge --sync
Since we chrooted, we'll need to redefine the bk function.
live # bk() {
> local s=${1}
> local d=${s}_orig_$(\ls -la --time-style="+%Y%m%d" ${s} | \awk '{print $6}')
> \cp ${s} ${d}
> }
Since the system is supposed to be as small as possible, I
did a lot of removing, and just a few additions. Obviously,
add and remove as much as you like- I added maildir support
since I suspect I might be installing postfix in the
future. Also- don't forget to add the livecd use
flag.
live # bk /etc/make.conf
live # cat >> /etc/make.conf <<EOF
# Remove these use flags
USE="-ipv6 -cups -postgres -slang -X -alsa -acpi -apm -oss \
-pcmcia -qt -gtk -gtk2 -kde -gnome -samba -doc -xmms \
-arts -foomaticdb -emboss -fortran -avi -eds -gif -gpm \
-gstreamer -imlib -jpeg -libwww -mad -mikmod -motif \
-mp3 -mpeg -nls -ogg -oggvorbis -opengl -pdflib -png \
-quicktime -sdl -truetype -truetype-fonts -type1-fonts \
-vorbis -xv -java"
# Add these use flags
USE="${USE} idn maildir usb livecd"
EOF
I decided to specify the glibc locales on the machine, since it would (hopefully) minimize the number of installed locales on the machine. I wasn't really sure if this would actually work, but it was worth a try. If anyone reads this who knows more on the subject, please drop me a line!
live # mkdir -p /etc/portage live # echo "sys-libs/glibc userlocales" >> /etc/portage/package.use live # bk /etc/locales.build live # cat >> /etc/locales.build <<EOF en_US/ISO-8859-1 en_US.UTF-8/UTF-8 EOF
I live in Stockholm, so I set my timezone to... Stockholm.
live # cp /usr/share/zoneinfo/Europe/Stockholm /etc/localtime
Besides the regular recommendations from Gentoo, you'll be needing squashfs compiled as a module. Also, save the kernel image in the boot directory as vmlinuz. Finally, write down the kernel name so we can refer to it later. In my case, I got '2.6.14-gentoo-r5' (we can't use uname -r, since we remounted proc... remember?).
live # emerge gentoo-sources live # cd /usr/src/linux live # make menuconfig # see notes below live # make live # make modules_install live # make install live # cp /usr/src/linux/arch/i386/boot/bzImage /boot/vmlinuz live # cp .config /boot/vmlinuz_config
Notes on compiling a kernel:
Another note: If you don't know the version of the kernel you've got, take a look at the top of the Makefile in /usr/src/linux. It will look something like this:
live # head -n 5 /usr/src/linux/Makefile VERSION = 2 PATCHLEVEL = 6 SUBLEVEL = 14 EXTRAVERSION = -gentoo-r5 NAME=Affluent Albatross
This shows that the version of the kernel I have is 2.6.14-gentoo-r5. It's project name, curiously, is Affluent Albatross :)
Now that we have a kernel, we'll need to emerge unionfs to get the unionfs module. Note: I know that we're not supposed to use the ACCEPT_KEYWORDS envvar, but I couldnt be bothered to look up how to use the /etc/portage/packages.keywords properly...
live # ACCEPT_KEYWORDS="~x86" emerge unionfs
When we boot the CD, we want the kernel (which resides in the boot filesystem) to be able to access the squashfs and unionfs modules. So lets copy them to the linux-live directory so it can be bundled by the scripts later on. Remember I told you to write down your kernel name previously? Well, we'll be using it now. Oh, and we need to gzip them as well.
live # LINUX_KERNEL="2.6.14-gentoo-r5" # use your kernel name!
live # LIVE_LIBDIR="/tmp/linux-live-5.1.8/initrd/kernel-modules/${LINUX_KERNEL}"
live # mkdir -p ${LIVE_LIBDIR}
live # for module in squashfs.ko unionfs.ko ; do
> cp $(find /lib/modules/${LINUX_KERNEL} -name ${module}) ${LIVE_LIBDIR}
> gzip ${LIVE_LIBDIR}/${module}
> done
At boot time, the linux-live scripts add all detected disks (hd*) to fstab. So you only need to have one line in fstab; namely the proc filesystem. I also added usbfs, since it will be useful for the testing phase later on.
live # bk /etc/fstab live # cat > /etc/fstab <<EOF proc /proc proc defaults 0 0 none /proc/bus/usb usbfs defaults 0 0 EOF
We'll also want to do the usual configuring of the configuration files. My machine is called helmer, and belongs to the domain yanson.org, so here goes:
live # cd /etc/conf.d live # bk hostname live # echo HOSTNAME=\"helmer\" > hostname live # bk domainname live # cat > domainname <<EOF OVERRIDE=1 DNSDOMAIN="yanson.org" EOF live # rc-update add domainname default
And now for network configuration. At home I use the a class A address range defined in RFC 1918. The real destination of the target OS is a server hall where I use a real (public) internet protocol address. Therefore I decided to set my public ip on interface eth0, and my private one on an aliased interface eth0:1.
I set the default gateway to my gateway ip in the network where I will place the nameserver. Why I won't set the gateway for my home with some smart script? Well, when I start the livecd in a machine from home, it will be for testing, and so I won't need to send packages beyond the scope of my LAN. So no need to set a backup default gateway with some nifty script.
live # bk /etc/conf.d/net live # cat > /etc/conf.d/net <<EOF config_eth0=( "194.68.48.117/24 " "10.0.0.30/8" ) routes_eth0=( "default gw 194.68.48.4" ) EOF live # rc-update add net.eth0 default live # bk /etc/hosts live # cat > /etc/hosts <<EOF 127.0.0.1 localhost helmer helmer.yanson.org EOF
K, now let's add users and set passwords. First set the root password. I normally don't let root log in via ssh, so lets add another user, as well as a password for her.
Don't forget to add your user to the group 'wheel', so that you can access root via the command 'sudo'.
live # # Set root password live # passwd live # # add regular user live # useradd -g users -G wheel -d /home/USERNAME -m USERNAME live # passwd USERNAME
Since my machine won't have a monitor, I want to be able to log in without a) having a spare monitor in my back pocket, or b) turning off the machine and moving it to the closest monitor.
Most modern computers still have serial ports (ahem. my main laptop doesn't...). You might want to activate the possibility of plugging your laptop to the machine using a serial cable. With my last laptop, and my last job, this was the definitive way of accessing a Cisco router if you didn't or couldn't connect via network.
live # bk /etc/securetty live # echo "tts/0" >> /etc/securetty
Just remember to emerge a simple serial port terminal program onto your laptop. I used to use minicom, but maybe there are better apps out there nowadays...
Now you'll want to configure some system settings in the files below. In my case I edited rc.conf, clock, and keymaps. But depending on your preferences you might want to play with keymaps as well.
I added /etc/conf.d/rc here, even if I didnt fiddle with it (this time). Im thinking of the variable RC_FORCE_AUTO, which is set to 'no' by default. Im guessing that the boot-up process might go more smoothely if its set to yes. But again- I didnt fiddle with that file.
live # bk /etc/rc.conf live # cat > /etc/rc.conf <<EOF UNICODE="no" EDITOR="/usr/bin/vim" EOF
live # bk /etc/conf.d/clock live # cat > /etc/conf.d/clock <<EOF CLOCK="local" CLOCK_OPTS="" CLOCK_SYSTOHC="no" SRM="no" ARC="no" EOF
live # bk /etc/conf.d/keymaps live # cat > /etc/conf.d/keymaps <<EOF # for Finnish/Swedish keyboard mappings KEYMAP="fi-latin1" SET_WINDOWKEYS="no" EXTENDED_KEYMAPS="" DUMPKEYS_CHARSET="" EOF
K, lets just zip through this, Im going to install some system utilities.
live # emerge syslog-ng logrotate vixie-cron at ntp live # rc-update add syslog-ng default live # rc-update add vixie-cron default live # rc-update add at default live # rc-update add ntp-client default live # rc-update add ntpd default live # emerge coldplug hotplug live # rc-update add coldplug boot live # rc-update add hotplug boot # see note(1) below live # emerge pciutils slocate app-admin/sudo live # emerge cdrtools # see note(2) below live # # woah! lets not forget adding sshd to default runlevel! live # rc-update add sshd default
I normally tighten sshd's security settings a bit.
live # cat > /tmp/mydiff <<EOF 37c37 < #PermitRootLogin yes --- > PermitRootLogin no 105a106,107 > AllowUsers myNonRootUser > # EOF live # patch /etc/sshd/sshd_config /tmp/mydiff
Also note that you should only run protocol 2, protocol 1 is proven breakable. This should be set by default nowadays:
live # grep Protocol /etc/sshd/sshd_config Protocol 2
I used the default values of syslog-ng, but you might want to consider sending all log events to another machine, which has a physical harddrive to save the logs in.
Since I didn't use networking as an alternative in syslog-ng, Im facing a potential full ramdisk if the logs get too full. This, in tech-lingo, is called a "bad thing". So I edited the logrotate conf so that it cycles the logs every two weeks instead of the default of four weeks.
live # bk /etc/logrotate.conf
live # cat > /etc/logrotate.conf<<EOF
weekly
rotate 2
create
compress
include /etc/logrotate.d
notifempty
nomail
noolddir
/var/log/wtmp {
weekly
create 0664 root utmp
rotate 2
}
EOF
As an afterthought, I realize that it would be wiser to tell logrotate to rotate logs when a file has reached a specific size. This way we won't be caught with our pants down half-way through a rotation period with very large logs and a full filesystem...
Since this machine will contain a server of some sort (at least for me) I need reliable timestamps on the logs. And I want ssh to act sanely. So we'll be needing the ntp services.
There are two files to mess with here:
/etc/ntp.conf and
/etc/conf.d/ntp-client. Hmm. just fyi, ntp-client
sets the date at boot time, once- even if your computer's
local time is completely off the mark. ntpd in turn sees to
that the time on your system doesn't drift, but if the local
time is very different from the upstream servers, it wont
start. So we need both.
live # bk /etc/conf.d/ntp-client live # cat > /etc/conf.d/ntp-client <<EOF NTPCLIENT_CMD="ntpdate" NTPCLIENT_OPTS=" -b -u your.favourite.ntp.server" NTPCLIENT_TIMEOUT=30 EOF
live # bk /etc/ntp.conf live # cat > /etc/ntp.conf <<EOF server your.favourite.ntp.server prefer server pool.ntp.org driftfile /var/lib/ntp/ntp.drift restrict default nomodify nopeer restrict 127.0.0.1 EOF
Note: remember that if you use dhcp to get an address, that dhcp will wipe your ntp.conf file unless you tell it not to in its configuration file. (I think it was the -N flag). Check the man page or the gentoo wiki for details.
For some reason I cant be bothered to figure out, some boot
script finds out that we're running on a CD and replaces the
/etc/issue file with some warnings. This error
message is harmless since Im running on a livecd, but it
doesn't look nice, so I added some code to
/etc/conf.d/local.start which replaces the
contents of /etc/issue
live # cat > /etc/conf.d/local.start <<EOF # Override the warning text cat > /etc/issue <<END_OF_MESSAGE This is my livecd server. This is \n.\O (\s \m \r) \t END_OF_MESSAGE EOF
Ok, so now lets secure the server a wee bit more. With our
current configuration, if some malicious script-kiddie manages
to get shell access on our box via, say a httpd or ntp
external exploit, they can try to take a shot at guessing the
root password using su. If they manage, they can
run /bin/su - to get root access. Ouch.
Enter sudo. If we let our regular user use sudo to access a
root shell, and remove the setuid bit on /bin/su,
we'll remove the risk of the above happening. This since we
set tight rules on who can sudo.
live # # Add your regular user to group wheel
live # groups=$(echo "$(groups) wheel" | tr ' ' "\n" | sort | uniq )
live # usermod -G $(echo ${groups} | tr ' ' ',') username
live #
live # # Make sure that /etc/sudoers allows wheel to sudo
live # cd /etc
live # chmod 0770 sudoers
live # bk sudoers
live # cat > sudoers <<END_OF_MESSAGE
live # Defaults env_reset
live # root ALL=(ALL) ALL
live # %wheel ALL=(ALL) ALL
live # EOF
live # chmod 0440 sudoers
live #
live # # Remove suid bit from /bin/su
live # chmod u-s /bin/su
Now, assuming that we fix all the problems and burn the CD, boot from it and it works, how to we make changes to the CD operating system persistent once we're running it live? After all, rebooting will wipe changes right?
There's lots of ways of doing this- via network, floppy, hard disk, etc. I went for the USB stick. Why?
What we'll need now is a set of functions which will enable us to easily copy changed files to the usb-stick. Likewise, we'll need tools which will let us copy the changed files back to the original location at boottime. We'll need a place to put the functions, and all the edited files as well. Hmm, what else do we need? Perhaps some defaults, if the USB stick isn't inserted at boot time. And finally, to make this a painless process, and an editable one as well, we'll stick all the functions and variables onto the usb-stick itself.
live # mkdir /home/usb live # cd /tmp live # wget http://blahonga.yanson.org/howtos/livecd/tools/usb_default.tar.gz live # tar zxvf usb_default.tar.gz live # mv usb_default/* /home/usb live # rm -r usb_default
What did we just fill the directory /home/usb
with? Theres three directories in the tarball:
bin, empty directory.
etc, location of some boottime scripts, as
well as an environment file for sourcing at login time. More on this later.
saved_states, place where we save data
between boots.
saved_states, but I decided to create this directory
anyway so that the bootup of the system goes smoothly.
Now its time to format the usb stick. Insert your USB stick
into your build machine. I'll just assume that the stick ends
up at /dev/sda1. Yours might end up someplace
else- in that case run dmesg to see where it is.
Run dmesg to make sure that the stick has settled. On my machine, looks like this (truncated output):
live # dmesg ------- snip ------- scsi0 : SCSI emulation for USB Mass Storage devices usbcore: registered new driver usb-storage USB Mass Storage support registered. usb-storage: device found at 2 usb-storage: waiting for device to settle before scanning ------- snip ------- EXT3-fs: mounted filesystem with ordered data mode. Vendor: SanDisk Model: Cruzer Micro Rev: 0.2 Type: Direct-Access ANSI SCSI revision: 02 SCSI device sda: 250879 512-byte hdwr sectors (128 MB) sda: Write Protect is off sda: Mode Sense: 03 00 00 00 sda: assuming drive cache: write through SCSI device sda: 250879 512-byte hdwr sectors (128 MB) sda: Write Protect is off sda: Mode Sense: 03 00 00 00 sda: assuming drive cache: write through sda: sda1 Attached scsi removable disk sda at scsi0, channel 0, id 0, lun 0 Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0, type 0 usb-storage: device scan complete ------- snip -------
Once the stick has settled, we can create a partition on it. Fire up cfdisk (or fdisk, or if you're a masochist, sfdisk) and create a partition on the usb disk. My disk looks like this:
cfdisk 2.12i
Disk Drive: /dev/sda
Size: 128450048 bytes, 128 MB
Heads: 8 Sectors per Track: 32 Cylinders: 979
Name Flags Part Type FS Type [Label] Size (MB)
------------------------------------------------------------------------------
sda1 Primary Linux ext3 [usb_save] 128.32
[Bootable] [ Delete ] [ Help ] [Maximize] [ Print ]
[ Quit ] [ Type ] [ Units ] [ Write ]
As you can see I have one partition on my usb stick, and its some 128M big. Ok, on to formatting the partition, and filling it with data. I chose the ext3 filesystem for its journalling capabilities.
live # mke2fs -j /dev/sda1 live # mount /dev/sda1 /home/usb live # cd /tmp live # wget http://blahonga.yanson.org/howtos/livecd/tools/usb_stick.tar.gz live # tar zxvf usb_stick.tar.gz live # mv usb_stick/* /home/usb live # rm -r usb_stick
Btw, if you're wondering why I placed the mount point in
/home as opposed to /mnt, its
because the linux-live scripts rolls the contents of
/home into the livecd, but not the contents of
/mnt. I could have edited the linux-live scripts,
but then I'd have to maintain that for future versions... I
can't be bothered.
cd over to /home/usb and take a
look around. There are three important files:
| Filename | Description |
/home/usb/etc/usb.env | Environment script. Holds important variables. |
/home/usb/etc/usb.func | Shell functions for saving and restoring files to/from the usb stick. |
/home/usb/etc/usb.bootscript | Script to run at boot time. |
So, to summarize, there are two copies of each script- one
inside the unmounted directory /home/usb and the
other in the usb stick, mounted onto the same. If you add new
scripts or modify the behaviour of the current script-set,
remember to add it to the CD if you need it to work even if
the USB stick is not inserted.
You'll want to source the functions when you log in as root so that you can easily add and remove stuff from the usb-stick.
live # cat >> /root/.bashrc < EOF # Source livecd functions source /home/usb/etc/usb.env source /home/usb/etc/usb.func echo echo "Enter <help_livecd> for help on livecd functions." EOF live #
In the next section we'll be looking (among other things) at how these scripts are called.
The scripts we added in the previous section are nice but completely worthless if we don't call them at boot time. So lets add one boot runlevel script and two configs for boot and shutdown to our target OS.
| Filename | Description |
/etc/init.d/usb_stick | Start/stop script |
/etc/conf.d/usb_stick.start | What usb_stick script should do at boot |
/etc/conf.d/usb_stick.stop | What it should do at shutdown |
So lets download the scripts and put them into place:
live # cd /tmp live # wget http://blahonga.yanson.org/howtos/livecd/tools/bootscripts.tar.gz live # tar zxvf bootscripts.tar.gz -C / live # rc-update add usb_stick boot
Since we'll be booting off a CD, we really don't want the OS to do root filesystem checking and such during bootstrap. So now we're going to remove this part of the bootup process.
live # rc-update del checkroot boot
live # rc-update del checkfs boot
live #
live # bk /etc/init.d/checkfs
live # bk /etc/init.d/checkroot
live # rm /etc/init.d/{checkfs,checkroot}
live #
live # # remove checkroot and checkfs from DEPEND block
live # # empty DEPEND blocks are ok.
live # cd /etc
live # list=$(grep -rlE 'checkroot|checkfs' *)
live # for n in ${list} ; do
> bk ${n}
> done
live # vi ${list}
live #
live # # remove checkroot and checkfs from CRITICAL_SERVICES (around line 128)
live # bk /sbin/rc
live # vi /sbin/rc
Finally, we'll need to fix some boot runlevel script
symlinks. You might not have the same problem I had, but I
noticed that the symbolic links didnt point at the correct
place in the filesystem, that is, /etc/init.d/. If
you run into the same problem, run this code:
live # cd /etc/runlevels/boot
live # for n in * ; do
> rc-update del ${n} boot
> rc-update add ${n} boot
> done
Sorry to let you down folks, this hasn't got anything whatsoever to do with Alan Turing's Halting Problem. Aw, shucks...
Anyway, the problem we have now is that if we create the
iso, burn the CD and boot off it, we'll notice that most
things work quite well (hopefully!). The thing is, the moment
you try to turn off the computer with any of the usual
commands halt, shutdown,
reboot, and poweroff, you'll notice
something. The machine will happily start turning off
services, and at the very end, INIT goes veggie- it writes one
last thing to tty: INIT: no more processes left in this
runlevel. Afterwards it just sits there in contentment,
forever.
Needless to say this was a major pita, especially for me since my livecd will be running in a colo, where I can't rush over and push the 'off' or 'reset' button, but I fixed it like this:
/etc/init.d/halt.sh
with this:
live # bk /etc/init.d/halt.sh live # cat > /etc/init.d/halt.sh <<EOF #!/bin/sh echo "Starting (modified) Slax live-cd halt procedure." /etc/rc.d/rc.6 EOF
Note that the script above just runs another script, namely
/etc/rc.d/rc.6, which is a modified version of
the Slax livecd halt script.
/etc/rc.d/rc.6. Note: I kept the
file location and filename of the original from the slax
distro so that I get forcibly reminded that this is an alien
part of my otherwise pure gentoo OS. A pretty crude method to
be sure, but it works for me :-).
live # mkdir /etc/rc.d live # cd /etc/rc.d live # wget http://blahonga.yanson.org/howtos/livecd/tools/rc.6.gz live # gunzip rc.6.gz live # chmod 755 rc.6
Why all this hassle? Well, initially I just went with the
default Gentoo /etc/init.d/halt.sh, which I
thought should do to job. Assuming that you add cdroot to the
kernel command at boot time, the script should take you to a
poweroff (or reboot, depending on your command).
But the thing is, since I decided to use the linux-live
scripts, the system is kick-started using a slax base
filesystem (which in turn is based on slackware). Which means
that all the apps (above all, busybox) are located in slightly
different places, resulting in broken
${PATH}s. Also, there seem to be general
inconsistencies between the slax and gentoo way of booting
(and thus shutting down) a livecd.
As a consequence of all this, I just couldn't restart the stupid computer. After lots of websearching, testing on my part, and unrebootable compact discs, this was the simplest result I could get. Ugly, but it works.
Ok, now I can get past INIT, but I noticed that when the reins get passed to BIOS, that it somehow couldn't start up the system. The kernel never gets loaded. As a matter of fact, it seemed to have hung at the bootloader. *grumble*.
Everything pointed at the isolinux bootloader. Isolinux is
a mini bootloader adapted for iso9660 filesystems- a small
replacement for grub or lilo (or silo!). Go look for it- some
of you will find that you have two copies of isolinux in your
linux-live scripts directory. This confused me for some time
but I finally took the time to read the scripts around them,
and I saw that the one ending with bi_ was a
backup of isolinux.bin, necessary for restoring isolinux.bin
after making a CD (it gets changed).
Anyway, my current situation was that I could not reboot. I searched the web a bit, and found a solution- replace isolinux.bin with a newer version.
I got mine from kernel.org, but discovered afterwards that it was in portage. *bonk on head* Always check portage before going out and downloading yourself :)
machine # # please note! OUTSIDE OF CHROOT
machine # emerge syslinux
machine # cp /usr/lib/syslinux/isolinux.bin ${PROOT}/tmp/linux-live-5.1.8/cd-root/boot/isolinux.bin
machine # cp /usr/lib/syslinux/isolinux.bin ${PROOT}/tmp/linux-live-5.1.8/cd-root/boot/isolinux.bi_
I did the emerge outside of my chroot since I don't want all the clutter of the accompanying files on the target OS.
Alright! Assuming that you followed all my instructions and that I remembered all the steps, you should be ready to assemble the livecd!
I've written a short shell function which does some basic checking, and if the tests succeed, it moves the larger unnecessary directories to our dmz directory. The 'unnecessary' directories are the kernel source directory, and the portage directory and db. These aren't needed at runtime. The function is included in the init_livecd.env script specified at the beginning of this document. The bash function is called prepare_for_linuxlive_scripts. Oh, and btw- this script should be called from outside the chroot.
machine # prepare_for_linuxlive_scripts Checking if linux-live scripts are installed...ok. Checking for mkisofs...ok. Checking if livecd USE flag is in make.conf...ok, found it. Are all references of checkroot and checkfs removed from init.d and rc? yes. Checking if checkroot and checkfs are removed from boot runlevel...ok. Checking if kernel is in boot partition...ok. Kernel version: 2.6.14-gentoo-r5 Checking kernel module squashfs.ko.gz... ok, found it. Checking kernel module unionfs.ko.gz... ok, found it. All ok. Setting name and version number into file /etc/livecd_version Cleaning up unnecessary files Moving kernel and portage to dmz... To move them back, use the dmz2root function. machine #
Once you get the ok from the script, you should be ready to roll your first iso!
live # cd /tmp/linux-live-5.1.8 live # ./runme.sh Changing current directory to /tmp/linux-live-5.1.8 Linux Live scripts were installed successfuly in / Creating LiveCD from your Linux some debug information can be found in /tmp/linux-live-debug.log copying cd-root to /tmp/live_data_8259, using kernel from /boot/vmlinuz Using kernel modules from /lib/modules/2.6.14-gentoo-r5 creating initrd image... creating compressed images... base/bin.mo base/etc.mo base/home.mo base/lib.mo base/opt.mo base/root.mo base/usr.mo base/sbin.mo base/var.mo creating LiveCD ISO image... mkisofs 2.01 (i686-pc-linux-gnu) Scanning . Scanning ./base Scanning ./boot Scanning ./boot/DOS Scanning ./tools Scanning ./tools/DOS Scanning ./modules Scanning ./rootcopy Scanning ./optional Writing: Initial Padblock Start Block 0 Done with: Initial Padblock Block(s) 16 Writing: Primary Volume Descriptor Start Block 16 Done with: Primary Volume Descriptor Block(s) 1 Writing: Eltorito Volume Descriptor Start Block 17 Size of boot image is 4 sectors -> No emulation Done with: Eltorito Volume Descriptor Block(s) 1 Writing: Joliet Volume Descriptor Start Block 18 Done with: Joliet Volume Descriptor Block(s) 1 Writing: End Volume Descriptor Start Block 19 Done with: End Volume Descriptor Block(s) 1 Writing: Version block Start Block 20 Done with: Version block Block(s) 1 Writing: Path table Start Block 21 Done with: Path table Block(s) 4 Writing: Joliet path table Start Block 25 Done with: Joliet path table Block(s) 4 Writing: Directory tree Start Block 29 Done with: Directory tree Block(s) 10 Writing: Joliet directory tree Start Block 39 Done with: Joliet directory tree Block(s) 9 Writing: Directory tree cleanup Start Block 48 Done with: Directory tree cleanup Block(s) 0 Writing: Extension record Start Block 48 Done with: Extension record Block(s) 1 Writing: The File(s) Start Block 49 7.89% done, estimate finish Mon Mar 13 10:50:36 2006 15.79% done, estimate finish Mon Mar 13 10:50:42 2006 23.66% done, estimate finish Mon Mar 13 10:50:40 2006 31.56% done, estimate finish Mon Mar 13 10:50:39 2006 39.43% done, estimate finish Mon Mar 13 10:50:38 2006 47.33% done, estimate finish Mon Mar 13 10:50:38 2006 55.21% done, estimate finish Mon Mar 13 10:50:37 2006 63.10% done, estimate finish Mon Mar 13 10:50:37 2006 70.98% done, estimate finish Mon Mar 13 10:50:37 2006 78.88% done, estimate finish Mon Mar 13 10:50:38 2006 86.75% done, estimate finish Mon Mar 13 10:50:38 2006 94.65% done, estimate finish Mon Mar 13 10:50:38 2006 Total translation table size: 2048 Total rockridge attributes bytes: 5590 Total directory bytes: 16384 Path table size(bytes): 120 Done with: The File(s) Block(s) 63207 Writing: Ending Padblock Start Block 63256 Done with: Ending Padblock Block(s) 150 Max brk space used 2e000 63406 extents written (123 MB) Your ISO is created in /tmp/livecd.iso live #
As you can see from the log, the image is 123M big (the
original, uncompressed filesystem is 995M excluding
/proc. FYI, if you don't remove the kernel and
portage tree, the total size is 290M. It might not sound like
much but when unpacked, 290M is approximately 1.8G.
The entire operation took under three minutes. Hats off to the linux-live script hackers!
CDs are cheap, but in the process of building this livecd I would have used some forty CDs if it hadn't been for qemu, and later on vmplayer. Both are x86 emulators, qemu being a F/OSS project, and vmplayer being a free but proprietary emulator. Vmplayer is quite a bit faster than qemu, but qemu is simpler to use if you like command line interfaces and a no-nonsense interface. I'll leave the decision to you.
Start the livecd in your favourite virtual environment, and test away. Remember that the USB support on the target OS is limited, since the usb devices are taken by the host system. To test this you'll just have to burn an actual CD and boot off it.
I use qemu to test all basic stuff, like if it boots, reboots, if all the basic services have started, if I've spelled something wrong in some script, etc and so forth. If something doesn't work as you think it should, just edit the target OS in the chrooted environment. I normally had the chrooted environment in one window, and unchrooted one in another, and qemu in a third window. My normal testing pattern:
Once the system seems sane in the virtual environment, you can burn a real, physical CD off the iso.
By the way, the version number can be found on the target
OS in the file /etc/livecd_version. Nice to have
if you're like me and you can't be bothered to mark the CDs
you burn... and you eventually have a pile of unmarked CDs on
your physical desktop and you're wondering which one was the
CD you burned last...
Anyway, things you might want to test:
add_to_usb. Reboot. See if the file is restored
at boot up.
Before you create your final iso of the livecd, remember to
update the slocate database. What I did was boot off the
penultimate iso in vmplayer (not qemu, since I couldn't be
bothered to fix the network connection in it... vmplayer was
easier) , and run /etc/cron.daily/slocate. Then I
took the slocate.db file and scp'd it to the same
directory in my target OS.
The shell function
prepare_for_linuxlive_scripts will, when called,
move the kernel and portage directories out of the target OS
directory space. You might want to look around to see if there
are any other files (or entire directories) you want to move
out of the target OS. Size matters here, since the smaller the
iso, the faster unionfs/squashfs can seek and find files in
the compressed file system. I seriously considered moving all
the man pages out, as well as the documentation in
/usr/share. In the end I didn't do this, since I
felt that the trade-off wasn't worth it. When I want
documentation, I want it now. I don't want to have to
log into another machine, or search the web for it.
At this point, you should have a complete install of a gentoo-ized livecd, which can be used as a base for further fiddling.
Ok, so now you have a livecd. Now what?
I wanted a livecd based name server, so I proceeded to copy
the entire project tree to a new directory, gave it a new
project name and reset the version number to 0 in
init_livecd.env.
Inside this new target OS, I removed ssmtp, installed
postfix, a chrooted named, added big brother for monitoring,
and added some simple iptable rules. I also removed all the
backup files I created during the creation of the first base
livecd (the ones created by my shell function
bk).
I also made logrotate rotate the logs more frequently than the default, and it emails me the logs instead of deleting them. I wrote a hack to monitor all logins into the system, and added a simple rootkit checker just to be on the safe side.
Afterwards, I did some testing and before long it was done. Today, its happily running at my colo. Nice!
Finally- if you find any errors in this documentation, or if you have some refinements- dont hesitate to email me at m at yanson dot org. Make sure you add the string "livecd howto" in the subject line somewhere so my filters can put it in the right directory.
If you have any questions, please check the various forums first before emailing me. The gentoo forums, the gentoo wiki and the linux-live forums are all excellent resources. If you're really stuck, you can email me. Just don't expect a prompt answer, and if the question is very newbiesque (i.e. 'What is a filesystem?' or 'How do I burn a CD?'), I'll probably just send a link to tldp.
My name is Mattias Jansson. I'm half-Japanese/half-Swedish, and live in Stockholm. My days are spent in disguise at a large Swedish bank where I administrate an interest rate derivative system, which runs on Solaris and Wintendo machines. In the evenings I throw off my disguise and play with GNU/Linux.
I've taught internet routing protocols at KTH (Royal Institute of Technology, Stockholm) in a previous life, and enjoy talking network protocols and operating systems the way ordinary mortals like playing FPSs or MMORPGs.
My favourite programming languages are currently perl and the bourne shell descendents (bash and ksh primarily), but I'd say I'm quite proficient at building multithreaded, networked java server applications. I like emacs-lisp, tho I never have time to learn it properly. I write C hacks when I have to. I avoid C++ at all costs, unless its a part of my job (its a part of my job).
My favourite non-computer interests are post-revolutionary European History and thus by implication political science. I like dabbling in metaphysics, with a special focus on the mind-body problem.
Link to my blog: http://blahonga.yanson.org.