ubuntu remaster script

techComments Off on ubuntu remaster script

Script to customise Linux. It’s generally a good idea to build the image from a virtual machine or livecd/udb that is the same as the one you are customising. Will customise iso and usb images and automatically write to usb or loop device file with optional encryption. PLEASE NOTE: Im still updating this script to install LUKS encrypted partitions to loop devices, but due to some confusion over cylinders, heads and sectors using fdisk, sfdisk, parted and kpartx I have hit a snag. I will update this when I have time.

Works with Ubuntu 10 & 11, Backtrack5R1 and probably others:

[expand title=”read source”]

#! /bin/sh

#this script remasters an ubuntu livecd please see the help for more info

_usage () {
[ -n "$1" ] && /bin/echo -e "n\033[31;1m$1\033[mn" >&2
cat >&2 < Usage: $0 [command] [options...]

Commands & options:
extract : create the work folder
options : /path/to/ubuntu.iso /path/to/working_area

chroot : start a shell in the WA
options : /path/to/working_area

release : umount special WA mounts
options : /path/to/working_area

regeniso : create new iso image
options : /path/to/working_area IMGNAME /path/to/new.iso

regenusb : create new usb image
options : [ vfat ext2 ext3 ext4 ] [compress uncompress] [write nowrite] [enc noenc] /path/to/working_area IMGNAME /path/to/new.iso [device]

listusb : list available usb devices
options :

rndusb : randomise usb device
options : /dev/device

checkdeps : check dependencies for encrypted install
options :

remove : remove software defined in the file 'kicklist'
options : /path/to/working_area

examples:

./remaster regen usb compress nowrite work/ EViL usbtest.img /dev/sdc

./remaster chroot work/

EOF

exit 1
}

# Check that we're the root user
[ x`id -u`-`id -g`y = x0-0y ] || _usage "This script requires root privileges"

check_deps () {
# Check that right packages installed

for command in syslinux extlinux hashalot lvm2 cryptsetup ; do
if ! which "$command" > /dev/null; then
echo -e "$command not found! Install? (y/n)"
read
if [ "$REPLY" == "y" ]; then
apt-get install $command
fi
fi
done
}

######### EXTRACT CD ####################

list_usb () {
echo available usb devices:
for devices in $( fdisk -l | grep -o /sd*. | uniq ) ; do
size=`fdisk -l | grep $devices | grep B | cut -d ' ' -f3-4 | cut -d ',' -f 1`
echo "/dev$devices : $size"
done

}

_finalize_extract () {
umount "$d/squashfs" >/dev/null 2>&1 && rmdir "$d/squashfs"
umount "$d/mnt" >/dev/null 2>&1 && rmdir "$d/mnt"

trap "" KILL EXIT QUIT TERM INT
}

extract_img () {
[ $# != 2 ] && _usage "Invalid number of arguments for extract command"

imgfile="$1"
d="$2"

[ -f "$imgfile" ] || _usage "Invalid ISO file $imgfile"
[ -d "$d/edit" ] && _usage "Destination working area $d already exists"

[ ! -d "$d" ] && mkdir "$d"

echo "# Preparing working area $d for $imgfile"

trap "_finalize_extract" KILL EXIT QUIT TERM INT
set -x

mkdir "$d/mnt"
mount -o loop "$imgfile" "$d/mnt"

mkdir "$d/extract-cd"
rsync --exclude=/casper/filesystem.squashfs -a
"$d/mnt/" "$d/extract-cd"

mkdir "$d/squashfs"
mount -t squashfs -o loop "$d/mnt/casper/filesystem.squashfs"
"$d/squashfs"
mkdir "$d/edit"
rsync -a "$d/squashfs/" "$d/edit"

# backsup conf data

mkdir "$d/orig"
cp -a "$d/edit/etc/hosts"
"$d/orig/etc-hosts" >/dev/null 2>&1 || true
cp -a "$d/edit/etc/resolv.conf"
"$d/orig/etc-resolv.conf" >/dev/null 2>&1 || true
cp -a "$d/edit/etc/mtab"
"$d/orig/etc-mtab" >/dev/null 2>&1 || true
tar cf "$d/orig/etc-apt-conf.tar" -C "$d/edit/etc" apt

}

######### CHROOT LIVE ####################

chroot_live () {
[ $# != 1 ] && _usage "Invalid number of arguments for chroot command"

d="$1"
[ -d "$d/edit" ] || _usage "Invalid working area $d"

cp /etc/resolv.conf /etc/hosts "$d/edit/etc/"
cp -fa /etc/apt "$d/edit/etc/"

mkdir -p "$d/edit/tmp/remaster-xchg"

/bin/echo -e "\033[1m"
cat >&2 < IMPORTANT: do not ever try to remove the $d (eg. rm -r) area
unless you ran either the 'release' or 'regen' command beforehand !
EOF
/bin/echo -e "\033[m"

[ -d "$d/edit/dev/usb" ] || mount --rbind /dev/ "$d/edit/dev"

env_key=`uuidgen`

main_sh="/tmp/remaster-xchg/main-$env_key.sh"

cp "$0" "$d/edit/$main_sh"
cat > "$d/edit/tmp/remaster-xchg/startenv-$env_key.sh" < echo `xauth nlist $DISPLAY` | xauth nmerge -
EOF

exec chroot "$d"/edit env ENV_KEY="$env_key" DISPLAY="$DISPLAY"
/bin/sh "$main_sh" _actual_chroot_
}

########## INSIDE CHROOT ################

_actual_chroot_ () {
[ -f /proc/cpuinfo ] || mount -t proc none /proc
[ -d /sys/class ] || mount -t sysfs none /sys

HOME=/root
LC_ALL=C
export HOME LC_ALL
unset XAUTHORITY

dbus-uuidgen > /var/lib/dbus/machine-id

. /tmp/remaster-xchg/startenv-$ENV_KEY.sh
rm -f /tmp/remaster-xchg/startenv-$ENV_KEY.sh
rm -f "$0"

cd /root
exec env debian_chroot="l33t" /bin/bash -l
}

################# RELEASE ###################

release_live () {
[ $# != 1 ] && _usage "Invalid number of arguments for release command"
d="$1"
[ -d "$d/edit" ] || _usage "Invalid working area $d"

while [ -d "$d/edit/dev/usb" ] ; do umount -fl "$d/edit/dev" 2>/dev/null ; done
while [ -d "$d/edit/sys/class" ] ; do umount -fl "$d/edit/sys" 2>/dev/null ; done
while [ -f "$d/edit/proc/cpuinfo" ] ; do umount -fl "$d/edit/proc" 2>/dev/null ; done
# er quick hack to fix something
umount -lf $d/edit/dev || true
echo 'dev, sys and proc unmounted, safe to proceed...'
}

################# REGEN ###################

prepare_img () {

## prepare image

echo "# Creating image $IMAGE_NAME as $outfile from $d"
release_live "$d"
set -x
chroot "$d/edit" aptitude clean

rm -f "$d/edit/etc/hosts" "$d/edit/etc/resolv.conf" "$d/edit/etc/mtab"
"$d/edit/root/.bash_history" "$d/edit/root/.Xauthority"
# rm -rf "$d/edit/etc/apt"
rm -rf "$d/edit/tmp/*"
rm -rf /var/lib/dbus/machine-id

cp -a "$d/orig/etc-hosts"
"$d/edit/etc/hosts" >/dev/null 2>&1 || true
cp -a "$d/orig/etc-resolv.conf"
"$d/edit/etc/resolv.conf" >/dev/null 2>&1 || true
cp -a "$d/orig/etc-mtab"
"$d/edit/etc/mtab" >/dev/null 2>&1 || true

# tar xf "$d/orig/etc-apt-conf.tar" -C "$d/edit/etc/"

chmod +w "$d/extract-cd"/casper/filesystem.manifest
chroot "$d/edit" dpkg-query -W --showformat='${Package} ${Version}n'
> "$d/filesystem.manifest"
cp -f "$d/filesystem.manifest"
"$d/extract-cd"/casper/filesystem.manifest
cp -f "$d/filesystem.manifest"
"$d/extract-cd/casper/filesystem.manifest-desktop"

rm -f "$d/extract-cd/casper/filesystem.squashfs"
mksquashfs "$d/edit"
"$d/extract-cd/casper/filesystem.squashfs" # -nolzma

# TODO: Edit extract-cd/README.diskdefines
set +x || true 2>/dev/null
for f in ubiquity casper live-initramfs user-setup discover
xresprobe os-prober libdebian-installer ; do
sed -i "/${f}/d" "$d/extract-cd/casper/filesystem.manifest-desktop"
done
set -x
rm "$d/extract-cd/md5sum.txt"
sh -c "cd '$d/extract-cd' && find . -type f -print0 | xargs -0 md5sum > md5sum.txt"
}

regenerate_iso () {

# options : /path/to/working_area IMGNAME /path/to/new.iso

[ $# != 3 ] && _usage "Invalid number of arguments for regeniso command"

# Determine absolute path to output ISO file (we'll do a chdir later)
od=`dirname "$3"`
od=`cd "$od" && /bin/pwd`
outfile="$od"/`basename "$3"`
d="$1"
IMAGE_NAME="$2"

[ -d "$d/edit" ] || _usage "Invalid working area $d"
[ -f "$outfile" ] && _usage "Destination iso file $outfile already exists"
here=`/bin/pwd`

prepare_img

cd "$d/extract-cd"

mkisofs -r -V "$IMAGE_NAME" -cache-inodes -J -l
-b isolinux/isolinux.bin -c isolinux/boot.cat
-no-emul-boot -boot-load-size 4 -boot-info-table
-o "$outfile" .

cd "$here"

}

regenerate_usb () {

# options : [ vfat ext2 ext3 ext4 ] [compress uncompress] [write nowrite] [enc noenc] /path/to/working_area IMGNAME /path/to/new.iso [device]
[[ $# != 7 && $# != 8 ]] && _usage "Invalid number of arguments for regenusb command"

type="$1"
compress="$2"
write="$3"
enc="$4"
d="$5"
IMAGE_NAME="$6"
od=`dirname "$7"`
od=`cd "$od" && /bin/pwd`
outfile="$od"/`basename "$7"`
device="$8"

[ "$type" == "vfat" -o "$type" == "ext2" -o "$type" == "ext3" -o "$type" == "ext4" ] || _usage "Invalid image type: $type"
[ "$compress" == "compress" -o "$compress" == "uncompress" ] || _usage "Invalid compression setting: $type"
[ "$write" == "write" -o "$write" == "nowrite" ] || _usage "Invalid write setting: $write"
[ "$enc" == "enc" -o "$enc" == "noenc" ] || _usage "Invalid encryption setting: $enc"
[[ -z "$device" ]] && [[ "$write" == "write" ]] && _usage "selected write but no device entered"
[ -n "$device" ] && [ "$write" == "nowrite" ] && _usage "selected nowrite but device entered"
[ "$write" == "nowrite" ] && device=loop

# specified hard disk and write
[ -n "$device" ] && [ "$write" == "write" ] && check_disk $device

here=`/bin/pwd`

prepare_img

# temp liveusb work
LIVEUSBROOT="$od/liveusb"
mkdir $LIVEUSBROOT
cd $LIVEUSBROOT

set +x
#determine rough image size
IMAGE_SIZE=$(( `du -hs "$od/$d""extract-cd" | cut -b1` + 1 ))G

# make loop device
echo making "$IMAGE_NAME", "$IMAGE_SIZE" loop device image
touch loop
dd if=/dev/zero of=loop bs=1 count=1 seek=$IMAGE_SIZE

# determine encryption options
if [ "$enc" == "enc" ] ; then
partition_drive "$device"
echo "do you want to write random data to this device first ? (y/n)"
read
if [ "$REPLY" == "y" ]; then
rnd_usb "$device"
fi
fi

if [ "$type" == "ext2" ] ; then
mkfs.ext2 -F -L "$IMAGE_NAME" -m 0 loop
fi

if [ "$type" == "ext3" ] ; then
mkfs.ext3 -F -L "$IMAGE_NAME" -m 0 loop
fi

if [ "$type" == "ext4" ] ; then
mkfs.ext4 -F -L "$IMAGE_NAME" -m 0 loop
fi

if [ "$type" == "vfat" ] ; then
mkfs.vfat -F 32 -n "$IMAGE_NAME" loop
fi

#mount loop and copy files
ln -s "$od/$d/extract-cd" tmp
mkdir mnt
mount -o loop loop mnt
echo "copying files..."
cp -a tmp/* mnt/

# make bootable
cd mnt
mv isolinux/ extlinux
mv extlinux/isolinux.cfg extlinux/extlinux.conf
mv extlinux/isolinux.bin extlinux/extlinux.bin
extlinux --install extlinux/
cd ..
umount mnt

cd "$here"

#compress or no ?
if [ "$compress" == "compress" ] ; then
#pack it up
echo "compressing image..."
gzip -cv1 "$LIVEUSBROOT/loop" > "$outfile.gz"
#write or no ?
if [ "$write" == "write" ] ; then
echo "writing $outfile to usb..."
zcat $outfile | tee $device >/dev/null
fi
else
# dont pack it up, use loop file
mv "$LIVEUSBROOT/loop" "$outfile"
#write or no ?
if [ "$write" == "write" ] ; then
echo "writing $outfile to usb..."
dd if=$outfile of=$device
fi
fi

#remove working directory...
rm -rf $LIVEUSBROOT
echo "now use burn_usb script"

}
checkMBR() {

bs=$(mktemp bs.XXXXXX)
dd if=$device of=$bs bs=512 count=1 2>/dev/null || exit 2

mbrword=$(hexdump -n 2 $bs |head -n 1|awk {'print $2;'})
rm -f $bs
if [ "$mbrword" = "0000" ]; then
echo "MBR appears to be blank."
echo "Do you want to replace the MBR on device $device? (y/n)"
read
if [ "$REPLY" == "y" ]; then
resetMBR $device
echo "MBR reset"
else
echo "MBR untouched..."
fi
fi

}

resetMBR() {
if [ "$type" == "vfat" ] ; then
cat /usr/lib/syslinux/mbr.bin > $device
else
echo extlinux will write itself once the image is done...
fi

}

partition_drive() {

checkMBR "$device"
size=`fdisk -l | grep $device | grep B | cut -d ' ' -f3-4 | cut -d ',' -f 1`

echo "/dev/$device : $size"

echo $IMAGE_SIZE

echo hello!
exit 1
}

check_disk() {

DEV=$1
udev=$(udevadm info --query=path --name $DEV) || true
[ -n "$udev" ] || _usage "Error finding block device of $DEV. Aborting!"
echo $udev
echo "disk exists..."
}

rnd_usb () {
[ $# != 1 ] && _usage "Invalid number of arguments for rndusb command"

DEV=$1
check_disk $DEV
echo "disk exists, i hope you know what youre doing..."
echo "are you sure you want to erase $DEV and randomise its blocks? (y/n)"
read
if [ "$REPLY" == "y" ]; then
echo this is going to take a long time, depending on the size of your disk...
dd if=/dev/urandom of=$DEV
else
echo "disk left alone..."
exit
fi

echo "disk finished..."
exit
}

remove_sfw () {
[ $# != 1 ] && _usage "Invalid number of arguments for remove command"

d="$1"
[ -d "$d/edit" ] || _usage "Invalid working area $d"

rm -rf "$d/edit/remove/"
# remove file
mkdir "$d/edit/remove"
cp -a "$d/uninstall_list"
"$d/edit/remove/" >/dev/null 2>&1 || true

cp /etc/resolv.conf /etc/hosts "$d/edit/etc/"
# cp -fa /etc/apt "$d/edit/etc/"

[ -d "$d/edit/dev/usb" ] || mount --rbind /dev/ "$d/edit/dev"

env_key=`uuidgen`

main_sh="/tmp/remaster-xchg/main-$env_key.sh"
cp "$0" "$d/edit/$main_sh"
cat > "$d/edit/tmp/remaster-xchg/startenv-$env_key.sh" < echo `xauth nlist $DISPLAY` | xauth nmerge -
EOF

exec chroot "$d"/edit env ENV_KEY="$env_key" DISPLAY="$DISPLAY"
/bin/sh "$main_sh" _actual_remove_
}

########## INSIDE REMOVE CHROOT ################

_actual_remove_ () {
[ -f /proc/cpuinfo ] || mount -t proc none /proc
[ -d /sys/class ] || mount -t sysfs none /sys

HOME=/root
LC_ALL=C
export HOME LC_ALL
unset XAUTHORITY

dbus-uuidgen > /var/lib/dbus/machine-id

cd /root
aptitude remove `cat $d/edit/remove/uninstall_list`
rm -rf "$d/edit/remove/"

echo done
}

########## HELP ################

help () {
echo "
1. extract the contents of an ubuntu CD ISO image to your hard drive

2. chroot to the working area containing the extracted CD
contents. This allows to do all the /etc configs, aptitude
purge/install, etc. to prepare the future ISO image

3. generate a new ISO or USB image containing your changes

./remaster extract /path/to/original/ubuntu.iso /path/to/working/area
=> This will extract the contents of the ISO into 2 subdirectories of
/path/to/working/area/ (should be empty initially):
extract-cd/: the files needed to boot (isolinux config,
kernel, initramfs, etc.), but not the root FS
edit/: the root file system (complete with /etc, /usr, etc.)

./remaster chroot /path/to/working/area
=> This will start a root bash shell inside the root file
system. Takes care of mounting /dev, /proc, /sys. In that
shell, you can aptitude purge/install, edit /etc files, create
users, even start X applications (xauth is configured),
etc. But do NOT edit anything in /dev, as this will affect
your host (the host's /dev is mounted inside the chroot:
anything changed in the chroot's /dev is also changed in the
host's /dev !): avoid doing rm in /dev at this point :) !
Beware also that issuing some aptitude install commands might
start /etc/init.d daemons which will run inside the chroot,
even after you exited from it: you might have to kill these
manually so that they don't interfere with the host
Note: This operation will also copy the host's apt config into
the working area, so that the same repositories can be
used. The original ubuntu apt config will be restored when the
regen command is issued

./remaster regeniso /path/to/working/area SOME_NAME /path/to/dest/my.iso
=> Generate a new ISO file from the extract-cd/ and edit/ dirs of
the working area. The SOME_NAME param is just a dummy tag for
the ISO image. The original apt configuration will be restored
before the image is generated. You will still be able to use
the chroot operation after a regen

./remaster regenusb /path/to/working/area SOME_NAME /path/to/dest/my.iso
=>
Generate a new USB file from the extract-cd/ and edit/ dirs of
the working area. The SOME_NAME param is just a dummy tag for
the USB image. The original apt configuration will be restored
before the image is generated. You will still be able to use
the chroot operation after a regen

./remaster listusb
=>
lists usb devices available tothe system
you should have written the partition table or be prepared to hand the whole
disk to the script for partitioning. the script doesnt do everything for you.
please know what you are doing when writing drives.

./remaster rndusb /dev/blah
=>
randomise data on usb stick to further increase security of encrypted lvm

./remaster checkdeps
=>
check dependencies required for encryption

./remaster release /path/to/working/area
=>
this command makes sure the
host will not be impacted by the removal of the working area
(eg. it will unmount the /dev mount in the chroot). After this
command, it is safe to remove completely the working area. You
will still be able to use the chroot operation after a release
but will always need to release it aferwards.

./remaster remove /path/to/working/area

removes programs from the file uninstall_list in the same dir as script

The 'extract' and 'regen' operations will be verbose and stop at the
first error that occurs. You can of course run the 'extract'
operation just once, and run the 'chroot' or 'regen' operations
thousands of times on the same working area. You can also run
several 'chroot' sessions in parallel. But be sure NOT to run a
'chroot' operation while a 'regen' is in progress.

"

_usage ;
}

############# MAIN ###############

case x"$1" in
xextract) shift ; set -e ; extract_img "$@";;
xchroot) shift ; set -e ; chroot_live "$@" ;;
xrelease) shift ; set -e ; release_live "$@" ;;
xregeniso) shift ; set -e ; regenerate_iso "$@";;
xregenusb) shift ; set -e ; regenerate_usb "$@";;
xlistusb) shift ; set -e ; list_usb "$@";;
xrndusb) shift ; set -e ; rnd_usb "$@";;
xrndusb) shift ; set -e ; rnd_usb "$@";;
xcheckdeps) shift ; set -e ; check_deps "$@";;
xhelp) shift ; set -e ; help "$@";;
x_actual_chroot_) _actual_chroot_ ;;
x_actual_remove_) _actual_remove_ ;;
*) _usage "Invalid command $1" ;;
esac

[/expand]

Click here to download the script.

Howto

1. extract the contents of an ubuntu CD ISO image to your hard drive

2. chroot to the working area containing the extracted CD
contents. This allows to do all the /etc configs, aptitude
purge/install, etc. to prepare the future ISO image

3. generate a new ISO or USB image containing your changes

./remaster extract /path/to/original/ubuntu.iso /path/to/working/area
=> This will extract the contents of the ISO into 2 subdirectories of
/path/to/working/area/ (should be empty initially):
extract-cd/: the files needed to boot (isolinux config,
kernel, initramfs, etc.), but not the root FS
edit/: the root file system (complete with /etc, /usr, etc.)

./remaster chroot /path/to/working/area
=> This will start a root bash shell inside the root file
system. Takes care of mounting /dev, /proc, /sys. In that
shell, you can aptitude purge/install, edit /etc files, create
users, even start X applications (xauth is configured),
etc. But do NOT edit anything in /dev, as this will affect
your host (the host’s /dev is mounted inside the chroot:
anything changed in the chroot’s /dev is also changed in the
host’s /dev !): avoid doing rm in /dev at this point :) !
Beware also that issuing some aptitude install commands might
start /etc/init.d daemons which will run inside the chroot,
even after you exited from it: you might have to kill these
manually so that they don’t interfere with the host
Note: This operation will also copy the host’s apt config into
the working area, so that the same repositories can be
used. The original ubuntu apt config will be restored when the
regen command is issued

./remaster regeniso /path/to/working/area SOME_NAME /path/to/dest/my.iso
=> Generate a new ISO file from the extract-cd/ and edit/ dirs of
the working area. The SOME_NAME param is just a dummy tag for
the ISO image. The original apt configuration will be restored
before the image is generated. You will still be able to use
the chroot operation after a regen

./remaster regenusb /path/to/working/area SOME_NAME /path/to/dest/my.iso
=>
Generate a new USB file from the extract-cd/ and edit/ dirs of
the working area. The SOME_NAME param is just a dummy tag for
the USB image. The original apt configuration will be restored
before the image is generated. You will still be able to use
the chroot operation after a regen

./remaster listusb
=>
lists usb devices available tothe system
you should have written the partition table or be prepared to hand the whole
disk to the script for partitioning. the script doesnt do everything for you.
please know what you are doing when writing drives.

./remaster rndusb /dev/blah
=>
randomise data on usb stick to further increase security of encrypted lvm

./remaster checkdeps
=>
check dependencies required for encryption

./remaster release /path/to/working/area
=>
this command makes sure the
host will not be impacted by the removal of the working area
(eg. it will unmount the /dev mount in the chroot). After this
command, it is safe to remove completely the working area. You
will still be able to use the chroot operation after a release
but will always need to release it aferwards.

./remaster remove /path/to/working/area

removes programs from the file uninstall_list in the same dir as script

The ‘extract’ and ‘regen’ operations will be verbose and stop at the
first error that occurs. You can of course run the ‘extract’
operation just once, and run the ‘chroot’ or ‘regen’ operations
thousands of times on the same working area. You can also run
several ‘chroot’ sessions in parallel. But be sure NOT to run a
‘chroot’ operation while a ‘regen’ is in progress.

 

Done

  • added choice of image type on regen – cd, usb, encrypted cd, encrypted usb.
  • added remove applications from a filed list automagically
  • fixed an error with dbus machine-id
  • fixed an error releasing the dev mount

To Do

  • add install from list
  • clean up and shorten chroot code
  • make some tea

 

sources:

http://peopleareducks.com/docs/vdg/output/Virtual-Disk-Operations.html#VDG-Concepts-Creation

 

» tech » ubuntu remaster script

, 07/07/2011

Comments are closed.