Category: Unix tricks


I’ve been messing around with my stack of WRT54G routers this weekend.  So far I have serial modded two out of the five that I have sitting here.  The neat thing about the serial mod is that it’s so easy to grab a console off of it without worrying about network parameters.  The bad thing is that your router may or may not be connected to the internet when you are on that console.  It’s pretty easy to hook up to another wireless router in client mode from the console.  I couldn’t find the following information all in one place so I’m going to hash out the quick version here:

iwconfig wlan0 essid router_name

iwconfig wlan0 key 0123456789 (I have a wep router handy for connecting older devices)

ifconfig wlan0 10.10.10.40 netmask 255.255.255.0 (no dhcp client on my router by default)

ifconfig wlan0 up

route add default gw 10.10.10.1

ifconfig wlan0 up

and finally add a known dns server (like 8.8.8.8) to your /etc/resolv.conf with vi

For advanced Unix users, none of this is anything new but hopefully this will help someone else out there who is struggling through an OpenWrt or Gentoo install or can’t figure out how to configure wireless on your Zipit after you’ve put an aftermarket root fs on it.  All of these settings will disappear when you reboot your device aside from editing the resolv.conf although if you are using a WRT54G series router, your edits to the resolv.conf will also disappear.

Installing Gentoo in a Fusion VM

The other day I installed Gentoo on an old Celeron 466MHz and it was quite an adventure.  Today, I’m attempting to install it in a far more useful capacity as a virtual machine under VMware Fusion.  The first thing I did was create a new virtual machine.  There is no template for Gentoo so I tweaked the settings a bit.  I gave it 512MB ram, 30GB hard drive and set it to “generic linux 2.6.X kernel”.  I set it to boot from the minimal ISO that I used to burn the CD for the other day. After I was booted up, I ran a couple of benchmarks.  Thankfully this system is several orders of magnitude faster than a Celeron 466MHz.

Using fdisk, I created my 3 partions:

/boot /dev/sda1

swap /dev/sda2

/        /dev/sda3

Then I did my mkfsing:

mke2fs /dev/sda1

mke2fs -j /dev/sda3

mkswap /dev/sda2 && swapon /dev/sda2

Now it was time to snag my stage file.  This time instead of wget, I used links:

links gentoo.org

I hit the downloads link, then hit the stages button next to i686 and found the most recent stage3-i686 tar.bz2 file and downloaded it.  After that, I untarred it:

tar -xpjf s<TAB>

Next, I changed snagged the latest portage via links and untarred that:

cd  usr

links distfiles.gentoo.org/snapshots

tar -xpjf p<TAB>

Then it’s time to chroot:

livecd / # mount -t proc proc /mnt/gentoo/proc

livecd / # mount -o bind /dev /mnt/gentoo/dev

livecd / # cp -L /etc/resolv.conf /mnt/gentoo/etc/

livecd / # chroot /mnt/gentoo /bin/bash

livecd / # env-update && source /etc/profile

Then I set the timezone:

cp /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

After that, it’s time to fix up the hostname:

cd /etc

echo “127.0.0.1 gentoo.notanon.com gentoo localhost” > hosts

sed -i -e ‘s/HOSTNAME.*/HOSTNAME=”gentoo”/’ conf.d/hostname

hostname gentoo

Now for the kernel.  I didn’t do a lot of tweaking with the kernel this time around.  I just added ext2 support and a couple of extra modules I want to experiment with.

emerge gentoo-sources

cd /usr/src/linux

make menuconfig

time make -j2

make modules_install

cp arch/i386/boot/bzImage /boot/kernel

Then I editted the /etc/fstab to look something like this:

/dev/sda1 /boot ext2 noauto,noatime 1 2

/dev/sda3 / ext3 noatime 0 1

/dev/sda2 none swap sw 0 0

Then I installed cron, syslog, grub dhcpcd:

time emerge syslog-ng vixie-cron grub dhcpcd

rc-update add syslog-ng default

rc-update add vixie-cron default

After that it’s time to nano /boot/grub/grub.conf:

default 0

timeout 10

title Gentoo

root (hd0,0)

kernel /boot/kernel root=/dev/sda3

Then I ran grub and entered:

root (hd0,0)

setup (hd0)

quit

After this, supposedly it’s time to unchroot, cross my fingers and reboot:

exit

umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo

reboot

Alas, it didn’t work.  I’ve yet to nail a Gentoo installation on the first attempt but this time I was much closer.  I ended up finding a wiki on installing Gentoo in a VM.  I rebuilt the kernel with some of the suggestions that were stated in the wiki and that did the trick.

I was listening to the pauldotcom.com security podcast the other day and heard then talking about netcat.  This peaked my interests and got me thinking that I should be able to print text files from any Unix-based system on which netcat could be installed/compiled.  At first I tried it the simple way:

cat file.txt | nc 10.10.10.201 9100

This did indeed send the file to the printer’s port 9100.  There were two problems though.  First off, the printer asked me if I wanted to print the buffer or wait until it was finished printing.  Pushing the button on the printer made the file print but is not as convenient as it should be.  This problem can be fixed with a “-w1” switch.  This tells netcat to quit if the input stream is idle for more than 1 second.

cat file.txt | nc -w1 10.10.10.201 9100

This prints the text file almost instantaneously but there is another problem.  All the text is stair stepped.  As it turns out, Unix uses only newline character at the end of each line.  In dos and apparently network laser printers, a newline and a carriage return are two separate things that need to be handled individually.  A little bit of sed trickery is just the thing to fix this problem.

sed ‘s/$'”/`echo -e \\\r`/” file.txt | nc -w1 10.10.10.201 9100

Obviously you will need to put your own printer’s IP address there.  This will probably only work with a decent networked laser printer.  I’ve tested this with my Lexmark Optra T644 and it works great.  Special thanks to rkdavis for helping sort out the sed bits.

Sed is a very powerful tool when used correctly but can also be a source of frusteration at times.  One problem with it shows up when you are trying to identify and replace non-printable characters.  Luckily sed has a nice command line switch that will help you debug certain problems with sed.

When you just use cat to display a file, you’ll see it as you are normally used to but this isn’t how sed sees files.

cat hello.c

When you use the -n ‘l’, you’ll see the file how sed sees it.  (It’s a lower case “L” in the single quotes)

sed -n ‘l’ hello.c

Now you can see the new line characters at the end of the lines and the tabs that proceed some of the lines.

I’ve been listening to earlier episodes of pauldotcom.com‘s weekly podcast and have continually heard Twitchy go on and on about how great Gentoo Linux is.  I finally got to episode 62 where Twitchy did a tech segment about portage which is the package management system for Gentoo Linux.  It SOUNDS fantastic.  Pretty similar to Macports but even better since EVERYTHING on your Gentoo system is a manageable package. When you upgrade packages on your system, new source code is downloaded and compiled locally(or can be distributed to other systems you run) and you can specify configuration options on the fly.  Portage keeps track of all installed packages and their dependencies(recursively).  If you want to install a package NOT already available in Gentoo, simply write an ebuild file and Portage will handle the rest of the process including keeping track of the package for you.  You are able to upgrade your entire system with a single emerge command every day, week, month, etc.  After hearing this, I felt it was finally time for me to try out Gentoo.

I consider myself a fairly seasoned Linux user.  I’ve been using Linux on and off for work and play since Yggdrasil Linux came out in 1993 or so and moved back and forth with Redhat and Debian over the years.  Gentoo SOUNDS like it may be the answer to many of my annoyances with the other systems mainly being the broken package management systems.  That being said, installing Gentoo Linux does not follow an obvious process.  Not to say it’s not easy but it’s doesn’t have a scripted install like Redhat or Debian and it’s most certainly not a GUI-based installer like Ubuntu..  When you put in the installation CD and boot it up, you end up at a live cd prompt.  What do you do from here?  You read the 7-page(tiny print) Gentoo Linux x86 Quick Install Guide of course…

My ancient Celeron 466MHz micro desktop system about to get Gentoo'd

Note: The following is a recap of MY personal adventure of installing Gentoo Linux.  It’s not meant as a replacement for their excellent documentation.  If you read the steps I followed though, you might find a couple of ways that I did stuff differently than stated in the installation guide.

Starting out at the top of the quick install guide, I see that the installation was timed on a MUCH quicker machine than mine.  When I type in:

grep bogo /proc/cpuinfo

I get a result back of 933.54.  The AMD 2000 1.6GHz system used for this guide is really old but not nearly as old as mine.  The result on the AMD was 3337.81 bogomips.  Hopefully my system ONLY takes 3x as long to install.  This little Celeron system is the same system that wouldn’t install Haiku for some reason.

Moving on down in the install guide, it looks like I already screwed up.  Apparently I should have used some switches at the first boot prompt when I started the system up.  When I booted, I should have type in:

boot: gentoo-nofb nodetect

That would disable X from trying to load and prevent a zillion kernel modules from loading but I think since my system is soooo old and all the drivers for my hardware are extremely mature by now, I didn’t hit any hang ups.  Doing a “ping go.com” at the command line netted me a favorable result so I’m just going to move on and call it good for now.

The first real and destructive step of this install is to partition your disk manually.  Disk druid?  I don’t think so.  It’s all about the fdisk.  On my system, /dev/hda is the hard drive.  I knew this because running a “df” command showed my CD-ROM as /dev/hdc.  So…

fdisk /dev/hda

The installation guide assumes that you know how to use fdisk.  Luckily I do.  I’m going to create three partitions for my installation as suggested.  One 128MB partition for boot where the kernel and lilo or grub will live, swap which I’ll make 256mb and a / (root) partition that uses up the remainder of the space on the drive.  I remembered to change the swap partition type to 82 and set the /boot partition bootable flag.  Now I need to commit the changes and format the paritions:

mke2fs /dev/hda1

mke2fs -j /dev/hda3 (-j for ext3)

mkswap /dev/hda2 && swapon /dev/hda2

Now mount the partitions in their proper locations:

mount /dev/hda1 /mnt/gentoo

mkdir /mnt/gentoo/boot

mount /dev/hda1 /mnt/gentoo/boot

cd /mnt/gentoo

After that, you are supposed to set the clock.  Since I’m on the net, I’ll run:

ntpdate pool.ntp.org

Now onto the less familiar stuff.  I need to wget the stage3 archive from a mirror.  I’m going to skip the step of finding a local mirror and simply use the default location:

wget ftp://distfiles.gentoo.org/pub/gentoo/releases/x86/current-stage3/stage3-i686-*.tar.bz2

This pulled a 130MB file down into the root directory of my freshly formatted hard disk.  Now to unpack it with:

tar -xjpf stag*

After that, I need to snag the latest portage build and unpack that so I can start managing packages:

cd /mnt/gentoo/usr

wget http://distfiles.gentoo.org/snapshots/portage-latest.tar.bz2

tar -xjf por*

Now to the weird stuff.  We need to chroot the filesystem.  Chroot fools bash into thinking that a sub directory is actually your root directory.  This allows you to type and run commands in a sandbox of sorts that shouldn’t be able to effect outside files.  In this case we have booted up a live CD and have sketched out enough of a root file system on our new disk to operate now so it’s time to switch into the new root file system in order to finish building it:

cd /

mount -t proc proc /mnt/gentoo/proc

mount -o bind /dev /mnt/gentoo/dev

cp -L /etc/resolv.conf /mnt/gentoo/etc/

chroot /mnt/gentoo /bin/bash

env-update && source /etc/profile

cp /usr/share/zoneinfo/America/Los_Angeles /etc/localtime

While I’m dealing with the timezone, I’ll “nano /etc/cron.d/clock” uncommenting the TIMEZONE line and changing “factory” to “America/Los_Angeles”  Then I’ll set up the hostname.  This is certainly not the most straight forward process:

cd /etc

echo “127.0.0.1 mybox.at.myplace mybox localhost” > hosts

sed -i -e ‘s/HOSTNAME.*/HOSTNAME=”mybox”/’ conf.d/hostname

hostname mybox && hostname -f

After all that, it’s time to build the kernel.  If you’ve never build a Linux kernel, you will probably find this step extremely overwhelming but hang in there.  You will learn the most about Linux in this single step.  Make use of the help that is embedded in the menuconfig script.  They used to be somewhat of a joke back in the days of the 2.0.X kernel but now most of the helps are actually very helpful.  Generally in the more confusing kernel options, it will say something like “if unsure, say Y(or N)”.  This will let you fake your way through the kernel config for the most part.  Don’t forget, you can always rebuild it later.

Note: The first time I ran through this, I forgot to include second extension filesystem.  This caused a non-bootable system since the /dev/hda1 block device is formatted ext2.  These errors are common and you’ll learn a lot from making them since something may not work correctly down the road.  Don’t get discouraged, just retrace your steps and you shouldn’t have a problem figuring out where you went wrong.  The command “dmesg” can be very helpful if you get booted.  If you DON’T get booted, whatever the kernel is hanging on should be printed on your screen.  On my Celeron 466MHz, a fairly stripped down kernel is taking me 100 minutes to build.  I’m sure yours is MUCH quicker so don’t be afraid to rebuild it a few times.

emerge gentoo-sources

cd /usr/src/linux

make menuconfig

make -j2

make modules_install

cp arch/i386/boot/bzImage /boot/

I would have thought that the next natural step in the installation process would be to install grub or lilo but oddly they have you jump to an entirely different topic which is fixing up the /etc/fstab to make it bootable.  So I need to run “nano /etc/fstab” and change BOOT to /dev/hda1, ROOT to /dev/hda3 and SWAP to /dev/hda2.  I’m skipping the network config for now and crossing my fingers that dhcp will do it’s job.

emerge dhcpcd

Also should install cron, syslog and grub:

emerge syslog-ng vixie-cron grub

rc-update add syslog-ng default

rc-update add vixie-cron default

Now I need to point grub to the kernel image that I built earlier.  Using nano I’ll want to edit /boot/grub/grub.conf.  Something like the following should work just fine for now as a grub.conf file:

default 0

timeout 10

title Gentoo

root (hd0, 0)

kernel /boot/bzImage root=/dev/hda3

After the file is saved, then I’ll run grub and walk through a few commands to write the bootloader to the MBR.  If I screw this up, I can just reboot from the live CD, chroot again and fix it up but let’s hope it just works right the first time:

grub

grub> root (hd0, 0)

grub> setup (hd0, 0)

grub> quit

Now for a final bit of housekeeping:

passwd

Time to see if all the hard work paid off.  I’m going to reboot and snag the CD out of the tray in the process:

exit

umount /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo

reboot

This next part is an account of MY troubleshooting process.  In most cases you will have your own set of problems that are different from mine.

DOH!  Kernel panic.  For some reason, my hard drive that was being detected as /dev/hda when I was installing is now being detected as /dev/sda so the boot loader passes the wrong root parameter to the kernel.  I suspect this is a problem with the way I built the kernel.  For now, I want to get the box running.  Time to go back to the live cd, boot and chroot.  First off, I’m going to install lilo since I’m a bit oldschool and I’m more familiar with it.

emerge lilo

mv /etc/lilo.conf.example /etc/lilo.conf

I edited the lilo.conf to point it at /dev/hda for the spot to write the bootloader but then pointed the root partition to /dev/sda3.  When I tried running lilo, it crapped out with a fatal error since /dev/sda didn’t exist when booted on the live cd.  I can do a quick, ugly thing to fix that:

ln -s /dev/hda /dev/sda

lilo

This successfully wrote the Gentoo option to the MBR.  When I tried to boot however, I still got a kernel panic so I rebooted again and typed the following at the lilo prompt:

gentoo root=”/dev/sda3″ boot=”/dev/sda1″

That seemed to work and the system finally booted!  Not ideal but now I’m to a point where I can troubleshoot without the hassle of using the live CD and chrooting.  Next I edited the lilo.conf to show /dev/sda as the boot device and reran “lilo” at the prompt.  When all of this was done it was time for a reboot and this time everything came up perfectly.

My first task with this newly working system was to bring it up to an entirely current, updated state.

emerge –sync

time emerge -u world

The picture speaks to the fact that this is an old, slow computer but the operation was successful in the end.

Final words

How many other modern, current and fully patched operating systems would legitimately be able to run on this computer?  Not many.  I’m not sure why Haiku wouldn’t run but it may have just been circumstantial.  Installing Gentoo was a bit of an adventure but using Gentoo makes Debian seem both bloated and outdated.

If you have been a casual Linux user and you want to learn WAY more about Linux, installing Gentoo is an excellent way to achieve that goal.  If you are a developer/hacker who always needs the latest and greatest packages at their disposal and doesn’t want to deal with dependency hell, you probably already use Gentoo.

I am currently searching for a 486/66 to add to my small cluster of old hardware.  After I find one, I will probably attempt to install Gentoo on that hardware and I expect to be successful with that endeavor.

Sometimes you come across a need to troubleshoot a misbehaving script or program things roll by on the screen too quickly.  The other day, I was troubleshooting a script that was blurting out an error and then continuing to open up a “dialog” screen which redraws the entire screen and clears the error.  I needed a way to send just the errors to a logfile.  If you are familiar with C programming, you’ll realize that there is STDOUT which writes to the console.  There is also STDERR which usually writes to the console but cannot be captured with a standard redirect such as:

program 1> logfile.txt

You’ll capture the STDOUT of that program but not any errors.  If you want just the errors, you’ll want to use:

program 2> logfile.txt

If you want to grab both the errors and the STDOUT, you can use:

program &> logfile.txt

Have you ever wondered what to do with that obscure long number that is sometimes seen posted with files that you download?  If you have ANY suspicion on whether a file is legitimate or not, you should use that number to make sure you are getting the file from the correct source.  Other than checking for trojan’d files and other dirty deeds you can also check to make sure you downloaded the WHOLE file.  Why waste a DVD by burning a corrupted ISO?  It’s easy enough to run the checksum so that you don’t have to guess if the file is good or not.  Open up a terminal and type:

md5 filename

See?  It’s dead simple.  Just make sure the number matches the one that is posted on the trusted site where you downloaded the file.  If you want to get really fancy with the process, there is an automator script available from apple called MD5 Checksum 1.0 that allows you to right click and check any file right in the finder.

Lynx text-based web browser. It's not as complicated as it looks.

So you’ve installed Macports now and probably met your needs for the moment.  Did you ever wonder what else was available as a port?  Here is a short list of some other gems:

John the ripper – This is a password cracker.  It’s highly flexible.  Very handy for testing the strength of your own passwords or recovering a password in certain instances.  For instance, I’ve used it when administrating a web server when I have forgotten the password to a protected directory but I still have ssh or ftp access.  I’ll just grab the .htpasswd file and run it through John the Ripper and presto, it will give me one or more passwords.  It’s also good as a sysadmin if you want to make sure people are using good passwords.  You can run it against the password file, see which user’s passwords are returned and notify them that they need to set something stronger.  The port package is simply called john.

wget – Wget is one of my favorite handy little command line utilities.  All it does is downloads something from the internet and saves it.  This can be an html page or any file available on the web.  I use it when I want to download a tarball into a specific directory.  Instead of thumbing through directory trees on firefox, I’ll just copy and paste the URL into a shell and wget it.  Then it’s exactly where I need it and ready to be untarred.  The port package is called wget.

lynx – If you’ve never used Lynx before, now may not be the time to start.  Lynx is a command line, text-based web browser.  Before the days of Mosaic and Internet Explorer 1.0, Lynx was the browser of choice for early users of the web.  Some folks even snobbed the newer browsers reasoning that pictures just messed up the overage web experience.  Lynx is useful for when you are ssh’ing into your machine remotely and need to grab/install a file off the web or just do something similar quickly.  It supports cookies and many other modern features surprising enough.  The port package is called lynx.

There are TONS of other good packages out there.  Some other honorable mentions are bzip2, hexedit, hping3, nmap & tcpdump.  Feel free to post in the comments whatever cool packages you have found available as a Macport.

When you install Ubuntu, it doesn’t give you the option of creating a root account.  Only a user.  Ubuntu is more desktop oriented so it pretends to not actually have a root account.  This only works so long as you don’t legitimately need it for something.  If you do, there is a simple trick to get a root shell.

Pop open a terminal window and type:

sudo passwd

You will be prompted for your current account’s password.  Next you will be prompted to set the root password and confirm it.  Now type:

su

You should be prompted for the password you just created and then you will be handed a root shell for your efforts.

Powered by WordPress. Theme: Motion by 85ideas.