Repair, Upgrade or Replace

[I had decided to not publish this, but with the LX brand revived, I thought it would be an interesting story post]

The perpetual question

Repair, upgrade or replace? It doesn't matter if we are talking about an appliance, a car, furniture or a laptop. Whatever it is, it will break or become obsolete. It's all a question of scale and time, and outside pressures.

What do I mean by outside pressures? Let's say you have a car, it is yours and it is working fine. You drive it 30,000 miles each year. It does 20mpg (US). Let's say gas price is $3.50 a US gallon, that's a tad over $5000 a year to make it run (not counting normal maintenance). A lot of money to go from point A to point B, but still within reason. The outside pressure is minor. But let's bump up gas price to $9.00 a gallon, and we are now talking over $13,000 a year. $27,000 over two years. ROI on getting a new car with twice the mpg is now pretty obvious.

But what if there are no options

In our example above we assume there is such a thing as a 40mpg vehicule that can replace the one you have. But what if I told you it has to be able to tow 6000lbs? There might not be any option that fit the bill. So you have to compromise in one way or another. And that is where it gets complicated.

Lightweight, but not lightweight

If you spend times going from one place to another doing presentations, no doubt you've welcomed tablets and ultra portable laptops. To be able to demo a specific n-tier setup or even a whole datacenter directly on a 13" laptop that weighs less than 3.5 lbs is definitely a great thing.

I've been using for a few years a Sony Vaio Z that is totally custom. Large SSD, large drive, multi core. Ram was the limitation back then, 8GB. But anybody familiar with Solaris or (open source solaris) IllumOS based OSes knows that thanks to cpu and memory sharing between each zones and the full virtual networking capability of crossbow, you can simulate a lot of machines within that 8GB.

My outside pressure

Linux zones are no longer available, so I have to rely on virtualbox or other similar solutions. They dont share ressources such as memory and are not lightweight solutions like zones. 8GB is no longer acceptable. CPU or disk is not a problem (not dealing with big data, just terabyte scale).

Lack of solutions

For some reason there are very few lightweight laptops with > 8GB. 16GB seems to be the next barrier, but typically with a very low screen resolution. Why is that? I don't know. So, for right now I have to choice but to wait on technology to catch up, assuming market demand. [Or, now, hope to get LX branded zones back. If somebody was looking for a good business case, I'm giving you one]



François
@f_dion

Bonjour / avahi / zeroconf

I use a few Raspberry Pi on my network. I gave them proper hostnames. I could map the DNS and all that, but I simply have avahi running on them, so I can access them with hostname.local.

Multicast

For OpenIndiana to support zeronconf, all that is needed is this:

fdion@p490s:~$ su
Password:
fdion@p490s:~# vi /etc/nsswitch.conf

[here, I just added mdns:

hosts:      files mdns dns
ipnodes:    files mdns dns

]
fdion@p490s:~# svcadm enable svc:/network/dns/multicast:default
fdion@p490s:~# exit

From that point on, I can simply use:

fdion@p490s:~$ ssh -X rpi01.local

And of course, I had to configure Avahi on the Raspberry Pi.

This also works with my Macs, I can use ssh mac01.local for example.

Mirrored Laptop with Compact Flash

An alternative to the USB stick to have a mirrored storage on a laptop is to use a PC card compact flash adapter, along with a compact flash card. These cards are available currently in capacities of up to 16GB, but are much more expensive than USB sticks.

For the purpose of demonstrating the concept, I used my old IBM microdrive (340MB) in CFII format. I think the largest ever made was 6 or 8GB, but with solid state cards capable of higher density, that is probably the end of the line for that technology. Still, I've had it for about 7 years now, so it has proven surprisingly reliable (unlike the iPaq this was in).

So, all you have to do is insert the PC card and Solaris hald (Solaris Nevada B56) picks up the microdrive, mounts it and is shown on the desktop.

bash-3.00# df -h
Filesystem size used avail capacity Mounted on
[...]
/dev/dsk/c1t0d0s2 3.6G 3.6G 0K 100% /media/SOL_11_X86
/dev/dsk/c8d0p0:1 341M 16K 341M 1% /media/MICRODRIVE

I then unmounted the microdrive. Assuming you have a free slice 7 on your internal hard disk (make sure you have an empty, pre-allocated slice), then you would do:

bash-3.00# zpool create internal mirror c0d0s7 c8d0p0
invalid vdev specification
use '-f' to override the following errors:
mirror contains devices of different sizes
bash-3.00# zpool create -f internal mirror c0d0s7 c8d0p0

bash-3.00# df -h
Filesystem size used avail capacity Mounted on
[...]
/dev/dsk/c1t0d0s2 3.6G 3.6G 0K 100% /media/SOL_11_X86
internal 304M 24K 304M 1% /internal

bash-3.00# zpool status
pool: internal
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
internal ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c8d0p0 ONLINE 0 0 0

errors: No known data errors


We can then proceed to set compression and mountpoint just like in the previous case we went through.

Stick to ZFS, or The laptop with a mirrored drive

My everyday laptop is a Dell Latitude D600. It's old but it has served me well. It started its life with Windows 2000 on its hard disk. I switched back and forth between Solaris 10 and Windows 2000 (Windows XP was just too frustrating for me and at some point I've triple booted Mandriva, Suse or Ubuntu), until I decided to go Solaris Express (aka Nevada, aka what will become 11) which is the base anyway for OpenSolaris. Since Nevada B53, I've never looked back. I removed all other partitions and dedicated it to Solaris.

Now, most laptop can only take one drive. This is the case with the Latitude.

Stick to ZFS

Assuming you have a slice available on your laptop hard disk and you have a memory stick, a lot of fun can be had with ZFS.

This came about as I was pondering about making my laptop a little more safe for travelling. If I lost a hard drive, I could buy a new one and then just reinstall Solaris. The latest Solaris (B56) even has a Sun Studio and Netbeans when you select Solaris Express Developer at install time. Email, source and other stuff are remotely stored on a server etc. The only issue is all my documents that I'm saving in the local user/Documents folder (I am not always online). Yes, I do a backup to an external USB device, but I would like for this to be transparent. I tought, ZFS could do this, along with a USB stick. There are some sticks that are so small, you can leave them in the USB port when putting the laptop in its bag.

The laptop with a mirrored drive.

So here is the recipe. I insert my memory stick and check its logical node:

bash-3.00# rmformat
Looking for devices...
1. Logical Node: /dev/rdsk/c4t0d0p0
Physical Node: /pci@0,0/pci8086,4541@1d,1/storage@1/disk@0,0
Connected Device: KINGSTON USB DRIVE 1.12
Device Type: Removable
Bus: USB
Size: 246.0 MB
Label:
Access permissions:

Above is my test memory stick. My internal disk is c0d0:

bash-3.00# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c0d0
/pci@0,0/pci-ide@1f,1/ide@0/cmdk@0,0
^C

I already know that slice 7 is unassigned but allocated and free. If I didn't know, I'd select the drive in format, then enter the partition mode and print the partition table. If the boot disk partition needs to be edited, you'll need to do that by booting Solaris from CD/DVD.

So, let us create a mirror:

bash-3.00# zpool create -f stick mirror c0d0s7 c4t0d0p0
bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
stick 240M 89K 240M 0% ONLINE -
usb 298G 182G 116G 60% ONLINE -
bash-3.00# zfs create stick/Documents
bash-3.00# zfs set compression=on stick/Documents
bash-3.00# zfs set mountpoint=/export/home/fdion/Documents stick/Documents
bash-3.00# cp /export/home/fdion/mp3/PeteNamlook\ and\ MixMaster\ Morris/*.mp3 .

While the copy was going on (Pete Namlook and Mixmaster Morris' Dreamfish album), I pulled the usb stick. Copy completed without problem:

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
stick 97.9M 110M 24.5K /stick
stick/Documents 97.7M 110M 97.7M /export/home/fdion/Documents

bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
stick 240M 97.9M 142M 40% DEGRADED -
usb 298G 182G 116G 60% ONLINE -

The mirror is degraded tough. Let's check the full status:

bash-3.00# zpool status
pool: stick
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
stick DEGRADED 0 0 0
mirror DEGRADED 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 UNAVAIL 0 306 0 cannot open

errors: No known data errors

Let's bring it online:

bash-3.00# zpool online stick c4t0d0p0
Bringing device c4t0d0p0 online

This took a little bit of time as the mirror finished fixing itself. Am I done? Let's check the status:

bash-3.00# zpool status
pool: stick
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: resilver completed with 0 errors on Fri Feb 9 14:22:09 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 306 0

errors: No known data errors

bash-3.00# zpool clear stick c4t0d0p0
bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:22:09 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

Alright, everything is ok now. Just a recap of what I've done up to now:

bash-3.00# zpool history stick
History for 'stick':
2007-02-09.14:08:34 zpool create -f stick mirror c0d0s7 c4t0d0p0
2007-02-09.14:10:54 zfs create stick/Documents
2007-02-09.14:11:06 zfs set compression=on stick/Documents
2007-02-09.14:11:59 zfs set mountpoint=/export/home/fdion/Documents stick/Documents
2007-02-09.14:20:14 zpool online stick c4t0d0p0
2007-02-09.14:25:26 zpool clear stick c4t0d0p0

Pretty slick. Let's export and load this in another computer.

bash-3.00# zpool export stick
bash-3.00#


On another machine, I go and plug the memory stick in and:

bash-3.00# zpool import stick
cannot mount '/export/home/fdion/Documents': directory is not empty

Ah, yes, that's an issue, but we are just testing. I probably should have mounted a different user.

bash-3.00# zfs set mountpoint=/export/home/fdlaptop/Documents stick/Documents
bash-3.00# zpool export stick
bash-3.00# zpool import stick

Data is there, everything is fine. But, how does it look from ZFS standpoint?

bash-3.00# zpool status
pool: stick
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver completed with 0 errors on Fri Feb 9 14:38:56 2007
config:

NAME STATE READ WRITE CKSUM
stick DEGRADED 0 0 0
mirror DEGRADED 0 0 0
c0d0s7 UNAVAIL 0 0 0 cannot open
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

This is as expected. Let's export again:

bash-3.00# zpool export stick

I put the stick back in the laptop, then

bash-3.00# zpool import stick
bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:41:42 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

Impressive, no?

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
stick 97.9M 110M 24.5K /stick
stick/Documents 97.7M 110M 97.7M /export/home/fdlaptop/Documents

BTW, I did have compression enabled, but since I copied MP3 files, I expect no compression whatsoever:

bash-3.00# zfs get compressratio stick/Documents
NAME PROPERTY VALUE SOURCE
stick/Documents compressratio 1.00x -

Sure enough. I then copied 20M of office documents and pdfs and removed the mp3 files:

bash-3.00# zfs get compressratio stick/Documents
NAME PROPERTY VALUE SOURCE
stick/Documents compressratio 3.53x -

At 3.5x and small USB sticks available in 8GB, that's potentially 28GB mirrored! Currently, the 8 GB are a bit too high, but 4GB is <$80 (I've seen as low as $40). In the newer Solaris builds, a mirror can be reverted back to a non mirrored zone: bash-3.00# zpool detach stick c4t0d0p0

I pulled the stick out.

bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:41:42 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0

errors: No known data errors

I now am back to a non mirrored mode. To recreate:

bash-3.00# zpool attach stick c0d0s7 c4t0d0p0

The only issue is if the two devices are not exactly the same size (or the memory stick is smaller than the hard disk slice), you will get:

cannot attach c4t0d0p0 to c0d0s7: device is too small

Just something to keep in mind.