LX zones back in illumos and OpenIndiana?

Branded Zones

If you've used Solaris and typed man lx, you saw:

Standards, Environments, and Macros lx(5)NAME

lx - Linux branded zone

DESCRIPTION


The lx brand uses the branded zones framework described in
brands(5) to enable Linux binary applications to run unmodi-
fied on a machine with a Solaris Operating System kernel.

The lx brand includes the tools necessary to install a Cen-
tOS 3.x or Red Hat Enterprise Linux 3.x distribution inside
a non-global zone. The brand supports the execution of 32-
bit Linux applications on x86/x64 machines running the
Solaris system in either 32-bit or 64-bit mode.

Supported Linux Distributions
The lx brand emulates the system call interfaces provided by
the Linux 2.4.21 kernel, as modified by Red Hat in the RHEL
3.x distributions. This kernel provides the system call
interfaces consumed by the glibc version 2.3.2 released by
Red Hat.

In addition, the lx brand partially emulates the Linux /dev
and /proc interfaces.


But if you've followed the story, lx branded zones became a mere historical reference. Until now.

Reintroduction

David Mackay posted the following webrev to the illumos developer mailing list:

http://cr.illumos.org/~webrev/DavidJX8P/lx-zones-restoration/


This is great, and I hope this gets reintegrated and pushed in the next openindiana!

François
@f_dion

Virtual Box with SMF control

SMF

The Service Management Facility (SMF), first introduced in Solaris Nevada, as project Greeline, and then later in the commercial Sun Solaris, is the modern way to manage system and application services in a self healing, fault tolerant way. It replaces the old init mechanism.

Some historical background information (including all the PSARCs) on SMF is still available at opensolaris.org.

Another good read is the Joyent wiki.

VirtualBox

Alexandre Dumont published some scripts for Virtual Box, to integrate with SMF. That was in 2009. Not too long ago, I was trying to find his blog and it was down.

I just stumbled upon:
https://forums.virtualbox.org/viewtopic.php?f=11&t=33249

And there is a sourceforge project for it, here:
http://sourceforge.net/projects/vboxsvc/

Installation

From the README, using the pkgadd in a global zone:

    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgadd -d /tmp/x -G

You probably want the -G flag. It doesn't block you from manually installing
the same package in a certain local zone where you'd use VirtualBox, but
it blocks automatic package propagation to those local zones which are
not expected to use and run VirtualBox. For us these zones are rare,
zero or one per machine (there is no definite/hardcoded limit though). YMMV.

To update the package you can simply remove the old version and install
anew, i.e.:
    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgrm COSvboxsvc
    # pkgadd -d /tmp/x -G

A cleaner way is to use an admin file to overwrite an existing package,
i.e. one from LiveUpgrade:
    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgadd -d /tmp/x -G -a /etc/lu/zones_pkgadd_admin

Also note that this package "depends" on SUNWvbox, so that should be
installed beforehand.

Netscape, Sun, Oracle. No, Heliod web server

Back in the days, the netscape web server (at some point branded iPlanet) was the reference for web serving. Of course, came Apache (a collection of patches to NSCA's HTTPs hence: a "patchy" server...). We are talking the 90s here to put things in perspective. I stuck to Apache for a while while IIS was being deployed all around me.

iPlanet

Although I felt a special attachment to Apache, knowing personally some of the original contributors (like Brian Behlendorf), I started doing speed comparisons, benchmarks, and the best performance I could get on my servers was from the iPlanet server. In the early 2000s this was known as Sun One Web Server and that's when I switched. Then the Java buzzword started infecting everything, including that web server, and ended up with the name Sun Java System Web Server. The web server part had no Java in it, but the bundled app server module did of course, along with the fancy web interface that was generated by that module.

The story doesn't end there however. Somebody (thank you, whoever you are) figured that before something bad happened to Sun, it would be wise to open source the server. Hence appeared the Open Web Server. You can read Joerg's blog comment on this here at c0t0d0s0.org

I grabbed all the resources that I could related to it, in case, tried to compile it, but the barrier to entry was high in term of the number of libraries. Still, I figured that I'd wait for the project to get some momentum before switching.

How wise that proved to be to open source the server... Soon after, Oracle bought Sun, closed most open source projects, and rebranded yet again the web server, now calling it Oracle iPlanet Web Server. It is now almost impossible to download, as it is hidden in the Fusion middleware cluster on Oracle's cloud delivery. It is mostly pushed by Oracle as a simple reverse proxy for weblogic.

Performance

Of course, with Oracle's approach, one would probably be wise to jump ship. But on a server with sufficient RAM (if you have a very low RAM scenario, lighttpd works well), it is the best performing of the current breed. I'm not talking about Apache. After all, it is 20 years old, so I expect something more recent to be faster. No, I'm talking about faster than lighttpd, faster than Cherokee. So I started again down the path of getting a stable build of Open Web Server last month. I want the performance, but I'd like the flexibility of having the source to figure out specific issues.

Made my day

Today, I got a nice surprise in my email from Dave Koelmeyer to the OpenIndiana discussion list:

Not related to OpenIndiana, but I thought it interesting stumbling across
this today: the open source guts of the "Sun Java System" web server
product which Oracle apparently swept under the carpet (the open source
bits that is, not the proprietary, rebranded offering).

http://173.255.252.27/jyri/articles/index.php/web-server/

That's pretty exciting news! There's even a sourceforge page: http://sourceforge.net/projects/heliod/ The name is Heliod

The future

I think I'll be running this web server pretty soon. A web interface to manage the complex configuration would probably be a good addition to it for those that are new to it, and if I get some time, I'll write one in Python (with web.py). I'll probably propose that project to our local Python User Group (PYPTUG).

Glassfish 3.1.2

Have you installed Glassfish 3.1.2 on a Solaris variant? For one thing, it has SMF integration, that is, it creates (optionally) a service that can be controlled by svcadm. And then there are the functionalities that are available on all platforms. For example, clustering (both on the open source edition and oracle supported versions). So, basically, it is just a matter of downloading the file and then, as root (or sudo or pfexec depending how you are set up):

# sh glassfish-3.1.2-unix-ml.sh

Extracting the installer archive...

Extracting the installer runtime...

Extracting the installer resources...

Extracting the installer metadata...

Welcome to GlassFish installer

Using the user defined JAVA_HOME : /

Entering setup... SwixML 1.5 (#144)

This will bring up the usual GUI installer (assuming you are installing in the global zone, or you ssh -X to your zone). Make sure you select custom instead of express install. Now, this is pretty straightforward, and wouldn't merit a blog entry, but I wanted to bring up at least one thing that might stump you if you are installing in a solaris container / zone. The web admin (port 4848) expects localhost, as in http://localhost:4848, else it will error out. If we are in a zone, it is not too convenient. Sure, you could do an ssh -X zone_name and then run firefox from the zone:

# ssh -X server1

Oracle Corporation SunOS 5.11 11.0 May 2012

user@server1:~$ firefox -no-remote

This will launch firefox from the zone. Without the -no-remote, firefox will run in the global zone, and the login to the admin console will fail: Error Configuration Error Secure Admin must be enabled to access the DAS remotely. Again, you could do that... A better approach is to enable access through the host (zone) name instead of localhost. How?

cd /opt/glassfish3/bin

./asadmin --host server1 --port 4848 enable-secure-admin

Replace server1 by the hostname of the zone and you are almost there. For the change to take place, a restart of the app server is required. Easiest is reboot of the zone, but hold on! We have to make sure the service is enabled:

user@server1:~$ svcs domain1Service

STATE STIME FMRI disabled 13:35:17 svc:/application/GlassFish/domain1Service:default

user@server1:~$ su

root@server1:~# svcadm enable domain1Service

root@server1:~# svcs domain1Service

STATE STIME FMRI offline* 15:15:10 svc:/application/GlassFish/domain1Service:default

root@server1:~# svcs domain1Service

STATE STIME FMRI maintenance 15:15:19 svc:/application/GlassFish/domain1Service:default

The first time around it will be in maintenance, because glassfish is already running. We issue a reboot command in the zone and once it comes back up:

user@server1:~$ svcs domain1Service

STATE STIME FMRI online 15:18:11 svc:/application/GlassFish/domain1Service:default

That is it. We can now access http://server1:4848

Solaris Express Developer Edition (2/07)

Well, it is finally available. I had mentionned before on SunQuebecthat the next release of Solaris Express would be interesting, and the reason is: the DVD now includes Sun Studio 11 and Netbeans 5.5. So it is now a complete environment.

It can be found here:
http://developers.sun.com/solaris/downloads/solexpdev/

This build also includes firefox 2, star office 8, and Sun is trying to appeal to LAMP developers with SAMP (Solaris, Apache, MySQL et PHP). Sun Web Server 7 is available and Postgresql has been included for a while now, so it's not a huge addition, but it makes for a pretty complete distribution.

Currently it is only x86/x64, but you can get equivalent functionality under Sparc with Solaris Express Community Edition B55 and above. I have B56, but B57 is now available.

The next Community Edition to keep an eye out for will be B60. This will include bootable ZFS for x86/x64, and I would guess it might appear in Solaris Express 4/07 if I understand the revision cycle, altough I now read that Solaris Express will now be quarterly so that might be 5/07.

François

Secure By Default

Learn to love SBD.

When doing the initial install of Solaris 10, select the secure by default (SBD) option. If this is Solaris Nevada, no option is given, it will be (mostly) secure by default. If the install is an upgrade, then it will not change the previous install. To activate the secure profile, run the following:

# /usr/sbin/netservices limited

Note that this will only allow local smtp and rpc, and only ssh remotely, so be careful if you are on a production box, everything else is tight.

Why should you care? Because you are safe from the latest Solaris 10 hack. If you haven't heard of it yet, you might want to search the web and learn more. Basically, it enables an attacker to telnet into a Solaris 10 machine.

I decided to head over the OpenGrok OpenSolaris source code browser, and look at login.c:
In login.c:

524 /* ONC_PLUS EXTRACT START */
525 /*
526 * validate user
527 */
528 /* we are already authenticated. fill in what we must, then continue */
529 if (fflag) {
530 /* ONC_PLUS EXTRACT END */
531 if ((pwd = getpwnam(user_name)) == NULL) {
532 audit_error = ADT_FAIL_VALUE_USERNAME;
533
534 log_bad_attempts();
535 (void) printf("Login failed: unknown user '%s'.\n",
536 user_name);
537 login_exit(1);
538 }
539 /* ONC_PLUS EXTRACT START */
540 } else {
541 /*
542 * Perform the primary login authentication activity.
543 */
544 login_authenticate();
545 }

So as long as the f flag is set, it never does the login_authenticate(). If that option was somewhat injected in a string...

1399 case 'f':
1400 /*
1401 * Must be root to bypass authentication
1402 * otherwise we exit() as punishment for trying.
1403 */
1404 if (getuid() != 0 || geteuid() != 0) {
1405 audit_error = ADT_FAIL_VALUE_AUTH_BYPASS;
1406
1407 login_exit(1); /* sigh */
1408 /*NOTREACHED*/
1409 }
1410 /* save fflag user name for future use */
1411 SCPYL(user_name, optarg);
1412 fflag = B_TRUE;
1413 break;

Quite the punishment for trying, no? This is similar to an old rlogin issue.

Sun Studio now has a thread analyzer that does race detection and other similar complex detection of problems in your code. Maybe security hole detection should be next? :)

So why did this come back?

54 * -f : This flag was introduced by PSARC 1995/039 in support
55 * of Kerberos. But it's not used by Sun's Kerberos implementation.
56 * It is however employed by zlogin(1), since it allows one to tell
57 * login: "This user is authenticated." In the case of zlogin that's
58 * true because the zone always trusts the global zone.

So if you are running Solaris secure by default and have not re-enabled telnet, you are safe.

Learn to love SBD.

______________________________________

Update:

For Tpatches and details from the sausage factory, see this blog:
Alan Hargreaves - the in.telnetd exploit

Mirrored Laptop with Compact Flash

An alternative to the USB stick to have a mirrored storage on a laptop is to use a PC card compact flash adapter, along with a compact flash card. These cards are available currently in capacities of up to 16GB, but are much more expensive than USB sticks.

For the purpose of demonstrating the concept, I used my old IBM microdrive (340MB) in CFII format. I think the largest ever made was 6 or 8GB, but with solid state cards capable of higher density, that is probably the end of the line for that technology. Still, I've had it for about 7 years now, so it has proven surprisingly reliable (unlike the iPaq this was in).

So, all you have to do is insert the PC card and Solaris hald (Solaris Nevada B56) picks up the microdrive, mounts it and is shown on the desktop.

bash-3.00# df -h
Filesystem size used avail capacity Mounted on
[...]
/dev/dsk/c1t0d0s2 3.6G 3.6G 0K 100% /media/SOL_11_X86
/dev/dsk/c8d0p0:1 341M 16K 341M 1% /media/MICRODRIVE

I then unmounted the microdrive. Assuming you have a free slice 7 on your internal hard disk (make sure you have an empty, pre-allocated slice), then you would do:

bash-3.00# zpool create internal mirror c0d0s7 c8d0p0
invalid vdev specification
use '-f' to override the following errors:
mirror contains devices of different sizes
bash-3.00# zpool create -f internal mirror c0d0s7 c8d0p0

bash-3.00# df -h
Filesystem size used avail capacity Mounted on
[...]
/dev/dsk/c1t0d0s2 3.6G 3.6G 0K 100% /media/SOL_11_X86
internal 304M 24K 304M 1% /internal

bash-3.00# zpool status
pool: internal
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
internal ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c8d0p0 ONLINE 0 0 0

errors: No known data errors


We can then proceed to set compression and mountpoint just like in the previous case we went through.

Stick to ZFS, or The laptop with a mirrored drive

My everyday laptop is a Dell Latitude D600. It's old but it has served me well. It started its life with Windows 2000 on its hard disk. I switched back and forth between Solaris 10 and Windows 2000 (Windows XP was just too frustrating for me and at some point I've triple booted Mandriva, Suse or Ubuntu), until I decided to go Solaris Express (aka Nevada, aka what will become 11) which is the base anyway for OpenSolaris. Since Nevada B53, I've never looked back. I removed all other partitions and dedicated it to Solaris.

Now, most laptop can only take one drive. This is the case with the Latitude.

Stick to ZFS

Assuming you have a slice available on your laptop hard disk and you have a memory stick, a lot of fun can be had with ZFS.

This came about as I was pondering about making my laptop a little more safe for travelling. If I lost a hard drive, I could buy a new one and then just reinstall Solaris. The latest Solaris (B56) even has a Sun Studio and Netbeans when you select Solaris Express Developer at install time. Email, source and other stuff are remotely stored on a server etc. The only issue is all my documents that I'm saving in the local user/Documents folder (I am not always online). Yes, I do a backup to an external USB device, but I would like for this to be transparent. I tought, ZFS could do this, along with a USB stick. There are some sticks that are so small, you can leave them in the USB port when putting the laptop in its bag.

The laptop with a mirrored drive.

So here is the recipe. I insert my memory stick and check its logical node:

bash-3.00# rmformat
Looking for devices...
1. Logical Node: /dev/rdsk/c4t0d0p0
Physical Node: /pci@0,0/pci8086,4541@1d,1/storage@1/disk@0,0
Connected Device: KINGSTON USB DRIVE 1.12
Device Type: Removable
Bus: USB
Size: 246.0 MB
Label:
Access permissions:

Above is my test memory stick. My internal disk is c0d0:

bash-3.00# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c0d0
/pci@0,0/pci-ide@1f,1/ide@0/cmdk@0,0
^C

I already know that slice 7 is unassigned but allocated and free. If I didn't know, I'd select the drive in format, then enter the partition mode and print the partition table. If the boot disk partition needs to be edited, you'll need to do that by booting Solaris from CD/DVD.

So, let us create a mirror:

bash-3.00# zpool create -f stick mirror c0d0s7 c4t0d0p0
bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
stick 240M 89K 240M 0% ONLINE -
usb 298G 182G 116G 60% ONLINE -
bash-3.00# zfs create stick/Documents
bash-3.00# zfs set compression=on stick/Documents
bash-3.00# zfs set mountpoint=/export/home/fdion/Documents stick/Documents
bash-3.00# cp /export/home/fdion/mp3/PeteNamlook\ and\ MixMaster\ Morris/*.mp3 .

While the copy was going on (Pete Namlook and Mixmaster Morris' Dreamfish album), I pulled the usb stick. Copy completed without problem:

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
stick 97.9M 110M 24.5K /stick
stick/Documents 97.7M 110M 97.7M /export/home/fdion/Documents

bash-3.00# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
stick 240M 97.9M 142M 40% DEGRADED -
usb 298G 182G 116G 60% ONLINE -

The mirror is degraded tough. Let's check the full status:

bash-3.00# zpool status
pool: stick
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
stick DEGRADED 0 0 0
mirror DEGRADED 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 UNAVAIL 0 306 0 cannot open

errors: No known data errors

Let's bring it online:

bash-3.00# zpool online stick c4t0d0p0
Bringing device c4t0d0p0 online

This took a little bit of time as the mirror finished fixing itself. Am I done? Let's check the status:

bash-3.00# zpool status
pool: stick
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: resilver completed with 0 errors on Fri Feb 9 14:22:09 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 306 0

errors: No known data errors

bash-3.00# zpool clear stick c4t0d0p0
bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:22:09 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

Alright, everything is ok now. Just a recap of what I've done up to now:

bash-3.00# zpool history stick
History for 'stick':
2007-02-09.14:08:34 zpool create -f stick mirror c0d0s7 c4t0d0p0
2007-02-09.14:10:54 zfs create stick/Documents
2007-02-09.14:11:06 zfs set compression=on stick/Documents
2007-02-09.14:11:59 zfs set mountpoint=/export/home/fdion/Documents stick/Documents
2007-02-09.14:20:14 zpool online stick c4t0d0p0
2007-02-09.14:25:26 zpool clear stick c4t0d0p0

Pretty slick. Let's export and load this in another computer.

bash-3.00# zpool export stick
bash-3.00#


On another machine, I go and plug the memory stick in and:

bash-3.00# zpool import stick
cannot mount '/export/home/fdion/Documents': directory is not empty

Ah, yes, that's an issue, but we are just testing. I probably should have mounted a different user.

bash-3.00# zfs set mountpoint=/export/home/fdlaptop/Documents stick/Documents
bash-3.00# zpool export stick
bash-3.00# zpool import stick

Data is there, everything is fine. But, how does it look from ZFS standpoint?

bash-3.00# zpool status
pool: stick
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver completed with 0 errors on Fri Feb 9 14:38:56 2007
config:

NAME STATE READ WRITE CKSUM
stick DEGRADED 0 0 0
mirror DEGRADED 0 0 0
c0d0s7 UNAVAIL 0 0 0 cannot open
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

This is as expected. Let's export again:

bash-3.00# zpool export stick

I put the stick back in the laptop, then

bash-3.00# zpool import stick
bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:41:42 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
mirror ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0
c4t0d0p0 ONLINE 0 0 0

errors: No known data errors

Impressive, no?

bash-3.00# zfs list
NAME USED AVAIL REFER MOUNTPOINT
stick 97.9M 110M 24.5K /stick
stick/Documents 97.7M 110M 97.7M /export/home/fdlaptop/Documents

BTW, I did have compression enabled, but since I copied MP3 files, I expect no compression whatsoever:

bash-3.00# zfs get compressratio stick/Documents
NAME PROPERTY VALUE SOURCE
stick/Documents compressratio 1.00x -

Sure enough. I then copied 20M of office documents and pdfs and removed the mp3 files:

bash-3.00# zfs get compressratio stick/Documents
NAME PROPERTY VALUE SOURCE
stick/Documents compressratio 3.53x -

At 3.5x and small USB sticks available in 8GB, that's potentially 28GB mirrored! Currently, the 8 GB are a bit too high, but 4GB is <$80 (I've seen as low as $40). In the newer Solaris builds, a mirror can be reverted back to a non mirrored zone: bash-3.00# zpool detach stick c4t0d0p0

I pulled the stick out.

bash-3.00# zpool status
pool: stick
state: ONLINE
scrub: resilver completed with 0 errors on Fri Feb 9 14:41:42 2007
config:

NAME STATE READ WRITE CKSUM
stick ONLINE 0 0 0
c0d0s7 ONLINE 0 0 0

errors: No known data errors

I now am back to a non mirrored mode. To recreate:

bash-3.00# zpool attach stick c0d0s7 c4t0d0p0

The only issue is if the two devices are not exactly the same size (or the memory stick is smaller than the hard disk slice), you will get:

cannot attach c4t0d0p0 to c0d0s7: device is too small

Just something to keep in mind.