Bonjour / avahi / zeroconf

I use a few Raspberry Pi on my network. I gave them proper hostnames. I could map the DNS and all that, but I simply have avahi running on them, so I can access them with hostname.local.

Multicast

For OpenIndiana to support zeronconf, all that is needed is this:

fdion@p490s:~$ su
Password:
fdion@p490s:~# vi /etc/nsswitch.conf

[here, I just added mdns:

hosts:      files mdns dns
ipnodes:    files mdns dns

]
fdion@p490s:~# svcadm enable svc:/network/dns/multicast:default
fdion@p490s:~# exit

From that point on, I can simply use:

fdion@p490s:~$ ssh -X rpi01.local

And of course, I had to configure Avahi on the Raspberry Pi.

This also works with my Macs, I can use ssh mac01.local for example.

Virtual Box with SMF control

SMF

The Service Management Facility (SMF), first introduced in Solaris Nevada, as project Greeline, and then later in the commercial Sun Solaris, is the modern way to manage system and application services in a self healing, fault tolerant way. It replaces the old init mechanism.

Some historical background information (including all the PSARCs) on SMF is still available at opensolaris.org.

Another good read is the Joyent wiki.

VirtualBox

Alexandre Dumont published some scripts for Virtual Box, to integrate with SMF. That was in 2009. Not too long ago, I was trying to find his blog and it was down.

I just stumbled upon:
https://forums.virtualbox.org/viewtopic.php?f=11&t=33249

And there is a sourceforge project for it, here:
http://sourceforge.net/projects/vboxsvc/

Installation

From the README, using the pkgadd in a global zone:

    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgadd -d /tmp/x -G

You probably want the -G flag. It doesn't block you from manually installing
the same package in a certain local zone where you'd use VirtualBox, but
it blocks automatic package propagation to those local zones which are
not expected to use and run VirtualBox. For us these zones are rare,
zero or one per machine (there is no definite/hardcoded limit though). YMMV.

To update the package you can simply remove the old version and install
anew, i.e.:
    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgrm COSvboxsvc
    # pkgadd -d /tmp/x -G

A cleaner way is to use an admin file to overwrite an existing package,
i.e. one from LiveUpgrade:
    # gzcat COSvboxsvc-0.16.pkg.gz > /tmp/x
    # pkgadd -d /tmp/x -G -a /etc/lu/zones_pkgadd_admin

Also note that this package "depends" on SUNWvbox, so that should be
installed beforehand.

Tribblix

 

 

OpenSolaris

When Solaris was open sourced, several distributions came to life from the OpenSolaris source code. Most of them however relied on closed bits provided by Sun. Then Oracle came and put Han Solo in a carbonite... err, wrong story. But you get the point.

With the emergence of Illumos (opensolaris with 100% open bits), we are starting to see again a surge in number of distributions based on it, and this is great news.

Now, there are several focusing on storage, and of course that is a natural application for an operating system based on Solaris technologies, such as ZFS.

But I want a desktop. I'm currently using OpenIndiana, and it works fine for my needs, better than any other OS on the planet, due to the fact I need to manage largish projects on my desktop. One in particular has over 3/4 million lines of code. Zones (clones) and snapshots are a must to tackle that. And dtrace to debug. But if you've ever used Solaris 10 or above you already know that.

Tribblix

Today, we are looking briefly at Tribblix.

The homepage is at http://www.tribblix.org/

What's in the name? Peter Tribble answers: "

Well, while it may seem rather vain, if I were to choose a name from scratch I would probably have come up with some obscure solar-powered pun. But at one of the OpenSolaris summits we took part in a Go Game, and one of the questions was:

Which of the following is not an OpenSolaris-derived distribution:

• Schillix
• Tribblix
• Belenix
• OpenSolaris 2009.06

I'm not sure that whoever set the question knew how close that was to the name of someone on the project, but that was how the name became planted in my consciousness."

Pretty funny... At that time, there was no Tribblix, but it sparked something, and Peter has now released Tribblix.

Speaking of Sparc

When asked on the illumos discussion list about availability of a Sparc version (Tribblix is x86), Peter indicated the following:

"

Do you have a SPARC ISO for your SVR4 distro, Peter?

Not yet. That shouldn't be too difficult. (It's much easier to develop
and test on x86, as I can repeatedly spin something up in VirtualBox
until I get it right. The differences in building the ISOs for sparc are
relatively small.)

As I recall, though, Martin said he had stripped out the IPS packaging
from his DVD, which I would need in order to convert all the packages."

Peter is referring to Martin Bochnig's own distribution that runs on Sparc systems. I will cover that one in the next blog post.

ISO Please

It is to be found here: http://www.tribblix.org/tribblix-0m1.iso

From OpenIndiana, it is a simple matter of issuing:

$ cdrw -i tribblix-0m1.iso

From any other OS, use your favorite method to burn an ISO. This is a CD image, btw, something rare nowadays.

Install

The installation process is relatively quick, but it is for people who are well versed in the Solaris OS. After choosing your country and language, you'll have to play with format and fdisk before starting the install process.

Instead of going through the steps here I will point you to Peter's install guide.

One thing I'll point out, when you are about to run the install script, make sure you type the following:

/.cdrom/live_install.sh c1t0d0s0 x11 retro-desktop develop xfce

That way, you install x11 etc. Else, you are 100% command line.

Desktop

When you boot Tribblix, you end up with a text login prompt. This is also the approach Raspbian are taking for the Raspberry Pi. I like that.

You login as jack initially (just as you would login as pi on a Raspbian distro). Then you can either startx, which will give you a spartan desktop called TWM. For some that might be enough.

TWM: Spartan and fast

You can also run wmaker.inst to modify your startx to use WindowMaker. I ran this with 512MB of ram, and it runs, a little tight.

WindowMaker

 XFCE

Peter did set it up so you could also start XFCE effortlessly. Instead of typing the typical startx,you would:

$ startxfce4

XFCE4: light yet full featured

What is interesting is that with 512MB of RAM, I still have 108MB free on the above screenshot.

There is potential thus to port this over to the Raspberry Pi. The Pi uses an ARM11 (ARM V6) processor, but OpenSolaris had been ported to ARM in the past: http://hub.opensolaris.org/bin/view/Project+osarm/installation. Another real plus to a port to the Raspberry Pi is that the hardware is pretty much set in stone, so no need for a bazillion sound and video drivers, for example.

Anyway, this concludes our brief review of Tribblix. It is very much a work in progress, but it shows that some interesting stuff can happen still with regards to Solaris and derived distributions.

That is so wrong...

At some point, somehow, somebody thought this would be a good idea...

Lots of new solaris

Oracle Solaris 11.1

First off, we will start with Oracle's new Solaris: Solaris 11.1

So, OK, it is not a brand spanking new OS, but it is an update.

According to Oracle, the new key features are:

• 8x faster database startup and shutdown and online resizing of the database SGA 
• Kernel Mode Acceleration for Oracle Real Application Clusters (Oracle RAC)
• 4x faster Oracle Solaris Zones updates
• Oracle Solaris Zones on Shared Storage

Also, some other new features:

New Logging Daemon

rsyslog, the popular daemon for message logging, has been included in Oracle Solaris 11.1. It is not enabled by default, but administrators can switch to this new logging daemon by disabling svc:/system/system-log:default and enabling svc:/system/system-log:rsyslog using SMF administrative utilities.

Simple SMF Manifest Creation Tool

The svcbundle(1M) command makes it easier to generate SMF manifests and profiles. By providing a few options on the command line, the user can generate a wide variety of manifests and profiles. This relieves the user from the burden of writing XML files.

File System Statistics for Oracle Solaris Zones

Until now, global zone administrators had no way of monitoring activity in a particular zone, nor could non-global zone administrators monitor fstype activity in their own zones. This update provides a per-fstype kstat (kernel statistic) for each zone. The global zone also has a kstat exclusively reporting its activity.

But most of the new features of interest are in the networking stack:

Edge Virtual Bridging (EVB)

EVB extends network virtualization features into the physical network infrastructure allowing users to manage bandwidth and to increase utilization of network resources. EVB is an IEEE standard that defines new protocols and mechanisms for exchanging information about virtual
links between hosts/stations and a switch/bridge. Support for the EVB Virtual Station Interface (VSI) Discovery and Configuration Protocol (VDP) and the Edge Control Protocol (ECP) are new in this release.

Using EVB, properties (such as the bandwidth limit) of the Oracle Solaris Virtual NIC (VNIC) can be exchanged with the switch so that the switch can be configured accordingly for the VNIC.

The standard is defined in the IEEE 802.1Qbg specification. Additional information can be obtained from evb(7P), vdpd(1M), and dladm(1M).

Data Center Bridging (DCB)

DCB allows the same network fabric to be used for both Ethernet and storage traffic reducing overall infrastructure costs in a datacenter. Support for enhanced transmission selection (ETS) based on IEEE 802.1Qaz has been added to the DCB functionality introduced in Oracle Solaris 11 11/11. This will provide guaranteed bandwidth and lossless Ethernet transport for
converged network environments where storage protocols share the same fabric as regular network traffic.

Oracle Solaris Link Aggregation to Span Across Multiple Switches

Link aggregation support in Oracle Solaris 11, being 802.3ad–compliant, does not allow aggregations to span across multiple switches. Thus, an administrator can either work with a single switch configuration (availability might suffer since the switch becomes a single point of failure) or use vendor-specific mechanisms to explicitly configure switches. This feature extends Oracle Solaris link aggregation to provide for a solution that is
independent of the switch vendor to span aggregations across multiple switches.

VNIC Migration

Associations between Physical NICs (PNICs) and Virtual NICs (VNICs) can now be changed without an outage in the network configuration. For example, if a PNIC becomes saturated, a VNIC associated with one key application can be migrated across to another PNIC (one with available bandwidth) without taking an outage.

But, I'm not here to just talk about the now closed source Solaris. There is a lot going on in the open source front too. The next two articles will demonstrate that clearly.

The Sidekick

Wrapping up the week, I just wanted to mention The Sidekick. No, not that phone thing. This one (raspberry-python blog). And the follow up article.

The sidekick is my desktop's sidekick. A secondary computer that complements my workstation. Not in term of speed or anything like that. But in term of adding a whole world of possibilities to interface with all kinds of stuff. Like a vinyl cutter, or a barometer.

Pi. Raspberry Pi

Quite the sidekick this little computer. If you've never heard of it, head out to Raspberrypi.org and read up on it, check the FAQ. Then come back to the Raspberry Pi Python Adventures to learn more.


[edit] forgot to publish this, so here you go.

ZFS send with built in buffer ( mbuffer ?)

Currently attending the live video feed from ZFS day. Somebody made the suggestion to add buffering to zfs send to resolve performance and mismatch issues.

Sounds like a great idea to me.

Over the years, in the past, mbuffer has been suggested as the solution, but not everyone is getting success with that:


[zfs-discuss] zfs send to remote any ideas for a faster way than ssh?

Alasdair offers this recipe with mbuffer that works for some. I'm thinking OpenIndiana specific, because it aborts using some other OSes.


Anyway, zfs send / receive should just work out of the box, so again, a great idea.

NTFS for OpenIndiana

In case you weren't aware, check out NTFS-3G on OpenIndiana. The support forum is here if you have any issues to report. It should just work, though.

OpenIndiana workstation, OI_151 a year later

Well, coincidentally, I just noticed that I've had OpenIndiana (OI 151) on my main workstation for a whole year exactly today (September 14th 2011, and today is September 14th 2012).

I also find that although it can multiboot (from the OI grub) Windows 7 and Linux Mint beside OI, I've not really spent any time in these other OSes. Just no need. I'll spend some time on the Mac workstation (my 8th Mac since the 80s), but mostly for XCode.

Outside of XCode though, I get back to OI almost immediately. The Mac, like my VMWare server (with various OSes) is used mostly as part of my build system, driven by Jenkins CI. And I got lazy to walk to the Mac, so I actually access it with VNC:

The VNC window is 1600x1200, that gives an idea of how spacious a 2560 x 1440 desktop really is. The monitor on the OpenIndiana workstation is a Catleap Q270 which uses the same LG IPS panel as the Apple 27", but without the awful super glossy. It still has a bit of reflection, but it is low gloss (I dont know of any matte version of that panel). It replaced two 20" 1600x1200 in a xinerama 3200x1200 desktop. I needed more vertical space, I can see more code at once like that.

In case anybody needs it, the Q270 doesn't provide the correct edid info, so you have to hardwire one in the /etc/X11/xorg.conf:

Section "Screen"
    Option "CustomEDID" "DFP-0: /etc/X11/edid-q270.bin"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
#    Option "UseEdid" "False"
#    Option "ExactModeTimingsDVI" "True"
    Option         "TwinView" "0"
    Option         "metamodes" "nvidia-auto-select +0+0"
    SubSection     "Display"
        Depth       24
        Modes       "2560x1440"
    EndSubSection
EndSection

and the edid as a hex dump:

00 FF FF FF  FF FF FF 00  04 62 9B 04  00 00 00 00
28 15 01 03  A5 3C 22 78  22 6F B1 A7  55 4C 9E 25
0C 50 54 00  00 00 01 01  01 01 01 01  01 01 01 01
01 01 01 01  01 01 56 5E  00 A0 A0 A0  29 50 30 20
35 00 55 50  21 00 00 1A  00 00 00 FC  00 51 48 44
32 37 30 0A  20 20 20 20  20 20 00 00  00 FC 00 0A
20 20 20 20  20 20 20 20  20 20 20 20  00 00 00 FC
00 0A 20 20  20 20 20 20  20 20 20 20  20 20 00 7E
0A 

You'll have to convert the hexdump back to bin and put it in /etc/X11/edid-q270.bin (or whatever you put in the xorg.conf). Bug me in the comments if you need the actual bin file, I'll post it somewhere.

I'll leave you with one of my favorite wallpapers (I have a bunch of wallpapers on autorotation, and different for each workspace):

Netscape, Sun, Oracle. No, Heliod web server

Back in the days, the netscape web server (at some point branded iPlanet) was the reference for web serving. Of course, came Apache (a collection of patches to NSCA's HTTPs hence: a "patchy" server...). We are talking the 90s here to put things in perspective. I stuck to Apache for a while while IIS was being deployed all around me.

iPlanet

Although I felt a special attachment to Apache, knowing personally some of the original contributors (like Brian Behlendorf), I started doing speed comparisons, benchmarks, and the best performance I could get on my servers was from the iPlanet server. In the early 2000s this was known as Sun One Web Server and that's when I switched. Then the Java buzzword started infecting everything, including that web server, and ended up with the name Sun Java System Web Server. The web server part had no Java in it, but the bundled app server module did of course, along with the fancy web interface that was generated by that module.

The story doesn't end there however. Somebody (thank you, whoever you are) figured that before something bad happened to Sun, it would be wise to open source the server. Hence appeared the Open Web Server. You can read Joerg's blog comment on this here at c0t0d0s0.org

I grabbed all the resources that I could related to it, in case, tried to compile it, but the barrier to entry was high in term of the number of libraries. Still, I figured that I'd wait for the project to get some momentum before switching.

How wise that proved to be to open source the server... Soon after, Oracle bought Sun, closed most open source projects, and rebranded yet again the web server, now calling it Oracle iPlanet Web Server. It is now almost impossible to download, as it is hidden in the Fusion middleware cluster on Oracle's cloud delivery. It is mostly pushed by Oracle as a simple reverse proxy for weblogic.

Performance

Of course, with Oracle's approach, one would probably be wise to jump ship. But on a server with sufficient RAM (if you have a very low RAM scenario, lighttpd works well), it is the best performing of the current breed. I'm not talking about Apache. After all, it is 20 years old, so I expect something more recent to be faster. No, I'm talking about faster than lighttpd, faster than Cherokee. So I started again down the path of getting a stable build of Open Web Server last month. I want the performance, but I'd like the flexibility of having the source to figure out specific issues.

Made my day

Today, I got a nice surprise in my email from Dave Koelmeyer to the OpenIndiana discussion list:

Not related to OpenIndiana, but I thought it interesting stumbling across
this today: the open source guts of the "Sun Java System" web server
product which Oracle apparently swept under the carpet (the open source
bits that is, not the proprietary, rebranded offering).

http://173.255.252.27/jyri/articles/index.php/web-server/

That's pretty exciting news! There's even a sourceforge page: http://sourceforge.net/projects/heliod/ The name is Heliod

The future

I think I'll be running this web server pretty soon. A web interface to manage the complex configuration would probably be a good addition to it for those that are new to it, and if I get some time, I'll write one in Python (with web.py). I'll probably propose that project to our local Python User Group (PYPTUG).

Glassfish 3.1.2

Have you installed Glassfish 3.1.2 on a Solaris variant? For one thing, it has SMF integration, that is, it creates (optionally) a service that can be controlled by svcadm. And then there are the functionalities that are available on all platforms. For example, clustering (both on the open source edition and oracle supported versions). So, basically, it is just a matter of downloading the file and then, as root (or sudo or pfexec depending how you are set up):

# sh glassfish-3.1.2-unix-ml.sh

Extracting the installer archive...

Extracting the installer runtime...

Extracting the installer resources...

Extracting the installer metadata...

Welcome to GlassFish installer

Using the user defined JAVA_HOME : /

Entering setup... SwixML 1.5 (#144)

This will bring up the usual GUI installer (assuming you are installing in the global zone, or you ssh -X to your zone). Make sure you select custom instead of express install. Now, this is pretty straightforward, and wouldn't merit a blog entry, but I wanted to bring up at least one thing that might stump you if you are installing in a solaris container / zone. The web admin (port 4848) expects localhost, as in http://localhost:4848, else it will error out. If we are in a zone, it is not too convenient. Sure, you could do an ssh -X zone_name and then run firefox from the zone:

# ssh -X server1

Oracle Corporation SunOS 5.11 11.0 May 2012

user@server1:~$ firefox -no-remote

This will launch firefox from the zone. Without the -no-remote, firefox will run in the global zone, and the login to the admin console will fail: Error Configuration Error Secure Admin must be enabled to access the DAS remotely. Again, you could do that... A better approach is to enable access through the host (zone) name instead of localhost. How?

cd /opt/glassfish3/bin

./asadmin --host server1 --port 4848 enable-secure-admin

Replace server1 by the hostname of the zone and you are almost there. For the change to take place, a restart of the app server is required. Easiest is reboot of the zone, but hold on! We have to make sure the service is enabled:

user@server1:~$ svcs domain1Service

STATE STIME FMRI disabled 13:35:17 svc:/application/GlassFish/domain1Service:default

user@server1:~$ su

root@server1:~# svcadm enable domain1Service

root@server1:~# svcs domain1Service

STATE STIME FMRI offline* 15:15:10 svc:/application/GlassFish/domain1Service:default

root@server1:~# svcs domain1Service

STATE STIME FMRI maintenance 15:15:19 svc:/application/GlassFish/domain1Service:default

The first time around it will be in maintenance, because glassfish is already running. We issue a reboot command in the zone and once it comes back up:

user@server1:~$ svcs domain1Service

STATE STIME FMRI online 15:18:11 svc:/application/GlassFish/domain1Service:default

That is it. We can now access http://server1:4848