Netscape, Sun, Oracle. No, Heliod web server

Back in the days, the netscape web server (at some point branded iPlanet) was the reference for web serving. Of course, came Apache (a collection of patches to NSCA's HTTPs hence: a "patchy" server...). We are talking the 90s here to put things in perspective. I stuck to Apache for a while while IIS was being deployed all around me.

iPlanet

Although I felt a special attachment to Apache, knowing personally some of the original contributors (like Brian Behlendorf), I started doing speed comparisons, benchmarks, and the best performance I could get on my servers was from the iPlanet server. In the early 2000s this was known as Sun One Web Server and that's when I switched. Then the Java buzzword started infecting everything, including that web server, and ended up with the name Sun Java System Web Server. The web server part had no Java in it, but the bundled app server module did of course, along with the fancy web interface that was generated by that module.

The story doesn't end there however. Somebody (thank you, whoever you are) figured that before something bad happened to Sun, it would be wise to open source the server. Hence appeared the Open Web Server. You can read Joerg's blog comment on this here at c0t0d0s0.org

I grabbed all the resources that I could related to it, in case, tried to compile it, but the barrier to entry was high in term of the number of libraries. Still, I figured that I'd wait for the project to get some momentum before switching.

How wise that proved to be to open source the server... Soon after, Oracle bought Sun, closed most open source projects, and rebranded yet again the web server, now calling it Oracle iPlanet Web Server. It is now almost impossible to download, as it is hidden in the Fusion middleware cluster on Oracle's cloud delivery. It is mostly pushed by Oracle as a simple reverse proxy for weblogic.

Performance

Of course, with Oracle's approach, one would probably be wise to jump ship. But on a server with sufficient RAM (if you have a very low RAM scenario, lighttpd works well), it is the best performing of the current breed. I'm not talking about Apache. After all, it is 20 years old, so I expect something more recent to be faster. No, I'm talking about faster than lighttpd, faster than Cherokee. So I started again down the path of getting a stable build of Open Web Server last month. I want the performance, but I'd like the flexibility of having the source to figure out specific issues.

Made my day

Today, I got a nice surprise in my email from Dave Koelmeyer to the OpenIndiana discussion list:

Not related to OpenIndiana, but I thought it interesting stumbling across
this today: the open source guts of the "Sun Java System" web server
product which Oracle apparently swept under the carpet (the open source
bits that is, not the proprietary, rebranded offering).

http://173.255.252.27/jyri/articles/index.php/web-server/

That's pretty exciting news! There's even a sourceforge page: http://sourceforge.net/projects/heliod/ The name is Heliod

The future

I think I'll be running this web server pretty soon. A web interface to manage the complex configuration would probably be a good addition to it for those that are new to it, and if I get some time, I'll write one in Python (with web.py). I'll probably propose that project to our local Python User Group (PYPTUG).

Glassfish 3.1.2

Have you installed Glassfish 3.1.2 on a Solaris variant? For one thing, it has SMF integration, that is, it creates (optionally) a service that can be controlled by svcadm. And then there are the functionalities that are available on all platforms. For example, clustering (both on the open source edition and oracle supported versions). So, basically, it is just a matter of downloading the file and then, as root (or sudo or pfexec depending how you are set up):

# sh glassfish-3.1.2-unix-ml.sh

Extracting the installer archive...

Extracting the installer runtime...

Extracting the installer resources...

Extracting the installer metadata...

Welcome to GlassFish installer

Using the user defined JAVA_HOME : /

Entering setup... SwixML 1.5 (#144)

This will bring up the usual GUI installer (assuming you are installing in the global zone, or you ssh -X to your zone). Make sure you select custom instead of express install. Now, this is pretty straightforward, and wouldn't merit a blog entry, but I wanted to bring up at least one thing that might stump you if you are installing in a solaris container / zone. The web admin (port 4848) expects localhost, as in http://localhost:4848, else it will error out. If we are in a zone, it is not too convenient. Sure, you could do an ssh -X zone_name and then run firefox from the zone:

# ssh -X server1

Oracle Corporation SunOS 5.11 11.0 May 2012

user@server1:~$ firefox -no-remote

This will launch firefox from the zone. Without the -no-remote, firefox will run in the global zone, and the login to the admin console will fail: Error Configuration Error Secure Admin must be enabled to access the DAS remotely. Again, you could do that... A better approach is to enable access through the host (zone) name instead of localhost. How?

cd /opt/glassfish3/bin

./asadmin --host server1 --port 4848 enable-secure-admin

Replace server1 by the hostname of the zone and you are almost there. For the change to take place, a restart of the app server is required. Easiest is reboot of the zone, but hold on! We have to make sure the service is enabled:

user@server1:~$ svcs domain1Service

STATE STIME FMRI disabled 13:35:17 svc:/application/GlassFish/domain1Service:default

user@server1:~$ su

root@server1:~# svcadm enable domain1Service

root@server1:~# svcs domain1Service

STATE STIME FMRI offline* 15:15:10 svc:/application/GlassFish/domain1Service:default

root@server1:~# svcs domain1Service

STATE STIME FMRI maintenance 15:15:19 svc:/application/GlassFish/domain1Service:default

The first time around it will be in maintenance, because glassfish is already running. We issue a reboot command in the zone and once it comes back up:

user@server1:~$ svcs domain1Service

STATE STIME FMRI online 15:18:11 svc:/application/GlassFish/domain1Service:default

That is it. We can now access http://server1:4848

My Laptop (sxce or solex)

I never did post this, but last year i installed Solaris 11 express on my laptop (an older dell m4400 with 8 gb of ram and Intel quad core extreme), and missed the look and feel of SXDE (the most productive os out of the box from sun/oracle to this day) and SXCE. You can see the result. Hard to tell it is solaris 11 express (solex).

Solaris Desktop on the move (ipad)

So as seen here in the picture, i'm still using solaris as my desktop. Big deal, right?

But this is on an Apple ipad 2. Using Oracle virtual desktop, served from a solaris 10 sun ray server, and to top it all off, from a vmware server. Oh yeah, and off verizon 3g or public wifi access points and VPN. What the Sun Microsystems sunray laptop could have been, plus relatively inexpensive

Since Apple will be making some announcements in a few hours (and who knows what will they announce- new ios or new iphone or just software ), I thought it might be a good time to mention this, for those that do not know about this solution. Granted, there are so many applications for the ipad / iphone and ipod, that it is easy to never see or hear about them.

Shortly, Solaris 11 fcs should be available, and SRS 5.2 will hopefully function properly on it, giving this a more modern desktop.

And now back to your regular programming... (no pun intended)

Where is the memory gone?

So you just installed Solaris 10 U6. Pretty exciting this zfs boot, no?

But you ran vmstat (or installed top on the system) and noticed you had 200MB free. Worse, you tried starting your Oracle database with a large SGA and it failed because it couldn't allocate the memory. What? This machine has 16GB! and barely anything running, I hear you scream. Where is the memory gone?

# echo "::memstat" | mdb -k
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 1717128 13415 83%
Anon 238964 1866 12%
Exec and libs 23450 183 1%
Page cache 19039 148 1%
Free (cachelist) 19243 150 1%
Free (freelist) 40453 316 2%

Total 2058277 16080
Physical 2054336 16049

The kernel is using 13GB?? Yes. You are hitting a default setting that's been around since ZFS was introduced to Solaris 10. It is the ZFS ARC. But dont complain too much because it is now easy to fix. When we first hit this issue way back when, we had to use mdb to set values at boot time, you couldn't just set something in the /etc/system file.

So what is ZFS ARC? In simple terms, it is memory that ZFS uses for cache. The default is for the cache to grow up to total memory - 1GB. The problem is that although it is supposed to free up memory when applications in user space request memory, in practice, it doesn't do this fast enough. Plus you end up with fragmented memory which is a huge problem for SHM (part of the SGA under Oracle).

In general, I reserve 2GB for the os and my apps. If I run Oracle and / or Sun App server, i'd also set aside the SGA and / or the java memory. Add it all up. Let's say you need 4GB total you dont want touched by ZFS, and you have 8GB, then you would set the maximum size for the ARC to be 4GB.

What if you dont run Oracle and the like? Still, if you run a graphical desktop or a Sunray server on your machine, leave 2GB untouched, so if you have just 4GB total, set the ARC to 2GB.

How?

edit /etc/system and add:

* Restrict ZFS ARC to 8GB
set zfs:zfs_arc_max = 8000000000


Now this is actually less than 8GB, but it is easier to read 8 followed by 9 zeros than 8 x1024x1024x1024. So for 2GB: 2000000000 and for 4GB: 4000000000

This will require a reboot.

Once rebooted you can verify it took the change by executing:

# kstat -m zfs
module: zfs instance: 0
name: arcstats class: misc
c 8000000000
c_max 8000000000
c_min 1000000000
...

Solaris 10 U6 and ZFS BOOT

This update is really great! Solaris 10 can now boot ZFS.

I wrote an overview of this version here (it is in french and way too long to translate - try babelfish or google translate on it):

http://www.sunquebec.com/forum/viewtopic.php?t=1643

If you've yet to download U6 (10/08) get it here:

http://www.sun.com/software/solaris/get.jsp

Project Indiana

In the same vein as Belenix which provides a live CD with an OpenSolaris base and XFCE or KDE as desktop environments, Project Indiana leverages the Distro Construction kit that was created for Belenix and provides a live CD with Gnome as the desktop environment.

It also supports a concept similar to pkg-get (as used with sunfreeware and blastwave on Solaris for many years) which is pkg:
pkg documentation

Now showing at a theather near you
It was released last night as a developer preview and you can get the preview ISO of Project Indiana here:
http://opensolaris.org/os/project/indiana/resources/getit/

It differs from Belenix in that it does require a strict minimum of 512MB.

Further reading (links):

Project Indiana documentation

Release notes