Paul Kraus via illumos-zfs
2014-06-26 16:33:51 UTC
tl;dr I am at a crossroads where I can change OS from FreeBSD to SmartOS or OmniOS, should I
I know that this is more of an opinion than a fact question, but I have an opportunity to choose an OS. I am building a new server for home use. Here are the limitations / requirements / desires:
*- Must use ZFS. My data is currently living in a 5 disk RAIDz2 pool running under FreeBSD 9.1 and I want to be able to just physically move the disks over and import the pool.
*- My current server is an HP Micro Proliant N36L, the new one is an N54L. 8 GB RAM (I would put in more but I am not sure the BIOS will support more than 8 GB). The new server has a pair of 120 GB SSD (Kingston V300 that I got relatively cheap) that can be used however is most beneficial.
*- The PRIMARY function is as a file server via both NFS (to Mac OS clients) and SMB (to Windows and older Mac OS clients).
*- Secondary functions include: DHCP server, DNS server, maybe mail server (I am considering moving all the mail out to a Joyent SmartOS instance so I do not have to permit SSL IMAP into my home network), maybe LDAP (authentication) service, and occasional Web/PHP/MySQL development.
*- My current server (N36L) will become my offsite backup server getting incremental zfs send / recv streams via an SSH tunnel.
The OS options currently on the table include:
FreeBSD 10.0
SmartOS (current)
OmniOS (current stable)
I have confirmed that all three are either supported or “reported to work” on my hardware (I have booted each and run them for a bit on the exact hardware I will be using).
It seems to me (and I may be wrong here) that SmartOS is further ahead in terms of ZFS development than either OmniOS or FreeBSD. But SmartOS is more designed for hosting VM’s (zones) than being a basic server. If I went the SmartOS route, the NFS file service would be in the Global Zone and all the other services in Local Zones (Containers). The SmartOS documentation makes it clear that you should run as little as you can in the Global Zone. I am not opposed to that model, that is what we did under Solaris 10 at my last long term contract.
I have been running a handful of FreeBSD systems in both production and development for about 2 years and have not had any bad experiences (if you ignore the FreeBSD VBox guest that kept losing it’s boot block with every restart of the VM). The robustness of ZFS has bailed my butt out on a number of occasions (yes, some of my configurations are less than best practice due to cost constraints).
Prior to working with FreeBSD I had been managing Solaris systems since 1995 and between about 2009 and 2012 had been managing a 250 TB zpool of mission critical storage.* So I am familiar with the Solaris way of doing things (SMF, ZFS, etc.). I currently have a couple Joyent SmartOS instances for client’s production services.
So what is the general opinion here … Which of the three am I better off with ?
NOTE: I am also evaluating whether it makes sense to convert our development server at the office from FreeBSD with VirtualBox to SmartOS with Zones and KVM. We have been recommending Joyent SmartOS instances for our clients when they go production, so having it in-house is a smart move.
* The 250 TB of storage was mostly small files, over 400 million of them when I left about 2 years ago. ZFS was the only FS when we started the designing the project in 2005 that could handle that many objects in one filesystem and not take a huge performance penalty. We had to present the data to the clients as one coherent view and did try breaking it up by logical groupings behind the scenes, that was a disaster in terms of manageability and scalability. One huge zfs zpool with separate zfs datasets (to allow for manageable backups … snapshot + send/recv) was the only sane way to go.
--
Paul Kraus
***@kraus-haus.org
I know that this is more of an opinion than a fact question, but I have an opportunity to choose an OS. I am building a new server for home use. Here are the limitations / requirements / desires:
*- Must use ZFS. My data is currently living in a 5 disk RAIDz2 pool running under FreeBSD 9.1 and I want to be able to just physically move the disks over and import the pool.
*- My current server is an HP Micro Proliant N36L, the new one is an N54L. 8 GB RAM (I would put in more but I am not sure the BIOS will support more than 8 GB). The new server has a pair of 120 GB SSD (Kingston V300 that I got relatively cheap) that can be used however is most beneficial.
*- The PRIMARY function is as a file server via both NFS (to Mac OS clients) and SMB (to Windows and older Mac OS clients).
*- Secondary functions include: DHCP server, DNS server, maybe mail server (I am considering moving all the mail out to a Joyent SmartOS instance so I do not have to permit SSL IMAP into my home network), maybe LDAP (authentication) service, and occasional Web/PHP/MySQL development.
*- My current server (N36L) will become my offsite backup server getting incremental zfs send / recv streams via an SSH tunnel.
The OS options currently on the table include:
FreeBSD 10.0
SmartOS (current)
OmniOS (current stable)
I have confirmed that all three are either supported or “reported to work” on my hardware (I have booted each and run them for a bit on the exact hardware I will be using).
It seems to me (and I may be wrong here) that SmartOS is further ahead in terms of ZFS development than either OmniOS or FreeBSD. But SmartOS is more designed for hosting VM’s (zones) than being a basic server. If I went the SmartOS route, the NFS file service would be in the Global Zone and all the other services in Local Zones (Containers). The SmartOS documentation makes it clear that you should run as little as you can in the Global Zone. I am not opposed to that model, that is what we did under Solaris 10 at my last long term contract.
I have been running a handful of FreeBSD systems in both production and development for about 2 years and have not had any bad experiences (if you ignore the FreeBSD VBox guest that kept losing it’s boot block with every restart of the VM). The robustness of ZFS has bailed my butt out on a number of occasions (yes, some of my configurations are less than best practice due to cost constraints).
Prior to working with FreeBSD I had been managing Solaris systems since 1995 and between about 2009 and 2012 had been managing a 250 TB zpool of mission critical storage.* So I am familiar with the Solaris way of doing things (SMF, ZFS, etc.). I currently have a couple Joyent SmartOS instances for client’s production services.
So what is the general opinion here … Which of the three am I better off with ?
NOTE: I am also evaluating whether it makes sense to convert our development server at the office from FreeBSD with VirtualBox to SmartOS with Zones and KVM. We have been recommending Joyent SmartOS instances for our clients when they go production, so having it in-house is a smart move.
* The 250 TB of storage was mostly small files, over 400 million of them when I left about 2 years ago. ZFS was the only FS when we started the designing the project in 2005 that could handle that many objects in one filesystem and not take a huge performance penalty. We had to present the data to the clients as one coherent view and did try breaking it up by logical groupings behind the scenes, that was a disaster in terms of manageability and scalability. One huge zfs zpool with separate zfs datasets (to allow for manageable backups … snapshot + send/recv) was the only sane way to go.
--
Paul Kraus
***@kraus-haus.org