I recently acquired a Gen8 HP Microserver for use as a new NAS. My existing NAS’ run variants of OpenSolaris (ie, NexentaStor community edition), but I wanted to get away from using Nexenta’s products for various reasons. I chose OmniOS because it’s based on Illumos, which NexentaStor is also based on, and because it has drivers for the BCM5720 on-board NICs.
One of the things I wanted to do with this nas was to use an SSD for both rpool and l2arc. I only have room for one SSD, and since the rpool gets very little I/O I thought I would add the L2ARC to the same SSD. This turned out to be fairly challenging, since I couldn’t find many instances of people doing it, and even less information on actually how to do it. I thought I’d share what I did, in the hopes it will benefit someone else.
After installing OmniOS, and specifying that rpool should consume 40GB of my 256GB SSD, the real fun began. Here are the steps to performing this task.
Run the format command, to change the partition size via fdisk (select the disk, in my case 4, then type fdisk). Originally it looked like this:
12 Total disk size is 39869 cylindersCylinder size is 12544 (512 byte) blocks
1234 CylindersPartition Status Type Start End Length %========= ====== ============ ===== === ====== ===1 Active Solaris2 1 6687 6687 17
But we need to change that so the partition uses 100% of the disk instead of 17%. So delete the partition and then create a new one. Note that this doesn’t remove any data, we’re only changing the partition table. Just make sure the starting cylinder is the same, and the new partition is larger than the old one. Now it will look like this:
12 Total disk size is 39869 cylindersCylinder size is 12544 (512 byte) blocks
1234 CylindersPartition Status Type Start End Length %========= ====== ============ ===== === ====== ===1 Active Solaris2 1 39868 39868 100
Our next step is to update the slices within this partition. Enter 6 in the fdisk menu to update disk configuration and exit, then enter the partition menu by typing p, and then p again to print the current table. You’ll see something like this:
123456789 SELECT ONE OF THE FOLLOWING:1. Create a partition2. Specify the active partition3. Delete a partition4. Change between Solaris and Solaris2 Partition IDs5. Edit/View extended partitions6. Exit (update disk configuration and exit)7. Cancel (exit without updating disk configuration)Enter Selection: 6
1 format> p
12345678910111213141516171819 PARTITION MENU:0 - change `0' partition1 - change `1' partition2 - change `2' partition3 - change `3' partition4 - change `4' partition5 - change `5' partition6 - change `6' partition7 - change `7' partitionselect - select a predefined tablemodify - modify a predefined partition tablename - name the current tableprint - display the current tablelabel - write partition map and label to the disk!<cmd> - execute <cmd>, then returnquitpartition> pCurrent partition table (original):Total disk cylinders available: 39866 + 2 (reserved cylinders)
1234567891011 Part Tag Flag Cylinders Size Blocks0 unassigned wm 0 0 (0/0/0) 01 unassigned wm 0 0 (0/0/0) 02 backup wu 0 - 39865 238.46GB (39866/0/0) 5000791043 unassigned wm 0 0 (0/0/0) 04 unassigned wm 0 0 (0/0/0) 05 unassigned wm 0 0 (0/0/0) 06 unassigned wm 0 0 (0/0/0) 07 unassigned wm 0 0 (0/0/0) 08 boot wu 0 - 0 6.12MB (1/0/0) 125449 unassigned wm 0 0 (0/0/0) 0
Notice that the second slice or partition has changed size already, but we’ve lost our root slice. Recreate it using the same starting cylinder it had before:
123 partition> 0Part Tag Flag Cylinders Size Blocks0 unassigned wm 0 0 (0/0/0) 0
1234567 Enter partition id tag[unassigned]: rootEnter partition permission flags[wm]:Enter new starting cyl[1]:Enter partition size[0b, 0c, 1e, 0.00mb, 0.00gb]: 6686Please specify units in either b(blocks), c(cylinders), e(end cylinder),m(megabytes) or g(gigabytes)Enter partition size[0b, 0c, 1e, 0.00mb, 0.00gb]: 6686c
Next we can create a slice to use for our l2arc. Any of the partition numbers between 3 and 7 can be used, but I chose 5, with a starting cylinder after our root slice. I also decided to give it to partition tag of var. There’s a limited set of partition tags one can use. It’s probably not the way an experienced Solaris sysadmin would do it, but it’s how I’ve done it.
Note that I wanted to only use 100GB of the SSD for the L2ARC. This leaves a number of unused blocks on the SSD, which will hopefully limit the performance drop when the L2ARC partition fills up.
1234567 Enter partition id tag[unassigned]: varEnter partition permission flags[wm]:Enter new starting cyl[1]: 6688Enter partition size[0b, 0c, 6688e, 0.00mb, 0.00gb]: 100gbpartition> pCurrent partition table (unnamed):Total disk cylinders available: 39866 + 2 (reserved cylinders)
1234567891011 Part Tag Flag Cylinders Size Blocks0 root wm 1 - 6686 39.99GB (6686/0/0) 838691841 unassigned wm 0 0 (0/0/0) 02 backup wu 0 - 39865 238.46GB (39866/0/0) 5000791043 unassigned wm 0 0 (0/0/0) 04 unassigned wm 0 0 (0/0/0) 05 var wm 6688 - 23406 100.00GB (16719/0/0) 2097231366 unassigned wm 0 0 (0/0/0) 07 unassigned wm 0 0 (0/0/0) 08 boot wu 0 - 0 6.12MB (1/0/0) 125449 unassigned wm 0 0 (0/0/0) 0
Finally, save the changes with the label command, press y to continue.
12 partition> labelReady to label disk, continue? y
123 partition> pCurrent partition table (unnamed):Total disk cylinders available: 39866 + 2 (reserved cylinders)
1234567891011 Part Tag Flag Cylinders Size Blocks0 root wm 1 - 6686 39.99GB (6686/0/0) 838691841 unassigned wm 0 0 (0/0/0) 02 backup wu 0 - 39865 238.46GB (39866/0/0) 5000791043 unassigned wm 0 0 (0/0/0) 04 unassigned wm 0 0 (0/0/0) 05 var wm 6688 - 23406 100.00GB (16719/0/0) 2097231366 unassigned wm 0 0 (0/0/0) 07 unassigned wm 0 0 (0/0/0) 08 boot wu 0 - 0 6.12MB (1/0/0) 125449 unassigned wm 0 0 (0/0/0) 0
Now we can specify this slice in our zpool add command. It will give us a warning that the partition overlaps with the backup partition, but partitions with the backup tag are required to specify the full disk capacity so I couldn’t see a way around that.
1234 root@micronas3:/root# zpool add storage cache c4t4d0s5invalid vdev specificationuse '-f' to override the following errors:/dev/dsk/c4t4d0s5 overlaps with /dev/dsk/c4t4d0s2
123456 root@micronas3:/root# zpool add -f storage cache c4t4d0s5root@micronas3:/root# zpool status -vpool: rpoolstate: ONLINEscan: none requestedconfig:
123 NAME STATE READ WRITE CKSUMrpool ONLINE 0 0 0c4t4d0s0 ONLINE 0 0 0
1 errors: No known data errors
123456789 pool: storagestate: ONLINEstatus: The pool is formatted using a legacy on-disk format. The pool canstill be used, but some features are unavailable.action: Upgrade the pool using 'zpool upgrade'. Once this is done, thepool will no longer be accessible on software that does not support featureflags.scan: none requestedconfig:
12345678910 NAME STATE READ WRITE CKSUMstorage ONLINE 0 0 0mirror-0 ONLINE 0 0 0c1t10d0 ONLINE 0 0 0c1t9d0 ONLINE 0 0 0mirror-1 ONLINE 0 0 0c1t11d0 ONLINE 0 0 0c1t12d0 ONLINE 0 0 0cachec4t4d0s5 ONLINE 0 0 0
1 errors: No known data errors
Job is done! I’m currently running some load tests to ensure that everything works as expected, but so far, so good!
Leave a Reply