Part 2: Upgrading Solaris Non-Global Zones

 

Introduction

In Part 1 of this series, I covered the process of upgrading the Global Zone from Solaris 11.4 SRU22 to SRU89. Once the Global Zone upgrade was completed successfully, the next task was upgrading the Non-Global Zones running on the same server.

According to Oracle's recommended approach, upgrading Non-Global Zones is usually a straightforward process. If the zone has sufficient free space available, the update can normally be completed by mounting the zone, updating the packages, and then booting the zone.

In many environments, this method works without any issues. However, during this project I encountered an unexpected problem. Even though several Non-Global Zones had more than 12 GB of free space available, the standard upgrade procedure repeatedly failed. This article documents the issue I encountered, the troubleshooting process, and the workaround that ultimately allowed the upgrades to complete successfully.

The Standard Upgrade Method

For zones with sufficient free space, the typical upgrade procedure is:

zoneadm -z gastest1 attach -u
zoneadm -z gastest1 boot

Based on available documentation, this approach should work for zones with more than approximately 7–8 GB of available space.

Several of my zones, including gastest1 and others, had more than 12 GB of free space available. Based on those numbers, I expected the update to complete successfully.

Why the Standard Method Failed

After upgrading the Global Zone, I attempted to use the standard zone update process with the attach-and-update option:

zoneadm -z gastest1 attach -u

The process started normally but failed during package synchronization:

NOTE: Always review the logs generated by the zoneadm attach -u command. They provide detailed information about every step of the attach operation and are essential for troubleshooting and identifying the root cause of any issues.

At first, the failure was confusing because the zone appeared to have more than enough free space. After investigating further, I found that the issue was related to how IPS performs package updates.

During a large SRU upgrade, such as SRU22 to SRU89, Solaris creates a temporary package image under /var/pkg. The package system downloads thousands of files, extracts package contents, stages updates, and retains existing package versions until the update transaction is completed.

As a result, the package update can require significantly more temporary disk space than the free space reported inside the zone.

Although my zones had more than 12 GB available, the zones themselves were limited to approximately 20 GB in total size. During the package synchronization stage, IPS simply did not have enough temporary working space to complete the update.

Instead of resizing the zone storage, I chose another approach: performing the update on a temporary copy of the zone filesystem located on a Global Zone filesystem with approximately 650 GB of available capacity.

Alternative Method: Relocating the Zone to Larger Storage

The idea is simple:

1. Mount the zone filesystem from the Global Zone.
2. Copy the zone to a location with sufficient free space.
3. Perform the package update there.
4. Synchronize the updated files back.
5. Restore ownership and boot the zone.

This approach provided enough temporary working space for IPS to complete the SRU upgrade successfully.

Step 1 – Prepare and Mount the Zone Filesystem

Temporarily remove the ZFS zone ownership restrictions and mount the datasets.

Temporarily remove the ZFS zone ownership restrictions and mount the datasets.

# Break the ZFS zone lock
zfs set zoned=off os_zone_test1/test1/rpool/ROOT/S11.4SRU22-1
zfs set zoned=off os_zone_test1/ test1/rpool/ROOT/S11.4SRU22-1/var

# Create mount location
mkdir -p /mnt_upd

# Mount datasets
zfs mount -o mountpoint=/mnt_upd os_zone_test1/ test1/rpool/ROOT/S11.4SRU22-1

zfs mount -o mountpoint=/mnt_upd/var os_zone_test1/test1/rpool/ROOT/S11.4SRU22-1/var

Verify that the filesystem is accessible before proceeding.

Step 2 – Copy the Zone to Temporary High-Capacity Storage

Create a temporary workspace on the large filesystem.

mkdir -p /var/share/temp_zone_root

Copy the entire zone filesystem.

rsync -aP /mnt_upd/ /var/share/temp_zone_root/

Depending on the size of the zone, this operation may take several minutes.

Step 3 – Perform the Package Update

Run the package update against the copied filesystem.

  pkg -R /var/share/temp_zone_root update --accept --no-be-activate

Since the update is now running on storage with hundreds of gigabytes available, IPS has sufficient temporary workspace to complete the package operation.

Wait for the update to finish completely before continuing

Step 4 – Synchronize the Updated Files Back

After a successful update, copy the updated files back to the actual zone filesystem.

# 1. Sync the updated files back
# The --delete flag ensures old, replaced files are removed to save space
rsync -aP --delete /var/share/temp_zone_root/ /mnt_upd/

The --delete option removes files that were replaced during the update process and keeps both filesystems synchronized.

Verify the updated release files:

ls -l /mnt_upd/etc/release

Step 5 – Restore Zone Ownership and Boot

Once synchronization is complete, restore the original ZFS configuration.

# Unmount datasets
zfs unmount /mnt_upd/var
zfs unmount /mnt_upd

# Restore zoned property
zfs set zoned=on os_zone_test1/test1/rpool/ROOT/S11.4SRU22-1
zfs set zoned=on os_zone_test1/test1/rpool/ROOT/S11.4SRU22-1/var
# Reattach and boot the zone.
zoneadm -z test1 attach -F
zoneadm -z test1 boot

After the zone starts successfully, verify the installed SRU version and perform any required application testing.

Conclusion

The standard Non-Global Zone upgrade process should always be the first option because it is simple, supported, and works well when adequate disk space is available.

In my environment, however, the standard approach failed even though the affected zones had more than 12 GB of free space. The root cause was not the reported free space itself but the temporary workspace required by IPS during a large SRU jump from SRU22 to SRU89.

By temporarily relocating the zone filesystem to a larger storage area, I was able to provide enough working space for the package update process to complete successfully. After synchronizing the updated files back to the original zone storage and reattaching the zone, the upgrade completed without requiring any changes to the existing zone layout.

If you encounter disk space errors during a major Solaris SRU upgrade, especially in environments where zone storage is tightly allocated, this method can be a practical alternative to resizing filesystems or rebuilding the zone.