OpenVMS Alpha Partitioning and Galaxy Guide
OpenVMS Galaxy Concepts
The Compaq Galaxy Software Architecture on OpenVMS Alpha lets you run
multiple instances of OpenVMS in a single computer. You can dynamically
reassign system resources, mapping compute power to applications on an
as-needed basis---without having to reboot the computer.
This chapter describes OpenVMS Galaxy concepts and highlights the
features available in OpenVMS Alpha Version 7.3.
2.1 OpenVMS Galaxy Concepts and Components
With OpenVMS Galaxy, software logically partitions
CPUs, memory, and I/O ports by assigning them to individual instances
of the OpenVMS operating system. This partitioning, which a system
manager directs, is a software function; no hardware boundaries are
required. Each individual instance has the resources it needs to
execute independently. An OpenVMS Galaxy environment is
adaptive in that resources such as CPUs can be
dynamically reassigned to different instances of OpenVMS.
The Galaxy Software Architecture on OpenVMS includes the following
hardware and software components:
The console on an OpenVMS system is comprised of an
attached terminal and a firmware program that performs power-up
self-tests, initializes hardware, initiates system booting, and
performs I/O services during system booting and shutdown. The console
program also provides run-time services to the operating system for
console terminal I/O, environment variable retrieval, NVRAM
(nonvolatile random access memory) saving, and other miscellaneous
In an OpenVMS Galaxy computing environment, the console plays a
critical role in partitioning hardware resources. It maintains the
permanent configuration in NVRAM and the running configuration in
memory. The console provides each instance of the OpenVMS operating
system with a pointer to the running configuration data.
Memory is logically partitioned into private and shared sections. Each
operating system instance has its own private memory; that is, no other
instance maps those physical pages. Some of the shared memory is
available for instances of OpenVMS to communicate with one another, and
the rest of the shared memory is available for applications.
The Galaxy Software Architecture is prepared for a nonuniform memory
access (NUMA) environment and, if necessary, will provide special
services for such systems to achieve maximum application performance.
In an OpenVMS Galaxy computing environment, CPUs can be reassigned
An OpenVMS Galaxy has a highly scalable I/O subsystem because there are
multiple, primary CPUs in the system---one for each instance. Also,
OpenVMS currently has features for distributing some I/O to secondary
CPUs in an SMP system.
One or more OpenVMS instances can execute without sharing any resources
in an OpenVMS Galaxy. An OpenVMS instance that does not share resources
is called an independent instance.
An independent instance of OpenVMS does not participate in shared
memory use. Neither the base operating system nor its applications
access shared memory.
An OpenVMS Galaxy can consist solely of independent instances; such a
system would resemble traditional mainframe-style partitioning.
Architecturally, OpenVMS Galaxy is based on an SMP hardware
architecture. It assumes that CPUs, memory, and I/O have full
connectivity within the machine and that the memory is cache coherent.
Each subsystem has full access to all other subsystems.
As shown in Figure 2-1, Galaxy software looks at the resources as if
they were a pie. The various resources (CPUs, private memory, shared
memory, and I/O) are arranged as concentric bands within the pie in a
specific hierarchy. Shared memory is at the center.
Figure 2-1 OpenVMS Galaxy Architecture Diagram
Galaxy supports the ability to divide the pie into multiple slices,
each of disparate size. Each slice, regardless of size, has access to
all of shared memory. Furthermore, because software partitions the pie,
you can vary the number and size of slices dynamically.
In summary, each slice of the pie is a separate and complete instance
of the operating system. Each instance has some amount of dedicated
private memory, a number of CPUs, and the necessary I/O. Each instance
can see all of shared memory, which is where the application data
resides. System resources can be reassigned between the instances of
the operating system without rebooting.
Another possible way to look at the Galaxy computing model is to think
about how a system's resources could be divided.
For example, the overall sense of Figure 2-2 is that the proportion
by which one resource is divided between instances is the proportion by
which each of the other resources must be divided.
Figure 2-2 Another Galaxy Architecture Diagram
2.2 OpenVMS Galaxy Features
An evolution in OpenVMS functionality, OpenVMS Galaxy leverages proven
OpenVMS Cluster, symmetric multiprocessing, and performance
capabilities to offer greater levels of performance, scalability, and
availability with extremely flexible operational capabilities.
Fifteen years of proven OpenVMS Cluster technology facilitates
communication among clustered instances within an OpenVMS Galaxy.
An OpenVMS Cluster is a software concept. It is a set of coordinated
OpenVMS operating systems, one per computer, communicating
over various communications media to combine the processing power and
storage capacity of multiple computers into a single, shared-everything
An OpenVMS Galaxy is also a software concept. However, it is a set of
coordinated OpenVMS operating systems, in a single computer,
communicating through shared memory. An instance of the operating
system in an OpenVMS Galaxy can be clustered with other instances
within the Galaxy or with instances in other systems.
An OpenVMS Galaxy is a complete system in and of itself. Although an
OpenVMS Galaxy can be added to an existing cluster as multiple cluster
nodes can be added today, the single system is the OpenVMS Galaxy
architecture focus. An application running totally within an OpenVMS
Galaxy can take advantage of performance opportunities not present in
Any instance in an OpenVMS Galaxy can be an SMP configuration. The
number of CPUs is part of the definition of an instance. Because an
instance in the OpenVMS Galaxy is a complete OpenVMS operating system,
all applications behave the same as they would on a traditional,
A CPU can be dynamically reassigned from one instance to another while
all applications on both instances continue to run. Reassignment is
realized by three separate functions: stopping, reassigning, and
starting the CPU in question. As resource needs of applications change,
the CPUs can be reassigned to the appropriate instances. There are some
restrictions; for example, the primary CPU in an instance cannot be
reassigned, and a CPU cannot specifically be designated to handle
Multiple instances of the OpenVMS operating system allow system
managers to reassign processing power to the instances whose
applications most need it. As that need varies over time, so can the
configuration. OpenVMS allows dynamic reconfiguration while all
instances and their applications continue to run.
2.3 OpenVMS Galaxy Benefits
Many of the benefits of OpenVMS Galaxy technology result directly from
running multiple instances of the OpenVMS operating system in a single
With several instances of OpenVMS in memory at the same time, an
OpenVMS Galaxy computing environment gives you quantum improvements in:
- Compatibility---Existing applications run without changes.
- Availability---Presents opportunities to upgrade software and
expand system capacity without down time.
- Scalability---Offers scaling alternatives that improve performance
of SMP and cluster environments.
- Adaptability---Physical resources can be dynamically reassigned to
meet changing workload demands.
- Cost of ownership---Fewer computer systems reduce system management
requirements, floor space, and more.
- Performance---Eliminates many bottlenecks and provides more I/O
The following descriptions provide more details about these benefits.
Existing single-system applications will run without changes on
instances in an OpenVMS Galaxy. Existing OpenVMS Cluster applications
will also run without changes on clustered instances in an OpenVMS
An OpenVMS Galaxy system is more available than a traditional,
single-system-view, SMP system because multiple instances of the
operating system control hardware resources.
OpenVMS Galaxy allows you to run different versions of OpenVMS (Version
7.2 and later) simultaneously. For example, you can test a new version
of the operating system or an application in one instance while
continuing to run the current version in the other instances. You can
then upgrade your entire system, one instance at a time.
System managers can assign resources to match application requirements
as business needs grow or change. When a CPU is added to a Galaxy
configuration, it can be assigned to any instance of OpenVMS. This
means that applications can realize 100% of a CPU's power.
Typical SMP scaling issues do not restrict an OpenVMS Galaxy. System
managers can define the number of OpenVMS instances, assign the number
of CPUs in each instance, and control how they are used.
Additionally, a trial-and-error method of evaluating resources is a
viable strategy. System managers can reassign CPUs among instances of
OpenVMS until the most effective combination of resources is found. All
instances of OpenVMS and their applications continue to run while CPUs
An OpenVMS Galaxy is highly adaptable because computing resources can
be dynamically reassigned to other instances of the operating system
while all applications continue to run.
Reassigning CPUs best demonstrates the adaptive capability of an
OpenVMS Galaxy computing environment. For example, if a system manager
knows that resource demands change at certain times, the system manager
can write a command procedure to reassign CPUs to other instances of
OpenVMS and submit the procedure to a batch queue. The same could be
done to manage system load characteristics.
In an OpenVMS Galaxy environment, software is in total control of
assigning and dynamically reassigning hardware resources. As additional
hardware is added to an OpenVMS Galaxy system, resources can be added
to existing instances; or new instances can be defined without
affecting running applications.
Cost of ownership
An OpenVMS Galaxy presents opportunities to upgrade existing computers
and expand their capacity, or to replace some number of computers,
whether they are cluster members or independent systems, with a single
computer running multiple instances of the operating system. Fewer
computers greatly reduces system management requirements as well as
An OpenVMS Galaxy can provide high commercial application performance
by eliminating many SMP and cluster-scaling bottlenecks. Also, the
distribution of interrupts across instances provides many I/O
configuration possibilities; for example, a system's I/O workload can
be partitioned so that certain I/O traffic is done on specific
2.4 OpenVMS Galaxy Version 7.3 Features
With OpenVMS Alpha Version 7.3, you can create an OpenVMS Galaxy
environment that allows you to:
- Run eight instances on an AlphaServer GS320
- Run four instances on an AlphaServer GS160
- Run two instances on an AlphaServer GS160
- Run three instances of OpenVMS on AlphaServer GS140 or 8400 systems
- Run two instances of OpenVMS on AlphaServer GS60, GS60E, GS80,
8200, 4100, or ES40 systems
- Reassign CPUs between instances
- Perform independent booting and shutdown of instances
- Use shared memory for communication between instances
- Cluster instances within an OpenVMS Galaxy using the shared memory
- Cluster instances with non-Galaxy systems
- Create applications using OpenVMS Galaxy APIs for resource
management, event notification, locking for synchronization, and shared
memory for global sections
- Use the Galaxy Configuration Utility to view and control the
OpenVMS Galaxy environment
- Run a single-instance OpenVMS Galaxy on any Alpha system for
2.5 Is an OpenVMS Galaxy for You?
For companies looking to improve their ability to manage unpredictable,
variable, or growing IT workloads, OpenVMS Galaxy technology provides
the most flexible way to dynamically reconfigure and manage system
resources. An integrated hardware and software solution, OpenVMS Galaxy
allows system managers to perform tasks such as reassigning individual
CPUs through a simple drag and drop procedure.
An OpenVMS Galaxy computing environment is ideal for high-availability
applications, such as:
- Database servers
- Transaction processing systems
- Data Warehousing
- Data Mining
- Internet servers
- NonStop eBusiness solutions
2.6 Why a Galaxy is a Good Business Choice
An OpenVMS Galaxy computing environment is a natural evolution for
current OpenVMS users with clusters or multiple sparsely configured
An OpenVMS Galaxy is attractive for growing organizations with varying
workloads---predictable or unpredictable.
2.7 Possible OpenVMS Galaxy Configurations
An OpenVMS Galaxy computing environment lets customers decide how much
cooperation exists between instances in a single computer system.
In a shared-nothing computing model, the instances do
not share any resources; operations are isolated from one another.
In a shared-partial computing model, the instances
share some resources and cooperate in a limited way.
In a shared-everything model, the instances cooperate
fully and share all available resources, to the point where the
operating system presents a single cohesive entity to the network.
2.7.1 Shared-Nothing Computing Model
In a shared-nothing configuration (shown in Figure 2-3), the
instances of OpenVMS are completely independent of each other and are
connected through external interconnects, as though they were separate
With Galaxy, all available memory is allocated into private memory for
each instance of OpenVMS. Each instance has its own set of CPUs and an
appropriate amount of I/O resources assigned to it.
Figure 2-3 Shared-Nothing Computing Model
2.7.2 Shared-Partial Computing Model
In a shared-partial configuration (shown in Figure 2-4), a portion of
system memory is designated as shared memory, which each instance can
access. Code and data for each instance are contained in private
memory. Data that is shared by applications in several instances is
stored in shared memory.
The instances are not clustered.
Figure 2-4 Shared-Partial Computing Model
2.7.3 Shared-Everything Computing Model
In a shared-everything configuration (shown in Figure 2-5), the
instances share memory and are clustered with one another.
Figure 2-5 Shared-Everything Computing Model
2.8 What Is a Single-Instance Galaxy?
A single-instance Galaxy
is for non-Galaxy platforms, that is, those without a Galaxy console.
Galaxy configuration data, which is normally provided by console
firmware, is instead created in a file. By setting the system parameter
GALAXY to 1, SYSBOOT reads the file into memory and the system boots as
a single-instance Galaxy, complete with shared memory, Galaxy system
services, and even self-migration of CPUs. This can be done on any
Single-instance Galaxy configurations will run on everything from
laptops to mainframes. This capability allows early adopters to
evaluate OpenVMS Galaxy features, and most importantly, to develop and
test Galaxy-aware applications without incurring the expense of setting
up a full-scale Galaxy platform.
Because the single-instance Galaxy is not an emulator---it is real
Galaxy code---applications will run on multiple-instance configurations.
For more information about running a single-instance Galaxy, see
2.9 OpenVMS Galaxy Configuration Considerations
When you plan to create an OpenVMS Galaxy computing environment, you
need to make sure that you have the appropriate hardware for your
configuration. General OpenVMS Galaxy configuration rules include:
- One or more CPUs per instance
- One or more I/O modules per instance
- Dedicated serial console port per instance
- Enough private memory for OpenVMS and applications
- Enough shared memory for the shared memory cluster interconnect,
global sections, and so on.
- Display for configuration management with either an Alpha or VAX
workstation running DECwindows or a Windows NT workstation with an X
For more information about hardware-specific configuration
requirements, see the chapter in this book specific to your hardware.
2.9.1 XMI Bus Support
The XMI bus is supported only on the first instance (Instance 0) of a
Galaxy configuration in an AlphaServer 8400 system.
Only one DWLM-AA XMI plug-in-unit subsystem cage for all XMI devices is
supported on an AlphaServer 8400 system. Note that the DWLM-AA takes up
quite a bit of space in the system because an I/O bulkhead is required
on the back of the system to connect all XMI devices to the system.
This allows only two additional DWLPB PCI plug-in units in the system.
2.9.2 Memory Granularity Restrictions
Private memory must start on a 64 MB boundary.
Shared memory must start on an 8 MB boundary.
All instances except the last must have a multiple of 64MB.
2.9.3 EISA Bus Support
The EISA bus is supported only on the first instance (instance 0) of a
Galaxy configuration. Due to the design of all EISA options, they must
always be on instance 0 of the system. A KFE70 must be used in the
first instance for any EISA devices in the Galaxy system.
All EISA devices must be on instance 0. No EISA devices are supported
on any other instance in a Galaxy system.
A KFE72-DA installed in other instances provides console connection
only and cannot be used for other EISA devices.
2.10 CD Drive Recommendation
Compaq recommends that a CD drive be available for each instance in an
OpenVMS Galaxy computing environment. If you plan to use multiple
system disks in your OpenVMS Galaxy, a CD drive per instance will be
very helpful for upgrades and software installations.
If your OpenVMS Galaxy instances are clustered together and use a
single common system disk, a single CD drive may be sufficient because
the CD drive can be served to the other clustered instances. For
operating system upgrades, the instance with the attached CD drive can
be used to perform the upgrade.
2.11 Important Cluster Information
This section contains information that will be important to you if you
are clustering instances with other instances in an OpenVMS Galaxy
computing environment or with non-Galaxy OpenVMS Clusters.
For information about OpenVMS Galaxy licensing requirements that apply
to clustering instances, see the OpenVMS License Management Utility Manual.
2.11.1 Becoming an OpenVMS Galaxy Instance
When you are installing OpenVMS Alpha Version 7.2--1, the OpenVMS
installation dialog asks questions about OpenVMS Cluster and OpenVMS
If you answered "Yes" to the question
Will this system be a member of a VMScluster? (Yes/No)
and you answered "Yes" to the question
the following information is displayed:
Will this system be an instance in an OpenVMS Galaxy? (Yes/No)
For more information, see OpenVMS Alpha Installation and Upgrade
For compatibility with an OpenVMS Galaxy, any systems in the VMScluster
which are running versions of OpenVMS prior to V7.1-2 must have a
remedial kit installed. The appropriate kit from the following list
must be installed on all system disks used by these systems.
(Later versions of these remedial kits may be used if available.)
Alpha V7.1 and V7.1-1xx ALPSYSB02_071
Alpha V6.2 and V6.2-1xx ALPSYSB02_062
VAX V7.1 VAXSYSB01_071
VAX V6.2 VAXSYSB01_062
2.11.2 SCSI Cluster Considerations
This section summarizes information about SCSI device naming for
OpenVMS Galaxy computing environments. For more complete information
about OpenVMS Cluster device naming, see the OpenVMS Cluster Systems manual.
If you are creating an OpenVMS Galaxy with shared SCSI buses, you must
note the following:
For OpenVMS to give the SCSI devices the same name on each instance
correctly, you will likely need to use the device-naming feature of
For example, assume that you have the following adapters on your system
when you enter the SHOW CONFIG command:
PKA0 (embedded SCSI for CDROM)
PKB0 (UltraSCSI controller KZPxxx)
PKC0 (UltraSCSI controller)
When you make this system a two-instance Galaxy, your hardware looks
like the following:
PKA0 (UltraSCSI controller)
PKA0 (embedded SCSI for CDROM)
PKB0 (UltraSCSI controller)
Your shared SCSI will be connected from PKA0 on instance 0 to PKB0 on
If you initialize the system with the LP_COUNT environment variable set
to 0, you will not be able to boot OpenVMS on the system unless the
SYSGEN parameter STARTUP_P1 is set to MINIMUM.
This is because, with the LP_COUNT variable set to 0, you will now have
PKB connected to PKC, and the SCSI device-naming that was set up for
initializing with multiple partitions is not correct for initializing
with the LP_COUNT variable set to 0.
During the device configuration that occurs during boot, OpenVMS will
notice that PKA0 and PKB0 are connected together. OpenVMS expects that
each device has the same allocation class and names, but in this case,
they will not.
The device naming that was set up for the two-instance Galaxy will not
function correctly because the console naming of the controllers has