HP OpenVMS Systems Documentation

Content starts here

HP OpenVMS Programming Concepts Manual

Previous Contents Index

3.3.1 Programming with Intracluster Communications

The following sections provide information on how to program with intracluster communictions (ICC) using ICC system services. ICC Concepts

The following terms and their definitions are central to creating and using intracluster communication.

An ASSOCIATION is a named link between an application and ICC. The association name, combined with the node name of the system on which the application is running, identifies this application to other applications that want to communicate within a node or cluster.

An ASSOCIATION NAME is a string that identifies an ASSOCIATION. Association Names are 1 to 31 characters in length and must be unique within a node. Association Names are case-sensitive. Associations are created by calling the SYS$ICC_OPEN_ASSOC service (or by the first call to SYS$ICC_CONNECT if OPEN was not previously called) and are identified by an Association Handle.

An ASSOCIATION HANDLE is an opaque identifier that represents an association to ICC. Association Handles are passed as unsigned longwords.

A NODE is either a standalone system or a member of an OpenVMS Cluster. It is identified by a one-to-six-character case-blind name that matches the SYSGEN parameter SCSNODE.

A CONNECTION is a link between two associations created by calling the SYS$ICC_CONNECT or SYS$ICC_ACCEPT service. An association may have multiple simultaneous connections at any given time. A connection is identified by a Connection Handle. A given application may choose to either only initiate connections, or initiate and accept connections as well. Connections support both synchronous and asynchronous communications.

A CONNECTION HANDLE is an opaque identifier that represents a connection to ICC. Connection Handles are passed as unsigned longwords.

In ICC terminology, a SERVER is an application that has declared to ICC that it is willing to accept connections. A server must call SYS$ICC_OPEN_ASSOC and supply the address of a connection routine. Upon receipt of a connection request AST, a server completes its side of the connection by calling the SYS$ICC_ACCEPT service. It may also choose not to accept the connection request by calling SYS$ICC_REJECT.

A CLIENT, in the ICC model, is an application that calls SYS$ICC_CONNECT to request a connection to a server. Note that a server may also be a client. A client need not call SYS$ICC_OPEN_ASSOC (although there are certain benefits to doing so) but may instead call SYS$ICC_CONNECT using the supplied constant Default Association Handle. An association opened in this manner is called the Default Association. Only one default association is permitted per process; however, any number of connections may be established even when there is only one association.

The ICC SIMPLE CLUSTERWIDE REGISTRY provides a method for servers to register their node and association names under a single logical name, thus eliminating the need for clients to determine on which node the server process is running. Multiple servers can register under the same logical name, and ICC will randomly select one from the list. The selection algorithm leads to load sharing, but not necessarily to load balancing. Design Considerations

This section contains information about ICC design considerations. Naming

An ICC server may have two visible names---the association name and, if the ICC registry is used, a registry name. The registry name may be the same as the association name.

If, however, you wish to run multiple copies of the server on a single node, the registry name should be the more general name, while the association name should reference a particular instance of the server. Message Ordering

ICC guarantees that messages will be delivered in the order in which they were transmitted. There is no provision for out-of-order delivery. Flow Control

ICC implements flow control on a per-association basis. You can change this value by specifying a value for the MAXFLOWBUFCNT parameter to the SYS$ICC_OPEN_ASSOC service.

In addition to flow control, ICC uses the value of MAXFLOWBUFCNT to determine preallocation of certain resources cached by the services for performance reasons. Increasing this value results in a larger charge against process quotas. The default value of MAXFLOWBUFCNT is 5.

Because all ICC connections within the association are subject to flow control, failure to receive data in a timely manner will eventually cause all connections in the association to stall. Transfer Sizes and Receiving Data

ICC supports two models for receiving data. In the simple receive model, a SYS$ICC_RECEIVE call either completes immediately if data is present, or else the receive buffer is queued up to be filled when data arrives from the sender. SYS$ICC_RECEIVEW does not return to the caller until either data is present, or the connection has disconnected.

The major disadvantage to this method is that the buffer you supply to SYS$ICC_RECEIVE must be large enough to receive the largest message the sender might transmit. Receipt of a message larger than the queued buffer causes ICC to disconnect the link to preserve order. There is no provision within ICC to break a single message over multiple receive calls.

The second model is data event driven. In this method, the application provides SYS$ICC_OPEN_ASSOC (parameter recv_rtn) the address of a routine to be called whenever a connection established over this particular association is the target of a TRANSMIT. The data routine is called at AST level in the original mode of the caller and supplied with the size of the incoming transfer, the connection handle for this transfer, and a user-supplied context value. Once notified of incoming data, the application must then allocate a sufficiently large buffer and issue a SYS$ICC_RECEIVE call to obtain the data. The maximum transfer size using this method is 1 Mb.

The SYS$ICC_RECEIVE call does not have to be made from the data routine; however, a receive request cannot be made before receipt of a data event. Therefore, receive operations are never queued within ICC when you use a data event.

Because there is a single data routine per association, all connections on the association share the same data routine. To have some connections use both methods, at least two associations must be opened. Transfer Sizes and Transceive

SYS$ICC_TRANSCEIVE is a single service that sends a message and completes when the other side responds with a call to SYS$ICC_REPLY. The maximum size of the return message is fixed by the size of the buffer the caller provides to the transceive service at the time the call is made. The maximum transmission size for both SYS$ICC_TRANSCEIVE and SYS$ICC_REPLY is 1 Mb. The minimum length of the reply transmission is zero bytes. Disconnection

In a properly coded communications protocol, responsibility for disconnecting the connection should be left to the side of the protocol that last received data. This rule assures that there is no data in transit in which the state may be uncertain at the time the disconnect event occurs.

An ICC connection may be disconnected under any of the following circumstances:

  • One side of the connection calls either SYS$ICC_DISCONNECT or SYS$ICC_CLOSE_ASSOC.
  • The client or server process in the connection is deleted.
  • The node on which one side of the connection is running is either shut down, crashes, or loses communication with the rest of the cluster.
  • An unrecoverable error within the ICC services occurs, and the only possible recovery by the service is to disconnect the link.
  • A pending receive buffer is too small to receive all the incoming data. To preserve message order, ICC will disconnect the link.

No matter what the cause of the disconnection, the application should be prepared to deal with disconnection other than that as an expected part of the application communication protocol. Error Recovery

Whenever possible, ICC services attempt to return an appropriate status value to the caller, either as the routine status value or in the status fields of the IOSB or IOS_ICC structure. Sometimes the only recovery method available to ICC is to disconnect the connection. ICC disconnects only when it lacks sufficient context to return status to the calling application. General Programming Considerations

The ICC public structures and constants are declared in the module $ICCDEF (MACRO), within STARLET.REQ for BLISS and ICCDEF.H in SYS$LIBRARY:SYS$STARLET_C.TLB for C. STARLET.H provides full system service prototypes for C. Servers

To open an association to ICC, call $ICC_OPEN_ASSOC, which provides the following parameters:

Parameter Description
assoc_handle Address to receive the association handle. (longword) Required.
assoc_name Address of a string descriptor pointing to the desired association name. If you omit this argument, the ICC default association will be opened. Association names are case sensitive. This is also the name used in the ICC security object for this association.
Address of string descriptors describing the logical name and logical table name for use by the ICC simple registry. The logical name table must exist at the time of the call, and the caller must have write access to the table. Unless your application requires a logical name table for exclusive use of the application, use of the default ICC$REGISTRY_TABLE is recommended.

The logical name is case sensitive. Table names are always converted to uppercase. The logical name supplied here is the name by which the client using the registry will know the server. If either of these arguments is supplied, they must both be supplied.

conn_event_rtn Address of a routine to be called whenever a client requests a connection to this server. A server may not omit this parameter. See the sections on connection and disconnection routines for more details about what actions a connection routine must take.
disc_event_rtn Optional address of a routine to be called when a disconnect event occurs. This may be the same routine as the connection routine. See the section on connection and disconnection routines for more details about what actions a disconnection routine may take.
recv_rtn Address of routine to be called whenever a connection receives data. Note that all connections on this association use the same routine. Please see the discussion on transfer sizes in Section for information on how having or not a having a data routine affects maximum receive size. Optional.
maxflowbufcnt Pass by value the maximum number of inbound messages per connection that ICC will allow before initiating flow control on the connection. The default value for this parameter is 5. Optional.
prot The default protection for this association. Refer to HP OpenVMS Guide to System Security for information on the security attributes of ICC associations.

This parameter is effective only if no Permanent Security Object has been created for this association.

If prot is zero (0) or not specified, all users may connect to this association.

If prot is one (1), only members of the same UIC group, or users with SYSPRV, may connect.

If prot is two (2), only the owner or users with SYSPRV may connect.

Object protection specified by a Permanent Security Object overrides this parameter. Additionally, if no Permanent Security Object exists, the value specified here may be overridden by the SET SECURITY command issued for the Temporary Security Object while the association is open. Connection Events

Once the association is opened, connection requests are delivered as asynchronous calls to the server's connection routine.

The connection routine (and disconnection routine, if supplied) are each called with seven arguments as described in the $ICC_OPEN_ASSOC system service, located in the HP OpenVMS System Services Reference Manual: GETUTC--Z.

A single connection or disconnection routine may distinguish between the events based on the first argument (event_type), which will either be the code ICC$C_EV_CONNECT or ICC$C_EV_DISCONNECT.

A connection routine must either call SYS$ICC_ACCEPT to complete the connection or SYS$ICC_REJECT to reject the connection. The EPID and username of the requesting process are supplied by ICC. Any additional information required by the application for verification either must be supplied by the client via the connection data parameter or must be obtainable by the server itself (for example, via SYS$GETJPI.)

Failure to ACCEPT or REJECT the connection will stall all additional attempts to connect to this server. Disconnection Events

A disconnection event signals that the partner application of the connection has requested a disconnect, the partner process has been deleted, or that the remote node has left the cluster. Upon receiving a disconnect event, the server should call SYS$ICC_DISCONNECT to clean up the server side of the connection. However, when the partner that received the disconnect event calls SYS$ICC_DISCONNECT, the connection has already been terminated and the link is already gone. The return status from this call to SYS$ICC_DISCONNECT should be SS$_NORMAL, but the completion status returned in the IOSB is SS$_LINKDISCON.

Failure to call DISCONNECT may leave resources allocated to the connection that are not released until the image terminates. In the case of inner mode associations, resources will be released at process termination. Clients

A simple client need not call SYS$ICC_OPEN_ASSOC although there may be some benefits for even the simplest clients. For a client, the major benefit of calling OPEN is declaring a data receive routine.

In cases where a client may wish to connect to multiple servers, calling OPEN_ASSOC multiple times can help isolate the routines related to data receipt and disconnection.

Call SYS$ICC_CONNECT[W] with the following arguments:

Argument Description
IOS_ICC Address of an IOS_ICC structure (defined in ICCDEF).
Optional AST address and parameter.
assoc_handle Either the value returned from the OPEN_ASSOC call or the constant ICC$DEFAULT_ASSOC if OPEN_ASSOC was not called.
conn_handle Address of a longword to receive the connection handle for this connection.
remote_assoc A string descriptor pointing to either the association name of the server or the registry name of the server if the server is using the ICC simple registry. Either use is case sensitive.
remote_node If omitted, then the assoc_name argument will be treated as a name to look up in the ICC simple registry. A zero length or blank string represents the local node. Any other string will be converted to uppercase and must match the name of a node in the cluster.
user_context A unique context value for this connection. This value will be passed to data event and disconnection routines.
Address and length of any connection data. This could include additional authentication or protocol setup data required by the application.
Address and length of a buffer to receive any data (accept or reject data) returned from the server. The buffer must be large enough to receive the maximum amount of data the server might return. The actual number of bytes will be written to retlen_addr.
flags The only valid flag is ICC$M_SYNCH_MODE. If you select synch mode, the data transmission routines (TRANSMIT, REPLY, and RECEIVE---TRANSCEIVE can never complete synchronously) may return the alternate success status SS$_SYNCH and not call the AST completion routine. If you do not specify synch mode, the AST routine, if provided, will always be called.

When the CONNECT call completes, the connection is established and useable only if success status was returned as both the return value of the service and in the status fields (ios_icc$w_status and ios_icc$l_remstat) of the IOS_ICC.

Chapter 4
Process Control

This chapter describes how to use operating system features to control a process or kernel thread. It contains the following sections:

Section 4.1 describes the control of a process or kernel thread to complete a programming task.

Section 4.2 describes how to use the operating system's process information services to gather information about a process or kernel thread.

Section 4.3 describes how to change a process's scheduling.

Section 4.4 describes the affinity and capability mechanisms for CPU scheduling.

Section 4.5 describes the class scheduler and how to use it on VAX, Alpha, and I64 systems.

Section 4.6 describes how to change a process's name.

Section 4.7 describes how to access another process's context.

Section 4.8 describes how to synchronize programs by setting specific times for program execution.

Section 4.9 describes how to suspend, resume, and stop program execution.

4.1 Using Process Control for Programming Tasks

Process control features in the operating system allow you to employ the following techniques to design your application:

  • Modularize application programs so that each process or kernel thread of the application executes a single task
  • Perform parallel processing, in which one process or kernel thread executes one part of a program while another process or kernel thread executes another part
  • Implement application program control, in which one process manages and coordinates the activities of several other processes
  • Schedule program execution
  • Dedicate a process to execute DCL commands
  • Isolate code for one or more of the following reasons:
    • To debug logic errors
    • To execute privileged code
    • To execute sensitive code

Among the services and routines the operating system provides to help you monitor and control the processes or kernel threads involved in your application are those that perform the following functions:

  • Obtaining process information
  • Obtaining kernel thread information
  • Setting process privileges
  • Setting process name
  • Setting process scheduling
  • Hibernating or suspending a process or kernel thread
  • Deleting a process
  • Synchronizing process execution

You can use system routines and DCL commands to accomplish these tasks. Table 4-1 summarizes which routines and commands to use. You can use the DCL commands in a command procedure that is executed as soon as the subprocess (or detached process) is created.

For process synchronization techniques other than specifying a time for program execution, refer to Chapters 6, 7, and 8.

Table 4-1 Routines and Commands for Controlling Processes and Kernel Threads
Routine DCL Command Task
SHOW PROCESS Return process or kernel thread information. SYS$GETJPI(W) can request process and thread information from a specific PID or PRCNAM. If no specific thread is identified, then the data represents the initial thread.
SYS$SETPRV SET PROCESS Set process privileges.
SYS$SETPRI SET PROCESS Set process or kernel thread priority. This service affects the base and current priority of a specified kernel thread and not the entire process.
SYS$SETSWM SET PROCESS Control swapping of process.
SET PROCESS Hibernate, suspend, and resume a process or kernel threads. These services hibernate, suspend, or resume all kernel threads associated with the specified process.
SYS$SETPRN SET PROCESS Set process name.
EXIT and STOP Initiate process and image rundown. All associated kernel threads of a specified process are run down and deleted.
SYS$DELPRC EXIT and STOP Delete process.
SYS$CANTIM CANCEL Cancel timer for process or kernel threads. This service finds and cancels all timers for all threads associated with the specified process.
SYS$ADJSTK SET PROCESS Adjust or initialize a stack pointer. Stack adjustments are performed for the kernel thread requesting the service.
SYS$PROCESS_SCAN SHOW PROCESS Scan for a process or kernel thread on the local system, or across the nodes in an OpenVMS Cluster system.
SYS$SETSTK None available Allow the current process or kernel thread to change the size of its stacks. This service adjusts the size of the stacks of the kernel thread that invoked the service.

By default, the routines and commands reference the current process or kernel thread. To reference another process, you must specify either the process identification (PID) number or the process name when you call the routine or a command qualifier when you enter commands. You must have the GROUP privilege to reference a process with the same group number and a different member number in its UIC, and WORLD privilege to reference a process with a different group number in its UIC.

The information presented in this section covers using the routines. If you want to use the DCL commands in a command procedure, refer to the HP OpenVMS DCL Dictionary.

Previous Next Contents Index