HP OpenVMS Systems Documentation
HP OpenVMS Programming Concepts Manual
8.2.2 Shared Data Access with Readers and Writers
The following are two types of shared data access:
If there is shared data access with multiple readers, your application must be able to tolerate having a stale counter that allows frequent looping back and picking up a new value from the code.
With multiple writers, often the AST is the writer, and the mainline code is the reader or updater. That is, the mainline processes all available work until it cannot dequeue any more requests, releasing each work request to the free queue as appropriate, and then hibernates when no more work is available. The AST then activates, pulls free blocks off the free queue, fills entries into the pending work queue, and then wakes the mainline code. In this situation, you should use a scheduled wakeup call for the mainline code in case work gets into the queue and no wakeup is pending.
Having multiple writers is possibly the most difficult to code, because
you cannot always be sure where the mainline code is in its processing
when the AST is activated. A suggestion is to use a work queue and a
free queue at a known shared location, and to use entries in the queue
to pass the work or data between the AST and the mainline code.
Interlocked queue routines, such as LIB$INSQHI and LIB$REMQTI, are
available in the Run-Time Library.
An AST routine might invoke subroutines that are also invoked by another routine. To prevent conflicts, a program unit can use the SYS$SETAST system service to disable AST interrupts before calling a routine that might be invoked by an AST. You use the SYS$SETAST service typically only if there is noninterlocked (nonreentrant) variables, or if the code itself is nonreentrant. Once the shared routine has executed, the program unit can use the same service to reenable AST interrupts. In general you should avoid using the SYS$SETAST call because of implications for application performance.
Implicit synchronization can be achieved for data that is shared for write by using only AST routines to write the data, since only one AST can be running at any one time. You can also use the SYS$DCLAST system service to call a subroutine in AST mode.
Explicit synchronization can be achieved through a lack of read-modify cells, in cases of where there is one writer with one or more readers. However, if there are multiple writers, you must consider explicit synchronization of access to the data cells. This can be achieved using bitlocks (LIB$BBCCI), hardware interlocked queues (LIB$INSQHI), interlocked add and subtract (LIB$ADAWI) routines, or by other techniques. These routines are available directly in assembler by language keywords in C and other languages, and by OpenVMS RTL routines from all languages. On Alpha systems, you can use PALcode calls such as load-locked (LDx_L) and store-conditional (STx_C) instructions to manage synchronization.
The VAX interlocked queue instructions work unchanged on OpenVMS I64 systems and result in the SYS$PAL_xxxxx run-time routine PALcode equivalents being called, which incorporate the necessary interlocks and memory barriers.
Whenever possible, the OpenVMS I64 BLISS, C, and MACRO compilers convert CALL_PAL macros to the equivalent OpenVMS-provided SYS$PAL_xxxx operating system calls for backward compatibility. Not all Alpha PAL operations are be implemented on I64; in some cases, programs calling PALcode directly may need to change. While all user-mode PAL operations that are supported on Alpha are supported on I64, there are some kernel-mode PAL operations that are not implemented on I64. You can see which PAL calls have macros supplied by looking in module PAL_BUILTINS.H in the text library SYS$LIBRARY:SYS$STARLET_C.TLB.
For details of synchronization, see Chapter 6. Also see processor
architecture manuals about the necessary synchronization techniques and
for common synchronization considerations.
OpenVMS asynchronous completions usually activate an inner-mode, a
privileged mode AST to copy any results read into a user buffer, if
this is a read operation, and to update the IO status block (IOSB) and
set the event flag. If a use-mode AST has been specified, it is
activated once all data is available and the event flag and IOSB, if
requested, have been updated.
The following lists common asynchronous programming mistakes and suggests how to avoid them:
8.4 Using System Services for AST Event and Time Delivery
The following list presents system services and routines that are used to queue the AST routine that determines whether an AST is delivered after a specified event or time. Note that the system service (W) calls are synchronous. Synchronous system services can have ASTs, but the code blocks pending completion, when the AST is activated.
If a program queues an AST and then exits before the AST is delivered, the AST is deleted before execution. If a process is hibernating when an AST is delivered, the AST executes, and the process then resumes hibernating.
If a suspended process receives an AST, the execution of the AST depends on the AST mode and the mode at which the process was suspended, as follows:
Generally, AST routines are used with the SYS$QIO or SYS$QIOW system
service for handling Ctrl/C, Ctrl/Y, and unsolicited input.
Each request for an AST is associated with the access mode from which the AST is requested. Thus, if an image executing in user mode requests notification of an event by means of an AST, the AST service routine executes in user mode.
Because the ASTs you use almost always execute in user mode, you do not
need to be concerned with access modes. However, you should be aware of
some system considerations for AST delivery. These considerations are
described in Section 8.7.
This section shows the use of the Set Time (SYS$SETIMER) system service as an example of calling an AST. When you call the Set Timer (SYS$SETIMR) system service, you can specify the address of a routine to be executed when a time interval expires or at a particular time of day. The service schedules the execution of the routine and returns; the program image continues executing. When the requested timer event occurs, the system "delivers" an AST by interrupting the process and calling the specified routine.
Example 8-1 shows a typical program that calls the SYS$SETIMR system service with a request for an AST when a timer event occurs.
8.7 Delivering ASTs
This section describes the AST service routine, some conditions
affecting AST delivery, and the affect of kernel threads on AST
delivery. The order of an AST delivery is not deterministic. The order
the ASTs are entered into the AST queue for delivery to the process is
not related to the order the particular operations that included AST
notification requests were queued.
An AST service routine must be a separate procedure. The AST must use the standard call procedure, and the routine must return using a RET instruction. If the service routine modifies any registers other than the standard scratch registers, it must set the appropriate bits in the entry mask so that the contents of those registers are saved.
Because you cannot know when the AST service routine will begin executing, you must take care that when you write the AST service routine it does not modify any data or instructions used by the main procedure (unless, of course, that is its function).
On entry to the AST service routine, the arguments shown in Table 8-3 are passed.
Registers R0 and R1, the program counter (PC), and the processor status longword (PSL) on VAX systems, or processor status (PS) on Alpha and I64 systems, and were saved when the process was interrupted by delivery of the AST.
The AST parameter is an argument passed to the AST service routine so that it can identify the event that caused the AST. When you call a system service requesting an AST, or when you call the SYS$DCLAST system service, you can supply a value for the AST parameter. If you do not specify a value, the parameter defaults to 0.
The following example illustrates an AST service routine. In this example, the ASTs are queued by the SYS$DCLAST system service; the ASTs are delivered to the process immediately so that the service routine is called following each SYS$DCLAST system service call.
8.7.2 Conditions Affecting AST Delivery
When a condition causes an AST to be delivered, the system may not be able to deliver the AST to the process immediately. An AST cannot be delivered under any of the following conditions:
If an AST cannot be delivered when the interrupt occurs, the AST is
queued until the conditions disabling delivery are removed. Queued ASTs
are ordered by the access mode from which they were declared, with
those declared from more privileged access modes at the front of the
queue. If more than one AST is queued for an access mode, the ASTs are
delivered in the order in which they are queued.
On Alpha and I64 systems with the kernel threads implementation, ASTs are associated with the kernel thread that initiates them, though it is not required that they execute on the thread that initiates them. The use of the kernel thread's PID in the asynchronous system trap control block (ACB) identifies the initiating thread. Associating an ACB with its initiating thread is required; the arrival of an AST is often the event that allows a thread, waiting on a flag or resource, to be made computable.
An AST, for example, may set a flag or make a resource available, and when the AST is completed, the thread continues its execution in non-AST mode and rechecks the wait condition. If the wait condition is satisfied, the thread continues; if not, the thread goes back into the wait queue.
On the other hand, if an AST executes on a kernel thread other than the one that initiated it, then when the AST completes, the kernel thread that initiated the AST must be made computable to ensure that it rechecks a waiting condition that may now be satisfied.
The queuing and delivery mechanisms of ASTs make a distinction between outer mode ASTs (user and supervisor modes), and inner mode ASTs (executive and kernel modes). This distinction is necessary because of the requirement to synchronize inner mode access.
With the kernel threads implementation, the standard process control block (PCB) AST queues now appear in the kernel thread block (KTB), so that each kernel thread may receive ASTs independently. These queues receive outer mode ASTs, which are delivered on the kernel thread that initiates them. The PCB has a new set of inner mode queues for inner mode ASTs that require the inner mode semaphore. With the creation of multiple kernel threads, inner mode ASTs are inserted in the PCB queues, and are delivered on whichever kernel thread holds the inner mode semaphore. Inner mode ASTs, which are explicitly declared as thread-safe, are inserted in the KTB queues, and are delivered on the kernel thread that initiates them.
If a thread manager declares a user AST callback, then user mode ASTs are delivered to the thread manager. The thread manager then is responsible for determining the context in which the AST should be executed.
There are significant programming considerations to be understood when
mixing POSIX Threads Library with ASTs. For information about using
POSIX Threads Library with ASTs, see the Guide to the POSIX Threads Library.
Before kernel threads, AST routine code of a given mode has always been able to assume the following:
Further, before kernel threads, user mode code could safely access data that it knows is only used by other user mode, non-AST level routines without needing any synchronization mechanisms. The underlying assumption is that only one thread of user mode execution exists. If the current code stream is accessing the data, then by implication no other code stream can be accessing it.
After kernel threads, this assumed behavior of AST routines and user mode code is no longer valid. Multiple user-mode, non-AST level code streams can be executing at the same time. The use of any data that can be accessed by multiple user-mode code streams must be modified to become synchronized using the load-locked (LDx_L) and store-conditional (STx_C) instructions, or by using some other synchronization mechanism.
Kernel threads assumes that multiple threads of execution can be active at one time and includes outer mode ASTs. Within any given kernel thread, outer mode ASTs will still be delivered serially. Also, the kernel thread model allows any combination of multiple outer mode threads, or multiple outer mode ASTs. However, outer-mode AST routines, as well as non-AST outer-mode code, has to be aware that any data structure that can be accessed concurrently by outer-mode code, or by any other outer-mode AST must be protected by some form of synchronization.
Before kernel threads, same-mode ASTs executed in the order that they were queued. After kernel threads and within a single kernel thread, that still is true. However, it is not true process-wide. If two ACBs are queued to two different KTBs, whichever is scheduled first, executes first. There is no attempt to schedule kernel threads in such a way to correctly order ASTs that have been queued to them. The ASTs execute in any order and can, in fact, execute concurrently.