HP OpenVMS Systems

ask the wizard
Content starts here

C and Virtual Memory Management?

» close window

The Question is:

Is there a way to determine dynamically how
much memory I can allocate through calloc ?
I have tried using getjpiw and then
calculating this amount of memory by
subtracting ppgcnt and gpgcnt from either
pagfilcnt or 1Gb, whichever is smallest,
but apparently I always seem to be off for
an indeterminate number of pages.
Also, calling calloc and free subsequently,
there appears to be less and less memory available using the above calculation.
After a number of calls though this seems to stabilize. Is there a dummy
 explanation for this?

The Answer is :

  There is no effective nor simple nor reliable means to predict if
  any particular calloc or malloc or lib$get_vm call will succeed,
  other than the simplest approach: try it, and capture any errors.
  With 32-bit addressing and the traditional address space segmentation
  model of OpenVMS virtual addressing, you are limited to a theoretical
  maximum of two (2) gigabytes (GB) of process virtual memory -- with
  the use of 64-bit addressing, rather more address space is available.
  That said, there is no entirely reliable means to determine how
  much memory is remaining in the heap, and the recommended approach
  is to simply capture and report an allocation error.  (Even if you
  could predict the remaining memory, there are things that can
  change between the check and the corresponding call, and there
  are also ways to allocate virtual memory that are not charged
  against your pagefile quota.)
  The same approach holds for checking for file access security and
  such -- while you can implement the security check in your code,
  you are duplicating the check that will be performed within OpenVMS,
  and your code must also still contend with potential changes to the
  object security profile that can occur between the application-internal
  security check and the actual access attempt.  In simplest terms, it is
  best simply to try the access, and to capture any potential failure(s).
  If you want to be absolutely certain that you have memory available,
  then you will want to pre-allocate the required memory.
  The OpenVMS Wizard prefers to use the RTL LIB$GET_VM and related
  calls, as these provide far better control over memory management
  than the inherently generic C memory management calls.  There
  are available capabilities -- such as the ability to flush an
  entire memory zone, useful when managing a pool of temporary
  memory as you don't have to track individual memory blocks for
  the deallocation -- that are entirely lacking with the C memory
  management calls.  With the statistics calls available, you can
  also track various of the memory-management related activities of
  your application using the RTL LIB$ VM.
  When you repeatedly allocate and freeing memory using the C calls
  and several sizes of memory blocks, you can run afoul of the
  documented and intended behaviour where the previous memory
  allocation remains available for immediate reallocation.  This
  long-standing allocation behaviour was deliberately implemented
  in support of (usually) older C code that implicitly assume it.
  Virtual address space is implemented using the process working set
  -- that part of process virtual memory that is currently loaded
  into physical memory -- and backing storage; that part of virtual
  memory that resides on disk storage in a section file or in a system
  pagefile. Operating systems mechanisms known as paging and swapping
  move the contents of physical memory in and out of backing storage,
  while updating the data structures known as the page tables to
  reflect the current state of virtual memory.  (The transition
  itself is known as a pagefault or a swap, though the difference
  between these two is largely irrelevent for this discussion.)
  Three process virtual address space ranges exist on OpenVMS: P0,
  P1, and P2 -- though OpenVMS Alpha prior to V7.0 and OpenVMS VAX
  lack P2 space.  P0 and P1 are the traditional portion of OpenVMS
  process address space and are one gigabyte in size each, while P2
  space is the rather larger process-specific portion of sixty-four
  (64) bit address space.
  The amount of pagefile storage that a process can allocate is
  limited by the process quota PGFLQUOTA.  The amount of section
  file storage is limited by the available disk space and (if
  enabled) by the user's disk quota settings.  Taken together,
  these two quotas are the basic limit on the amount of virtual
  address space that can be written to and can be modified beyond
  the process working set storage.
  PPGCNT is a count of pages in the process working set; the amount
  of physical memory.  GPGCNT is a count of global pages, which is
  effectively physical memory that is not charged against the process
  pagefile quota.  Neither of these are specific to the amount of
  remaining process virtual address space, which is the limit behind
  the C and RTL LIB$ VM memory management calls.  The amount of process
  virtual address space is (coursely) limited by the VIRTUALPAGECNT
  system parameter and (for that part of the virtual memory that is
  backed there) by the amount of process pagefile quota available.
  You can track peak process virtual memory use with the VIRTPEAK
  item available on the $getjpi.
  The only available estimate for remaining P0 virtual address space
  -- and it is a poor estimate -- is based on the virtual address
  (hex) 3FFFFFFF (the architected top of P0 space) and the FREP0VA
  address (the first free address within P0 space).  This potential
  allocation is limited by the remaining pagefile quota -- and this
  calculation will overestimate remaining address space, as there is
  overhead within individual memory allocations, and as there is
  likely other code active that can be consuming and releasing heap
  space.  Further, applications can also use P1 or P2 space.
  Some of the related topics include (1661), (3257), and (7552).
  Also of interest will be (2486), (3115), (3748), (3764), and (5455).
  For details of virtual memory, paging, swapping, pagefaults and
  pagefault handlers, page tables, and other memory-related operating
  system details, please see one of the Internals and Data Structures
  Manuals for OpenVMS.

answer written or last revised on ( 9-SEP-2002 )

» close window