The Summarize Journal Entries function allows you to 'net' the
journal entries so there is only a single entry for each relative
record number in a file/member. This reduces the number of journal
entries that must be applied if recovery is needed. A separate
command is used to apply the entries.
To use SUMJRNENT, you must first convert the journal entries to a
data base file using the i5/OS DSPJRN command. DSPJRN allows you to
control what file/entries you want to be converted, the journal
receivers to be used, the journal entry to start at etc.
*****************
* *
* Journal *
* Receiver *
* *
*****************
*
*
DSPJRN ....... ENTDTALEN(xxx)
*
*
DSPJRN OUTFILE
*************************************************************
* 125 Bytes * Nbr of bytes per ENTDTALEN parm *
* Standard fields ***************************************
* eg File, member, * * *
* RR number ... * Physical record image * *
*************************************************************
A typical command would be to convert all of the journal entries for
the current journal receiver such as:
DSPJRN JRN(JOURNALX) ENTDTALEN(500) JRNCDE(R)
OUTPUT(*OUTFILE) OUTFILE(LIB1/JRNOUTP)
The ENTDTALEN parameter is very important. It must be specified to
hold the largest physical record image (record length) being
journaled. If it is less than the largest physical record length,
the APYSUMJRN command will not apply changes for that file.
The use of JRNCDE(R) only converts the 'record' entries that appear
in the journal. These are the only journal entries processed by
SUMJRNENT. If other entries exist, they will be bypassed. See the
later discussion on the specific entry types processed.
A typical command to summarize the changes for a specific outfile
would be:
SUMJRNENT DSPJRNOUT(JRNOUTP) SUMENTP(SUMENTP)
This merges the contents of the latest converted journal entries in
JRNOUTP with the existing summarized information in SUMENTP. The new
'net' results are stored in SUMENTP. You must initially create this
file (See the implementation section).
In a typical environment you would keep summarizing the changes until
the file had been saved again. Each save of the file being journaled
represents a synchronization point to recover from. When the file is
saved, you would use CLRPFM to clear the SUMENTP file. If recovery
is needed, the latest saved version of the file would be restored and
the summarized journal changes would be applied using the APYSUMJRN
command. See the later section on SUMJRNENT scenario for more
details.
A typical command to apply the changes for a specific file would be:
APYSUMJRN FILE(FILEA)
A report is printed to QPRINT which describes the number of entries
applied for each member. If an error has occurred on a member, the
problem is identified and no further updates to that member will
occur.
Because the update occurs by relative record number the function is
year 2000 ready.
SUMJRNENT scenario
------------------
The SUMENTP file will contain the summarized changes. When the file
being journaled is saved, you need to clear the SUMENTP file. Each
time the file is saved represents a new recovery or synchronization
point.
A typical scenario would be a daily routine (or multiples times per
day) of:
- Change to a new journal receiver
- Use DSPJRN to convert the entries to an outfile
- Summarize the entries with SUMJRNENT
- Backup the SUMENTP file
- Optionally backup the journal receiver
Periodically you would establish a new synchronization point by:
- Backing up the file being journaled
- Use CLRPFM against SUMENTP
If recovery is needed, you would:
** Do the daily routine to merge in the last journal receiver.
If you are recovering because of a problem which occurred at a
specific time of day, you would only want to convert the
entries to that specific point using DSPJRN.
** Restore the last backup of the file being journaled
** Use APYSUMJRN for the file
If you reorganize or initialize the file you should then backup the
file and clear SUMENTP to establish a new synchronization point.
SUMJRNENT is dependent on consistent relative record numbers. Note
that neither the RGZPFM nor INZPFM entries in the journal will be
processed by SUMJRNENT.
If you use a 'put direct' function in your application or
REUSEDLT(*YES), see the later section.
One or multiple SUMENTP files
-----------------------------
Normally, you would have multiple files being journaled. The SUMENTP
file can contain journal entries for more than one file. You can
have multiple SUMENTP files, but each should be in a separate
library.
APYSUMJRN only operates on a single file per use of the command. To
recover multiple files, you would need to run the command for each
file to be recovered.
In a typical environment, you would backup all files where the
journal entries were being summarized at the same point (e.g. Sunday
night). Since you have a common synchronization point (the point of
the save), only a single SUMENTP file is needed. You could consider
multiples for your own reasons.
If you do not have a common backup point for the files where the
journal entries are being summarized, you need at least one SUMENTP
file for each backup group. For example, if you normally backup
files A, B and C on one day and D, E and F on a separate day, you do
not have a common synchronization point for all of the files. You
would need at least two SUMENTP files so that CLRPFM can be used at
the same time as the save.
Note that the SUMENTP2 file described in the implementation steps
does not have to be cleared. This occurs automatically by use of the
SUMJRNENT command.
Error conditions
----------------
Some error conditions will be found by SUMJRNENT. However, unlike
the TAA tool APYUSRCHG, there are several conditions which can only
be assumed to be valid. Because only the last journal entry is kept
for a unique relative record number, the following examples are
considered valid:
** An update entry exists for a non-existent relative record
number. It is assumed the add has occurred previously,
therefore the update is turned into an add.
** An add entry exists for an existing relative record number.
It is assumed that a delete has occurred previously, therefore
the add is turned into an update.
The following types are considered errors:
** The length of the file to be updated differs from the length
described in the JOENTL field of the journal entry.
** For any records added by PT entries, the relative record
number of the journal entry is compared with the relative
record number found in the file feedback area after the add
occurs. If a difference exists, the file restored (or copied
back), is not synchronized with when CLRPFM of the SUMENTP
file occurred.
** The maximum record length of the program has been exceeded.
See the later discussion of how to modify the program.
** The file/member could not be opened. The member must exist
and the user must be authorized.
** There are two special errors which can occur if you attempt to
delete a record that does not exist in the file. This would
be caused by the fact that an add occurred as a prior journal
entry, but the summarization function has removed the add and
kept only the delete. In this case the SUMJRNENT function
must place a deleted record into the file so that any
subsequent additions will be added to the proper relative
record number.
The INZPFM command supports the capability to initialize a
specified number of deleted records. If records exist in the
file, only the additional number of records required will be
initialized. For example, if 100 records exist in the file
and the user requests to initialize 110 records, records 101
thru 110 would be initialized. Records 1 thru 100 would be
left as is.
The SUMJRNENT command uses the relative record number of the
delete journal entry type to request INZPFM. For they typical
use of SUMJRNENT, the only valid condition is when INZPFM
causes a single record to be initialized. Therefore, two
error types are possible which indicate that the CLRPFM of the
SUMENTP file was not synchronized with the restored backup
version.
No records were initialized which means that more records
exist in the file than are accounted for by the delete
relative record number.
If more than 1 record was initialized, it indicates that not
enough records exist in the file.
Entry types processed
---------------------
SUMJRNENT will only process the JOCODE = R type journal entries from
the DSPJRN outfile. If other entries exist, they will be bypassed.
The Entry types processed are:
PT Put. Add a new record. This type of entry always
causes a record to be added at the end of the file.
UP Update. Update a record. The entire record image
exists in the journal entry.
DL Delete. Delete a record.
DR Delete at rollback. This entry only occurs if you
are using commitment control. As part of the commit
cycle, a record was added and then a rollback
occurred. This entry causes the record that was
added to be deleted.
UR Update at rollback. This entry only occurs if your
are using commitment control. As part of the commit
cycle, a record was either deleted or updated and
then rollback occurred. This entry causes the
original value of the record to be written back to
the data base.
PX Put direct. This entry is caused by the use of a
write (add) to a specific relative record number.
The record number written to must be a deleted
record. The record could have been deleted by
either a delete operation or initializing the file
to deleted records.
Most HLL usage cause PT entries to occur when adding
to a file. To request 'put direct' in RPG, you must
use the RECNO file continuation function.
When 'Put direct' is used, the apply function logic
must change because the relative record numbers of
new records will not appear in sequence. Fewer
logical errors can be checked when PX entries exist.
The most typical use of PX entries occurs when the
file is specified as REUSEDLT(*YES). PX entries are
written into the deleted record slots (if they
exist) instead of being added to the end of the
file.
Because the process differs, APYSUMJRN requires a
special parameter to be specified to process PX
entries. You must specify PXUSD(*YES) if PX entries
exist in the journal.
The following chart describes the processing of the entries. The
'Record Exists' information describes whether the relative record
number of the journal entry exists as an active (not deleted) record
in the data base.
Record does not exist
Entry Record ---------------------------
type exists PXUSD(*NO) PXUSD(*YES)
------- -------- ------------ ------------
PT Update Add at EOF Add at EOF
UP Update Add at EOF Put direct
DL Delete Inlz 1 at EOF Note 1
DR Delete Inlz 1 at EOF Note 1
UR Update Add at EOF Note 3
PX Update Note 2 Note 3
(Note 2)
Note 1. Initialize by specifying the number of records as
described by the RR number.
Note 2. If PXUSD(*NO) is specified, an error occurs if any
PX entries exist.
Note 3. Try Put Direct. Monitor for CPF5006 which occurs if
the RR is beyond the EOF. If so, initialize as
described by the RR number. Try Put Direct again.
If it fails the second time, abort
The UP = Update Before journal entry caused by commitment control is
bypassed by SUMJRNENT.
Recovery with Put Direct journal entries
----------------------------------------
Because of the logic difference required in applying entries if the
PX (Put Direct) entries exist, you must specify PXUSD(*YES) on the
APYSUMJRN command.
Before you apply the entries, you should ensure the file is
initialized as it was at the time of the synchronization point.
There are two typical uses of 'Put direct'.
** You have a file that is specified as REUSEDLT(*YES). The
system will add new records to any existing deleted record
slots instead of at the end of the file.
** You have a file that uses a relative record number to access
records. Normally some algorithm would be used to convert a a
key like customer number to a relative record number and then
you would access the record. Your program logic would have to
handle the case of multiple customer numbers generating the
same relative record number. This type of file would normally
be initialized to deleted records. For example, you might
initialize a file with 10,000 entries even though you had only
5,000 records to be added. When you add a record to the file,
you want to write it to one of the deleted record slots.
If you save the file, the deleted records are saved.
Therefore, a restore to the synchronization point will work
properly. If you use some other technique of recovering the
file to the synchronization point, you may need to initialize
the file again to allow the PX entries to be properly applied.
SUMJRNENT command parameters *CMD
----------------------------
DSPJRNOUT The qualified name of the data base file that was
specified as the output file for DSPJRN. The
library defaults to *LIBL.
SUMENTP The file containing the summarized changes. The
file must be initially created by you. See the
implementation section.
APYSUMJRN command parameters *CMD
----------------------------
JRNLDFILE The qualified name of the data base file that was
journaled. Only a single file may be named for each
use of APYSUMJRN.
JRNLDMBR The member of the JRNLDFILE to be applied to. The
default is *ALL. A specific member may be named.
SUMENTP The qualified file name of the file containing the
summarized changes. The file contains data written
by the SUMJRNENT command. The library defaults to
*LIBL.
STRNBR The start relative record number to begin applying
at for the named file. The default is *BEGIN which
applies any relative record number found.
STRNBR is intended for recovery purposes. If an
error occurs after some of the entries have been
applied, you may be able to make a manual change.
If you wish to restart the process at a specific
relative record number, you can use this parameter.
Since the records are in relative record number
order in the SUMENTP file, the relative record
number is used for a restart rather than the journal
entry ID.
PXUSD This is a *YES/*NO value with *NO as the default.
Specify *YES if you have made PX entries in the
journal for the file.
PX entries are caused by the 'put direct' function.
Additions to a file normally occur at the end of the
file and generate PT entries. A 'put direct'
function can be specified to write directly to a
specific relative record. The record on the data
base must contain a deleted entry. This would be
caused by either a prior operation that deleted the
record or by use of the INZPFM RECORDS(*DLT) option.
PX entries also occur if the file is specified as
REUSEDLT(*YES) and a record is added by the system
into an existing deleted record slot.
If PX entries are found in the journal and
PXUSD(*NO) is specified, an error occurs and the
remaining entries will not be applied. If
PXUSD(*YES) is specified, the logic for handling
certain types of journal entries changes. See the
earlier discussion.
Restrictions
------------
** The apply function will apply to a library/file/member of the
same name as originally journaled. The file does not have to
be the same file as was originally journaled nor does it have
to be journaled at this time. The only requirement is that it
be the same name. See the earlier section on error conditions
that will be checked.
** A physical file may not exceed 2000 bytes in record length.
If you have files that exceed this length, you can change the
program to allow for it. See the later section.
** Like the i5/OS APYJRNCHG command, SUMJRNENT is totally
dependent on consistent relative record numbers to apply. If
you reorganize the file or copy it without COMPRESS(*NO), the
relative record numbers are likely to change. Although some
logical errors will be found by APYSUMJRN, you must be careful
in how you are performing the apply.
The APYJRNCHG command has a similar set of restrictions, but
also prevents the apply from crossing a boundary such as
RGZPFM. SUMJRNENT assumes that you are in control of when
RGZPFM, CLRPFM and INZPFM have occurred and are working with
the proper set of journal entries to be applied.
Prerequisites
-------------
The following TAA Tools must be on your system:
SNDCOMPMSG Send completion message
SNDESCMSG Send escape message
SNDSTSMSG Send status message
SORTDB Sort data base
Implementation
--------------
The tool is ready to use, but the journal entries must first be
converted to a data base file with the DSPJRN command. When you
specify DSPJRN, you must use an entry for the ENTDTALEN parameter
that describes the largest record length that will be in the file.
The use of JRNCDE(R) should also be used to reduce the number of
records in the outfile that will be read by SUMJRNENT.
If your largest record length is 500 bytes, a typical command would
be:
DSPJRN JRN(JOURNALX) ENTDTALEN(500) JRNCDE(R)
OUTPUT(*OUTFILE) OUTFILE(LIB1/JRNOUTP)
The SUMJRNENT command requires two files be created. The first can
be any name, but SUMENTP is suggested. The second must be named
SUMENTP2 and must be in the same library as the SUMENTP file. The
purpose of the second file is to hold the merged output. It is then
used to copy back (automatically) to SUMENTP with the new net
entries.
It is best to create these files using CRTDUPOBJ based on the file
you named in the DSPJRN file for the OUTFILE parameter. The outfile
will have the proper description of the fixed fields and the length
you specified for the data by the ENTDTALEN parameter. Both files
should have the exact same definition.
CRTDUPOBJ OBJ(JRNOUTP) FROMLIB(xxxx) OBJTYPE(*FILE)
TOLIB(xxxx) NEWOBJ(SUMENTP)
CRTDUPOBJ OBJ(JRNOUTP) FROMLIB(xxxx) OBJTYPE(*FILE)
TOLIB(xxxx) NEWOBJ(SUMENTP2)
You should increase the record capacity size of the two files
according to how many summarized entries you will accumulate. If you
want to allow for 200,000 entries, you would specify:
CHGPF FILE(SUMENTP) SIZE(200000)
CHGPF FILE(SUMENTP2) SIZE(200000)
Handling larger than 2000 byte records
--------------------------------------
To provide better performance, SUMJRNENT is written to handle
physical record lengths up to 2000 bytes. The following describes
the changes needed to increase the size.
For the TAAJRNHR program:
** At about statements 2.00, 3.00 and 4.00 are the file
descriptions. The record size is 2125. Each journal entry
has 125 bytes of heading information. If your largest record
length is 5000 bytes, change the '2125' values to 5125.
** At about statement 33.00 is the input field description of the
JOESD field. This is the length of your largest data record.
The 'from position; should remain 126. The 'to position'
should be changed. If your largest record size is 5000 bytes,
change the '2125' value to 5125.
** At about statement 52.00 is the input field description of the
JOESD2 field. This is the length of your largest data record.
The 'from position; should remain 126. The 'to position'
should be changed. If your largest record size is 5000 bytes,
change the '2125' value to 5125.
** At about statements 54.00, 56.00 and 58.00 are the JOESD,
JOESD2 and JOESDX fields described as data structure fields to
avoid the RPG maximum field length restriction. If your
largest record length is 5000, change the '2000' values to
5000.
** At about statement 113.00 is the output location for the field
JOESDX. If your largest record length is 5000, change the
'2000' value to 5000.
For the TAAJRNHR2 program:
** At about statement 2.00 is the file descriptions for the
SUMENTP file. The record size is 2125. Each journal entry
has 125 bytes of heading information. If your largest record
length is 5000 bytes, change the '2125' value to 5125. If the
size of your SUMENTP file is less than 2000 bytes, you do not
need to change the value.
** At about statements 3.00 and 8.00 are the file descriptions
for the file being applied to. Program described files are
used since the actual file is controlled by your command
specification. A second file is needed to handle the PX
entries. The record size for both files is 2000. If your
largest record length is 5000 bytes, change the '2000' values
to 5125.
** At about statement 36.00 is the input field description of the
JOESD field. This is the length of your largest data record.
The 'from position; should remain 126. The 'to position'
should be changed. If your largest record size is 5000 bytes,
change the '2125' value to 5125.
** At about statement 46.00 is the JOESD field described as a
data structure field to avoid the RPG maximum field length
restriction. If your largest record length is 5000, change
the '2000' value to 5000.
** At about statement 136.00 is the test to ensure that the
program is not being requested to update a record greater than
the maximum record length specified in the program. If your
largest record length is 5000, change the '2000' value to
5000.
** At about statements 367.00 and 369.00 are the output locations
for the field JOESD. If your largest record length is 5000,
change the '2000' values to 5000.
Security considerations
-----------------------
Normal security is used. The user of the commands must be authorized
to make changes to each of the files to be applied to.
You may want to control who is authorized to use the commands and
prevent public use.
Objects used by the tool
------------------------
Object Type Attribute Src member Src file
------ ---- --------- ---------- ----------
SUMJRNENT *CMD TAAJRNH QATTCMD
APYSUMJRN *CMD TAAJRNH2 QATTCMD
TAAJRNHC *PGM CLP TAAJRNHC QATTCL
TAAJRNHC2 *PGM CLP TAAJRNHC2 QATTCL
TAAJRNHR *PGM RPG TAAJRNHR QATTRPG
TAAJRNHR2 *PGM RPG TAAJRNHR2 QATTRPG
Structure
---------
SUMJRNENT Cmd
TAAJRNHC CL
TAAJRNHR RPG
APYSUMJRN Cmd
TAAJRNHC2 CL
TAAJRNHR2 RPG
|
