Useful Utilities Code that makes our lives easier Paul Rose
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Useful Utilities
Code that makes our lives easier
Paul Rose
EOTC May 2019
Version 1.0Some Stats
75 Systems - 18 Labs, 18 Development, 39 Live
35 SA Plexs
17 SysPlexs
‘4 site’ GDPS environment.
Working on fully automated ‘Stretch Sysplex Siteswap’ so we can move to
alternate location without outage.
Standard code on all systems.
2 | Useful Utilities | May 2019
Restricted - External£SYSINFO - System Information
Although our setup is standard on all systems, not all systems are standard.
This provides a central method of storing and querying system information,
enterprise wide.
Hard coded Data includes;
Domain name, SAPlex name, Initiator Class, Storage Managment Class, DASD
Volumes, System type (LABS DEV LIVE), RACFPlex name, 4 Way Data Sharing,
etc
Dynamic Data;
Sysplex name, Processor & LPAR name, Region and Site, CMR,
State (UP SHUT DOWN IPL STRT)
3 | Useful Utilities | May 2019
Restricted - External£SYSINFO - Usage
£SYSINFO has a powerful query function:
• Named fields or global search
• Pattern matching
• AND and OR
• NOT
Any data can be returned.
Call from command line, as REXX Function or via PIPE.
4 | Useful Utilities | May 2019
Restricted - External£SYSINFO - Examples
Query all the UP systems and domains names in the current SAPLex:
£SYSINFO SAPLEX:GZPLEX+STATE:UP DOMAIN
ZACOA ZACOB ZACOC ZACOD ZACOE ZACOF
Systems not UP:
£SYSINFO STATE:¬UP SYSTEM+STATE
GZF2+DOWN
Systems running on a processor and their LPARs:
£SYSINFO PROCESSOR:BONDI SYSTEM+LPAR
GZT1+BONDB1 GZT3+BONDB2 GZT5+BONDB3 GZFO+BONDB7
GZK1+BONDA1 GZM2+BONDBE ZCT8+BONDC2
5 | Useful Utilities | May 2019
Restricted - External£SYSINFO – How does it work?
On system start-up the dynamic data for the current system is determined.
This data is passed to all other defined systems. In return the other systems
return their dynamic data. Any systems not in UP state are returned by the
first system queried. This ensures time is not wasted trying to communicate
to these systems.
From then on, the only value that can change is the STATE. When this
changes it is passed out to all other systems to keep them up to date.
STATE is a key value that is used by lots of automation routines. Especially in
alerting for preventing alerts when systems are closed/re-IPLd.
6 | Useful Utilities | May 2019
Restricted - External£SYSINFO - State
UP VTAM and IP communications are available
SHUT System shutdown in progress – JES2 Desired UnAvail.
DOWN System has left the SysPlex;
– IXC101I SYSPLEX PARTITIONING IN PROGRESS FOR system REQUESTED BY jobname REASON: reason
– IXC113I BCPII CONNECTION TO SYSTEM system RELEASED
– ISG013I SYSTEM sysname - PURGED FROM GRS COMPLEX
Using these messages means we can also trap system failure.
IPL System is being IPLd;
– AOFA0900 proc.lpar xx &SYSNAME. = "xxxx"
– MII0908I system xxxx reported ACTIVE
STRT IPL has come through and the system is starting;
– MII0366I system xxxx joined established MIM complex
7 | Useful Utilities | May 2019
Restricted - External£SYSINFO – IPL Complete
£SYSINFO also tracks IPL Completion.
This is a local system feature only. It is not shared with other systems.
The UP state only tells us the core communications interfaces are available –
it doesn’t mean the system is fully UP.
For IPL Complete, £SYSINFO initially tracks the XEQ/APL. This is a child of all
the main application components of the system. Once this is SATISFACTORY
AVAILABLE it tracks any other APLs that are being started are SATISFACTORY
AVAILABLE and MTRs are NORMAL. Once all these are in place the IPL is
deemed Complete.
8 | Useful Utilities | May 2019
Restricted - External£RMTCMD – Issue command to remote system
Utility for issuing commands on a remote system.
Traps and handles all identified errors for easy error handling.
Uses £SYSINFO STATE so only allows communications to UP systems.
Can be used as a REXX function and within PIPEs.
Uses multiple dedicated autotasks so very little command queueing at busy
periods.
9 | Useful Utilities | May 2019
Restricted - External£ALL – Issue command to ALL systems
Interface to issue a command to multiple systems and collate the responses.
System selection is via the £SYSINFO query language so its easy to target
system subsets.
Returned data can be; displayed on console (message colouring is preserved)
or saved into a dataset. Data can be prefixed with the system or domain
name for easy post-processing.
Asynchronous mode for handling long running commands.
Can be used on command line or PIPE.
10 | Useful Utilities | May 2019
Restricted - External£ALL – Example
£ALL PRESYS SAPLEX:GZPLEX MVS D SYMBOLS,SYMNAME=LPARID
11 | Useful Utilities | May 2019
Restricted - External£NOTIFY – Tell users about events
This is separate to Operator alerting.
£NOTIFY is used to tell support teams of important messages in real time.
The message are sent to a central system for immediate TSO Broadcast to
relevant users. Messages can be are saved to the broadcast dataset so users
can receive the messages when logging on after time out of the office.
Notifications are saved into Audit log datasets which allows for historical
data and trending.
Team specific setups. Uses RACF Groups so users are not hard coded.
Allows central event logging without user notification.
Integration with the MAT and other execs.
12 | Useful Utilities | May 2019
Restricted - External£WTO - WTO issuing utility
The WTO text and characteristics are stored in NetView panels. This ensures
the WTO and the Help panel data match.
Placeholders in the panel data allow for easy variable data.
• Descriptor and Route code specification.
• Automatic timed DOM.
• Multiline and WTOR handling.
• Validation of WTOR replies and timed WTOR replies.
• Save SMSGID in Global variable for later DOMing.
• Automatic notification (using £NOTIFY).
• Delayed WTOs
• Post message to SA APL via AOFCPMSG call
13 | Useful Utilities | May 2019
Restricted - External£WTO - Example
/* AOxxxnnna text %P1% text %P2%
/* DESC(n)
/* ROUT(n)
/* DOM(secs)
/* WTOR(replies)
/* WTORAUTO(n reply)
/* NOTIFY(group)
/* GLBLDOM(glblname)
/* DELAY(hh:mm:ss)
/* TOKEN(token)
/* SAMSG(jobname)
***
+msgid %Online Message Help
% msgid text
%
14 | Useful Utilities | May 2019
Restricted - External£CARLA – RACF Query/Update
Interface to RACF CARLA which can be called direct from the command line
or as REXX function (data is saved in SAFEs).
Pass the CARLA command only. Setting up the CARLA environment and error
handling is all contained within the utility.
£CARLA S C=USER S=BASE MASK=AUTINIT*;DISPLAY KEY
Profile key
AUTINIT1
AUTINIT2
15 | Useful Utilities | May 2019
Restricted - External£APLACTS - APL Actions
Advanced actions needed when starting or stopping APLs.
Started Job handling.
When starting and stopping APLs we regularly have to run one or more
started jobs to start/stop remote links. These are not queriable so cannot be
setup as normal APLs. Using the utility allows us to define the jobs within
UserData so everything is contained within the APL definition.
Dynamic Relationships.
Some APLs need their stop to be delayed on system shutdown. We can’t use
normal relationships as these will prevent them being closed during normal
operations. Uses MDFYSHUT to delay the shutdown and INGSTOBS to
monitor specific APLs for a specific state to allow shutdown to continue.
16 | Useful Utilities | May 2019
Restricted - External£APLACTS Examples
Once APL has started during IPL, signon to remote service
POSTSTART: £APLACTS NAME:POSTSTART
SPT9ACT3 £APLACTS
GZT3_POSTSTART_1= IF GLBL BAP.£SYSINFO.IPLING Y :GOOD NEXT
GZT3_POSTSTART_2= JOB KIT9UPD1 00:01:00 :GOOD NEXT :BAD ACTION 90
GZT3_POSTSTART_3= JOB KIT9UPD2 00:01:00 :GOOD NEXT :BAD ACTION 90
GZT3_POSTSTART_4= JOB KIT9UPD3 00:01:00 :GOOD EXIT :BAD ACTION 90
GZT3_POSTSTART_90= £WTO AOVSA001A SignOn
Hold shutdown of APL on system shutdown
AONVEAS SHUTNORM
CMD=(PASS1,'£APLACTS')
CMD=(PASS2,'MVS P &SUBSJOB')
CMD=(PASS4,'MVS C &SUBSJOB')
AONVEAS £APLACTS
ALL_STOP_1=IF DESIRED JES2 UNAVAILABLE :GOOD NEXT :BAD ACTION 99
ALL_STOP_2=SAFREEZE 01:00:00 :GOOD NEXT :BAD ACTION 99
ALL_STOP_3=INGSTOBS TCPA OBSERVED STOPPING 01:00:00 :GOOD ACTION 99 :BAD
ACTION 99
ALL_STOP_99=SANEXT :GOOD EXIT :BAD EXIT
17 | Useful Utilities | May 2019
Restricted - External£DUMMY
Used to fool SA into thinking an APL is AVAILABLE, when it isn’t.
Typically used in DR Testing where APLs that have children are not wanted.
With DUMMY we can allow the children to be started. Eg Hardware
encryption tasks
Also used for started jobs that cannot be ‘monitored’ but need start and stop
commands. These are a bit like Transient, but they do have shutdown jobs,
so we cant define as transient. Once the ‘start’ started job has been run turn
on DUMMY so we can tell SA it is active. We can then run the ‘stop’ started
job on system shutdown.
How?
£DUMMY as Monitor Routine.
£DUMMY DUMMY &SUBSAPPL to flag as active/available.
£DUMMY CLEAR &SUBSAPPL to clear flag
18 | Useful Utilities | May 2019
Restricted - External@RESSTRT Resource Start
Used to add votes/suspends at SA initialisation or Refresh prior to SA taking
any actions.
Run from SYS entry Exit Name(s): definition. Needs AOFSERXINT = 0 to
serialise exec execution.
At this point SA is not taking actions, but is initialised sufficiently for ING
commands.
Reads member which defines actions to be taken. STOP and DUMMY in DR
are the most common.
We can also dynamically add definitions. Eg to add STOP votes for new APLs,
or add STOPs if policy is backed out (where a deleted APL is re-added).
19 | Useful Utilities | May 2019
Restricted - External@AUTAPLY – Automatic Apply
Performs all the actions needed when new code and policy is applied or
restored.
The members shipped are identified and compared against a list of actions.
The actions are then done in the required order. Eg: MEMSTOUT, MAT loads,
NPS refresh, MPF load, ACF REFRESH etc etc.
Additional actions can be defined with the changes as they are shipped, so
one off actions can also be included. E.g. using @RESSTRT to add votes for
new APLs.
With this our weekly change rollout is simple.
20 | Useful Utilities | May 2019
Restricted - ExternalPDB Utilities - Find
List APLs containing specified text in APL name and/or description.
Eg: CTL to find all IMS Control regions
IMLBCTL IMLB Control Region||||
IMLCCTL IMLC Control Region||||
IML2CTL IML2 Control Region||||
IML6CTL IML6 Control Region||||
IML8CTL IML8 Control Region||||
IML9CTL IML9 Control Region||||
IMSACTL IMSA Control Region||||
IMSBCTL IMSB Control Region||||
21 | Useful Utilities | May 2019
Restricted - ExternalPDB Utilities – Group Membership
List the APLs membership of groups. Filter using Group Name or APL Name.
Easy to find which groups an APL is defined to.
Particularly useful for sysplex groups that have definitions spread over
several RESOURCES screens.
CAPPROV_M/APG CAPPROV/APG/BBL1 2000
CAPPROV_M/APG CAPPROV/APG/BBL5 700
CAPPROV_M/APG CAPPROV/APG/DET4 2000
CAPPROV_M/APG CAPPROV/APG/DET7 700
CAPPROV_M/APG CAPPROV/APG/GZT1 2000
CAPPROV_M/APG CAPPROV/APG/GZT2 700
Vs SA;
Entry Type : ApplicationGroup PolicyDB Name : LIVE
Entry Name : CAPPROV_M Enterprise Name : BARCLAYS
Action Sysplex Selected Resources
GZPLEX 2
LIVEBB 2
LIVEDE 2
******************************* Bottom of data *******************
22 | Useful Utilities | May 2019
Restricted - ExternalPDB Utilities - Owner
We use Resource Security (via SYSAUTO RACF) to control which users have
the ability to control resources. We have built an automated system based
around the APL OWNER, so when we load a new policy a process traps the
updated APLs (via AOFEXC25) and updates the RACF accordingly (using
£CARLA)
This view allows us to see all the defined owners for every APL. We use this
to ensure owner is defined and is valid.
23 | Useful Utilities | May 2019
Restricted - ExternalPDB Utilities - Relationships
From within the PDB its only possible to see relationships from the defined
resource. With this display we can see relationships from the supporting
resource.
This is useful when changing a mid-tree resource as we can identify which
resources have relationships defined to the APL being changed.
Resource Typ Rel Type
Supp Resource A Condition
VTAM
-------------------- --- ---------------- --------------------- - ---------
AAUTSKLP APL HASPARENT VTAM/APL/= StopsMe
ABCACIC APL HASPARENT VTAM/APL/=
ABENCIC APL HASPARENT VTAM/APL/=
ABL6ACT1 APL HASPARENT VTAM/APL/=
ADSM APL HASPARENT VTAM/APL/=
AFL2ACT2 APL HASPARENT VTAM/APL/=
AFL3ACT1 APL HASPARENT VTAM/APL/=
AFL3ACT4 APL HASPARENT VTAM/APL/=
AFL3ACT6 APL HASPARENT VTAM/APL/=
etc.
24 | Useful Utilities | May 2019
Restricted - ExternalPDB Utilities - Gateways
Defining Gateways is complex as it needs a separate definition for each local
system. This means its easy to define one end but not the other. To help with
this we have a display that summarises the Gateway definitions and checks
each end is correctly defined.
Sys GWay FromNTW Opposite NTW
---- ----- ---- -------------- --------------
BBKS BACOA BBL1 BB_TWSLINK BB_TWSFP
BBKS BACOB BBL2 BB_TWSLINK BB_TWSFP
BBKS SACO1 STL1 BB_SDFLINK STL1_SDFTWSFP
BBKS SACO3 STL3 BB_SDFLINK STL3_SDFTWSFP
BBK1 SACO1 STL1 BB_LINK_GL STL1_SDFTWSFP
BBK1 SACO3 STL3 BB_LINK_GL STL3_SDFTWSFP
BBK2 SACO1 STL1 BB_LINK_GL STL1_SDFTWSFP
BBK2 SACO3 STL3 BB_LINK_GL STL3_SDFTWSFP
PS. IBM did state some years ago they would remove the need for Gateways
for TWS outside a sysplex. Any movement on this?
25 | Useful Utilities | May 2019
Restricted - ExternalYou can also read
