Implementing Linux on IBM System z: Best Practices - August 2006
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
August 2006
Implementing Linux on
IBM System z:
Best Practices
Jason C. Hortman
Richard G. Young
IBM Client Technology Center
IBM Systems and Technology Group
jhortman@us.ibm.com
ryoung1@us.ibm.comImplementing Linux on IBM System z: Best Practices
TOC
Table of Contents
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 2
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 2
1. A Brief Overview of Linux on System z (The why, what, and how of Linux on
System z) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 3
1.1. Why Linux? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 3
1.2. How Linux runs on the System z . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 5
1.3. Software Packages Available for Linux on System z . . . . . . . . . . . . . . . . . . . . . . Page 5
2. Installation of Linux on System z – A Brief Overview . . . . . . . . . . . . . . . . . . . . . . Page 7
2.1. Know your objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 7
2.2. Choose a distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 7
2.3. Choose a hosting configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 8
2.4. System z Host Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 9
2.5. Post Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 10
3. Linux Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 11
3.1. Basic System Administration Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 11
3.2. Advanced System Administration Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 13
4. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 16
4.1. Network Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 16
4.2. Linux System Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 17
5. WebSphere Application Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 18
5.1. Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 18
5.2. Installing WebSphere Family Products on Linux on System z . . . . . . . . . . . . . . Page 18
5.3. Best Practices for WebSphere on Linux for System z . . . . . . . . . . . . . . . . . . . Page 19
6. Creating a Highly Available Environment with Linux for System z . . . . . . . . . . . . Page 21
6.1. The Definition of High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 21
6.2. Hardware Failure Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 22
6.3. Software Failure Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 23
7. Virtualization Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 27
7.1 Basic z/VM Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 27
8. Monitoring and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 28
8.1. Recommended Monitoring tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 28
8.2. Performance Tuning Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 31
9. Customer Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 34
Customer XYZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 34
Appendix A – External Materials and Web sites Referenced in this White paper . . . . Page 36
Section 1: A Brief Overview of Linux on System z . . . . . . . . . . . . . . . . . . . . . . . . . Page 36
Section 2: Installation of Linux on System z . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 36
Section 3: Linux Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 36
Section 4: Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 36
Section 5: WebSphere Application Server on Linux for System z . . . . . . . . . . . . . . . . Page 37
Section 6: Creating a Highly Available Environment with Linux for zSeries . . . . . . . . . Page 37
Section 7: Virtualization Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 37
Section 8: Monitoring and Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 37
Additional Helpful Reference Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Page 38Implementing Linux on IBM System z: Best Practices
Page 2
Abstract
This purpose of this paper is twofold. First, we have attempted to draw from customer
experiences and the latest IBM System z™ solutions to present best practices information for a
Linux® on System z installation, and more specifically a Linux installation on System z that
will support a WebSphere® environment. Second, we recognize that a great deal of
documentation and assistance already exists and is available on the Internet for Linux on
System z installations, but can be difficult to locate whenever you need it. In recognition of
that fact, the second purpose of this paper is to serve as a “reference hub” to help a customer
locate the documentation (Redbook™, Redpaper, IBM Manual, etc.) for a particular issue or
product he or she is looking for.
This paper represents general “best practices” information for all implementations of Linux
on System z, but when conflicts have arisen regarding recommendations that might be better
for one Linux configuration versus another, preferential consideration has been given to
those recommendations which would be most beneficial and appropriate in a typical Linux
on System z installation that supports a WebSphere environment.
As the term “best practices” implies, the recommendations outlined in this paper are
intended to be starting points: guidelines for building and/or tuning a System z Linux
environment. As such, the effectiveness of some of these recommendations will vary from one
environment to another. We have attempted to note how each guideline can vary based on
environment, but practical experience in your own environment is the best teacher. The most
effective application of this paper and the documents referenced herein will come when you
combine the guidelines given with your own experience, gleaned from your environment.
Also, it would be virtually impossible to mention every aspect of Linux on System z, or to
mention every option available for a given scenario, in this one paper. The guidelines and
discussion presented in this paper are intended to be a starting point. Please contact the
authors for additional assistance or questions regarding any topics presented here.
Finally, some notes of clarification: When a sentence contains a specific “best practice”
recommendation, that sentence is italicized. Also, links to specific IBM documentation are
listed in this paper as hyperlinks. With that said, let’s start the fun!
Acknowledgements
Our sincere thanks to those who have graciously given of their time to contribute to this
paper:
Phil Anselm Steve Wehr Carlos Ordonez
Eva Yan Jon vonWolfersdorf Stanley Jones
Kevin Curley Romney WhiteImplementing Linux on IBM System z: Best Practices
Page 3
1. A Brief Overview of Linux on System z (The why, what, and how of Linux on
System z)
Linux is one of the fastest growing operating systems in the world. Over the past few years,
that growth has expanded to include adoption by enterprise customers in their vital
application hosting environments. This first section is a brief overview of why IBM believes
customers are choosing to implement Linux on System z in their organizations. The
assumption is made that the reader of this paper has a basic to moderate understanding of
Linux and the System z platform. For a thorough introduction to the basics of Linux and the
System z platform, please see the Redbook Linux for zSeries and S/390®, document number
SG24-4987.
1.1. Why Linux?
Before getting into the technical details of Linux on System z, it is important to understand
why IBM believes businesses choose Linux as their enterprise platform of choice. If the main
“selling point” of Linux on System z was technological curiosity, or the novel concept of
having open source software running on what used to be a purely proprietary hardware
platform, then Linux would probably not have gotten to its current place and degree of
acceptance. There must be some driving reason, some vital business need, for which a Linux
solution makes the most business sense.
In fact, there are a plethora of reasons that customers may choose a Linux on System z
solution. Not all of the reasons listed below hold true in every situation, but at least one of
these reasons usually applies.
• Helping to Make e-business Infrastructure Simpler and More Efficient
In the on demand business world that your organization operates in today, simpler is
typically better when referring to infrastructure. Get the job done as well as possible, while
keeping it as simple as possible. After all, the infrastructure is not supposed to be the star
of your show – it’s the business application that runs your business! (Pardon the pun.) As
long as your applications are up and available to customers and business partners, the
infrastructure that runs those applications is less important.
In recognition of this fact, many customers are looking at how their on demand business
infrastructure can better integrate into their critical business data flow. If a customer’s
critical business data already lies on the System z platform (in a z/OS® environment),
hosting their primary e-business applications on Linux for System z can be a natural,
cost-effective step towards simplifying their e-business environment.
When an application running on a distributed server is moved to the System z
environment, less end-to-end network “hops” are required (helping to reduce overall
response time), and the business application can access the critical business data at
memory-bus access speeds (Gigabytes per second). These potential savings are usually
non-trivial, and can make strategic, long-term sense for a business looking to simplify
overall infrastructure.
• In addition, Linux on System z can provide easy integration into a corporate-wide Linux
adoption policy. For those workloads which might make more sense to run on a distributed
platform, for example, Linux on an Intel® processor-based machine is available, and is
operationally virtually identical to a Linux on System z machine. (That is, any applicationImplementing Linux on IBM System z: Best Practices
Page 4
that will run on a Linux on System z machine is expected to run unchanged on a
Linux/Intel machine – in the vast majority of cases it will simply have to be recompiled.)
This cross-platform compatibility also makes Linux a clearly strategic choice for
organizations looking to simplify their overall IT infrastructure and make more efficient
use of the server management team’s efforts.
• Helping to Ease Integration Into a High Availability Environment
Most organizations are looking to make their production environments highly available in
the on demand world, and hosting your line-of-business applications on Linux for System
z can help make the path to high availability straighter and shorter for your environment.
An in-depth discussion of high availability is included in section 6 below, but the main
value proposition that Linux on System z brings to the HA discussion is the ability to
integrate with current HA and disaster recovery plans that already exist for the traditional
z/OS environment. This includes participation in a Geographically Dispersed Parallel
Sysplex™ (GDPS®) and backup and recovery using tools in an existing z/OS system. Or, if
your organization has had a pre-existing z/VM® system prior to Linux on System z being
installed, then the HA integration may be even easier. There are many considerations
when implementing a HA environment, of course, and those considerations are discussed
in more detail in section 6.
• Server Consolidation and Total Cost of Ownership
Wait! Before you tune out this section and move on thinking, “Here we go again with the
old ‘TCO argument’,” let me explain the server consolidation concept and value-add it
brings as clearly as we can. Many distributed servers (Linux, UNIX®, or otherwise) can be
consolidated onto one System z machine, which can lead to substantial cost savings. In
fact, many Linux on System z customers have done just this, and realized substantial TCO
cost savings. If your server farm of 100-200 distributed servers can be consolidated onto 1
or 2 System z machines hosting 50-60 virtual Linux servers in a highly available
environment that is highly scalable as your business grows, then that consolidation effort
may mean a savings in systems management overhead, computer resource overhead, and
turnaround time overhead for scaling and implementing new environments.
• The simple fact that server consolidation can be done, however, does not mean that it
should be done in every case. If your current distributed production environment uses
100% of an 8-way modern UNIX server, for example, it would be unwise to expect to
consolidate that application with 5 or 10 others onto a 3-way System z machine (an 8 or 10
way System z machine might be more appropriate for that situation). If, however, your
production application peaks at 30-40% of that 8-way server, and the other 5-10
applications use marginal amounts (5-10%) of each of their own distributed servers, then
you might have a better case for consolidation onto a System z machine with 3 or more
IFLs. (This would be particularly true if those applications were WebSphere applications,
and some or most of them could be consolidated onto one WebSphere instance in the
Linux on System z environment.) There are many permutations to the sever consolidation
argument that can be discussed. Each organization’s environment is unique, however, and
the bottom line is that a server consolidation study should be undertaken with both eyes
open, with the assistance of the IBM server consolidation experts. These experts, including
the SIZE390 team (who do sizing estimates for workloads of all kinds) and teams that
specialize in server consolidation and enterprise infrastructure assessment studies (called
“scorpion” teams) can be engaged by your local IBM client team and IBM Techline. In
addition, a more thorough discussion of server consolidation on Linux on System z can be
found in Redpaper REDP0222, entitled Linux on IBM zSeries and S/390: Server
Consolidation with Linux for zSeries.Implementing Linux on IBM System z: Best Practices
Page 5
1.2. How Linux runs on the System z
The most important facet of Linux on System z to understand is that Linux on System z is a
native implementation of the OS. That is, Linux can run on the “bare hardware” of the
System z machine without any special software or support required. The hosting options for
Linux on System z that are mentioned below (namely, LPAR and z/VM hosting) are
recommended as best practices options for the sake of feasibility and cost-effectiveness, but it
is important to realize that Linux doesn’t require any other supporting software to run on the
System z. The major positive implication of native implementation is that Linux on System z
does not have a degraded performance potential as compared to any other System z OS or
even Linux on any other platform. As you will see stated several times in this paper (and in
other documents), “Linux is Linux is Linux.” This means that, aside from the benefits that
the hardware platforms themselves provide, the Linux system that runs on your Intel desktop
has the same performance potential as the Linux system that runs on your System z machine.
Another very important implication of the “Linux is Linux is Linux” proposition is that the
actual internals of the Linux OS are identical from platform to platform. The only changes
that are made in the System z implementation of Linux relate to the actual instructions that
the operating system passes to the hardware. (And indeed, this layer of communication with
the hardware must be included for any platform that Linux runs on.) This means, among
other things, that even on the System z platform, Linux is an ASCII operating system. This
concept of “Linux is Linux is Linux” also greatly simplifies porting efforts from one Linux
platform to another. Because the Linux code is virtually identical on every platform, the vast
majority of applications simply have to be recompiled when they are moved to Linux on
System z. (The exceptions to this rule would primarily be any software that communicates
directly with the hardware layer, such as some database packages.) For those who have tried
porting to a z/OS UNIX environment in the past, there is no concern about ASCII-EBCDIC
translation when moving an application to Linux on System z, because Linux is a native pure
ASCII environment.
The benefits of using a native ASCII environment on the mainframe are clear when
considered from a product life cycle perspective. Business applications can developed, tested,
QA’ed, and moved into production all within the same general operating environment, even if
one or more of those environments are actually hosted on different hardware platforms.
Clearly, Linux is a very strategic environment for the System z platform.
1.3. Software Packages Available for Linux on System z
No one runs just a bare operating system in a business environment, so it is important as well
to know what software packages are available to assist with implementing e-business and
on-demand solutions with Linux on System z. There are too many individual applications to
list in this paper, but links to pages that have comprehensive lists are included below. The
specific packages named below are the ones that are most commonly used.Implementing Linux on IBM System z: Best Practices
Page 6
1.3.1. IBM Products
IBM software products that are available for Linux on System z include:
• WebSphere® (Application Server, Portal, and Commerce Suite, among others)
• DB2®/ DB2 Connect™
• Tivoli® (Backup and Enterprise Monitoring products)
• Lotus® Domino™
• Rational® ClearCase®
More information on the IBM products currently available for Linux on System z is available
at IBM Software for Linux on System z.
1.3.2. ISV Products
In addition to the IBM products available for Linux on System z, many Independent Software
Vendors (ISV’s) have extended support for their software packages to Linux on System z. To
find out if the ISV product you are using is available for Linux on System z, you can either
contact the ISV directly or refer to Software Developer Products for Linux on IBM eServer
zSeries and S/390. This page has the most recent list of ISV supported software that has
been ported to Linux on System z.
If you do not see a product that your organization uses on this list, it is an excellent idea to
contact the ISV directly to ask about support options for Linux on System z. This is because
an ISV may wait to support that option until a “critical mass” of customers have requested
Linux on System z support. By registering your interest with your ISV, you may help increase
the chances that the software you want to use will be supported on Linux on System z.Implementing Linux on IBM System z: Best Practices
Page 7
2. Installation of Linux on System z – A Brief Overview
We now move on to the first logical topic – a brief discussion of the installation
considerations and procedures for Linux on System z.
2.1. Know your objectives
Before installation, learn as much as you can about the systems you are installing. How many
Linux images will you need initially, and in the near future? What will they be used for? What
resources do they need to perform their processing? Mid-tier applications such as SAP may
be very communication-intensive, while Web servers, such as Apache, may be very I/O
intensive in addition to using network resources intensively. Try to get estimates of the type
of user load that is expected during prime shift. If possible, try to determine the I/O,
network, memory, and processing requirements for these units of work. Even extremely rough
estimates can be used in this phase. The numbers will be used to size the Linux images and
to choose the best configuration options for communications, memory, processor share, and
the DASD subsystem.
These are just a few of the considerations that come into play when architecting a Linux on
System z environment. A very helpful Redbook™ to use both when architecting and
implementing an enterprise Linux on System z solution is Linux on IBM eserver zSeries and
S/390: Large Scale Deployment, Redbook Number SG24-6824.
As part of knowing your objectives and planning your implementation you may want to ask
your local IBM account team about a Solution Assurance Review for your z/VM and Linux
installation, especially if it is a new or complex implementation. In this review IBM will
involve a subject matter expert to help you evaluate your readiness to implement and support
your new Linux environment.
2.2. Choose a distribution
There are many Linux distributions available to choose from when installing Linux, and
many things may influence the choice of which distribution to use. The Redbook Linux for
IBM eserver zSeries and S/390: Distributions, Document Number SG24-6264, discusses the
installation and features of many of these distributions, including SUSE, TurboLinux, Red
Hat, the original Marist distribution, and Debian/Caiman. (The existence of Caiman cannot
be confirmed at this time, but the Debian project, on which Caiman is based, has officially
released a version of Debian for the System z, which is available at their Web site.) This
Redbook has valuable information regarding the configuration of LPARs and VM to prepare
for installation. It also covers the actual install of the various distributions. Of course, new
releases of all these distributions are published on an ongoing basis, so be sure to reference
the Web site of your distribution vendor for the most current installation documentation.Implementing Linux on IBM System z: Best Practices
Page 8
The Redbook also discusses a Linux installation procedure for hosting under the IBM Virtual
Image Facility™ (VIF) in addition to the VM and LPAR modes. VIF is now obsolete, and is no
longer supported by IBM. As a result, VIF should no longer be considered a viable hosting
option for any Linux installation.
The pros and cons of one distribution versus another have been debated widely in
publications and message boards on the Internet, and will not be addressed here. All
distributions provide a kernel, various software packages, and a bootstrap to be IPL’ed for
configuring the new system. Some of them use GUI interfaces for configuration; others use
script files. For most distributions, the installations have common steps:
• Configure the host environment (whether VM or LPAR mode). The Linux partition will
need network connectivity through TCP/IP for telnet access to Linux. Disk space estimates
are generally provided.
• Obtain the distribution(s) that you have chosen (FTP from a Web site, or obtain CDs
directly from the distributor). The distributions usually come in CD image format.
• Make the CD image files accessible to the new Linux partition via NFS or FTP.
• IPL from the initial ramdisk image
• Provide the information needed to configure the partition (through scripts or interactively
through ssh/telnet)
2.3. Choose a hosting configuration
• LPAR Mode
Running in LPAR mode can eliminate the overhead associated with hosting under z/VM.
Be aware, however, that there is a limit to the number of LPAR’s than can be created, and
this will limit horizontal scalability in a large Linux environment. This option may be
appropriate, however, if you know for a fact that you will be running a relatively few
number of Linux images, and those images will each be using a large amount of processing
power, or will require a very large amount of dedicated memory.
• VM Guests
This option allows for thousands1 of Linux guests per IFL, and will probably be the “best
fit” for many installations. When Linux images are processing unpredictable workloads, or
when migrating from a large number of distributed servers, this is expected to be the most
versatile option. Hosting under z/VM does, however, introduce some special considerations
when defining your Linux guests, most notably with regards to defining memory and swap
space. This is because Linux will use all the memory it is assigned for its own cache and
buffers if that space isn’t taken up by the running application(s). If these memory pages
aren’t frequently referenced by Linux (as is usually the case with Linux cache), z/VM will
page them out. If Linux then needs that memory for other processes, it will usually move
that data to its swap devices. This causes VM to page the memory page in, just so Linux
can swap it out. If this process happens a lot, this can cause a significant performance
penalty. To help avoid this situation, it is important to not allocate too much memory for
the Linux guest (as opposed to a typical distributed Linux server, which gets a lot of
memory). Give the Linux guest the minimum amount of memory that it needs, which can
1
“How Many Linux Systems Can Dance on a Single S/390?” David Boyes, SHARE July 2001, and
“Test Plan Charlie Unplugged: An Interview with David Boyes,” Linux Planet, March 21, 2001Implementing Linux on IBM System z: Best Practices
Page 9
prevent unnecessary z/VM paging. Also, when assigning swap volumes for Linux guests,
assign a small swap device as a z/VM VDISK and make it the highest priority Linux swap
device; this will help to reduce I/O time required if the Linux guests themselves have to
swap a small amount. It’s also recommended to keep overall swap volume size small, since
Linux swapping is not recommended at all on the mainframe for the reasons stated above.
A good rule of thumb is for total swap size to be half of overall allocated memory size. With
regards to I/O, you also need to choose the access methods (“disciplines”) that Linux will
use for its I/O. ECKD (Extended Count-Key-Data), FBA (Fixed Block Architecture), and
Diagnose are the options available. Using Diagnose tells Linux to let the CP perform the
I/O through VM high-level routines, while FBA is the Linux architecture for VDISK and
SCSI disk access and ECKD is the disk architecture for traditional mainframe I/O.
Changing these parameters to DIAGNOSE may gain significant performance improvements,
and is recommended. The Redbook Linux on IBM eServer zSeries and S/390: Performance
Measurement and Tuning, Number SG24-6926, discusses many of these performance
options at length. This Redbook should be referenced to plan the file systems, memory,
and relative processor shares beforehand. This planning step is vital, since it is much easier
to select the correct options at initial installation, rather than reinstalling the Linux image
later.
2.4. System z Host Preparation
2.4.1. Configure environments
If the decision has been made to install Linux in LPAR mode, the LPARs must be configured
with HCD, as would be necessary for any LPAR. If the plan is to install the Linux subsystems
as VM Guests, the VM options for each guest must be set in that guest’s User Directory entry.
• Memory
– For z/VM installations, ensure that the minimum amount of memory required is defined
to the Linux guest. See the performance and tuning section of this paper for additional
guidance on overcommitting memory in z/VM.
• Processor share
– When defining LPAR’s, remember that zAAPs, zIIPs, ICFs, and IFLs are shared out of
the same processor pool, and must be calculated together for processor weighting
purposes
• Disk requirements
– The disk requirements for a Linux image can vary significantly, depending on the
installation options you select. A full installation of Linux requires about 6 Gigabytes of
space. Note that this does not include planning for any additional products you plan to
install (WebSphere, DB2, etc.). Planning for these products should be done separately.
• Network connectivity
– Your Linux machine might or might not require network connectivity to install, but
network connectivity is virtually mandatory for the server to be useful. When installing
in an LPAR, dedicated OSA cards are required for the Linux image. When installing
under z/VM, however, the OSA cards can be shared between guests via a z/VM Guest
LAN or VSWITCH. In the absence of any other mitigating factors, a VSWITCH should
always be used instead of a Guest LAN, due to the lower overhead required to implement
it.Implementing Linux on IBM System z: Best Practices
Page 10
2.4.2. Obtain and install the distribution
When installing in LPAR mode, CDs may be ordered or created, and used in the HCD
console to IPL the initial image. On some systems, depending on your local network
configuration, you may be able to IPL via FTP to a machine with the CD images. The CD
images may also be loaded to tape or disk, if the disk has been prepared with ICKDSF. When
installing under z/VM, the installation files may be accessed by z/VM’s NFS or FTP programs,
where they are loaded onto the VM guest’s minidisks, A “CP IPL” can then be performed
from the minidisk.
2.5. Post Installation
Review the Redbooks listed above for installation and performance options. When installing
under z/VM, it is important to be aware of the implications of some installation options.
QDIO is the default for network I/O. It works well, and allows the VM address spaces to share
OSA cards. There may be situations, however, where it is desirable to have a Linux guest
connect directly to an OSA card. In this situation, the default could be overridden to use
Linux native networking, thereby eliminating one layer of software interface.
After your base system is up and running, you can then implement your preferred method of
collecting performance data under Linux (SAR, RMFPMS, etc). This step is discussed in
further detail in the performance section of this paper.Implementing Linux on IBM System z: Best Practices
Page 11
3. Linux Administration
3.1. Basic System Administration Topics
There are many activities that can fall under the “system administration” moniker. In this
paper, we have included the topics that we find customers most often ask about or need
assistance with. The basic administration of a Linux on System z system is not at all
dissimilar from a Linux system on any other hardware platform. “Linux is Linux is Linux,”
and the basic administration setup tasks outlined below should be virtually identical to any
other Linux machine your company has installed. If you currently have a standard Linux
build and/or security policy at your company (along with administrators who set up the Linux
machines), then the following setup tasks should be able to proceed virtually unaltered on
Linux on System z:
3.1.1. Setup of Users and Groups
The setup of users and groups on Linux on System z is virtually identical to the process that
is used on every Linux platform. Of course, the preferred tool changes from distribution to
distribution (YAST on SUSE, for example), but the creation of user IDs and group IDs is the
same. You have the standard /etc/passwd and /etc/shadow files, and the location of home
directories for users is the same.
Most companies that already have Linux installed already have a policy for creating users and
groups, and assigning one to the other. If your company already has a user/group policy,
integration into the existing policy is generally recommended. If you are installing Linux for
the first time in your company, however, or if you are just creating a user policy, the following
is recommended:
First, it is usually better to create groups based on functional role in the company, and not
necessarily based solely on departmental or corporate classification. Security needs are often
one of the primary concerns that dictate the group creation policy. For example, in a
WebSphere support environment, instead of creating separate groups for the Houston,
Atlanta, and Charlotte offices, it might be more efficient to create separate groups for
developers, WAS administrators, and end users (if end users will be accessing a Linux user ID
at all), since those groups will need different security permissions. Conversely, depending on
what role the technical personnel play in the development and deployment environment, it
might make more sense to create groups based on current projects in development and
production. For example, if all the developers on each project also act as WAS administrators
for that project, but you want security access to each project discretely separated, you might
want to create one group for each project (HR Web, Online Accounts Payable, etc.). Of
course, each user can be assigned to multiple groups, so if you had a very complex
environment and wanted to create and assign groups based on both functional role and
project, or based on whatever criteria you wish, you could do exactly that. The bottom line in
group creation and delegation is that you should have a specific and stated design for your
system that outlines your needs, along with a group scheme that addresses those needs.Implementing Linux on IBM System z: Best Practices
Page 12
Second, no uid’s for users should be assigned below the 500 level, to avoid the risk of
duplicating a uid that is used by a system service or application. There is no upper limit on
uid’s that can be used, so we typically like to start numbering uid’s at 10000, just to make
sure they are out of the range of any uid’s that would have been assigned by pre-installed
software packages.
Finally, it is important to note that, as your Linux infrastructure grows, maintenance of the
user registry can be a large and burdensome task. Many customers, to simplify and centralize
their user registry infrastructure, choose to set up a central LDAP user registry, which all
Linux images can point to when authenticating users and retrieving group information (i.e.
which group(s) the user belongs to). LDAP databases are also useful because they are
platform-independent, and can be used as a centralized user registry by any platform that will
support external user authentication. Instructions on creating an LDAP database is beyond
the scope of this paper, but configuring Linux to use the LDAP database for user
authentication is fairly easy. Only a few modifications are required to the default Linux PAM
configuration. The location of the PAM configuration files can vary based on distribution,
but in SLES9, the files are located in the /etc/pam.d/ directory. Information on the PAM
LDAP module can be found at PADL Software Pty Ltd. In addition, Redpaper REDP-0221-0 is a
good reference on integrating z/OS LDAP and RACF with Linux authentication.
3.1.2. Basic Linux Security
The basic form of security on Linux on System z, like Linux on all other platforms, is the
system of permission bits that are assigned to every directory and file on the system. We will
not go into great depth discussing the permission bit system in this paper, as information is
available on this subject. The book LPI Linux Certification In a Nutshell from O’Reilly, for
example, is an excellent reference book on general Linux information, including security and
permission bits. (Or if you just go to any search engine, Google, for example, and search for
“Linux Permission Bits,” you’ll get several hits that will explain how the permission bit
system works.) It is important, however, to note that you should attempt to avoid assigning
the “setuid” bit or “setgid” bit to any directories or files unless absolutely needed. (Some
software products require that these bits be activated for the product’s binary directories, but
the assigning of these bits should be severely limited whenever possible. Ideally, they would
not exist.)
In addition to basic “permission bit” security, the 2.6 Linux kernel introduced two new
functions that can be used with supporting filesystems, extended attributes and Access
Control Lists (ACL’s). Extended Attributes, while not solely having security applications,
allow you to tag files with additional information, such as their encoding. Extended Attributes
can be queried or set with the “getfattr” and “setfattr” commands. Access Control Lists,
which actually are a type of extended attribute, allow you more flexibility in assigning access
to files than the traditional owner, group, world scheme. ACL’s can be queried or set with the
“getfacl” and “setfacl” commands.Implementing Linux on IBM System z: Best Practices
Page 13
3.1.3. Shell Environment
The shell environment that is available on Linux on System z is identical to the shell
environments that are available on all other platforms. Bash, sh, ksh, csh, and tcsh are all
included with every distribution that is available, and any additional shell environment that
can be used on other Linux platforms can be downloaded and installed in Linux on System z.
3.1.4. Password Administration
This might seem like too simple a topic to cover in a best practices paper, but proper
password administration is vital in helping to address security on any Linux machine,
including one on Linux on System z. If the wrong potential intruder gains access to a user’s
account, then your entire Linux machine can be compromised. It is recommended that you
encrypt your passwords using the /etc/shadow file (this is the default is most Linux
distributions). In addition, a policy on password changing and security should be implemented
at your organization, if one is not already in place. Linux users should be forced to change
their password on a regular interval, and the passwords they create should be subjected to
basic “dictionary checks” to make sure that the passwords are resistant to potential hacking
attempts. Tools to do this are packaged into Linux, and they can be accessed using a user
administration tool (YaST, Webmin, etc.). Some of the tools that can be used manually are
passwd (for changing passwords), chage (for changing user and password expiration
information), and pwconv/grpconv (for implementing shadowed/encrypted passwords for
users and groups), among others.
3.2. Advanced System Administration Topics
3.2.1. Scheduling
• Job Scheduling - There are several options available for scheduling work on Linux. A
simple, but stable and functional scheduler daemon that is packaged with Linux is Cron.
Cron allows an administrator (or any user, theoretically) to run scripts and other
commands on a preset schedule. Cron is open source, and has a very simple functionality.
For more robust Linux scheduling, there are several scheduling products that have been
ported to Linux on System z by Independent Software Vendors (ISV’s). A comprehensive
list of these products by category is available at ISV Solutions for Linux on System z.
• I/O Scheduling - Beginning with the 2.6 kernel, there are four different I/O schedulers
available in Linux: noop, deadline, anticipatory (as), and completely fair queue (cfq). An
in-depth discussion of each of these options is beyond the scope of this paper. In
summary, however, the noop scheduler only performs request merging, while the deadline
scheduler is designed to avoid request starvation, and the cfq scheduler is designed to give
all users of a particular drive/volume equal access to that drive/volume. Finally, the as
scheduler is designed for use with physical disks and not disk subsystems, so it should not
be used with Linux on System z. The default scheduler for SLES 9 and RHEL 4 is the cfq
scheduler. To verify this, you should be able to find a message indicating which scheduler
was activated at boot time in the dmesg (RHEL) or boot.msg (SLES) file. You can also
specify which scheduler to use in the zipl.conf file via the “elevator=” parameter. While there is no one scheduler that will provide the best
performance in every conceivable situation, the deadline scheduler has been shown,
through various IBM benchmark tests, to typically provide results that are equivalent to or
better than other schedulers. Note that the deadline scheduler is not the default schedulerImplementing Linux on IBM System z: Best Practices
Page 14
for SLES and Red Hat installations. It is important to note that the results favoring
“deadline” are merely based on benchmark workloads, and that the best scheduler for
your environment will be highly dependent upon the workload being run. As a result, it is
recommended that you engage in an iterative process, testing your Linux workload with
multiple I/O schedulers, and ultimately choosing the one that best meets your needs based
on your testing. In the absence of any data suggesting otherwise, however, the deadline
scheduler is a good place to start.
3.2.2. Backup and Recovery
Backup and recovery is one of the most vital areas of preparation that every Linux
installation should have completed before any environment is put into production. Most
organizations who already have a production z/OS or z/VM environment are accustomed to
preparing and executing a backup and recovery strategy, and have very little trouble with
adapting that strategy for use on Linux on System z. If your organization is new to the System
z platform, however, then you might find creating a backup and recovery strategy a little
challenging.
Hardware choice is not normally a vital concern for developing a backup strategy (other than
the fact that some type of hardware is needed!), because Linux and z/VM generally support
any existing tape management system that your company currently uses.
As it relates to software, however, there are basically three options for backing up and
restoring data on a Linux on System z system. There is no one “most recommended” backup
method, but it is highly recommended that a backup strategy be implemented. The strategy
your organization chooses should be based on your business needs, available resources,
existing skills, and strategic direction.
• The first option is to use backup tools that reside on z/VM to backup and restore Linux
data. This is generally the most common strategy that customers already familiar with
z/VM use for Linux and z/VM backups. In general, it is important to note that z/VM
backup tools usually only provide volume-level backups, not file-level backups. That is,
you can back up and restore entire minidisks, but not individual files. To restore individual
files in this scenario, customers typically restore the minidisk that the file in question was
on, and then mount that volume to the Linux guest and retrieve the lost file. Tools that fall
into the z/VM-resident category include the z/VM built-in DASD Dump/Restore (DDR)
Facility, and a number of tools listed at the ISV Information for z/VM web page.
• The next basic option for Linux backups is to modify an existing z/OS backup strategy to
include Linux volumes. The benefit of using existing z/OS backup facilities (most notably
DFSMSdss) is that Linux DASD volumes can generally be treated like any other volume
that is dumped and restored using dss. Although z/OS can’t access the data in the volume
itself, the volume can be dumped and restored just like any other DASD volume.
Obtaining a "clean" backup with this method takes some coordination, since the Linux
system should be down or the file system/volume in question should be unmounted during
backup.
• The backup strategy that is quickly gaining momentum is to use tools that run on Linux
itself to backup Linux volumes and files. Obviously, the most attractive argument for this
option is that these backup tools provide easy access to both file-level and volume-levelImplementing Linux on IBM System z: Best Practices
Page 15
backups. IBM’s Tivoli Storage Manager™ product fits into this category. TSM can send the
backup data to a TSM server on any platform or perform a "LAN FREE" backup and send
data directly to a SAN. A good number of ISV’s have already ported their own Linux
backup tools to the Linux on System z platform, as well. The number of additional
products being ported is increasing regularly, also, so you can visit ISV Solutions for Linux
on System z to see the latest list of software packages that have been ported to Linux on
System z. If you currently have distributed Linux machines installed in your environment
and have a backup strategy and product already implemented, this option would probably
be the easiest method for you to use when doing a server consolidation.
3.2.3. Operational Considerations
• Startup and Shutdown - Proper operational procedures should be implemented for a safe
and orderly startup and shutdown process. Linux has a built-in mechanism for handling
this process, utilizing scripts in either the /etc/init.d or /etc/rc.d directories (depending on
your distribution). This process is designed to ensure that the products you use start (or
stop) in the proper sequence. After adding a startup/shutdown script to the correct
directory, you can add that script to the startup and shutdown sequences by using the
chkconfig command. Controlled startups and shutdowns are important, because software
may be required to start in a certain sequence (e.g. DB2 must start before WebSphere
Portal). Orderly shutdown can help ensure that things such as databases or journaled
filesystems don't have to go through a lengthy recovery at startup time. When shutting
down a Linux guest under z/VM, the z/VM SIGNAL SHUTDOWN command is a utility
that can send a signal to the guest operating system to indicate that it should begin its
shutdown sequence. From a z/VM administrator’s perspective, using this command can be
more convenient than actually having to log onto a Linux guest just to then shut it down.
• NPTL - Native POSIX Thread Libraries have been integrated into Linux beginning with
the 2.6 kernel, and are the strategic direction for Linux threading. The NPTL libraries
replace the native “Linux Threads” model that has been used in previous versions of the
Linux kernel, and have generally helped increase Linux scalability and throughput. In rare
situations, programs that are being migrated to the 2.6 kernel will suffer unexpected effects
due to the new thread model. If this happens, you can choose to revert to the old “Linux
Threads” model by setting the following environment variable for the running program:
LD_ASSUME_KERNEL=2.4.21. It is not recommended, however, to use this workaround
unless it is absolutely required.
• System Call Emulation and s390 Tools - All 2.6-based Linux kernels running on the
System z platform are supported only when running in a 64-bit mode. Many products,
however (including most WebSphere products), still run in a 31-bit mode. The 64-bit
“s390x” kernel for Linux on System z provides a System Call Emulation layer that allows
31/32-bit programs to run, unaltered, on the 64-bit kernel. In addition, there is a set of
s390(31-bit)-related tools that are available for Linux on System z. These tools are not
always installed by default, but are included on all distribution CD’s as the “s390-32...”
package. This package is recommended because it includes tools that allow the user to
instantiate a 31-bit shell environment that is useful for, among other things, installing
WebSphere Portal in the 64-bit Linux environment.
• New sysfs filesystem - Along with the new 2.6 Linux kernel came a new device filesystem
known as sysfs. An in-depth discussion of sysfs is beyond the scope of this paper, but for
additional information on sysfs and how it relates to devices on the System z platform,
please see the Redbook Linux on System z9 and zSeries - Device Drivers, Features, and
Commands.Implementing Linux on IBM System z: Best Practices
Page 16
4. Security
The security concerns facing a complex organization in the on-demand age cannot be
adequately covered in the scope of this paper, nor will we try to address every topic involved
in securing the Linux on System z environment. Entire Redbooks can be (and have been, and
are being) written about the different aspects of securing Linux on System z. Instead of
attempting to write a treatise on security, we will discuss the issues that we have had to assist
with most often when working with Linux on System z customers.
4.1. Network Security
Network Security is almost universally (and surprisingly) a topic on which “new ground” has
to be broken when moving applications to Linux on System z for the first time. This is due to
the fact that at most customer locations, a security team that is accustomed to architecting
network security for a distributed environment is charged with securing a Linux on System z
solution. Of course, there’s nothing wrong with this situation – it’s just that some information
about your new Linux on System z environment needs to be shared with the security team.
(Communication! Communication is the key to keeping security, sysadmins, and developers
happy and working together.)
First, it is important to understand that when one VM guest communicates with another
guest, it is possible for that communication to be kept inside the System z machine, That is,
the communication itself is designed so that it cannot be tapped into by any external process
or “sniffer” program. (Extremely technical z/VM discussion deleted here to save space – and
to keep the reader from dozing off.) Due to customer requests for an auditable internal z/VM
network, an internal network sniffer is available beginning with z/VM Version 5.2. This
sniffer is able to capture internal network traffic running over a VM Guest LAN or VSWITCH.
Sometimes, depending on the level of security desired, the security team will want to put a
firewall between different layers of an application. If this is simply to protect the network
layer, then this firewall might not be needed with zLinux. At other times, however, it may be
necessary to protect one layer of the application from another in case of hacking or other
corruption (for example, between an HTTP server in a DMZ and a WebSphere back-end
server or back-end DB2 database). Another example of a time when separation might be
required is if Linux guests are on different VLAN’s, representing different security levels, but
still need to communicate with each other. At these times, an additional firewall might be
warranted, and would even be a prudent course of action.
When securing parts of your organization’s network with firewalls, it is important to note that
there are firewall products currently available that can be installed in a z/VM Linux guest.
Moving at least one of the firewall layers to the Linux on System z environment can help bring
many benefits, which can include reduced response time due to the need for fewer network
hops outside the z/VM environment (especially if one or more of your servers in the DMZ are
hosted on Linux on System z). At the time of this writing, one firewall package is available for
the Linux on System z environment: Stonegate Firewall (developed by Stonesoft Software). It
is our understanding that other ISV’s are in the process of porting their products to Linux onImplementing Linux on IBM System z: Best Practices
Page 17
System z, however, and if your organization strictly uses a different firewall product, it is
recommended that you check the ISV Solutions for Linux on System z - Security
Applications regularly to see if your firewall product has been ported to Linux on System z. It
is also suggested that you contact your firewall vendor to request that their product be ported
to Linux on System z if you are interested in this functionality, since an ISV may wait until a
“critical mass” of customers have requested their products on a new platform to release a
version for that platform.
4.2. Linux System Security
After a discussion of network security logically comes the discussion on securing the Linux
image itself. While many books have been written on securing a Linux system, we will point
out the following issues that will be most important in a Linux on System z environment.
We have already mentioned some security considerations in the system administration
discussion earlier. Another option that organizations with traditional z/OS environments are
using to enhance Linux security is the ability to use their z/OS security product (RACF®,
ACF2, or Top Secret) as an LDAP authentication server for their Linux machines. This is a
recommended way to help further secure your Linux environment. The major potential benefit
of this approach is that user and group administration is centralized in a security-rich LDAP
server, and individual password/shadow files do not need to be maintained on each Linux
machine. This can help to significantly reduce administration overhead, especially if you have
a large number of Linux guests. The major drawback of this option, however, is that a
learning curve exists when trying to setup and configure the LDAP server in your z/OS
security product. (And most z/OS shops are squeamish about altering any part of their
security product to begin with.) A very good document on this topic is Linux on IBM zSeries
and S/390: Securing Linux for zSeries with a Central z/OS LDAP Server (RACF), document
number REDP0221. This option may not make sense for every organization, but is definitely
an option to consider if your organization is planning a large-scale Linux deployment.
To help further secure your Linux guest, any ports that are not specifically needed by a
software service running on the Linux image should be closed. Closing off unused ports helps
reduce the exposure of your Linux image to hackers, and in general makes your systems
much more efficient from a security standpoint. In addition, the default telnet and FTP
programs packaged with most distributions should not be used (they are usually disabled by
default). Instead, it is highly recommended that more secure access methods be used, such as
SSH and SCP/SFTP, respectively. This is especially important for any Linux image that
receives traffic directly from the Internet or is in the DMZ. Many online tools (PuTTy,
FileZilla, etc.) are available to communicate via SSH and/or SCP/SFTP to your Linux server
from your Windows desktop.Implementing Linux on IBM System z: Best Practices
Page 18
5. WebSphere Application Server
WebSphere Application Server (WAS) is one of the most common middleware products that
is being run on Linux on System z today. WebSphere is a widely-used Java™ application
server, and many organizations use Java applications hosted on WebSphere to connect to data
housed on the mainframe. Because the data that the Java app accesses is housed on the
mainframe, it makes logical sense to move the application from a distributed WAS server to a
WAS server on Linux on System z. The case for “moving the application closer to the data”
has been made many times in the past, and is a very compelling reason to move the
application to the mainframe. No matter what reasons drive your organization’s decision to
move applications to Linux on System z, you have to go through the process of installing,
configuring, and tuning WebSphere for your Linux on System z environment. This section is
designed to give you best practices information to use during initial WebSphere installation
on Linux on System z.
5.1. Prerequisites
Before you go about installing WebSphere Application Server, your initial Linux environment
needs to be set up, and some initial planning must be done. Obviously, you must have a
Linux guest to install WAS on. In addition, you will need to decide ahead of time whether or
not your installation will need to be a “base” installation or a “Network Deployment”
installation. That is, will your WebSphere servers be clustered together across multiple
systems, or will you want to install multiple nodes over one or more systems? If the answer to
either of those questions is yes, you will need the Network Deployment version of
WebSphere. If you answered “no” to those questions, then a “base” WebSphere installation
might work best for your organization.
In addition, it is highly recommended that you check the latest WebSphere Application Server
System requirements before attempting to go forward with a WebSphere implementation.
Make sure that all software that will be used in your environment will work with your desired
version of WebSphere.
5.2. Installing WebSphere Family Products on Linux on System z
5.2.1 WebSphere Application Server
An excellent walk through is available online in the WebSphere Online Library for installing
WAS for Linux on System z. The primary recommendation that is important to point out, and
which is not mentioned in the installation documentation, is that you should not, if at all
possible, use built-in GUI tool to install WebSphere for zLinux. The x-Server based GUI install
tends to be very CPU intensive. Instead of using the GUI, it is recommended that you
customize the sample install response script that is included with the product for your
environment and run the install process based on that script using the “silent” option. (Don’t
worry about the fact that it’s called a “silent” option… errors during the process will still get
logged so you can troubleshoot the installation process if needed.) This tends to be the most
efficient installation method for WebSphere on Linux on System z, and it is the only method
that IBM recommends.You can also read
