# Running IBM Parallel Environment Applications

## Introduction

The IBM Parallel Environment (PE) Eclipse plug-in, part of the Parallel Tools Platform (PTP), allows you to run a parallel application using IBM Parallel Environment from within the PTP framework. In addition to running applications, you can monitor their status, control their execution, and view their stdio output from within PTP. You can also debug a parallel application using the built in PTP parallel debugger.

You can run and debug parallel applications either as local projects or as remote projects, using the remote development environment support that is part of PTP.

Optionally, you can use IBM Tivoli WorkloadScheduler LoadLeveler to manage your application's access to system resources

This plugin consists of two parts. The first is the Eclipse PTP plugin code which is responsible for the PTP views presented to the user and the user's interaction with those views. The second part is a resource manager (proxy), which is responsible for communicating with the PTP PE plugin code, invoking the parallel application, and reporting application status to the PTP PE plugin.

Note: PTP 6.0 does not require the creation of separate Resource Managers and instead, the target configuration is specified on the Run Configuration dialog. This document describes the legacy (proxy-based) Resource Managers supported in versions of PTP prior to PTP 6.0. Target System Configurations replace Resource Managers in PTP 6.0 and beyond.

- For more up-to-date information on launching IBM PE jobs with PE, see Launching jobs with IBM PE

## Requirements

In addition to having Eclipse and PTP installed, you need to have the following software packages installed.

- CDT, as required by the current release of PTP.
- Parallel Environment 4.3.1 or later, available from IBM
- LAPI 2.4.4.0 or later, available from IBM
- IBM Tivoli WorkloadScheduler LoadLeveler 3.4 or later, available from IBM, if you plan to use LoadLeveler to manage your application's access to system resources

You should review the following information about Parallel Environment installation and usage

- IBM Parallel Environment for AIX and Linux Installation Guide
- IBM Parallel Environment for AIX 5L and Linux Operation and Use, Vol 1, Using the Parallel Operating Environment

If you are using LoadLeveler to manage your application's access to system resources, then you should review the following LoadLeveler documentation

- Tivoli Workload Scheduler LoadLeveler V3.4 Installation Guide
- Tivoli Workload Scheduler LoadLeveler V3.4 Using and Administering Guide
- Tivoli Workload Scheduler LoadLeveler V3.4 Diagnosis and Messages Guide

## Installation

If you will be running applications using PE with a host list and not using LoadLeveler to manage resource allocation, then installation is complete once you complete the PTP plugin installation procedures. If you plan to use LoadLeveler to manage resource allocation, then you will need to rebuild the Parallel Environment proxy by following these steps

1. Make sure you have installed LoadLeveler and that the llapi.h include file exists on your system. It normally is located in the ./full/include directory within a LoadLeveler installation. LoadLeveler is normally installed in /usr/lpp/LoadL on AIX systems and in /opt/ibmll/LoadL on Linux systems. LoadLeveler may also be installed in a private directory if you choose to do so.
2. cd to the org.eclipse.ptp.ibmpe.proxy directory inside your plugins directory.
3. Invoke the configure script in that directory with the --with-ll-header flag, specifying the path to the directory containing llapi.h. (Use configure --help to see a complete list of configure script options)
4. Invoke make in the same directory.

## Getting Started

Define an appropriate project, either a C/C++ project using CDT, or a Fortran project using Photran. You can define either a standard make project or a managed make project.

When you define the project, you should specify the project location as a directory which is shared among all nodes where the parallel application will run to ensure Parallel Environment has access to the executable on all nodes. The following image shows how a project can be created using a directory visible to all nodes.

You should use the following compiler invocation commands to invoke the compiler, either in your makefile, or when specifying the compiler and linker in a managed make project.

- AIX

- mpcc_r for C programs
- mpCC_r for C++ programs
- mpxlf_r for Fortran programs

- Linux

- mpcc for C programs
- mpCC for C++ programs
- mpfort for Fortran programs

The following image shows how the compiler is specified for a managed make project. The correct linker should also be specified by clicking the settings for the linker.

Once you have created your project, you should switch to the PTP runtime perspective, and set up the default preferences for Parallel Environment and specify the required resource manager. This setup needs to only be done once.

You specify the default preferences by opening the preferences dialog, opening the PTP node, opening the Resource Managers node, and then clicking the Parallel Environment entry. The PE preferences panel is displayed. Here, you specify the default settings for a resource manager to be used for Parallel Environment.

The settings that can be specified here are:

- The pathname to the PE proxy executable. This proxy handles the interaction between the Eclipse PTP session and your parallel applications.
- Multiplexing options
- Whether the proxy is started manually or is started automatically when Eclipse is started
- Whether LoadLeveler is used to manage access to system resources
- If LoadLeveler is used:

- Whether PTP forces LoadLeveler to operate in multi-cluster mode, single (local) cluster mode, or to use the mode LoadLeveler was configured for
- The minimum interval that the proxy will poll LoadLeveler to query node status
- The maximum interval that the proxy will poll LoadLeveler to query node status
- The interval that the proxy will poll LoadLeveler for job status
- Alternate directory for the LoadLaveler interface library. This option is useful if LoadLeveler is not installed on the node where the proxy runs, and you have installed your own copy of the LoadLeveler interface library libllapi
- Whether a second proxy process is started when the PTP PE proxy shuts down. This second proxy is useful in cases where an application writes data to STDIO. If the PE PTP proxy shuts down without starting the second proxy, then the application may suspend since there is no process intercepting the STDIO output from the application. If this option is selected, a second, smaller, proxy is started. This second proxy intercepts the STDIO output from the application and writes it to files in /tmp. This prevents the application from blocking when the pipes connected to the application's STDIO file descriptors are full.
- Specify the level of tracing performed by the proxy. This option is useful when attempting to diagnose problems with the PTP PE plugin

Once you have set up preferences, create the resource manager for Parallel Environment. To do this, right-click in the resource managers view and select Add Resource Manager from the menu. A resource manager wizard will be displayed. Select PE from the list of resource manager types and click next.

The next panel displayed is the first wizard panel which configures the PE resource manager.

This panel is the standard PTP wizard resource manager configuration wizard panel which specifies the location and connection attributes for a proxy.

Select the Remote service provider and proxy server location from the dropdown lists. You can also create a new proxy server location by clicking the New button in this pane. Specify the path to the proxy server executable, or just accept the value filled in as default from the preferences panel. Uncheck the Launch server manually and click next again to proceed to the next wizard panel

This panel is used to specify the proxy options for this particular resource manager configuration. Most of the options on this panel are identical with the corresponding option in the PE plugin preferences panel. This panel has one additional option, which specifies whether the proxy execution is to be suspended immediately after it starts. This option is intended to be used in debugging the proxy in cases where it is necessary to suspend proxy execution before the first command sent to the proxy is processed, so that a debugger can be attached to the proxy process

Once the fields on this panel have been filled in, click next to proceed to the next wizard panel

On the final page of the resource configuration manager wizard, either accept the default resource manager name, or uncheck the checkbox and enter a name and description for the resource manager. Then click finish to create the resource manager entry.

You may have as many resource managers defined as you require for your system configuration.

Once you have defined a resource manager, you must start it before it can run a parallel application using that resource manager. To start a resource manager, right-click over the name of the resource manager in the resource manager view then select Start resource manager from the popup menu. Once started, the icon for the selected resource manager will turn green.

If necessary, you can stop a resource manager by right-clicking over its name in the resource manager and selecting Stop resource manager from the popup menu. Normally, a resource manger will shutdown automatically when PTP is shutdown, and if it was running when PTP was shutdown, then it will automatically start when PTP is started again.

You can edit resource manager configuration for a resource manager that is in the stopped state by double clicking the resource manager name or by right clicking the resource manager name and selecting Edit Resource Manager from the drop down menu.

## Running a Parallel Application

Once an application has been compiled, the first step in running the application is to be sure the resource manager for Parallel Environment is running on the node where you intend to start the application. To do this, check the resource manager's entry in the resource manager view. The icon for the resource manager should be green. If it is not green, then right click over the resource manager's name and select Start resource manager from the popup menu to start the resource manager.

Before running an application, it must have a launch configuration defined for it. A launch configuration contains all of the settings required to run the application. The values entered in a launch configuration are saved so that the next time the application is run using the same launch configuration, those values do not need to be entered again.

To create a launch configuration, click the Run menu in the Eclipse menubar and select Open Run Dialog. Alternatively, click the dropdown next to the run icon in the Eclipse toolbar and select Open Run Dialog. When the run dialog is displayed, right click over Parallel Application and click New from the menu. This will create a new launch configuration. Set its name to the desired value. Select the Main tab and fill in the name of the project and the application program. Make sure that the debugger tab has an appropriate debugger selected. Fill in the Arguments tab with any application program arguments. Fill in the Environment tab with any environment variable settings required by the application. If you are using Parallel Environment in basic mode, described below, then you should not set any Parallel Environment environment variables (starting with MP_) in the Environment tab and not specify any options specific to the poe command in the Arguments tab, since those settings may conflict with settings in the Parallel tab and cause application invocation to fail.

The Resources tab is the tab where invocation options unique to Parallel Environment are specified. When it is initially displayed, it will appear as follows.

The first step is to select a resource manager from the dropdown list at the top of the resources tab. This list contains entries for each running resource manager.

The next step is to select the run mode for the resource manager by checking or unchecking the Advanced Mode checkbox.

When this checkbox is checked, the resource manager is running in advanced mode. In this mode, you need to have an existing setup script created. This is a text file, similar to a .rc or .profile script, where you specify Parallel Environment environment variable settings. Each setting is specified as a keyword-value pair such as

`MP_PROCS=2
MP_HOSTFILE=/home/deepview/hostfile`

This file should contain only Parallel Environment environment variable settings. Anything else is ignored. Environment variable settings in this file are not validity checked. You may also specify Parallel Environment environment variables in the Environment tab in this mode, as well as specifying parameters to the poe command in the Arguments tab in advanced mode. Any settings in these tabs will not be checked, and may result in conflicts with what is specified in the setup script.

When using advanced mode, the Environment Setup Script will be enabled, and you must fill in the pathname to the setup script.

If the Advanced Mode checkbox is not checked, then the resource manager is running in basic mode. In this mode, the tabbed pane at the bottom of the Parallel tab will be enabled, and you can fill in the required values for running your application.

In basic mode, each Parallel Environment option has a field in the tabbed pane at the bottom of the parallel tab. Each field has flyover help, where you can see that help by moving the mouse over the field in question. Each field is validity checked where possible to reduce the probability of invoking an application with invalid parameters.

If any value in the parallel tab is invalid, an error message will appear in the upper left corner of the launch configuration dialog and the apply and run buttons will be disabled. Once all fields have valid values, then you can click the Applybutton to save the current settings for the launch configuration, or click the Run button to invoke the application.

As a minimum, the launch configuration or the Parallel Environment setup script must specify the path name of a valid hostfile (MP_HOSTFILE) and number of parallel tasks (MP_TASKS)

### Debugging a Parallel Application

You can debug a parallel application using the built in PTP parallel debugger. You can do this using the same run configuration that you use for running your application. You can debug the application by clicking the debug icon in the Eclipse toolbar or by selecting Debug Configurations from the Eclipse run menu. Before you debug a parallel application for the first time, you should make sure that you have properly configured the debugger by opening the run configuration and selecting the debug tab then making sure the debugger settings are correct.

## Redirecting stdio

The default behavior is that data the application writes to stdout is sent to the PTP GUI. That data can be viewed by double clicking the task icon for the task, normally task 0, in the jobs view row for the application. This will open a process view window showing the data. By default, the application stderr output is written to the console where Eclipse was invoked.

If the application writes a large amount of data to stdout or stderr, the default behavior of sending that data to the PTP GUI for handling can cause the GUI to become slow or unresponsive. Alternatively, stdout and stderr can be redirected to files on the node where the resource manager process is running. This is done by specifying the path name for stdout or stderr in the fields labelled Stdout Path or Stderr Path in the parallel tab of the launch configuration.

## Monitoring Application Execution

Once you click run to invoke the application, the resource manager will monitor the application's execution. When Parallel Environment allocates tasks to nodes, the default machine view for the resource manager will be updated to show the nodes that are currently in use. If you move the mouse over a node icon, in the machine view, you will see the hostname for that node. If you double click a node icon in the machine view, the node attributes and process info windows in the machine view will be updated to report current status for that node. Once a job completes, the machines view will be updated to show the current status. As applications complete, nodes may disappear from the machine view if no other parallel tasks invoked by you are using those nodes.

Once a parallel application starts, the jobs view will be updated to show an entry for that job, along with icons for each task in the application. The job and task icons will change state once Parallel Environment setup is complete to show the job is running, and again when the application exits, to show that the job has completed. At any time that a job is shown in the jobs view, you can double click a task icon to show the stdout output for that task if stdout has not been redirected to a file.

## Restrictions

- Applications cannot read input from stdin at this time
- Debugging applications is not supported at this time
- All pathnames specified in the Parallel tab of the launch configuration must be absolute pathnames
- When using LoadLeveler, you can only viw stdout and stderr output for applications started by you using this plugin. You cannot view output from other applications even though you can see their process icons in the PTP jobs view.