Writing OGSI compliant CCA Components Using XCAT3


CONTENTS


INTRODUCTION

This tutorial gives an overview of how to write CCA Components that are compatible with the Open Grid Services Infrastructure (OGSI) specification using the XCAT3 toolkit.

For an overview of the XCAT project, visit our webpage here. Interested readers are also encouraged to visit our Grid Web Services project pages here, specifically those belonging to XSOAP-GSX which has been used for OGSI compatibility. We also recommend reading the following papers for a technical description of the XCAT(3) framework.

REQUIREMENTS

INSTALLATION

WRITING OGSI COMPATIBLE CCA COMPONENTS

In this section, we show how OGSI compatible CCA components can be written using the XCAT3 framework. This involves writing the port interfaces, implementing the components and the ports, and writing scripts for running them.

As an example, we describe how to write two components, one providing a service and another using it. The service being provided is a simple convertor of temperature from celsius to fahrenheit scales. Although this is not a typical use-case for XCAT3, the same concepts will apply even if more complicated components are to be written. The sources for this example are in the $XCAT_HOME/src/java/samples/conversion directory.

Writing Port Interfaces

Every Provides Port in XCAT3 is exposed as an OGSI Compliant Grid service. To make this happen, every interface definition for a Port has to extend from intf.ports.XCATPort, which adds methods defined by the OGSI GridServicePort to a regular CCA Port. The interface for the ConversionPort is defined as follows:

package samples.conversion;

import intf.ports.XCATPort;

/**
 * The interface definition for the Convert Port
 */
public interface ConversionPort extends XCATPort {

  /**
   * Method to convert a value in celsius to centigrade
   */
  public float centigradeToFahrenheit(float celcius);
}

Implementing the Component and Ports

We describe how to write two OGSI compatible components: ProvidesConversionComponent and UsesConversionComponent, and also the implementation of the OGSI compatible ports ConversionPortImpl and GoPortImpl (which is used for bootstrapping execution, as defined by the CCA specification).

We start with the implementation of the Ports. As an example, we will look at ConversionPortImpl. The implementation of GoPortImpl is on similar lines. The java documentation in the following code illustrates how to write an OGSI-compliant Port implementation.

package samples.conversion;

import xcat.ports.BasicPortImpl;
import soaprmi.util.logging.Logger;

/**
 * This is the implementation of the ConversionPort.
 * The port implementation has to extend from the BasicPortImpl which has
 * been provided by XCAT3. The BasicPortImpl class implements all methods
 * defined by the OGSI GridServicePort, obviating the need for the user to
 * implement any of them.
 */

public class ConversionPortImpl extends BasicPortImpl
  implements ConversionPort {

  // Logger provided by the XSOAP toolkit for logging information
  private static Logger logger = Logger.getLogger();

  /**
   * Default constructor.
   * This has to throw java.lang.Exception because the superclass constructor
   * has been declared to throw it.
   */
  public ConversionPortImpl() throws Exception {
    super();
  }

  /**
   * Method to convert a value in celcius to centigrade, which is the only
   * method defined by the ConversionPort
   */
  public float centigradeToFahrenheit(float celcius) {
    logger.finest("ConversionPortImpl: Received celcius value: " + celcius);
    float fahr = ( ( 9f / 5f ) * celcius ) + 32;
    logger.finest("ConversionPortImpl: Returning fahrenheit value: " + fahr);
    return fahr;
  }
}

Next, we write the two components themselves: ProvidesConversionComponent and UsesOGSIConvertComponent. As an example, we look at the UsesOGSIConvertComponent, which shows how to add both provides and uses ports, how to get a reference to a uses port and how to make remote calls using the same. As usual, the details are explained by the javadoc comments.

package samples.conversion;

import gov.cca.Component;
import gov.cca.Services;
import gov.cca.TypeMap;

import soaprmi.util.logging.Logger;

import xcat.exceptions.UnexpectedException;

/**
 * The implementation of the UsesConversionComponent.
 * Every XCAT3 Component implements the gov.cca.Component interface.
 */
public class UsesConversionComponent implements Component  {

  private static Logger logger = Logger.getLogger();

  // Every component contains an instance of gov.cca.Services
  private Services usesConvertCore;
    
  // Instance of a GoPort for this component to bootstrap execution
  private GoPortImpl goPort;

  /**
   * Default empty constructor: Required for instantiation
   */
  public UsesConversionComponent() {
    logger.finest("called");
  }

  /**
   * The only method defined by the Component interface
   * @param cc the services object to initialize the component with
   */
  public void setServices(Services cc) 
    throws gov.cca.CCAException {
    logger.finest("called");

    // set the Services object to the one passed
    usesConvertCore = cc;

    // Addition of a uses port requires a gov.cca.TypeMap object.
    // The TypeMap object needs to contain the portClass entry with the
    // fully qualified class name of the interface defining the port.
    TypeMap uMap = usesConvertCore.createTypeMap();
    uMap.putString("portClass",
		   ConversionPort.class.getName());

    // param[0] : name of port to register
    // param[1] : namespace for the port
    //            THIS HAS TO BE THE SAME AS THAT OF MATCHING PROVIDES PORT
    // param[2] : typeMap object for the port as defined above
    usesConvertCore.registerUsesPort("convertUsesPort",
				     "http://www.extreme.indiana.edu/xcat/ports/convert",
				     uMap);

    // create an instance of the GoPort
    try {
      goPort = new GoPortImpl(this);
    } catch (Exception e) {
      logger.severe("can't instantiate GoPort", e);
      throw new UnexpectedException("can't instantiate GoPort", e);
    }
    
    // Addition of a provides port requires a gov.cca.TypeMap object.
    // The TypeMap object needs to contain the portClass entry with the
    // fully qualified class name of the interface defining the port.
    TypeMap pMap = usesConvertCore.createTypeMap();
    pMap.putString("portClass",
		   intf.ports.XCATGoPort.class.getName());

    // param[0] : the instance of the port to add
    // param[1] : name of port to register
    // param[2] : namespace for the port
    // param[3] : typeMap object for the port as defined above
    usesConvertCore.addProvidesPort(goPort,
				    "providesGoPort",
				    "http://www.extreme.indiana.edu/xcat/ports/go",
				    pMap);
  }

  /**
   * This method represents whatever computation the component needs to do.
   * This method is called by the GoPort's go method
   */
  public void startExecuting() {
    try {
      // Get a reference to a usesConvert port to invoke remote methods
      ConversionPort usesConvert = (ConversionPort)
	usesConvertCore.getPort("convertUsesPort");
            
      // Make a few calls
      logger.finest("UsesConversionComponent: Celcius : 0, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(0f));
      logger.finest("UsesConversionComponent: Celcius : 20, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(20f));
      logger.finest("UsesConversionComponent: Celcius : 40, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(40f));
      logger.finest("UsesConversionComponent: Celcius : 60, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(60f));
      logger.finest("UsesConversionComponent: Celcius : 80, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(80f));
      logger.finest("UsesConversionComponent: Celcius : 100, Fahrenheit: " +
		    usesConvert.centigradeToFahrenheit(100f));

      // Release the port when you are done using
      // If it is not released, another getPort with the
      // same arguments will block
      usesConvertCore.releasePort("convertUsesPort");
    } catch (Exception e) {
      logger.severe("Exception in startExecuting", e);
    }
  }
}

Running the Components

The sample components can be compiled by running ./build.sh samples inside the XCAT_HOME directory. XCAT3 uses Apache Ant for compilation. The script used for compilation is build.xml, located inside the XCAT_HOME directory.

The conversion.py script located in $XCAT_HOME/src/jython/samples can be used to create, connect and bootstrap execution of the above OGSI-enabled components. Relevant sections from the script are shown below. The documentation associated with the respective steps highlight the important details.


# use the TypeMap for component instantiation
# create the TypeMap for the provider component
# required fields are: className, execDir and execName
#
# className: Fully Qualified classname of the component
# execDir:   The execution directory, which is the directory where the
#            scripts reside
# execName:  The script which launches the component
providerMap = cca.createTypeMap()
providerMap.putString("className",
		      "samples.conversion.ProvidesConversionComponent")
providerMap.putString("execDir",
		      xcatHome + "/src/java/scripts")
providerMap.putString("execName",
		      "ContainerLauncher.sh")

# create the TypeMap for the user component
userMap = cca.createTypeMap()
userMap.putString("className",
		  "samples.conversion.UsesConversionComponent")
userMap.putString("execDir",
		  xcatHome + "/src/java/scripts")
userMap.putString("execName",
		  "ContainerLauncher.sh")
The above snippet sets the environment for execution of the components, e.g it tells the system that the directory for execution is $XCAT_HOME/src/java/scripts (using the execDir variable) and the script that is called to instantiate the component is ContainerLauncher.sh. Note that this information pertains to the location where the component is to be instantiated. In other words, if the component is to be instantiated locally then the above information should be valid for the local machine, and if the component is to be instantiated remotely the information should be valid on the remote machine. Make sure that the JDK variable is set correctly in the ContainerLauncher.sh to point to a valid Java installation on the target machine.
# create component wrappers
# these are holders for the component instance
provides = cca.createComponentWrapper("provider", providerMap)
uses = cca.createComponentWrapper("user", userMap)

# assign a machine name
cca.setMachineName(uses, "k2.extreme.indiana.edu")
cca.setMachineName(provides, "k2.extreme.indiana.edu")

# set a creation mechanism to in-process (local)
# other possible mechanisms are gram (only if globus is installed) or ssh
cca.setCreationMechanism(uses, "local")
cca.setCreationMechanism(provides, "local")
The above snippet tells the system that the components are to be launched in the same process. If these components have to be launched remotely, then set the creation mechanism to be ssh [or gram if you want to use Globus GRAM to launch your jobs. However, since this is a basic tutorial, we don't provide any instructions on how to use our system with Globus; but if you are interested in doing so, drop us a note].
# set a creation mechanism to be ssh (as described above)
cca.setCreationMechanism(uses, "ssh")
cca.setCreationMechanism(provides, "ssh")

If you are using the ssh creation mechanism, you will also have to tell the system where to find the ssh installation, and you can do that by adding it as environment variables as follows:

providerMap.putString("sshHome",
                      "/usr/local/bin/ssh")
userMap.putString("sshHome",
                  "/usr/local/bin/ssh")
If you are using ssh as the creation mechanism, the stdout and stderr for the components can be seen as .out and .err files in the directory pointed to by the execDir variable on the target (remote) machine. To change the location where the outputs should be dumped, you can set the stdOut and stdErr variables as follows:
userMap.putString("stdOut",
		  xcatHome + "/user.out")
userMap.putString("stdErr",
		  xcatHome + "/user.err")
Make sure that the above locations are writable by you. Additionally, make sure that you can ssh to the target machine without password (via an ssh-agent). Another thing to note is that CTRL-C does not kill the remote components, it only terminates the local jython interpreter. Hence, you will have to kill your components manually when you are done. In this example the components kill themselves when they are done, but in the general case you may have to terminate them manually.

The following code snippet creates live instances of the components on the target machines using the specified creation mechanism, connects the ports, and bootstraps the execution of the application (by invoking the go method on the XCATGoPort implementation of the uses-component).

# create live instances
cca.createInstance(uses)
cca.createInstance(provides)

# connect their ports, so that the UsesConversionComponent can connect
# to the ProvidesConversionComponent to make remote calls
cca.connectPorts(uses, "convertUsesPort", provides, "convertProvidesPort")

# start the components
# For goPort
portClassName = "intf.ports.XCATGoPort"
portType = "http://www.extreme.indiana.edu/xcat/ports/go"
providesPortName = "providesGoPort"
methodName = "go"
# no parameters
methodParams = zeros(0, Object)
cca.invokeMethodOnComponent(uses, 
			    portClassName, 
			    portType, 
			    providesPortName, 
			    methodName, 
			    methodParams)
print "Done"

The above script can be run from XCAT_HOME by typing run.sh script ./src/jython/samples/conversion.py. Along with a lot of other debug information, you should see:

[ 23:23:28.850 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 0.0
[ 23:23:28.857 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 32.0
[ 23:23:28.862 Thread-0: samples.conversion.UsesConversionComponent.java:77 startExecuting ] UsesConversionComponent: Celcius : 0, Fahrenheit: 32.0
[ 23:23:28.868 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 20.0
[ 23:23:28.874 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 68.0
[ 23:23:28.875 Thread-0: samples.conversion.UsesConversionComponent.java:79 startExecuting ] UsesConversionComponent: Celcius : 20, Fahrenheit: 68.0
[ 23:23:28.876 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 40.0
[ 23:23:28.877 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 104.0
[ 23:23:28.878 Thread-0: samples.conversion.UsesConversionComponent.java:81 startExecuting ] UsesConversionComponent: Celcius : 40, Fahrenheit: 104.0
[ 23:23:28.879 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 60.0
[ 23:23:28.880 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 140.0
[ 23:23:28.881 Thread-0: samples.conversion.UsesConversionComponent.java:83 startExecuting ] UsesConversionComponent: Celcius : 60, Fahrenheit: 140.0
[ 23:23:28.882 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 80.0
[ 23:23:28.882 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 176.0
[ 23:23:28.883 Thread-0: samples.conversion.UsesConversionComponent.java:85 startExecuting ] UsesConversionComponent: Celcius : 80, Fahrenheit: 176.0
[ 23:23:28.884 Thread-0: samples.conversion.ConversionPortImpl.java:28 centigradeToFahrenheit ] ConversionPortImpl: Received celcius value: 100.0
[ 23:23:28.885 Thread-0: samples.conversion.ConversionPortImpl.java:30 centigradeToFahrenheit ] ConversionPortImpl: Returning fahrenheit value: 212.0
[ 23:23:28.886 Thread-0: samples.conversion.UsesConversionComponent.java:87 startExecuting ] UsesConversionComponent: Celcius : 100, Fahrenheit: 212.0

APPENDIX

Handle Resolution

XCAT3 uses a two-level naming scheme for all remote services, as specified by the OGSI specification: a Grid Service Handle (GSH) that serves as an immutable location-independent name for a service, and a Grid Service Reference (GSR) that provides a precise definition for accessing a service at any point of time. It uses a Handle Resolver provided by GSX to map from a GSH to the most recent GSR.

The location of the Handle Resolver can be provided to the XCAT3 framework using the System property HANDLE_RESOLVER_URL. However, if this URL is not provided, a temporary Handle Resolver is started automatically by the Jython scripts that launch the component. The components can then use this Handle Resolver for resolving a GSH to a GSR. On the flip side, this Handle Resolver is terminated when the Jython scripts terminate. If these scripts are used for launching long-running component instances which outlive the life of the Jython script, the component instances may not be able to resolve a GSH to a GSR. In this scenario, it is advisable to start a stand-alone Handle Resolver service and notify the framework of its location using the HANDLE_RESOLVER_URL property.

A stand-alone Handle Resolver service can be started by changing to the XCAT_HOME directory, and executing the following command: run.sh handle_resolver. This starts an OGSI compliant Handle Resolver, using an in-memory handle resolution table. It is also possible to create a MySQL-based Handle Resolver service which stores the handle resolution tables in a MySQL database. Before this can be done, the appropriate tables need to be set up in a MySQL database (version 4.x, which can be downloaded from here).

You would need to create a database, say XCAT, and a user, say xcat_user, with permissions to create, drop, select, insert, delete and drop. You can do these as follows:

    bin/mysql -u root -h localhost -p
    create database XCAT;
    grant create, drop, insert, select, update, delete on XCAT.* to xcat_user@"%" 
    identified by 'xcat-password';
You can set the tables as follows:
    CREATE TABLE handle_table (
           handle_id BIGINT(10) NOT NULL PRIMARY KEY AUTO_INCREMENT,
           handle_uri TEXT)
    CREATE TABLE gsr_table (
           gsr_id BIGINT(10) NOT NULL PRIMARY KEY AUTO_INCREMENT,
           gsr_hashcode BIGINT(10),
           gsr_xml TEXT)
    CREATE TABLE map_table (
           handle_id BIGINT(10) NOT NULL,
           gsr_id BIGINT(10) NOT NULL,
           PRIMARY KEY(handle_id, gsr_id))
Then you can start the Handle Resolver as follows: run.sh handle_resolver -dburl jdbc:mysql://localhost/XCAT -dbuser xcat_user -dbpassword xcat-password.

Passing Parameters

As we have seen earlier, TypeMap objects are used to set properties for components. These properties are made available to the component, and can be used to pass parameters to it. e.g. a property can be set as follows from the Jython script.

userMap.putString("className",
		  "samples.conversion.UsesConversionComponent")
The above property can be retrieved from inside the component as follows.
gov.cca.TypeMap props = ((intf.ccacore.XCATServices) usesConvertCore).getProperties();
String className = props.getString("className", "None"));
In this case, None is the default string which will be returned if the property className is not found. Also note that these properties are set *after* the setServices method is invoked, and hence will not be available during the same.

Connecting to Standard Web Services

Using XCAT3, components can connect to standard Web services, in addition to remote provides ports. In order to do so, we have added a special type of port called WS Port. WS Ports function similar to Uses ports, except that they are connected to remote Web services - not Provides ports. A component can register a WS port, just like it would register a Uses port, as follows:

// Create a TypeMap object to set port properties
TypeMap wMap = usesConvertCore.createTypeMap();

// Add the portClass property containing the fully qualified
// class name for the remote Web service
wMap.putString("portClass",
	       ConversionPort.class.getName());

// Add the WS port, passing (i) portName, (ii) unique port identifier,
// and (iii) port properties
((XCATServices) usesConvertCore).
   registerWSPort("convertWSPort",
		  "http://www.extreme.indiana.edu/xcat/ports/convert",
		   wMap);
Since XSOAP currently does not have complete WSDL support, you need to specify the Java interface to interact with the remote service. This interface needs to extend the soaprmi.Remote base interface.

Before a WS port is retrieved for remote invocations, it needs to be connected to a Web service supporting the same interface. This can be done using the Jython script as follows:

# connect the WS port of the uses component to a Web service
cca.connectPortToWS(uses, "convertWSPort", endPointLocation)
The endPointLocation parameter is the URL where the remote Web service is located. The WS port registered earlier, can be retrieved by the component as follows:
// Get a reference to a usesConvert port to invoke remote methods
ConversionPort wsConvert = (ConversionPort) 
   ((XCATServices) usesConvertCore).getRemoteRef("convertWSPort");
Remote methods can be invoked on the port, once it has been retrieved from the framework. Once the methods have been invoked, the port should be released as shown.
// Release the port when you are done using
// If it is not released, another getRemoteRef with the
// same arguments will block
((XCATServices) usesConvertCore).releaseRemoteRef("convertWSPort");
The semantics of the get/releaseRemoteRef are similar to those of the get/releasePort methods.

Java API Documentation

To build the Java API documentation for XCAT3, run the following command from the XCAT_HOME directory: build.sh javadocs. This should build the javadocs for the code inside the ./docs/javadocs subdirectory.

FEEDBACK

We are very happy that you share our interest in XCAT3! Please feel free to send any comments and/or suggestions to our mailing list.


Sriram Krishnan
Last modified: Tue Sep 7 15:16:51 EST 2004