XCAT Tutorial


The following tutorial comes with a package containing the scripts, C, Java and C++ source code, as well as all the .jar's, necessary to run the demo examples. You may wish to download and install this before proceeding.


Index

  1. Introduction
  2. Using the Application Manager
    1. The Jython Control Script
      1. The Basic Wrapper Template
      2. Adding Events
      3. Monitoring the Exec'd Process: ProcessManager
      4. Limitations to the Wrapper Approach
    2. The Jython Launcher Script
      1. The XML Description File
      2. The Basic Launcher Template
      3. Connecting AppMan Ports
      4. Asynchronous (Batch-Adapted) Launching
  3. Building Ad Hoc XCAT Components
    1. Using Java [JDK 1.3 or later]
      1. The CCA Component Object Interface
      2. Converting a Java Class into a Component
      3. Defining Port Interfaces
      4. Implementing Provides Ports
      5. Setting Up and Running the Component
      6. Retrieving Component IDs and Uses-Ports
      7. Instantiating a Component from within a Component
    2. Using C++



INTRODUCTION

The XCAT implementation of the CCA component specification can be utilized to build Grid applications in two basic ways:

We will walk through both of these options in some detail.

NOTE: This tutorial assumes that the user will be operating in a UNIX environment which is also Grid-enabled (i.e., with the proper Globus security mechanisms in place).

[For another exposition of this material, along with a few more diagrams, which may be useful to consult together with this tutorial, see Component Technology. For further information on this laboratory's work on Science Portal Technology, see Science Portals Project.]

Back to Index


USING THE APPLICATION MANAGER

[See also the related discussion The XCAT Scriptable Application Managers. We have adopted the term "Application Manager" along with the abbreviation "appMan" for what is elsewhere also referred to as the "Generic Scriptable Proxy".]

In this model, calls to native code and other system calls are wrapped into one or more Jython scripts; these in turn are uploaded into one or more application managers; if the managers need to communicate, they are connected; then the entire application is set in motion. Hence the work-flow would look like this (* = optional):

  1. create instance(s) of application manager
  2. (*) add ports to manager(s) as necessary
  3. (*) connect manager ports as necessary
  4. upload control script(s) into manager(s)
  5. start manager(s) executing

[ Note (1): In a Web Services environment, it is possible that one would simply locate an existing instance of the application manager rather than creating a fresh one. This would be especially true if one were hoping to find a particular subclass of the application manager which might have additional functionality required to run the type of application one is interested in, rather than having actually to implement the subclass itself. In such an environment, the work flow could also conceivably be taken care of by a Web Services mechanism such as WFDL, but for the moment it is embodied by another Jython script (the top-level or application-launching script) which most commonly would be run from some kind of portal. ]

[ Note (2): This model is still available for use; however, the next release of the Application Manager will also have a generic communication port through which XML-string messages can be passed. ]

This approach is extremely flexible, allowing for the rapid adaptation of any existing shell scripts responsible for setting up and running executables. Let us examine a very simple mock example to see how this works.

Suppose we have a parameterized, precompiled executable which writes its output to a file (the most widespread programming model for scientific computation). The following (admittedly silly) piece of C code is just that: it takes the names of environment variables as command-line arguments and prints them with their values to the file "env-vars".

printenv.c

Now suppose this executable were called from inside a c-shell script that sets a number of environment variables first and then determines the arguments to the C binary. Here it is:

printenv.csh

Let us pretend that this is the Grid application we wish to run using the application manager. What we now need to do to adapt this code to XCAT is as follows:

    1. write a Jython script which wraps the csh script;
    2. write another Jython script which feeds this into the application manager, and starts it running.

Back to Index


The Jython Control Script

This script is in principle extremely easy to write: since Jython is pure Java, it can directly invoke the Java Native Runtime exec method on the shell script. In reality, however, there may be a number of other steps required for successful execution, such as cd'ing to the proper directory, and setting up the command so that the Java method understands it (for instance, it is advisable to send system calls in UNIX environments using a 3-string array which begins with an explicit invocation of the shell, such as: {"/bin/sh", "-c", "arg0 ..."}.)


The Basic Wrapper Template

Step-by-step instructions for writing the "wrapper":

  1. import the necessary modules from sys and java:
    
                import sys
                from java.lang import *
                from jarray import zeros
            
  2. construct a string array of arguments to give to the shell which executes the csh script (we use /bin/sh); we do this using the Jython jarray "zeros" method:
    
                ## construct the shell command arguments
                ## shell script assumed to be in current directory
                command = zeros(3, String)
                command[0] = String("/bin/sh")
                command[1] = String("-c")
                command[2] = String("./printenv.csh")
            
  3. pass this to the exec method
    
                ## call Java's Runtime exec method
                rt = Runtime.getRuntime()
                try:
                   p = rt.exec(command)
                except Exception, e:
                   System.err.println("exec: " + e)
                   p.destroy()
                   System.exit(-1)
            
  4. wait for the process to exit:
    
                ## wait for process to return
                try:
                   p.waitFor()
                except Exception, e:
                   System.err.println("waitFor: " + e)
                   p.destroy()
                   System.exit(-1)
            
  5. report the exit value and quit
    
                System.err.println(command + " returned exit value of " 
                                           + p.exitValue())
    
                ## make the application manager exit
                System.exit(0)
            

The preceding scriptlet can serve as the most basic "template" for all wrappers needing to be run by an application manager. Note, however, that messages from the script are directed to stderr using Java's "System.err.println" method; obviously, this will not be very practical in most cases, since the output will not be visible to the remote user, or it will get logged to the same file that may be capturing all of the stderr stream for the entire job. What is required here is separate logging or -- better -- an event-handling mechanism.

Back to Index


Adding Events

Assuming that the user has installed our xEvents package and has registered a channel, all that needs to be done is to convert the print calls above to calls to "publish" an event to this channel (then another listener can be used to monitor that channel for events of this type).

The following modifications to the script thus need to be made:

[ For further information on setting up and using event channels and listeners, see: xEvents, along with the README for the xEvents distribution package.]

Back to Index


Monitoring the Exec'd Process: ProcessManager

One could very well stop here, if it were not necessary to monitor the progress of the process; in most instances, however, intermediate results are being produced or errors could potentially be thrown which are needed for diagnosis or for maintaining the proper flow of control. It is therefore advisable to report these as well. Java's Process class provides a way to capture the streams produced by the process; it is not, however, necessary to implement this code, because there already exists a utility class in the Java XCAT distribution which takes care of this for you -- xcat.framework.util.ProcessManager (view the source code here).

This class has three different constructors, depending on what one wishes to do with the process output (what java.lang.Process refers to as the "input" and "error" streams, which are usually associated with stdout and stderr, respectively). One can log this output to a separate file by passing in a String with the name of the file (a "log" subdirectory is created in the current working directory, and the file is placed in that subdirectory); or one can pass in a constructed EventChannel object to which the ProcessManager will publish events; or, finally, one can pass in both a file name and an EventChannel, plus a boolean indicating whether only the location of the file should be sent as an event when the process exits (= true/1), or whether all output should be replicated to both the file and the channel (= false/0). All constructors take as their first argument a String naming the source of the call to the ProcessManager.

With a ProcessManager object constructed, one need only call its "exec" method on the same array of Strings one would have passed to Runtime exec (as above). This method waits for the process to exit and returns its exit value, at the same time launching two separate threads to monitor the two process output streams (these threads are terminated when the process exits). The output is translated into XML-style name-value pairs. Possible names are:

source who instantiated this ProcessManager
localhost host name where running
ipAddress host address where running
timestamp timestamp of event
method name of method executed
args arguments to method
status status of execution
stream name of process output stream
output data from process output stream
try-block try-block where exception was thrown
Exception stringified exception message
process exit value stringified integer
log file address hostname:path

Instead of the code given above for steps 3-5, then, we would have just the following:

The final .py script is provided here. If you have installed this demo package to run on your system, the script may be executed by invoking "run.sh script printenv.py" from the scripts directory.

Back to Index


Limitations of the Wrapper Approach

There is one main disadvantage to this kind of global wrapper: we are wholly dependent upon the data management internal to the shell script and executables for any information concerning state. Now the accessibility and/or granularity of such information may be acceptable for some applications, but will more often be inadequate, especially in the case of a long-running process that needs to be checked periodically. When the original job does not publish events, we are constrained to rely on the wrapper to do so; and obviously, the smaller the pieces which are being launched and monitored from that wrapper, the more information we can access directly without having to do remote logins, and the more control we have over the flow of execution.

Of course, the optimal, albeit certainly unrealistic, solution is to require the bottom-level script to be written in Jython, with XCAT incorporated from the start. Short of this solution, some amount of interaction between the design of any shell scripts and that of the Jython wrapper may still be desirable; in such instances, the process of creating an XCAT infrastructure will involve something more than the procedure as we have described it here; e.g., modifying the shell scripts and/or executables to produce more error or status output that can be parsed when sent as an event by the ProcessManager, or breaking up a single shell script into several smaller ones.

[We are currently considering the possibility of a mini-compiler which would facilitate the conversion of shell scripts -- perhaps manually preprocessed in some simple way -- directly into a Jython script .]

Back to Index


The Jython Launcher Script

This script is also fairly straightforward to code for the most common scenarios. The simplest case is when the wrapper script, such as the one we just wrote, merely needs to be executed without further interaction. A slightly more complicated case is when there are two scripts running in parallel which need to communicate with each other. Finally, there are special provisions for when the Application Manager needs to be launched through a batch system.


The XML Description File

To continue with our example: before writing the Jython script to launch the application manager, we must provide an .xml file describing the static component information and execution environment in the location where it will be launched. Multiple hosts on the same distributed file-system with a common directory structure can be listed under a given execution environment; all modes of execution enabled for those hosts may also be listed. The "exec-fqn" name-value pair contains the fully-qualified name for the .class file defining the component -- in this case, the application manager. Here is what such a file might look like for our sample program (with a fairly extensive list of hosts on a single NFS): appMan.xml. [Note that the last two name-value pairs are needed only for launching via ssh. For more detailed look at the XML file, see XML for XCAT.]

Back to Index


The Basic Launcher Template

[For a related exposition of the following material, see also: Using the XCAT Jython Builder.]

Once this file is set up, one can proceed to the script itself, which should:

    1. create a component wrapper for the appMan from the XML
    2. instantiate the appMan using the component wrapper
    3. upload and run printenv.py

Step-by-step:

That's all there is to it. (Again, if the demo package has been installed, you can test this launchAppMan.py script by calling "run.sh script launchAppMan.py" from the scripts directory.)

Of course, the same application manager can be reused to run any number of scripts in succession.

Back to Index


Connecting AppMan Ports

In the case of two or more application managers which must communicate with each other, ports may be dynamically added and connected through this script (by "dynamically", we mean that the port is registered -- not defined -- at run-time; obviously, there must already exist and be available to the script some precompiled implementation of whatever port type you might wish to add to the application manager). Adding a port also involves invoking a method on the component.

Here is an example from a chemical engineering application developed by IU and NCSA for the Alliance portal effort (script author Sriram Krishnan):

  1. Each of two application managers, "mc" and "fd", acquire a file-transfer uses and provides-port. Note that the signature of "addProvidesPort" calls for four String parameters: the name of the port, the specific sub-type, the name of the interface defining it, and the fully qualified class name for the implementation; "addUsesPort" only requires the first three of these arguments:
    
                # file transfer provides-ports
                usesPortClassName = "samples.idl.scriptPort.UsesScriptPort"
                usesPortType = "http://www.extreme.indiana.edu/xcat/samples/wsdl#scriptPort"
                providesPortName = "scriptPortProvidesPort"
                methodName = "addProvidesPort"
                methodParams = zeros(4, Object)
                methodParams[0] = String("providesFileTransfer")
                methodParams[1] = String("myFileTransfer")
                methodParams[2] = String("samples.idl.fileTransfer.ProvidesFileTransfer")
                methodParams[3] = String("samples.appMan.FileTransferImpl")
    
                cca.invokeMethodOnComponent(mcComponentProxy, 
                                            usesPortClassName, 
                                            usesPortType,
                                            providesPortName, 
                                            methodName, 
                                            methodParams)
                cca.invokeMethodOnComponent(fdComponentProxy, 
                                            usesPortClassName, 
                                            usesPortType,
                                            providesPortName, 
                                            methodName, 
                                            methodParams)
                
                # file transfer uses-ports
                methodName = "addUsesPort"
                methodParams = zeros(3, Object)
                methodParams[0] = String("usesFileTransfer")
                methodParams[1] = String("myFileTransfer")
                methodParams[2] = String("samples.idl.fileTransfer.UsesFileTransfer")
                
                cca.invokeMethodOnComponent(mcComponentProxy, 
                                            usesPortClassName, 
                                            usesPortType,
                                            providesPortName, 
                                            methodName, 
                                            methodParams)
                cca.invokeMethodOnComponent(fdComponentProxy, 
                                            usesPortClassName, 
                                            usesPortType,
                                            providesPortName, 
                                            methodName, 
                                            methodParams)
            
  2. To connect these ports, simply do:
    
                cca.connectPorts (mcComponentProxy, "usesFileTransfer", 
                                  fdComponentProxy, "providesFileTransfer")
                cca.connectPorts (fdComponentProxy, "usesFileTransfer", 
                                  mcComponentProxy, "providesFileTransfer")
    
            

To be precise, the "connection" established here is a virtual one: it tells the framework how to associate ports between two components. The actual network connection is done at the moment one component invokes a method on the other's port. One could actually "connect" two components' ports before instantiating the components themselves, provided a "proxy" wrapper or handle for the component has been constructed (see again: Using the XCAT Jython Builder).

[For another example of connecting components dynamically, see below.]

Back to Index


Asynchronous (Batch-Adapted) Launching

Let us step back now a moment and review the sequence or "stack" of processes generated by our sample program:

host:level script/executable method call which exec's successive process
1:1 run.sh system call to java
1:2 ui.python.RunScript.main
  • org.python.util.PythonInterpreter
    • .execfile(launchAppMan.py)
  • >> cca.createInstance
  • >> ui.python.CompositionTool.instantiate
  • >> ui.python.ComponentWrapper.createInstance
  • >> xcat.framework.creation
    • .CreationService.createInstance
  • >> org.globus.gram.Gram.request
2:0 [Globus Gatekeeper daemon]  
2:1 [Globus Job Manager]  
2:2 javaComp.sh system call to java
2:3 xcat.framework .utilSoapComponentInstantiator.main calls SoapConnectionClient.sendMyComponentID while CreationService.createInstance waits to receive it by blocking on SoapConnectionServerImpl.getComponentID for a predetermined timeout
1:2 ui.python.RunScript.main
  • org.python.util.PythonInterpreter
    • .execfile(launchAppMan.py)
  • >> cca.invokeMethodOnComponent
  • >> ui.python.CompositionTool.executeMethod
  • >> java.lang.reflect.Method.invoke
2:3  
  • samples.appMan.ScriptPortImpl.runScript
  • >> samples.appMan.AppManComponent.runScript
  • >> org.python.util.PythonInterpreter
    • .execfile(printenv.py)
  • >> xcat.framework.util.ProcessManager.exec
  • >> java.lang.Runtime exec
2:4 printenv.csh system call
2:5 printenv.out ...


Here is the same sequence represented diagrammatically:


What we wish to focus on here is the fact that the SoapComponentInstantiator sends the componentID (i.e., a remote reference) back to the process which launched the Gram.request -- in this case, the JVM in which the Java Python-interpreter object is running the launcher Jython script. The call to CreationService.createInstance, which is responsible for invoking Gram.request, also sets up a thread which waits for this ID to arrive, but whose time-to-live is relatively short (15 seconds).

Now suppose the Gram submission were to a batch system. Waiting for the componentID in this case will probably not be desirable, since the period of time between the submission of the request and the actual instantiation of the appMan could be quite lengthy, and even if we were assured of a reliable network connection, increasing the waiting thread's time-to-live could still result in failure, since ultimately the interval in question is indeterminate. Clearly some asynchronous version of the appMan launch is needed in such a case.

Such an alternative has been implemented as the main method of the appMan component itself. By calling samples.appMan.AppManComponent from the command-line, the work usually done by the generic SoapComponentInstantiator is now done in the new JVM (this bootstrapping model is actually what has in fact been followed in implementing XCAT in C++, since the latter has no "reflection" capabilities); but instead of returning the componentID (there is in this case nothing to return it to), the appMan registers it with an Ldap service. (In the current version of the appMan, the main method also automatically calls "RunScript" with the pathname of the Jython script as one of the arguments passed to appMan.sh, so the execution is entirely independent of the caller; however, should that remote script wish to invoke "RunScript" or some other method on the ports of the appMan, all it needs to do is to retrieve the apposite componentID from the Ldap service.)

The "batch-adapted" version thus has the following structure:

host:level script/executable method call which exec's successive process
1:1 run.sh system call to java
1:2 ui.python.RunScript.main
  • org.python.util.PythonInterpreter
    • .execfile(launchAppManRSL.py)
  • >> org.globus.gram.Gram.request
2:0 [Globus Gatekeeper daemon]  
2:1 [Globus Job Manager]  
2:2 appMan.sh system call to java
2:3 samples.appMan.AppManComponent.main
  • samples.appMan.ScriptPortImpl.runScript
  • >> samples.appMan.AppManComponent.runScript
  • >> org.python.util.PythonInterpreter
    • .execfile(printenv.py)
  • >> xcat.framework.util.ProcessManager.exec
  • >> java.lang.Runtime exec
2:4 printenv.csh system call
2:5 printenv.out ...


And again, the same sequence represented diagrammatically:


This option is best utilized when there is also some form of asynchronous messaging (event) system by which the original caller can be notified when the appMan actually begins to run. It would then be appropriate to include the publication of an initial event to this effect in the script which is to be uploaded and executed by the appMan, or perhaps by the appMan itself (the latter option would entail customizing the component or subclassing it to include this special feature).

We have provided a Jython script, launchAppManRSL.py, which demonstrates this batch-adapted version of the Application Manager. Like the other scripts, this can be called from the scripts directory using "run.sh scripts launchAppManRSL.py" if you have properly installed the demo package.

Notice that because the "main" method of appMan.AppManComponent is set up to take command-line parameters, and these (in particular, the path of the script to be uploaded) need to be passed to the call, the Jython cca module methods can no longer be used. Hence, the launch script imports the Globus Gram module, constructs the RSL and invokes "Gram.request" directly. Notice also that instead of javaComp.sh as the executable, we use appMan.sh, which takes care of setting up the registry host and port parameters so that the appMan knows how to contact the Ldap in order to register its component ID.

For the full functionality of the Application Manager, you may wish to consult the javadoc pages.

Back to Index


BUILDING AD HOC XCAT COMPONENTS

The second alternative -- "rolling your own" -- will sometimes be the preferred method for creating a system of CCA-compliant components using XCAT. This can be done in either Java or C++. The former case is relatively uncomplicated, whereas the latter involves considerably more work. We will begin by discussing the procedure in Java.


Using Java [JDK 1.3 or later]

"Componentizing" a Java class is simpler than doing so for a C++ class because of the existence (in JDK version 1.3 and later) of the built-in dynamic proxy generator (java.lang.reflect.Proxy) which exploits reflection to serialize objects on the fly, eliminating the need for defining the traditional "skel" and "stub" classes containing the serialization and deserialization routines for the associated object class. This functionality has already been incorporated into the way the XCAT framework deals with a component's ports (cf. xcat.framework.ccacore.ProvidesDynamicSkelImpl and xcat.framework.ccacore.UsesDynamicStubImpl) via XSOAP (e.g., soaprmi.soaprpc.SoapServices, which in turn uses soaprmi.soaprpc.SoapDispatcherImpl and soaprmi.soaprpc.SoapInvokerImpl), so building a Java component essentially reduces to the following steps:

  1. define and implement the component
  2. define the component's port interfaces and implement them

If the component is entirely new, it would perhaps be more precise to think of the interfaces for both the component and its ports as elaborated simultaneously, since there is a necessary interconnection between them; but it is also possible (and very likely will not be uncommon in practice) simply to export a pre-existent class's interface through the port interface(s), as will be seen shortly.


The CCA Component Object Interface

The CCA specification is actually rather loose in terms of what it will accept as a component. In essence, a component must:

    1. have an empty constructor, which is called by the framework at instantiation;
    2. implement the Component interface, which means providing a "setServices" method that defines and registers all ports on the component.

As seen above in the case of the appMan, it is possible, but not necessary, to provide a main method in Java for the component. On the other hand, the CCA specification discourages constructor overloading. If the component must be initialized, this should be done through a separate method (e.g., "public void init") which can be invoked from a port.

Back to Index


Converting a Java Class into a Component

One result of this minimalist definition is that the conversion of a pre-existent Java class into a component does not involve further modifications to its public interface, much less deeper changes to the way it is implemented. Since the component is not a remote object, but rather a local one whose functionality is exported, all one needs to do is enfold its public API into one or more CCA-compliant provides-ports (in our implementation, these extend XSoap's RemoteObject class). Hence, only two changes are to be effected to the potential component class itself:

In order to illustrate this process, we will presume to have a Java source file, called PrintEnv.java, which defines an object whose functionality is analogous to that of our running example: like printenv.c, it prints to a file the System properties whose names are passed into it as arguments:


        import java.io.*;

        public class PrintEnv 
        {
	   private String [] args;
	   private PrintWriter pw;
	
	   public PrintEnv (String [] _args)
	   {
              args = _args;
              pw = null;
              try {
                 FileOutputStream fos = new FileOutputStream("env-vars");
                 pw = new PrintWriter(fos,true);
              } catch (IOException ioex) {
                 System.err.println("PrintEnv constructor: " + ioex);
              }
	   } // constructor
	
	   public void write()
	   {
              if (pw == null) {
              System.err.println 
                 ("PrintEnv.write:  PrintWriter was not initialized");
                 return;
              }
              for (int i = 0; i < args.length; i++)
              {
                 pw.print(args[i] + ":\t");
                 pw.println(System.getProperty(args[i], "Undefined"));
                 if (pw.checkError) {
                    System.err.println
                       ("PrintEnv.write: error printing value of args["+i+"]");
                    return;
                 }
              }
	   } // write
        
        } // class PrintEnv
      


[ Strictly speaking, an exact Java rendering of the sample C code should really look like this:


        import java.io.*;

        public class PrintEnv 
        {
           public static void main (String [] args)
	   {
              PrintWriter pw = null;
              FileOutputStream fos = null;
              try {
                 fos = new FileOutputStream("env-vars");
                 pw = new PrintWriter(fos,true);
              } catch (IOException ioex) {
                 System.err.println("PrintEnv: " + ioex);
                 return;
              }
            
              for (int i = 0; i < args.length; i++)
              {
                 pw.print(args[i] + ":\t");
                 pw.println(System.getProperty(args[i], "Undefined"));
                 if (pw.checkError) {
                    System.err.println
                       ("PrintEnv.write: error printing value of args["+i+"]");
                    return;
                 }
              }
	   } // main
        
        } // class PrintEnv
    

We have given a more object-like version in order to follow through with our point concerning parameterized constructors. Converting a Java class such as this one, however, is no more difficult: instead of an "init" method, one could substitute a "runComponent" method, which would be called by the control port and would be equivalent to the "main" above; or, one could break up "main" into an "init" method and a "run" method, with the former called through the control port's "sendParameter" method and the latter activated by the control port's "start" method (here the semantics of the class suggest that the file-writing functionality is once-per-process, not a separate procedure, so we would probably not export it through another provides-port). For port methods, see step 2 of the walk-through (immediately following), as well as the section on Implementing Provides Ports.]


We now walk through the conversion process (at the end of this section, we will also add an event system to this object to make it conform to the example in the first part of this tutorial).

  1. Get rid of the constructor with an argument by replacing it with an empty constructor, and by moving the constructor code over into an "init" method (which we also have throw an Exception):
    
                import java.io.*;
    
                public class PrintEnv 
                {
                   private String [] args;
                   private PrintWriter pw;
    
                   // empty constructor, as required by CCA specification
                   public PrintEnv () {}
                
                   public void init (Strings [] _args) throws Exception
                   {
                      args = _args;
                      pw = null;
                      try {
                         FileOutputStream fos = new FileOutputStream("env-vars");
                         pw = new PrintWriter(fos,true);
                      } catch (IOException ioex) {
                         throw new Exception("PrintEnv.init: " + ioex.toString());
                      }
                   } // init
    	
                   public void write()
                   {
                      if (pw == null) {
                      System.err.println 
                         ("PrintEnv.write:  PrintWriter was not initialized");
                         return;
                      }
                      for (int i = 0; i < args.length; i++)
                      {
                         pw.print(args[i] + ":\t");
                         pw.println(System.getProperty(arg[i], "Undefined"));
                         if (pw.checkError) {
                            System.err.println
                               ("PrintEnv.write: error printing value of args["+i+"]");
                            return;
                         }
                      }
    	       } // write
            
                } // class PrintEnv
            
  2. Implement the Component interface by adding the setServices method, whose purpose is to register with the framework both the uses-ports and provides-ports on the component.

Back to Index


Defining Port Interfaces

Next, we need to define the port interfaces through which the local public methods of the component will be exported. We will assume, for the sake of simplicity, that these interfaces, together with the component itself, are all in the same package, called "demo".

Unlike the component itself, the provides-ports are remote objects, and it is a peculiarity of the current version of XSoap that remote objects may implement only a single interface; if multiple interfaces must be implemented, a derived interface should be defined which extends all the others, and the port object should then implement that.

Both of our provides-ports implement the ProvidesPort interface required by the framework; in addition, the control port implements the standard interfaces defined in the xcat samples package (for reasons we will not go into here, these are given the tag "_idl"; port interfaces are also commonly grouped together in a separate package labeled idl), while the second port defines the method (whose name for convenience mirrors the component's) whereby the component's file-writing functionality is accessed remotely.

Since, to be useful, the port interfaces should be defined in uses/provides pairs, it is good design to create a base-class interface for the file-writing ports, similar to that for the control ports, which defines the remote write method, and have both uses- and provides-versions of this port extend this interface. In conformity with praxis, we tag this interface "_idl" (there is no need for its write method to throw a RemoteException, since the underlying method will not throw any exceptions):


        package demo;
        
        public interface PrintEnv_idl
        {
           public void write();
        }
    

The control-port interfaces will thus look like this,


        package demo;
        
        import xcat.framework.ccacore.UsesPort;
        import samples.idl.control.*;
        import samples.idl.echo.*;
        
        public interface UsesParamControl
               extends samples.idl.control.Control_idl, 
               samples.idl.echo.Parameter_idl, 
               UsesPort {}
    

        package demo;
        
        import xcat.framework.ccacore.ProvidesPort;
        import samples.idl.control.*;
        import samples.idl.echo.*;
        
        public interface ProvidesParamControl
               extends samples.idl.control.Control_idl, 
               samples.idl.echo.Parameter_idl, 
               ProvidesPort {}
    

while the file-writing-port interfaces will look like this:


        package demo;
        
        import xcat.framework.ccacore.UsesPort;
        
        public interface UsesPrintEnv
               extends demo.PrintEnv_idl, UsesPort {}
    

        package demo;
        
        import xcat.framework.ccacore.ProvidesPort;
        
        public interface ProvidesPrintEnv
               extends demo.PrintEnv_idl, ProvidesPort {}
    

Back to Index


Implementing Provides Ports

Now we come to the actual implementation of the provides-port interfaces. In our example, there are two specific classes to be built: "PrintEnvProvidesImpl" and "PrintEnvControlImpl".

The first has a single method, which calls the component's method by the same name. Its constructor takes a reference to the underlying component. Note that the class extends soaprmi's UnicastRemoteObject; its constructor is therefore required to invoke the empty form of "super" explicitly:


        package demo;
        
        import soaprmi.RemoteException;
        import soaprmi.server.UnicastRemoteObject;
        
        public class PrintEnvProvidesImpl extends UnicastRemoteObject 
                                          implements demo.ProvidesPrintEnv
        {
           private PrintEnvComponent pec = null;
        
           //  Constructor has to call the superclass
           //  constructor explicitly, as this is a remote object
           public PrintEnvProvidesImpl (PrintEnvComponent _pec) 
                  throws RemoteException 
           {
              super();
              this.pec = _pec;
           } // constructor
 
        
           //////////////////////////
           // PrintEnv_idl Interface
           //////////////////////////
        
           public void write()
           {
              pec.write();
           } // write
        } // class PrintEnvProvidesImpl
     

For the second, we have already hinted above that its parameterized method will call the component's "init", while "kill" will stop the JVM; note also that "start" is given an empty implementation, since there is nothing to do.

A remark on the "sendParameter" argument: the array of Objects represents the parameters of the method which is to be called -- in this case, the component's "init". Since this latter method takes a single argument, an array of Strings, we downcast the first element of the "sendParameter" array to this type and then pass it on to the component method.


        package demo;
        
        import soaprmi.RemoteException;
        import soaprmi.server.UnicastRemoteObject;
        
        public class PrintEnvControlImpl extends UnicastRemoteObject 
                                         implements demo.ProvidesParamControl
        {
           private PrintEnvComponent pec = null;
    
           //  Constructor has to call the superclass
           //  constructor as this is a remote object
           public PrintEnvControlImpl (PrintEnvComponent _pec) 
                  throws RemoteException 
           {
              super();
              this.pec = _pec;
           } // constructor
        
        
           /////////////////////////
           // Control_idl interface
           /////////////////////////

           public int start() 
           {
              //nothing to do
              return 0;
           } // start
               
           // Wait for a few seconds, then stop the JVM.
           public int kill() 
           {
              new Thread() 
                 {
                    public void run() 
                    {
                       try {
                          Thread.sleep(3000); 
                       } catch (InterruptedException ie) {}
                          System.exit(0);
                    }     
                 }.start();
              return 0;
           } // kill

           ///////////////////////////
           // Parameter_idl interface
           ///////////////////////////

           // Initializes component.
           public int sendParameter (Object [] param) 
           {
              String [] args = (String []) param[0];
              try {
                 pec.init(args);
              } catch (Exception e) {
                 System.err.println ("sendParameter: " + e);
                 return -1;
              }
              return 0;
           } // sendParameter  
      
        } // class PrintEnvControlImpl
    

Back to Index


Settup Up and Running the Component

To run this component, one would follow steps similar to those described above for the launcher script: write an XML description of the component's environment, and use the cca methods to instantiate and initialize it. These two files have been provided with this demo (see below); special attention should be drawn to the triply-nested array constructed for the "sendParameter" invocation:


        ###########################################################
        ## The array nesting here reflects the following semantics:
        ## invokeMethodOnComponent takes an array of Objects
        ## representing the parameters of the method, in this case
        ## "sendParameter"; this is itself just a single array of 
        ## Objects, which in turn represents the arguments of
        ## the method to be called by "sendParameter": that is,
        ## in this instance, the component's "init", which again
        ## takes a single argument, an array of Strings.
        ###########################################################
        methodName = "sendParameter"
        methodParams = zeros(1, Object)    #invokeMethodOnComponent
        methodParams[0] = zeros(1, Object) #sendParameter
        args = zeros(4, String)            #component's init
        args[0] = String("HOME")
        args[1] = String("log")
        args[2] = String("java.compiler")
        args[3] = String("PYTHONPATH")
        methodParams[0][0] = args
        

Of course, it is possible to avoid the second singleton array of Objects and just pass the array of Strings as an upcast array of 4 Objects to "sendParameter", but this would also require that either the implementation of "sendParameter" downcast each member individually, constructing a new array of Strings to give to "init", or that the "init" method take an array of Objects intead of Strings and do this downcasting itself. Moreover, this would actually violate the semantic generality of the "sendParameter" array, which is meant to reflect the calling sequence of the method to which it is passing its arguments, and so in this case should really have only one member.

Note also that, unlike with the Application Manager, where "System.exit" is called in the script it runs, here we have to kill the component explicitly; we do so by invoking the control port's method.

Adding an event system to the component is also similar to what was done in the control script:

Depending on how the "publish" method will be used, one might wish to wrap the initialization and retrieval calls into it (a less robust solution would be to do this once at component initialization, with the concomitant risk of not being able to recover the connection should the channel go down then come back up). In the present case, the additional overhead of initialize-fetch on each call to "publish" will not noticed; if, on the other hand, many calls must be made to this method, then one might wish to spin off a thread whose sole task is to reinitialize the mappings and refresh the connection to the channel every so often; this would also require synchronization on any access to the channel variable. (We have left this as an exercise for the reader).

The full implementation of this code may be examined in the following files:

In addition, you may view javadoc pages for a summary of the PrintEnv API. To run the component, call "run.sh scripts launchJava.py" from the scripts directory (after having properly installed the demo package; consult these installation instructions should you also wish to (re)build the Java classes).

Back to Index


Retrieving Component IDs and Uses-Ports

When launching a component using a Jython script, some useful functionality provided by xcat.ccacore.Services is masked by the methods imported from the cca module (which in turn call the classes defined by the ui [user interface] package of the XCAT Java implementation). Two examples of such functionality which may be necessary to utilize inside components are getting a component's ID and getting a uses-port reference. In reality, this functionality is available to a caller whether or not it is a component (e.g., a Jython script); the trick is simply to make the framework treat the caller as if it were a component.

By way of demonstration, we provide below a method, "getUsesPort", which returns a reference to our component's UsesPrintEnv port. This method could be incorporated into any Java object (or a similar one could be written in Jython). In the case that the object does implement Component, it is not strictly necessary to get a new Services object; a reference to the one passed in through the "setServices" method could be retained. Similarly, the component may have already registered in the "setServices" method the uses-port it will be retrieving here, though as stated above, port registration can take place dynamically, in any of the object's methods. In the interest of fully illustrating the process in question, we will assume that this method is part of a Java object which is not itself a Component.

Any component (or "pseudocomponent", as we will call this object), can obtain its own ID from the framework. Access to the ID of another component, on the other hand, is not possible unless:

Thus in the following code, we assume that the object already has a reference to the PrintEnvComponent, printEnvID, obtained in one of these two ways.

The class's import statements should include the following:


        ...
        
        import xcat.framework.ccacore.Services;
        import xcat.framework.ccacore.ComponentID;
        import xcat.framework.ccacore.PortInfoImpl;
        import xcat.framework.ccacore.UsesPortMapping;
        import xcat.framework.ccacore.ServicesImpl;
        import xcat.framework.connection.ConnectionService;
        
        import demo.UsesPrintEnv;
        
        ...
    

Notice next how:

  1. the object obtains an ID for itself;
  2. the port of the ConnectionService is obtained;
  3. the uses-port is registered, as if in a setServices method;
  4. the object is (virtually) connected to the component so that it will be able to invoke the methods of the corresponding provides-port (cf. the related discussion at the end of Connecting AppMan Ports);
  5. the uses-port is retrieved and returned.

These are marked in the code.


        private UsesPrintEnv getUsesPort()
        { 
           Services pseudocore = null;
           UsesPrintEnv printEnvPort = null;
           if (printEnvID == null) return null;
      
           try {
              // (1) get reference to this object's ComponentID
              pseudocore = new ServicesImpl();
              ComponentID pseudoID = pseudocore.getComponentID();

              // (2) get the ConnectionService port
              ConnectionService connectionService =
                 (ConnectionService) pseudocore.getPort("ConnectionService");

              // (3) register the uses-port
              UsesPortMapping.addMapping
                 ("printEnvUsesPort",
                  "http://www.extreme.indiana.edu/demo/wsdl#provides",
                  new Class [] {UsesPrintEnv.class});
              pseudocore.registerUsesPort
                 (new PortInfoImpl("printEnvUsesPort",
                  "http://www.extreme.indiana.edu/demo/wsdl#provides"));

              // (4) connect this object (virtually) to the component 
              connectionService.connect(pseudoID, "printEnvUsesPort",
                                        printEnvID, "printEnvProvidesPort");

              // (5) fetch and return the uses-port reference
              printEnvPort 
                 = (UsesPrintEnv) pseudocore.getPort("printEnvUsesPort");
           } catch (Exception e) {
              System.err.println("getUsesPort: " + e);
           }
        return printEnvPort;
        } // getUsesPort
    

Back to Index


Instantiating a Component from within a Component

In the same way that ports can be registered and retrieved dynamically, components can actually instantiate other components dynamically -- with the same reservation we stated above concerning the Application Manager's ability to add provide-ports: a .class file for the component to be created must exist and be available to the caller.

The usefulness of this capability might seem limited, but there are at least two scenarios in which such a need might arise:

There is essentially no difference in what needs to be done inside a Java component from what is done when the launcher script is used. Furthermore, there is nothing preventing us from directly invoking the user-interface (ui) classes which lie just below the line with respect to the Jython cca API.

In the following table, we give the Jython script version on the left, and the Java equivalents on the right.

Jython script Java component
  • import cca
  • import ui.python.*;
  • xmlpath = "../xml/appMan.xml"
  • String xmlpath =
    • new String("../xml/appMan.xml");
 
  • CompositionTool ct =
    • new CompositionTool();
  • ct.initialize():
  • componentProxy =
    • cca.createComponent(xmlpath)
  • ComponentWrapper componentProxy =
    • ct.create(xmlpath);
  • cca.setMachineName
    • (componentProxy, "k2.extreme.indiana.edu")
  • componentProxy.setMachineName
    • ( "k2.extreme.indiana.edu")
  • cca.setCreationMechanism
    • (componentProxy, "gram")
  • componentProxy.setCreationMechanism
    • ("gram")
  • cca.createInstance(componentProxy)
  • ct.instantiate(componentProxy)

Short of using the ui.python interface, one would have to implement the calls to the framework methods directly, as well as employ an XML parser to convert the .xml file into the EnvObj (environment object) needed by the call to xcat.framework.creation.CreationService.createInstance (this object holds the name-value pairs which will go into the RSL in the case of a Gram invocation). For most purposes, however, the provided interface should be sufficient.

The ui.python classes, in fact, also provide a wrapper around the connection routines which we implemented directly in the example from the section Retrieving Component IDs and Uses-Ports. Indeed, if those references are not required for future calls, one does not have to go through the bother of fetching them explicitly; all that would be necessary is, once again, to use the ui.python versions of the Jython methods illustrated previously in the section Connecting AppMan Ports.

For the sake of completeness, we adjoin to the preceding table an example of both a connection call and a method invocation (these are fabricated and do not correspond to any of the working code furnished with this demo). The variables named "...Proxy" are assumed to be of the type ComponentWrapper; "ct" continues to refer to the CompositionTool object. Note that we do not need direct access to the Component ID, though it is a member of the wrapper object (e.g., "componentProxy.theComponent") and could be extracted if so desired:

Jython script Java component
  • cca.connectPorts
    • (thisProxy, "usesPrint")
    • (printerProxy, "providesPrint")
  • ct.connect
    • (thisProxy, "usesPrint")
    • (printerProxy, "providesPrint");
  
  • usesPortClassName =
    • "printer.idl.UsesPrint"
  • String usesPortClassName =
    • "printer.idl.UsesPrint";
  • usesPortType =
    • "http://www.extreme.indiana.edu"
    • + "/printer/#printPort"
  • String usesPortType =
    • "http://www.extreme.indiana.edu"
    • + "/printer/#printPort";
  • providesPortName =
    • "providesPrint"
  • String providesPortName =
    • "providesPrint";
  • methodName = "print"
  • String methodName = "print";
  • methodParams =
    • zeros(1, Object)
  • Object [] methodParams =
    • new Object[1];
  • methodParams[0] =
    • String("Hello World")
  • methodParams[0] =
    • "Hello World";
  • cca.invokeMethodOnComponent
    • (printerProxy,
    • usesPortClassName,
    • usesPortType,
    • providesPortname,
    • methodName,
    • methodParams)
  • ct.executeMethod
    • (printerProxy,
    • usesPortClassName,
    • usesPortType,
    • providesPortname,
    • methodName,
    • methodParams);


Caveat: this discussion does not address issues of security and/or (Globus) proxy delegation which might arise when one component instantiates another remotely. (Up until Globus 1.4, it was possible to set the proxy delegation to "unlimited", but this option has been deprecated and furthermore need not have been ubiquitously supported in prior installations of Globus. The only way, unfortunately, to know whether it is possible is to try it out on the hosts in question.)

Back to Index


Using C++

COMING SOON!

arossi@indiana.edu
Last modified: Tue Jan 15 11:29:34 EST 2002