[hyades-dev] Bound and unbound HCE agents

[Allan K Pratt <apratt@xxxxxxxxxx>]

At today's HCE protocol meeting I agreed to write up something about 
agents which are "bound up with" specific processes.

WHY I'M TALKING ABOUT THIS

The HCE protocol document describes commands to start and stop an agent. 
There is also a reference to agents that aren't started and stopped this 
way, in the Agent Management Interface section.

What is not in the HCE document yet is any mention of whether and how the 
HCE can start a "target application" such as a Web server or a J2SE 
application. If it did mention this, it would have to discuss the 
interactions between agents and target applications, and particularly the 
way that some types of agents can only be launched in tandem with the 
processes they monitor. I'd like to get this entanglement out in the open 
and talk about it.

I also want to be clear about another assumption I have going into this 
project. I assume the whole point of the metadata and the HCE is so a 
client can be created without private knowledge of agents and, using only 
metadata and HCE API calls, start and control agents for data collection. 
For example, a client can query the HCE for a list of agents that can be 
started, show that list to the user, and start those agents that the user 
selects, all without special knowledge of how to drive any particular 
agent.

UNBOUND AGENTS

The agents I call "unbound" are not tightly coupled with any specific 
process. You can start one up and shut it down separately from other 
processes. In between, it will do its thing: collect data, monitor, 
respond to HCE and client commands, whatever. Some agents like this 
collect system-wide data, like CPU load or network bandwidth used. Some 
collect data by monitoring particular log files or other monitoring data 
provided by other processes, but are not "bound" to the target process by 
PID or other inextricable links.

BOUND AGENTS

The agents I call "bound" do not have an independent existence as a 
process. They are not started separately; they are started as a 
side-effect of starting a target process that they are meant to monitor.

The canonical bound agent is a JVMPI agent for monitoring a Java process. 
Such an agent is started using special command line options on the "java" 
command line (or equivalent). The agent loads as libraries inside the Java 
process. When the Java process exits, the agent goes with it.

The same paradigm exists for agents using the Microsoft .Net Profiling 
Interface for monitoring .Net managed programs, but the startup system is 
different: they are started based on the settings of certain environment 
variables during .Net process startup.

Another kind of bound agent is one that acts as a "wrapper" for the target 
process. On some Unix systems, for example, if you want to monitor a 
program "foo" using the "truss" monitoring program (which produces a trace 
of system-call activity), you use the command line "truss foo" to start 
both the data collection agent and the target process.

The major distinction is that a bound agent can not be started or stopped 
independently from the processes they are bound with.

WHAT IT ALL MEANS

In a world that only includes unbound agents, the client scenario I 
painted above is easy: a user can pick from a list of agents, and the 
client can tell the HCE to start or stop the selected agent, or control it 
in other ways. 

But in the real world of bound and unbound agents, the concept of 
"starting" the agent is completely different: it's encumbered with the 
concept of starting a target process. It's not as simple as knowing the 
command line to execute.

As a first cut, I think it would be good to specify in the agent metadata 
whether an agent is unbound - that is, if it can started independently. 
This enables the user scenario sketched above, at least for unbound 
agents.

I think it's much harder to use metadata to describe bound agents. The 
launch scenarios are highly variable. It might not be reasonable to extend 
metadata to include enough information to describe the launch protocol for 
a bound .Net agent, which involves setting three or more environment 
variables to particular values. And also to describe a Java bound agent, 
with VM arguments for launching it. And a wrapper agent, complete with 
notes on what wrappers may or may not appear with what other wrappers, in 
what order. Plus the types we haven't thought of yet.

I don't have the answers (except for flagging an agent as bound or 
unbound); I just wanted to spark the discussion.

-- Allan Pratt, apratt@xxxxxxxxxx
Rational software division of IBM