By far, the biggest new feature in CPN Tools 4 is simulator extensions. Simulator extensions makes it possible to add functionality to CPN Tools written in Java and also makes it possible for third parties to add functionality without having to bother with the rather esoteric languages Beta and Standard ML, which are used for the CPN Tools GUI and simulator.
As effort was ramped up for CPN Tools 4, I had a short-list of things that would be useful. A lot of those required doubling effort for reimplementing things already available in Java libraries or reimplementing the entire simulator in Java. Simulator extensions were designed as a solution that allowed reusing Java code without rewriting the entire simulator.
Many features of CPN Tools 4 rely critically on simulator extensions to work. This includes, message-sequence chart drawing, time intervals, Declare constraints, and PNML export. This feature describes the basics of simulator extensions: their architecture, the available communication patterns, and the Java interface for extensions. In later installments, we dig deeper into the available features.
Architecture
CPN Tools has hitherto consisted logically of two processes ((There is actually a third process mediating the communication between the two main processes, but let’s ignore that as it is not essential for our understanding.)): a user interface, the GUI, and a simulator doing the actual simulation and analysis:
The Access/CPN library makes it possible to replace the GUI part with any component written in Java:
To maintain as much backwards compatibility as possible, extensions communicate with the simulator:
As many extensions should react to user input or show output to users, it would seem adding another connection between the extensions server and the GUI, but instead we mediate all this thru the simulator, and get an architecture which conceptually looks like this:
This effectively allows the GUI to call extensions, but simplifies the GUI code, which only has to deal with messages to and form one channel. This also means that the GUI does not have to know whether a functionality is implemented by the simulator directly or by an extension. It is thus possible to prototype a feature as an extension and later move it directly into the simulator for improved performance. This introduction is not about explaining why the architecture is as it is; for that you can refer to this post.
Communication Patterns
CPN Tools prior to version 4 allowed three basic communication patterns:
Patterns 1 and 2 comprise of the GUI making a request to the simulator; the request is handled and a response is sent back. The simulator supports two kinds of requests. Pattern 1 handles basic remote procedure calls (RPCs) where a predefined call is handled. Such requests includes syntax checking and simulating CPN models. Pattern 2 allows the GUI to evaluate any code in the CPN simulator. This is used when a user manually evaluates ML code, e.g., to evaluate a state-space query, but also behind the user’s back to instantiate the state-space tool.
Pattern 3 is less used in versions of CPN Tools prior to 4, but allows predefined calls from the simulator to the GUI. Before version 4, this was only used by Access/CPN 2.0 to implement cosimulation, but gets a much more prominent role with CPN Tools 4 as the GUI implements more callbacks.
CPN Tools 4 introduces a number of new patterns. The first 3 just mirror patterns 1-3 to allow the extension mechanism to take on the same roles as the GUI:
Patterns 4 and 5 allow extensions interact with models as they see fit and to inject code into a running simulator. Such injected code can for example make a call back to the extension using pattern 6. The code can be called using pattern 2, essentially allowing a user to invoke code in the extension using the GUI. Such code can also be invoked using a code segment of a transition. We see that this already allows a lot of extension to take place as we can inject library code to perform any action in the extension at the request of the user.
The next two patterns allow the simulator to forward requests between the GUI and extension manager:
Pattern 7 is mostly useful for me as a CPN Tools developer. Any use of this requires changes to the CPN Tools GUI. I shall therefore not talk more about this, but mention that this is what makes the Declare functionality and PNML export work.
Pattern 8 along with another new feature of CPN Tools is very useful for extension developers. CPN Tools 4 allows such callbacks to directly display graphics in the user interface; this can be done from the simulator or – more conveniently – from an extension. We’ll get back to this in much more detail later.
The last pattern allows extensions to listen in on the communication between the GUI and the simulator. It essentially extends pattern 1 to also include extensions. The pattern comes in two variants:
Pattern 9a starts as pattern 1, but after the request is handled, it is forwarded to the extensions. They can then react however they want with the knowledge of the simulator’s response. The response of the extensions is passed on to the GUI. This allows extensions to have full knowledge of any model changes made in the GUI (as they are sent to the simulator for syntax check) and even to alter the response sent back to the GUI from the simulator. This is used by the Declare extension to alter which transitions are shown as enabled in the GUI.
Pattern 9b takes this even further, and forwards the requests to extensions before it is handled. The extension can react to or filter the request, and the altered request is then handled by the simulator, after which is is processed like pattern 9a. The reason for having 3 variants, 1, 9a and 9b is efficiency. While everything could be handled by variant 9b by just using identity filters and handlers, forwarding messages would lead to increased latency. This filtering ability is extremely powerful, and makes it possible to completely alter the simulator’s view of a model. In CPN Tools 4 this is used to implement time intervals without any changed to the GUI nor the simulator. Time intervals are simply recognized by the filter function and translated to something CPN Tools understands (actually, pattern 4 is used in the filter function to introduce a new variable in the model without the GUI knowing). This can also be used to remove net elements form the simulator and handle them in extensions (e.g., to create modules that are handled by external web-services) or to add new elements (e.g., to implement synchronous channels/transition fusion using simple unfolding behind the back of the user).
Interface
All of the complexity of all of these communication patterns is nicely encapsulated a couple of fairly simple interfaces in the extension server. In fact, most of the communication is handled automatically, and only rarely does a programmer need to think about the above patterns explicitly. We get more into details about the helper mechanisms later; here we’ll just look at the lowest level of support a user gets.
The basic interface that all extensions must implement is the Extension interface:
[java] package org.cpntools.simulator.extensions;public interface Extension {
int getIdentifier();
String getName();
List<Command> getSubscriptions();
Extension start(Channel c);
Packet prefilter(Packet request);
Packet handle(Packet p, Packet response);
Packet handle(Packet p);
Object getRPCHandler();
String inject();
List<Option<?>> getOptions();
<T> void setOption(Option<T> option, T value);
List<Instrument> getInstruments();
void invokeInstrument(Instrument i, Element e);
}
[/java]
Extensions have two identifiers, and integer id (l. 4) and a user-readable name (l. 5). for testing, extensions can use the special identifier 9999, but for any distribution, a unique id must be obtained as detailed here.
Extensions need to explicitly subscribe to messages forwarded using pattern 9 above. This is done using the getSubscriptions method (l. 6). The methods simply returns a list of the commands is is interested in. A command simply described which kinds of packages from CPN Tools the extension is interested in:
[java] public class Command {public Command(final int command, final int subcommand, final boolean prefilter) {
…
}
}
[/java]
Most of the times, using this functionality is for advanced uses. Commands and subcommands can be looked up in the protocol manual here. The prefilter boolean indicates whether variant 9a (false) or 9b (true) is to be used and can be omitted for the default value of false. The subcommand can also be omitted for the default Command.ANY, meaning that any subcommand with the specified major command is used. Using this functionality should only be necessary for deep integration and advanced usage; the extension server contains a log of mechanism for abstracting this away. Any subscribed command will be handed off to the prefilter (l. 8) and handle (l. 9) methods as shown for pattern 9 (using the name prefilter instead of filter).
Extensions are not connected to the simulator immediately on creation (some setup takes place in the background including the getSubscriptions method). When an extension is successfully connected, the start method (l. 7) is called. The Channel parameter is a representation of the connection to the simulator and GUI. The Channel looks like this:
[java] package org.cpntools.simulator.extensions;public interface Channel {
Packet send(Packet p) throws IOException;
String evaluate(String expression) throws Exception;
void subscribe(Command command, SubscriptionHandler h) throws IOException;
<T extends Extension> T getExtension(Class<T> clazz);
}
[/java]
The send (l. 4) and evaluate (l. 5) allow extensions to implement communication patterns 4 and 5 respectively. send is also responsible for pattern 8 depending on the packet sent, but this is in general handled more abstractly. We’ll get to that in a later installation. subscribe (l. 7) allows extensions to subscribe to command on-the-fly. This is useful if an extension only wants to listen to messages when necessary (to avoid the latency overhead if not needed). The getExtension method (l. 9) allows an extension to get an instance of another extension communicating with the same model. This makes it possible for extensions to build on the functionality of others, which is useful for abstraction. This, too, is the topic of another installment.
Going back to the Extension interface, we’ve already treated the prefilter and one of the handle methods (ll. 8-9). The one parameter handle method (l. 10) takes care of implementing communication pattern 7. The connection is made using the extension id (returned from getIdentifier in l. 4). As this is mostly interesting for me, I won’t go into much detail about this, but just state that this is very similar to pattern 9a except the simulator is in no way involved.
So far, we have treated how to handle communication patterns 4, 5 and 8 (ll. 4-5 of Channel), 7 (ll. 4 and 10 of Extension), and 8 and 9 (ll. 6 and 8-9 of Extension and 7 of Channel). Patterns 1-3 do not include extensions, so that just leaves us with pattern 6. As we expect that the most common use of this will be calling simple methods in extensions, this has been completely encapsulated.
The simplest use of pattern 6 is the getRPCHandler method (l. 12 of Extension). It returns any object (or null) to the extension manager. The extension manager then uses reflection to find any defined public methods, generates code to automatically invoke this, and injects the code into the simulator. If the object returned implements the NamedRPCHandler, it is generated using a user-visible name and otherwise it uses a generated name. This is as much of this section also a topic for later.
The inject method (l. 13) allows extensions to easily inject code into the simulator. Much like how getSubscriptions had a counterpart in the subscribe method of the Channel class, the inject method is the counterpart to the evaluate method of the Channel class. It implements a simple way to do communication pattern 5 once on extension start-up.
The last methods of Extension (ll. 15-19) also allow higher level integration with the CPN Tools GUI. As the names suggest, they allow extensions to add options to the CPN Tools GUI and even to add new instruments. As much else in this installment that is a topic for later.
Conclusion
In this part, we have gone over the basics of CPN Tools 4 extensions. We have seen the architecture of CPN Tools and how this has been extended with extensions. We have seen a number of communication patterns supported by extensions, and we have tied them to code. In the next installment, we show how to get started with extension development, and in future installments we introduce all the abstractions provided to make your life easier as an extension developer.
