You may have read my posts on conveniences and new features for CPN Tools. If you haven’t, you totally should, because both are very well-written and humorous 🙂 You may also have seen on my list of features for CPN Tools 4.0 that a feature “simulator extensions” is mentioned.
The thing is that I’d like to extend the simulator with some features that are 1000 times easier to code in Java than in SML. This includes a web-service interface and integration of the Declare formalism into the tool. In my post on new features, I argue that in order for this to be a reality, I would have to rewrite the simulator in Java.
This is still to a certain extent true, but for a pretty good solution, the important part is only that the new features are written in Java. This got me thinking…
In order to understand the following, here is a bit of background about how CPN Tools works. CPN Tools consists of two parts, a simulator and a GUI. They communicate over TCP using a two-way custom RPC protocol. This means the GUI can invoke methods in the simulator to syntax check and simulate CPN models and the simulator can invoke methods in the CPN Tools GUI. The latter is rarely used (actually, it is only used by Access/CPN to update the simulation graphics during cosimulation – se an example here). Graphically, think of CPN Tools looking like this:
Now, we’d like to extend this to, e.g., communicate with a web-service. It must communicate with (at least) one of the existing components. The CPN Tools GUI is written in Beta and the simulator in SML/NJ. It may come as a chock, but neither of those languages are very well-known, supported, nor modern, so there is no language reason to pick one over the other. If we connect web-services to the GUI, it will be possible to use the web-services while the web-service is actually running at a web-server, as this requires the GUI to run as well. This is sub-optimal, so we have to add the interface to the simulator. We would prefer not to make an external program using something like Access/CPN to make it possible to interactively set up and test models using or exposing web-services.
Now, SML/NJ does not (really) have support for concurrency. There is an extension called concurrent-ML (CML), which makes it possible to have light-weight threads. It includes a completely new thread-safe(ish) version of (parts of) the standard library. This cannot make use of operating system threads and hence no multi-computing for you, my friend. If we want the simulator to passively listen for connections, threads are almost a must. It seems like there is no way…
If we look to previous tools trying to tackle similar problems, BRITNeY Suite and Access/CPN both operate in two ways. The simplest let them replace the CPN Tools GUI. In the case of BRITNeY it furthermore injects some code allowing communication in the other direction (using XML-RPC) whereas Access/CPN allows bi-directional communication much in the same way as the CPN Tools GUI.
Both tools also work in another mode, allowing them to act as a proxy for the simulator. This mode is shown below. This allows the tools to observe and modify the commands sent to the simulator. BRITNeY used this to inject extra code into the simulator for acting as stub for the animation calls, and to learn the model being simulated, making it possible to automatically generate animations, e.g., an automatically generated message sequence chart for the simulation. Access/CPN uses this also to learn models making it possible to do cosimulations and showing feedback in the CPN Tools GUI by exploiting CPN Tools’ support for bidirectional communication.
It seems like an obvious idea to abuse the same trick again, but we would rather not. The reason is that we intend to alter the simulation, including the fast-forward step. In the fast-forward step, communication only takes place immediately before and immediately after the simulation. It is therefore not easy to influence the simulation in-between without resorting to tricks with severe negative impact on speed.
As a design goal is to make this all work without the GUI (so Access/CPN can benefit as well), there is really only one solution: swapping the last two boxes, yielding:
This requires changes to the SML code but not (necessarily) the Beta code. Which is good as changing SML is easier than changing Beta. After all, SML has dosens if not hundreds of users world wide, whereas Beta has, well, me.
The protocol used for communication between the CPN Tools GUI is called BIS. Instead of the old modem protocols (e.g., V22bis or V32bis), which uses the french word for “again” or “one more time” to indicate a not-really-a-standard-but-widely-used extension of standards, this is an abbreviation for Boolean, Integer, String, and indicated the package format. A package contains a number of booleans, integers, and strings and a message is encoded therein. Typically an integer indicates the kind of message (syntax check, simulation, performance analysis, etc), i.e., the module to call, and a second integer indicate the command, or more or less which function to call. The remaining integers and all booleans and strings are function-specific parameters.
Now, the first idea is to add a new module for calls. On top of that, we add an integer describing which extension to call. We’ll probably control these centrally to avoid conflicts if users start writing extensions. Next is an integer indicating the function (though the exact use is really up to the specific extension). Remaining values are parameters. This makes it possible to communicate using the exact same protocol always used inside the CPN Tools GUI (and Access/CPN) and furthermore to later internalize functionality in the simulator instead of extensions. This is also fairly fast and simple and allows the simulator to blindly send a package onto the extension handler without any processing. This is great for executing web-services from CPN Tools. We can implement the GUI and simply syntax such modules by making a standard call. During interactive simulation, we simply indicate we execute a web-service. All of this is just passed onto the simulator extension which does the heavy lifting (calling a TLA with SOAP and WSDL and XML) and responds back.
Other extensions require tying a bit closer into the simulator. Some extensions may wish to know about the net. Others may wish to be consulted during simulation. For example, a Declare extension would need to be consulted when computing enabling; both when simulation happens interactively and when it happens automatically. A web-service extension would probably like to know when tokens are produced around the external transition. A message-sequence chart generator might like to know about all transitions and maybe even places in addition to notifications when transitions are executed. An extension may sign up for such services high-level(ish) services or simply request a copy of a specific kind of packages. A Declare extension would probably also like to influence the syntax check (to mark a transition as incorrect in case surrounding constraints are). The exact nature of such services are still a bit fuzzy and will probably be defined as I go along. Requests are welcome! Extensions subscribe to such services in a hand-shake between the simulator and the extension manager taking place before any syntax check and simulation.
While making communication for each step is no problem during interactive simulation, it is not a good idea for a fast simulation or during state-space analysis. It is therefore possible for the simulator to request extensions to send it code for automatic simulation. The reason for not doing this during interactive simulation is that it is slower so the overhead does not matter, and generating the simulator code may be expensive, and interactive simulation is often used intermixed with model alterations, so generating the full simulation alteration code for each step is too expensive.
Extensions may wish to prompt the user for options. They can publish such options during the handshake as well. CPN Tools can then display them in the GUI and notify the simulator and by extension the extensions when they are changed. Options can be boolean, integers, or strings. This is, e.g., useful for a web-service extension, which wish to export a web-service and prompts for a port and namespace. Additionally, extensions can be enabled/disabled manually in the GUI (for efficiency).
Extensions may wish to expose functions to models. This can be done using the JavaExecute functionality of Access/CPN and should be allowed for extensions. They simply publish functions communicating directly with the extension to do complex computations, or other things most easily done in Java.
The calls supported by CPN Tools should be extended with drawing primitives. They should be able to create a page (and get a handle) and on that page create, style, move, and delete auxiliary graphics. All call-backs supported by CPN Tools should be possible to do from extensions. That way, it is possible to create a library for drawing message sequence charts by making all the logic in Java, doing the drawing in CPN Tools using these new calls and even exposing primitives to allow models to call the high-level functions from CPN Tools. That way an extension is written in Java but to the user seems like it is written in SML.
The calls to the external code should be as asynchronous as possible. For example, any extension subscribing to syntax checks are notified as soon as the syntax check message is received by the simulator and reply is only checked when the syntax check inside CPN Tools has completed. This reduces the latency problems incurred by TCP and allows modern computers to make use of SMP, making CPN Tools automatically make use of (at least) 3 CPUs (GUI, SML simulator, Java simulator extension).
Extensions will be defined in a functional manner. This removes the need for extensions to take care of concurrency themselves and imposes this on the extension manager. Extensions will not be allowed to store global data, but are instead given an environment they can use for data storage. We may either create a generic environment (read hash map) or allow extensions to make one themselves (which will be more efficient as we don’t have to go thru a hash map for every data lookup). An extension may be invoked for different models (there is no need to start the extension manager more than once for each model).
These are the needs I’ve identified until now: special RPC namespace for extensions, extension subscription to high-level events or low-level BIS packages, simulator request for simulation alteration code from extensions, option handling, code injection from extensions, and forwarding of GUI calls from extensions (and adding new calls for drawing). The design principles build upon asynchronous calls and functional extensions with an explicit environment. Let me know if I’ve forgotten anything.
