Architecture

Overall architecture

The SIGN is an application framework that gives the developer a set of support classes for developing applications for Image-Guided Therapy. The framework provides means for event handling (from GUI, trackers, imagers, and possibly others), thread handling (for background information acquisition such as real-time imaging), and render synchronization. In addition, the framework includes 2D and 3D viewers, mechanisms for fiducial input, and several useful extension classes to VTK. A simple application using The SIGN looks similar to any simple VTK application, except that a context is created, and this context is used to parse a MRML file and an OpenTracker configuration file. Nodes in these two graph descriptions of respectively image data (volumes and models) and tracking setup are matched (see illustration), and connected to viewers. Control of both data-elements and viewes can thus be given to trackers (or other devices) or vica versa. The program subsequently enters into a loop where the further propagation of the application will be determined by events and the defined data and device configuration. The process of any application is determined by a 'process' class. This implements a workflow description that closely matches a story board description of the procedure. A derived process class will be implemented for every application where the rules for changing the states of the software are encoded.

Context

The context is the main application manager. It is responsible for several tasks in the framework:

• Initialization and parsing of two sub-contexts: the MRML context and the tracker context.
• Building the internal VTK pipeline from the MRML tree.
• The main event loop.
• Cleanup.

Initialization includes parsing the XML files and creating the objects that will handle the different contexts. The MrmlContext manages the MRML tree. The SignOTModule (otm) is an OpenTracker module that also handles the OpenTracker context. When a MRML node is created, a helper node is created - this will perform three actions on each node:

• Read data. Read any data from file such as images or models. Does nothing for nodes where this is not applicable.
• Setup. Inserts the node into the VTK pipeline. This can include making an actor and adding that to the appropriate viewer.
• Connect tracker / transform. Sets the user transform of the node to be the transform in the MRML tree or the transform from a tracker with matching name.

The event loop is implemented in the function handle_idle(). This function makes sure that pending GUI events are processed, calls pushEvents() and pullEvents() on the OpenTracker context to process all tracker events, and requests rendering of all viewers that need to be updated. To avoid unnecessary calls to Render(), there is a map called renderWindowsUpdate where other parts of the software can set a flag to indicate if a RenderWindow needs to be redrawn. This map is checked once for every iteration of the main loop. Once the main loop is started, every action will be triggered by a tracker or a GUI event.

Viewers

The SIGN has two types of viewers: 2D viewers and 3D viewers. These viewers are stored in a vector in the context. The Sign3DViewer class has functionality for annotation in 3D and stores a collection of volumes that are displayed in the viewer. Since time-varying volumes are supported, there is also functionality for cycling the volumes (to set the current timepoint). The volumes are managed by the vtkSignVolume class which holds the pipeline for the image data to be displayed. This class also handles the type of volume rendering being used. The Sign2DViewer class has functionality for annotation in 2D, coordinate conversion between viewer and world coordinate systems, and functions for handling callbacks related to common operations. What data to display in what viewers is determined by the configuration XML file. The MRML file also contains viewer related information, like the reformat orientation for 2D viewers.

Workflow elements

Workflow elements encapsulate functionality that is relevant during specific steps of the clinical workflow. Such steps can be tool calibration, fiducial identifictation, or registration. A workflow element can depend on the completion of several other workflow elements. When all preliminary elements are completed, a workflow element will be activated. This is illustrated in the figure. In the illustration, workflow elements 1 and 2 are completed. Workflow elements 4, 5 and 6 are waiting for element 3 to complete, and element 7 is waiting for elements 4,5 and 6 to complete. This can be a powerful way to encode the steps of an application, and to make sure that a step is not executed unless the necessary previous steps are completed. The flow is typically encoded in the LinkWorkFlowElements() function in a class derived from SignProcess. Here is an example:

In this example, a linear workflow starting with identification of fiducial markers in a 2D viewer, followed by tool calibration, followed by acquiring the position of the fiducials with a tracker, followed by registration of the fiducial set is defined. All elements are deactivated when they are finished, except for the identify fiducials element which is deactivated after the acquisition of fiducials is completed. The basic functionality is implemented in the SignElement superclass. Each derived class will implement their own ActivateElement() and DeActivateElement() functions.

Fiducials

Fiducial markers are used to perform registration between coordinate systems based on matching pairs of corresponding fiducial markers. They could possibly have other uses as well, since they are principally just points. Every MrmlNode that inherits from MrmlVirtualObjectNode has a fiducial collection. Fiducials can be added to or removed from this collection. Fiducials are added by using one of two workflow elements; SignElementIdentifyFiducials (inserting fiducials using a Sign2DViewer) or SignElementAcquireFiducials (inserting fiducials using a tracked instrument). In the first case, the fiducials will be added to the fiducial collection belonging to the volume displayed in the 2D viewer. In the latter case, fiducials will be added to the fiducial collection belonging to the model that the tracker is connected to (as defined in the OpenTracker configuration file). Fiducials will show up as crosshairs in the 2D viewers, small spheres in the 3D viewer, and as labels in a list in the left panel. The fiducials will be colored to easily illustrate the correspondence.