A quick tour over MDSplus concepts
In this section we'll discover the basic concepts of MDSplus through some working examples. We shall skip here most details, covered by the following sections of the tutorial, in order to concentrate on the main concepts.
To run the examples we need first a running installation of MDSplus. The reference system for the presented examples is Linux, although MDSplus can run also on Windows and MacOS. The preparatory steps for this overview are:
- Installation of MDSplus. MDSplus can be installed on several platform and for Linux platforms you can also download the sources and then build the system. Refer to Download Page to install MDSplus on your system.
- Installation of the running examples. The examples can be downloaded as a tar file.
You can download the tar file using your browser and expand it or use a linux shell command such as:
wget -O - https://github.com/MDSplus/MDSplusTutorial/archive/master.tar.gz | tar zxf -
Once the examples have been downloaded and extracted, cd to the MDSplusTutorial-master directory:
and we are ready to start. There are two examples, activated by commands start_tutorial_1 and start_tutorial_2, respectively. The first tutorial will introduce some basic concepts of MDSplus and is intended for every MDSplus user. The second tutorial covers more advanced features of MDSplus and may be of interest to system developers willing to try MDSplus to set-up a complete data acquisition system. This section covers the first tutorial, while the second one will be presented in a later section, before entering in detail into more advanced MDSplus concepts.
Tutorial 1: MDSPlus Experiment Model and pulse file and their interfaces
Let's start with the first one and make sure first that:
- TCP port 8001 is not currently used in the system
- we are running an up-to-date version of MDSplus
Provided the two facts above, type
If TCP port 8001 is already used on your system, locate a free TCP port and call the tutorial script with the following command
./start_tutorial_1 <New TCP Port>
You will see the activation several graphical interfaces and a X Terminal. Let's concentrate on the following window showing a hierarchical structure.
That tree structure displays the hierarchical structure of an Experiment Model, that is the database containing all the data dealt with by the system. Data are stored in database in a hierarchical organization, reflecting the normal way information is arranged in scientific applications. A MDSplus database takes two names, that refer to two different stages of the experimental sequence: the Experiment Model and the Pulse file. Consider the execution of a set of measurements that we shall call "experiment". Typically such execution produces data that need to be collected to be analyzed. Experimental data alone however may be not enough, because it is also necessary to know the configuration of all the involved devices in order to give a meaning top the acquired values. A subset of experimental data will be therefore be represented by configuration information and will be defined before the experiment. This is the typical content of the MDSplus Experiment Model, that is a template of the full database containing configuration data. Observe that typically what happens during an experiment will depend on the actual configuration, i.e. on the data defined in the experiment model. The experiment model is then filled with experimental data, eventually producing a database containing the complete description of one experiment, that is, the Pulse File. In a typical sequence of repeated experiments, the current Experiment Model is copied into a new (incomplete) pulse file just before the experiment sequence. The jTraverser application provides a graphical browser to experiment models, showing the underlying structure and the contained data.
A name and a number are associated with Pulse Files. The number is normally called shot number and refers to the instance number of the pulse file. As an experiment is typically repeated several times, a sequence of pulse files with different shot numbers will be produced as well. The conventional shot number for the Experiment Model is -1.
A fairly complete set of data types are supported in MDSplus, reflecting the variety of data types dealt with in experiments. Let's have a look at the simplest types and expand PARAMETERS node.
The PARAMETER node has two children nodes: GAIN and OFFSET containing scalar data. You can look at their content via the popup menu as shown in the figure.
More complex data types can be defined, too. The list of data types supported in MDSplus is very long. More in general, any data item can be represented by an expression that is a combination of operators, numbers and references to other data items in the pulse file. Take for example the content of node TIMING.CLOCK shown below
This data item describes a timebase and when evaluated (expressions are always evaluated on the fly by MDSplus when read) it returns the array of times corresponding to the specified clock. Its definition is of the form
describing a clock starting at StartTime, ending at EndTime and with period Period. Here StartTime, EndTime and Period are not directly represented by numbers, but they refer to other data items in the TIMING subtrees, i.e. TRIGGER, DURATION and FREQUENCY. This particular expression defines a clock which starts at the time specified in TRIGGER item, lasts the time specified in DURATION item and whose period is the reciprocal of the frequency value specified in FREQUENCY item.
Let's now expand subtree RAW: a sequence of nodes named SIG_1 to SIG_16 is shown, with a different associated icon. That icon means that those nodes are designed to containing a Signal data type. Without entering into details for now, the Signal data types defines an array of values, that is, the samples of the acquired signal, and an array of times, that is, the time of every sample. Users are normally not concerned with the details of the Signal representation because signals object are managed by MDSplus providing the samples and the times when requested.
Try now to look at the content of SIG_1 node with the "Display Data" popup option. There is no data! This is however not a surprise if you recall the meaning of the Experiment Model (this jTraverser window is open at shot -1). Signals are typically acquired during the experiment and therefore they will be eventually available in the Pulse File. In our example they have eben put in the RAW subtree, suggesting that these signals will contain raw samples. Normally the actual values of a signal are the result of some sort of processing from the raw samples. For example, ADC data typically require calibration, that is, adding and offset and multiplying them for a gain. In MDSplus this sort of processing becomes natural thanks to expressions. Have a look at the signals defined in the CALIBRATED subtree, which are intended, in this example, to represent the calibrated version of the the signals in RAW subtree. In most data acquisition systems, calibrated data are stored as an array which si different from the raw data. In MDSplus you can avoid this, just defining the expression which converts raw ata into calibrated ones. Have a look at the expression defined fir CALIBRATED:SIG_1
The shown expression specifies that the raw signal in node RAW:SIG_1 is multiplied by 10./4096 (this is the conversion for a +- 5V, 12 bits ADC), that the offset value specified in node PARAMETERS:OFFSET is subtracted and the result multiplied by the value stored in node PARAMETERS:GAIN. Observe that what is stored in node CALIBRATED:SIG_1 is just the expression definition, not the actual value array. When CALIBRATED:SIG_1 is evaluated, e.g. when it is read by a user program, MDSplus performs an evaluation on the fly, returning the actual array of converted values for that signal. Observe also that, while RAW:SIG_1 node is empty, CALIBRATED:SIG_1 is not empty and contains already the expression definition in the experiment model. If evaluated before data are written in RAW:SIG_1 MDSplus will raise an error stating that the expression cannot be evaluated. After raw data have been written, the evaluation of CALIBRATED:SIG_1 will yield the correct result.
The few examples presented so far have illustrated the flexibility on data representation offered by MDSplus. However data dealt with in an experiment can be even richer. Consider for example the required configuration of the hardware devices involved in data acquisition: configuration parameters will be stored in the experiment model in order to properly configure the experiment. Since a set of data items will specify the configuration of a given device, and several instances of the same device type will be likely defined in the experiment, it is useful to provide a sort of "encapsulation" for sets of related data items (such as the configuration parameters of a given device instance). MDSplus allows this encapsulation process by allowing users define a set of device classes, which are then recognized by the system and handled accordingly. Let's expand the DEVICES subtree in our experiment model. Nodes ADC_1 to ADC_4 are shown with a new icon and clicking on any of these nodes we shall discover that it is not a data item, but the root node of a subtree, as shown below.
If we expand any ADC node in DEVICES we shall discover that all them have the same subtree structure. ADC_1 to ADC_4 are indeed instances of the same device type. If an instance of a different device type were present here, its subtree will likely have a different structure. We may browse the content of devices exactly at the same way as browsing the rest of the experiment model, but MDSplus can recognize devices and present a more specialized interface, activated via the Setup Device popup menu item. Below is the interface for the device class used in this tutorial, basically exposing the content of the underlying subtree, but with a more user friendly interface.
A set of MDSplus devices is already available in the MDSPlus distribution, but new devices can be defined by users thus providing a personalisation of MDSplus for the specific system. Section DevelopingNewDevices explains the steps required for developing new devices in MDSplus.
Accessing Acquired Waveforms
We have seen so far how the jTraverser tool allows browsing the experiment model. If jTraverser is open with a shot number different from -1 , a pulse file is open instead, possibly with mode data items filled. In the tutorial there are two shots filled with data, let's open shot 1 (recall, the name of the pulse files is "tutorial") and try to look at data in RAW:SIG_1. The window below is shown, rather disappointing, isn't it?
The signal is indeed there, but it would be too large to be displayed by jTraverser, which displays the content of the nodes in a textual form. However, looking at a loooong list of number is perhaps not the intended way of looking at signals, which are better displayed by an oscilloscope-like interface. This is exactly what the jScope tool does. Let's have a look first at the jScope Window with title jScope/Tutorial_1.jscp.
Here several waveforms are displayed in four panels. If you look at the panel configuration, which is activated via the popup "Setup Data Source..." menu item you will discover that the panels on the left contain the waveforms RAW:SIG_1 to RAW:SIG_16 and the panels on the right contain waveforms CALIBRATED:SIG_1 to CALIBRATED:SIG_16. Observe that the only difference between raw and calibrated is just the Y scale, not a surprise if you recall how calibrated signals are derived from raw ones. Observe also that a waveform is displayed for calibrated signals, even if the corresponding data items do not contain arrays, just expressions. That expressions are however evaluated on the fly when accessing these nodes thus producing the displayed waveforms. If you change the bottom shot number to -1, jScope will fail showing waveforms because raw data are not present in the experiment model, and the evaluation of calibrated fails as it refers to missing raw data.
You can now interact with waveforms and discover the many features offered by jScope. The radio buttons on the bottom left part of jScope define the mode of operation. In point mode a cursor can be dragged over the waveform, getting information about the actual waveform values; in Zoom mode it is possible to enlarge parts of the waveforms; in Pan mode waveforms can be dragged and in Copy mode, it is possible to copy and paste panels using MB1 and MB2 mouse buttons, respectively. The combined usage of the different modes allows discovering interesting features in the acquired signals. A typical use case is to zoom a waveform in order to enlarge an interesting part of a given waveform describing some interesting waveform, then moving to Point mode and selecting popup menu item "All Same X Scale". so that all the other displayed waveforms are re-scaled accordingly in order to discover possible relationships among acquired signals.
In order to understand how waveforms are defined in jScope panels right click the mouse over a jScope panel and select "Setup Data Source..." menu item. The Dialog below is shown:
Here you will see a list of node references in the pulse file, but in the Y and X field of the Setup Data Dialog it is possible to write any expression, which is evaluated by jScope in order to display the resulting array. Two fields (X and Y) define what is displayed in the jScope panel. Here only the Y field is filled because the referenced data item contains a Signal, i.e. a MDSplus data type which brings also information on the sample times (the X axis).
While it is possible to type in Setup Data Dialog the path names of the data items to be displayed in jScope, there is a faster way to define waveforms in jScope. Try selecting a signal in jTraverser (in CALIBRATED or RAW subtrees) and drag it into a jScope panel. The new signal will replace the previous one. If you keep the Ctrl key pressed, the dragged signal will be added to the those already defined in the panel.
Once jScope has been properly configured, it can be saved in a configuration file, so that you can start a new jScope application passing the configuration file in the command line or retrieving it using "Customize->Use saved setting from ..." menu option.
jScope is able to display different kinds of signal. If you look at the jScope window with title "- Scope - jScope/Tutorial3.jscp" you will see other waveforms, with a difference in the X labels showing now an absolute time indication. The displayed signals have been stored in this case with a different notation of time, i.e. specifying an absolute time, that is, date and time. A typical use case for this kind of waveforms is represented by trend signals such as temperature or vacuum level which are continuously acquired along a long period of time.
jScope can also display sequence of frames, as shown in the jScope window labeled "- Scope - jScope/Tutorial2.jscp". You can analyze the image on the left in several ways. For the single image you can zoom into it, or also look at the X and Y profiles by selecting "Show profile Dialog" popup menu item and moving the cursor in Point mode. You can also look the sequence of images using the "Start Play" popup menu item. The most useful ability of jScope in handling sequence of frames is by correlating them to other signals with the same time base. The displayed example shows the inside of a Tokamak device during a plasma discharge, and the corresponding plasma current. When moving the cursor over the plasma current in Point mode, the corresponding frame is displayed.
Let's consider now in more detail how jScope takes data. MDSplus provides a data access Application Programming Interface (API) in a variety of languages, including Java, so in principle jScope could directly read data stored in pulse files. MDSplus provides also a remote data access layer so that data from pulse files on separate machines over Internet can be read as well. MDSplus provides a variety of possible configuration for remote data access (seeRemote Data Access in MDSplus
When remote data access is performed in MDSplus, a server application must be running on the target machine hosting data, which exports MDSplus data over mdsip, that is the MDSplus data access protocol, normally built over TCP/IP. The jScope applications in this tutorial are configured for remote data access, and the mdsip data server has been activated and is displayed in the XTerm activated by the start script. If you have already looked at some waveform in the displayed jScope windows, the mdsip XTerm will look like:
The displayed messages describe the connections performed by the jScope applications. Since different MDSplus data providers may be available on the network, jScope can be connected to a variety of different data sources (see jScope Tutorial). The currently selected data provider is shown in the bottom right part of jScope window and in this tutorial it defines a local mdsip connection over port 8001, served by the process running in the XTerm application.
It is possible to switch from one data provider in jScope via the "Network->Servers" menu item. Of the displayed providers in the tutorial jScope applications only two will likely work on your system: "local mdsip" (the currently running one) and "Local". When the latter data provider is selected, jScope access data directly from disk. Try to switch from one data server to another (you need to hit the Apply button again). Maybe you will not perceive any difference, but what is happening behind the curtains is completely different: direct disk read with Local Data Server and network communication with local mdsip Data Server.
For the curious ones
At the end of this first tutorial you should have a first idea about the data management capabilities of MDSplus, even if many details have been skipped and will he presented in depth in the following sections. There are however a couple of questions that may have arisen and worth to be answered now.
- Where are the pulse file and the experiment model shown in the tutorial? MDSplus stores experiment models into triplets of files. Pulse files are then generated by cloning the experiment model into a new triplet of files with a different name which are them filled by the data acquisition system. The naming convention for these files is <experiment_name>_[model | shot number].[tree|characteristics|datafile]. For example, the tutorial experiment model is stored in files tutorial_model.tree, tutorial_model.characteristics, tutorial_model.datafile. We shall see in the following sections that the structure of experiment models and pulse files can be more complicated, and given subtrees can be stored in different triplets of files, possibly on remote machines.
- Where does the data displayed by jScope applications come from? The displayed waveforms represent real signals acquired in a nuclear fusion experiment. Therefore they have been copied from the pulse files for that experiments into the tutorial pulse files. This has been carried out by a python program (python is one of the supported languages of MDSplus), which is included in the tutorial distribution in python/fillTutorial.py. You may have a look at it just to get an impression on how easy is managing data using the python MDSplus API (just 74 lines of code). Unfortunately you won't be able to run it since the pulse files from which real data are taken are not included in the tutorial distribution.