Documentation:Tutorial:RemoteAccess - MdsWiki
Navigation
Personal tools

From MdsWiki

Jump to: navigation, search

Remote Data Access in MDSplus

In MDSplus remote data access is natively available. Data communication is based on TCP/IP and uses a mdsip protocol. A mdsip server has to be running at the data server side (i.e. where the pulse files are hosted).

The data client then communicates with the data server to retrieve or store data. There are two possible configurations for the data client:

  • Distributed Client
  • Thin Client

In the distributed client configuration most data access operations are carried out at the client site, except for the disk I/O operations which are demanded to the server. In this configuration no changes are required in the code, being data redirection carried out only by re-defining the environment variable <experiment name>_path. More precisely, the variable can contain a search list, separated by semicolons (not colons as for Linux search path). Every element of the search list is composed of three elements in the form:

<mdsip server IP address>:<port>::<directory>

The first element is the IP address of the machine hosting the pulse files and running the mdsip server. The second part is optional and specifies the port number of the mdsip server. The third part is the directory containing the pulse files. More flexibility can be achieved using filemasks, which allow defining directory definitions depending on experiment name and the shot number. In the directory definition you can add special replacement symbols to be replaced by either digits of the shot number or the experiment name. The available replacement symbols are:

  • ~a gets replaced by the one's digit of the shot number
  • ~b gets replaced by the ten's digit
  • ~c gets replaced by the hundred's digit
  • ~d gets replaced by the thousand's digit
  • ...
  • ~j gets replaced by the 10th digit of the shot number
  • ~t gets replaced by the experiment name

For example. assuming that the experiment name is my_tree, directory definition:

my_tree=/trees/~j~i/~h~g/~f~e/~d~c/~t

would cause the following locations of the trees:

  • shots 1-99 in /trees/00/00/00/00/mytree
  • shots 100-199 in /trees/00/00/00/01/mytree
  • shots 10000-10099 in /trees/00/00/01/00/mytree
  • shot 5987654321 in /trees/59/87/65/43/mytree

As an example,the following definition of cmod_path for experiment cmod:

cmod_path=alcdata::/cmod/trees/test/~t/;alcdata::/cmod/trees/new/~t/

instructs MDSplus to search first machine alcdata: if a mdsip server is running and listening to port 8000 (the default port of mdsip), and the pulse file exists in directory /cmod/trees/test/cmod, then it is accessed, otherwise the second element in the list is accessed, that directory /cmod/trees/new/cmod on the same machine.

In the Thin client configuration, almost all operation are carried out at the data server site. This configuration can be seen as a Remote Expression Evaluation call. The client prepares a TDI expression and then sends it to the server, which actually evaluates it and sends back the evaluation results. Recall that everything in MDSplus is an expression, including the specification of a data item in the pulse file. For write operations, a write expression whose argument is the data to be written is executed by the data server.

Unlike the Distributed Client configuration, the Thin client configuration requires a different Application Programming Interface. Class Connection is the communication broker. Its constructor requires the specification of the IP address of the machine hosting the mdsip server and optionally of its port (separated by a colon). The Connection methods are listed below

  • openTree(tree, shot) which remotely opens the specified tree
  • closeTree(tree, shot) which remotely closes the specified tree
  • closeAllTrees() which closes all open trees
  • setDefault(char *path) which sets the default position in the currently open tree
  • Data get(expression, arguments) which performs remote expression evaluation. The expression is passed as string argument, but may contain optional arguments, which are then passed as an array of Data objects.
  • put(path, expression, arguments) which writes in the specified item (identified by argument path) the passed expression, containing also optional arguments. Arguments are specified in an expression as dollars. For example the following code snippet

C++:

Connection *conn = new Connection("my_server_machine");
Data *args[] = new Data *[2];
args[0] = new Int32(2);
args[1] = new Int32(10);
Data *result = conn->get("(MY_NODE:DATA * $1)*$2", args, 2);

Java:

Connection conn = new Connection("my_server_machine");
Data result = conn.get("(MY_NODE:DATA * $1)*$2", new Data[]{new Int32(2), new Int32(10)});

Python:

>>> conn = Connection('my_server_machine')
>>> result = conn.get("(MY_NODE:DATA * $1)*$2" , Int32(2), Int32(10))

will return the result of the evaluation (on data server my_server_machine, port 8000) of the passed expression, that is the content of data item MY_NODE:DATA added by 2 and multiplied by 10.

The following example will instead write the data specified by my_data Data object in node specified by tag name \MY_NODE:

C++

Connection *conn = new Connection("my_server_machine");
Data *my_data;
... //Fill my_data instance
Data *args[] = new Data[1]
args[0] = my_data;
conn->put("\\MY_NODE", "$", args, 1);

Java

Connection conn = new Connection("my_server_machine");
Data my_data;
... //Fill my_data instance
conn.put("\\MY_NODE, "$", new Data[]{my_data});

Python:

>>> conn = Connection('my_server_machine')
... get my_data Data instance
>>>conn.put("\\MY_NODE", "$", my_data)

Observe the double backslash required to handle tag names in all languages (all of them use the C convention). (NOTE: In Python, one can optionally use "raw strings", e.g. conn.put(r"\MY_NODE", "$", my_data), where the first quote in the string is prefaced by the letter 'r', then only one backslash is required.)

Using the Distributed Client configuration has the advantage of not requiring any change in the code, but has a penalty over slow connections. In fact the number of network transactions carried out is much higher than in Thin Client configuration. The reason is that several Disk I/O operations are required per data access, and every one requires a network transaction. On the other side, in Thin client configuration, only one transaction is carried out per remote expression evaluation. As a rule of thumb, Distributed Client configuration is the preferred configuration for data access in the same LAN, while Thin client may provide much faster data access over WAN.

Important Note: when using the Thin Client configuration the internal data structure is possibly lost when retrieving signals which are not simple scalars or arrays. In fact the evaluation of a remote expression produces either a scalar or (possibly multidimensional) array which is returned over the network. For example, when accessing a signal using local data access (or the Distributed Client remote access) you may get an instance of class Signal, which brings information on both the signal samples and the time. Accessing the same signal via a Connection object will only return the array of signal samples, and not the associated times. Times can be than retrieved via a new call to the get() method of the Connection object, specifying in the expression that the dimension of the signal has to be evaluated (i.e. using the TDI function DIM_OF).

It is then possible to assemble the returned data objects to create, for example a new Signal instance.

Configuring the mdsip server as a xinetd service on Linux

It is possible to run a mdsip server as xinetd service following the steps described below. This configuration has two advanatges:

  • mdsip must not be manually launched anymore.
  • Every mdsip connection is served by a separate process. If for any reason a mdsip process crashes, it does not affect the other active connections.

The configuration of the mdsip xinetd is performed by a script in $MDSPLUS_DIR/rpm

$MDSPLUS_DIR/rpm/post_install_script

It is in general necessary that a set of environment variables is defined when the mdsip process is executed. For example the <tree name>_path variables must be defined for the trees accessed by the mdsip server. The mdsip xinetd service will search for such definitions in the following files:

  • $MDSPLUS_DIR/local/envsyms
  • /etc/mdsplus.conf
  • $HOME/.mdsplus

Environment variable definition in any of the above files is in the form of a line containing the pair

<variable name>  <variable definition>

If the value has to be appended to the head or to the tail of the current variable definition followed by a colon (e.g. for PATH variables), specifies this in the following way

<variable name>  <variable definition> >:

to append to the tail

<variable name>  <variable definition> <:

to insert to head. If you are specifying a path variable for pulse file directories, which are separated by semicolons, use either >; or <;.

When installed, the mdsip service is not yet active. To make it active you need to edit the following file

/etc/xinetd.d/mdsip

and replace line

disable = yes

with line

disable = no

In case you want to use a port different from 8000 for the mdsip process, you need to edit file

/etc/services

and change the port number in the line specifying the mdsip service. Finally, you must restart xinetd with the command (under root privilege)

/etc/init.d/xinetd restart