SVE - initial comments on API

Description
We are still tweaking the SVE API but I thought it might be useful to briefly describe the capabilities of our shared variable system. This should provide some input for the design of the OCS and other SISPI applications.

The core of the shared variable system is the SVE (Shared Variable Engine) which is the central clearing house for published information and subscriptions. This is completely transparent to the application programmer. A user will interact with the SVE through an interface library. Here are the basic calls:

a) Initialization
mySVE = SVE(...) # create a (local) instance of the SVE object that will perform the data transfer to and from the SVE

mySV = SharedVariable(name, group, some options) # creates a shared variable object

b) a SV be used to either publish or to subscribe. The user chooses by calling either
mySV.publish()
or
mySV.subscribe()

Subscribe has a mode argument. A standard subscription just gets the latest value (from the publisher). If a new value arrives it overwrites the current value whether the subscriber has read it or not. A guaranteed subscription differs in that updates from the publisher are queued up until the subscriber has read each value. In this mode no update gets lost (-> guaranteed delivery)

c) A publisher updates a shared variable using the write method:
mySV.write(value)

[an SV used for a subscription cannot be written to]

d) A subscriber gets the value of a Shared Variable using the read method:
new_value = mySV.read()

[reading from an SV used for publishing returns the current (local) value]

read() supports two modes: blocking and non-blocking

non-blocking is the default - the read() call just returns with the current value. Multiple read calls between updates return the same value.
In the blocking mode read() blocks until a new value is available. A timeout is possible. Each value can be read only once.

An SV object contains additional information that can be accessed and used by the application programmer when needed. This is still in flux and I don't want to discuss this in detail but here are some of the thinks we are considering:
Timestamp of last update (there could be two, one for the actual "measurement - think ICS) and one for the mySV.write() call)
ConnectionAlive (and similar flags to describe the status of the connections to the SVE)
Publisher (An SV (identified by group and variable name) can only be published by one publisher. However, multiple subscriptions to the same
SV are possible)
UpdateAvailable (a flag to test if an update is available)



I hope this brief introductions gives you an idea of what can and cannot be done with our shared variables. Let me know if you have questions or suggestions.
Question
Remember that the database will need some information that tells it which table this variable has to end up in. Maybe this could be one of your options. Then for variables that need to be stored the application can include that information. Also for things like the telemetry data it would be nice to have the concept of a giving the SVE some kind of array of values. It will be very painful to update the telemetry info if you have to set 200 some variables one call at a time. I could imaging having something like

mySVArray = SharedVariableArray(vector of names,options)
mySVArray.write(vector of values)

Is something like this possible? This would be very useful for groups of variables that need to be stored in the database in the same table at the same time.

Answer
The shared variable I described is an object that can represent different values. I should have included in my note that right now an SV can be any native python type (int, strings, float, lists (arrays) etc) - but probably not more complicated things such as classes [we don't use pickle to transfer the data but a simple string representation]

One of our design principles is that the publisher should not have any knowledge about the subscribers and in general what happens to the SV once it's written. So a publisher and hence the SV object doesn't know anything about database tables.
But that's not a problem because in our design the database will subscribe to the variables it wants [ the appropriate db server gets this from a configuration file which if we want could be even kept in the db itself]. That file cannot only specify what to get but also what to do with it - ie the table name.