Data Acquisition using the
LabOne User and Programming Interfaces


Tino Wagner, Zurich Instruments

Scanning Probe Microscopy User Meeting, ETH Zürich, April 2019

Overview

  • The LabOne Python API
  • The Node Concept
  • High-speed data streaming
  • Introducing the Data Acquisition Module
  • Data Acquisition in the User Interface
    • LabOne UI Demo for image acquisition
  • Automating acquisition with the Python Programming Interface
    • Continuous acquistion of data
    • Acquistion of 2D data for imaging

Nomenclature:

  • UI - User Interface
  • API - Application Programming Interface
  • DAQ - Data Acquisition

The LabOne Python API

  • Install with pip: pip install --upgrade zhinst
  • The zhinst module contains the actual API (zhinst.ziPython), utility functions (zhinst.utils), and examples (zhinst.examples).

The zhinst module:

In [2]:
import zhinst
help(zhinst)
Help on package zhinst:

NAME
    zhinst - Zurich Instruments LabOne Python API

DESCRIPTION
    Contains the API driver, utility functions and examples for Zurich Instruments
    devices.

PACKAGE CONTENTS
    examples (package)
    utils
    ziPython

DATA
    __all__ = ['ziPython', 'utils']

FILE
    /Users/tinow/.pyenv/versions/3.7.1/Python.framework/Versions/3.7/lib/python3.7/site-packages/zhinst/__init__.py


Running API examples (I)

  • Examples can be found under zhinst.examples
In [3]:
import zhinst.examples
help(zhinst.examples)
Help on package zhinst.examples in zhinst:

NAME
    zhinst.examples - Zurich Instruments LabOne Python API Examples.

PACKAGE CONTENTS
    common (package)
    deprecated (package)
    hdawg (package)
    hf2 (package)
    uhf (package)
    uhfqa (package)

DATA
    __all__ = ['common', 'deprecated', 'hdawg', 'hf2', 'uhf', 'uhfqa']

FILE
    /Users/tinow/.pyenv/versions/3.7.1/Python.framework/Versions/3.7/lib/python3.7/site-packages/zhinst/examples/__init__.py


Running API examples (II)

  • Each example is a separate Python module with a run_example method
In [4]:
help(zhinst.examples.common.example_connect)
Help on module zhinst.examples.common.example_connect in zhinst.examples.common:

NAME
    zhinst.examples.common.example_connect - Zurich Instruments LabOne Python API Example

DESCRIPTION
    Demonstrate how to connect to a Zurich Instruments device via the Data Server
    program.

FUNCTIONS
    run_example(device_id)
        Run the example: Create an API session by connecting to a Zurich Instruments
        device via the Data Server, ensure the demodulators are enabled and obtain a
        single demodulator sample via getSample(). Calculate the sample's RMS
        amplitude and add it as a field to the "sample" dictionary.
        
        Note:
        
          This is intended to be a simple example demonstrating how to connect to a
          Zurich Instruments device from ziPython. In most cases, data acquisition
          should use either ziDAQServer's poll() method or an instance of the
          ziDAQRecorder class, not the getSample() method.
        
        Requirements:
        
          Hardware configuration: Connect signal output 1 to signal input 1 with a
          BNC cable.
        
        Arguments:
        
          device_id (str): The ID of the device to run the example with. For
            example, `dev2006` or `uhf-dev2006`.
        
        Returns:
        
          sample (dict): The acquired demodulator sample dictionary.
        
        Raises:
        
          RuntimeError: If the device is not "discoverable" from the API.
        
        See the "LabOne Programing Manual" for further help, available:
          - On Windows via the Start-Menu:
            Programs -> Zurich Instruments -> Documentation
          - On Linux in the LabOne .tar.gz archive in the "Documentation"
            sub-folder.

DATA
    print_function = _Feature((2, 6, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0)...

FILE
    /Users/tinow/.pyenv/versions/3.7.1/Python.framework/Versions/3.7/lib/python3.7/site-packages/zhinst/examples/common/example_connect.py


Connecting to the Device

Let's see which device data is available from the device from the API.

To get started, let's import the Zurich Instruments LabOne Python Module and create a connection to the Data Server running on this PC:

In [5]:
import zhinst.ziPython

server_host = 'localhost'
server_port = 8004
api_level = 6
daq = zhinst.ziPython.ziDAQServer(server_host, server_port, api_level)
print('Running LabOne API ', daq.version(), ', ', daq.revision(), sep='')
Running LabOne API 19.05, 60891

Then we can connect our device to the Data Server:

In [6]:
dev = 'dev21'
daq.connectDevice(dev, '1gbe')

Device Settings and Data: The Node Tree (1)

The Data Server organises the data and settings in a hierarchical structure called the "node-tree".

Here are the top-level branches for our device:

In [7]:
from zhinst.ziPython import ziListEnum

flags_listNodes = ziListEnum.absolute
node_list = daq.listNodes(f'/{dev}/', flags_listNodes)
print('\n'.join(sorted(node_list)[0:10])); print("...")
/DEV21/AUCARTS
/DEV21/AUPOLARS
/DEV21/AUXINS
/DEV21/AUXOUTS
/DEV21/AWGS
/DEV21/BOXCARS
/DEV21/CLOCKBASE
/DEV21/CNTS
/DEV21/DEMODS
/DEV21/DIOS
...

Device Settings and Data: The Node Tree (2)

The data and settings are leaves in the node tree:

In [8]:
flags_listNodes = ziListEnum.absolute
demod_node_leaves = daq.listNodes(f'/{dev}/demods/0/', flags_listNodes)
list(sorted(demod_node_leaves))
Out[8]:
['/DEV21/DEMODS/0/ADCSELECT',
 '/DEV21/DEMODS/0/BYPASS',
 '/DEV21/DEMODS/0/ENABLE',
 '/DEV21/DEMODS/0/FREQ',
 '/DEV21/DEMODS/0/HARMONIC',
 '/DEV21/DEMODS/0/ORDER',
 '/DEV21/DEMODS/0/OSCSELECT',
 '/DEV21/DEMODS/0/PHASEADJUST',
 '/DEV21/DEMODS/0/PHASESHIFT',
 '/DEV21/DEMODS/0/RATE',
 '/DEV21/DEMODS/0/SAMPLE',
 '/DEV21/DEMODS/0/SINC',
 '/DEV21/DEMODS/0/TIMECONSTANT',
 '/DEV21/DEMODS/0/TRIGGER']

Device Settings and Data: The Node Tree (3)

  • Nodes are documented in the User Manual, for example:

nodedoc

  • Using help command provided by API:
In [9]:
daq.help(f'/{dev}/demods/0/sample')
/DEV21/DEMODS/0/SAMPLE
Contains streamed demodulator samples with sample interval defined by the
demodulator data rate.
Properties: Read, Stream
Type: ZIDemodSample
Unit: Dependent

  • Interactively from the LabOne UI:

    • Tooltips: tooltip

    • Command log: command-log

Reading Values from Nodes

  • Reading single values with getDouble, getInt, getComplex, getString, getByte
    • Blocking session until Data Server responds with value
    • Reads value cached by Data Server; kept up-to-date by the device pushing up data
In [12]:
daq.getDouble(f'/{dev}/oscs/0/freq')
Out[12]:
999999.9999969589
  • Additional commands for special node types: getSample, getAuxInSample
In [13]:
daq.getSample(f'/{dev}/demods/0/sample')
Out[13]:
{'timestamp': array([2545514643640], dtype=uint64),
 'x': array([0.0676666]),
 'y': array([-0.01383496]),
 'frequency': array([999999.99999696]),
 'phase': array([2.1640634]),
 'dio': array([0], dtype=uint32),
 'trigger': array([33556032], dtype=uint32),
 'auxin0': array([0.00030518]),
 'auxin1': array([0.])}
  • Non-blocking variant: getAsEvent → result is obtained in next poll() command
    • Request to fetch new data goes down to the device
In [14]:
path = f'/{dev}/oscs/0/freq'
daq.getAsEvent(path)
# Poll until we find path in result data
value = None
while True:
    poll_data = daq.poll(0.1, 100, 0, True)
    if path in poll_data:
        value = poll_data[path]
        break
value
Out[14]:
{'timestamp': array([2547405841312], dtype=uint64),
 'value': array([999999.99999696])}
  • Higher-level get supports wildcards and can return multiple values
    • Returns cached values if available, otherwise performs read from device
In [15]:
daq.get(f'/{dev}/oscs/*/freq', True)
Out[15]:
{'/dev21/oscs/0/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([999999.99999696])},
 '/dev21/oscs/1/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/2/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/3/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/4/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/5/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/6/freq': {'timestamp': array([2548970068128], dtype=uint64),
  'value': array([9999999.99999517])},
 '/dev21/oscs/7/freq': {'timestamp': array([2548970654552], dtype=uint64),
  'value': array([9999999.99999517])}}

Setting Values of Nodes

  • setDouble, setInt, setComplex, setString, setByte
    • Blocking session until the Data Server returns value
  • Synchronous set commands: syncSetDouble, syncSetInt, syncSetString
    • Blocking session until the device returns value
    • Ensures that data server is in sync with the value on the device
  • Setting multiple values: set([path1, path2, ...])
  • Sync command: sync()
    • Ensures that pending data is processed and commands are completed
    • Flushes all buffers: poll() data then contains only data obtained after call to sync()
    • Expensive (~100 ms); often it is sufficient to perform last set command as syncSet...

High Speed Data: ZI's "Streaming" Concept

  • Nodes which provide data at high rates are "streaming nodes". Here are all the streaming nodes available on our device:
In [16]:
flags_listNodes = ziListEnum.recursive | ziListEnum.absolute | ziListEnum.streamingonly
node_list = daq.listNodes(f'/{dev}/', flags_listNodes)
node_list = [node for node in node_list if '/0/' in node]
print('\n'.join(sorted(node_list)[0:]));
/DEV21/AUCARTS/0/SAMPLE
/DEV21/AUPOLARS/0/SAMPLE
/DEV21/AUXINS/0/SAMPLE
/DEV21/BOXCARS/0/SAMPLE
/DEV21/CNTS/0/SAMPLE
/DEV21/DEMODS/0/SAMPLE
/DEV21/DIOS/0/INPUT
/DEV21/INPUTPWAS/0/WAVE
/DEV21/OUTPUTPWAS/0/WAVE
/DEV21/PIDS/0/STREAM/ERROR
/DEV21/PIDS/0/STREAM/SHIFT
/DEV21/PIDS/0/STREAM/VALUE
/DEV21/SCOPES/0/STREAM/SAMPLE
/DEV21/SCOPES/0/WAVE

Obtaining Streaming Data Using the subscribe and poll Commands (1)

In [17]:
import zhinst.utils

# Load a device settings file.
zhinst.utils.load_settings(daq, dev, "./notebook_resources/beat_demod0.xml")

# Subscribe to one or more signals.
daq.subscribe(f'/{dev}/demods/0/sample')
daq.subscribe(f'/{dev}/demods/1/sample')
daq.sync()  # Flush the Data Server's buffers.

# Wait and accumulate data
time.sleep(2.0)

# All the data is returned since the sync() command was executed:
data = daq.poll(0.1, 100, 0, True)
daq.unsubscribe('*')

Obtaining Streaming Data Using the subscribe and poll Commands (2)

  • The data is a dictionary whose entries correspond to the subscribed paths.
  • The entries are also dictionaries which contain the fields of a demodulator sample.
  • Each field of the demodulator sample is a numpy array.
In [18]:
data.keys()
Out[18]:
dict_keys(['/dev21/demods/0/sample'])
In [19]:
demod_path = f'/{dev}/demods/0/sample'
data[demod_path].keys()
Out[19]:
dict_keys(['timestamp', 'x', 'y', 'frequency', 'phase', 'dio', 'trigger', 'auxin0', 'auxin1', 'time'])
In [20]:
ts = data[demod_path]['timestamp']
x = data[demod_path]['x']
y = data[demod_path]['y']
R = np.abs(x + 1j*y)

Obtaining Streaming Data Using the subscribe and poll Commands (3)

In [21]:
fig, ax = plt.subplots(figsize=(7,4))
clockbase = daq.getInt(f'/{dev}/clockbase')
plt.plot((ts - ts[0]) / clockbase, R)
plt.xlabel('Time (s)'); plt.ylabel(r'Amplitude ($V_{rms}$)');

The Data returned by the subscribe and poll Method

  • These nodes typically provide data at a tact that is ultimately defined by the instrument's ADCs.

title

  • The sample points from these nodes might not have common timestamps → interpolation required.
  • Poll does not guarentee that the data is aligned → for continuous acquisition, book-keeping is required.
  • Previously, subscribe() and poll() commands were the only way to obtain continuous data from the device, now they are only recommended when extremely high performance is required.

LabOne API Modules

  • Modules: high-level software components, which are common to all APIs
  • Each module has its own API connection (session) to the data server
  • Modules use an asynchronous (non-blocking) interface; they run in an own thread on the PC
  • Examples: Sweeper, DAQ, Scope, AWG, etc.

The Data Acquisition (DAQ) Module

The Data Acquistion (DAQ) Module:

  • Offers functionality analogous to that of lab oscilloscopes.
  • Provides continuous or triggered data acquisition.
  • Aligns data.
  • Reduces the amount of data saved to disk.
  • Simultaneously acquires time and frequency domain data.

The Data Acquisition Module was introduced in LabOne Release 17.12. It combines and improves the Software Trigger Module and the Spectrum Modules.

UI Demo

Let's have a look at the DAQ Module in the LabOne UI …

  • Configure instance of a DAQ module from LabOne user interface:
    • Settings → Trigger Level = 10 mV
    • Settings → Hysteresis = 5 mV
    • Grid → Mode = Linear
    • Grid → Duration = 2 s
  • Observe API command log to see how the module was configured.

title

Continuous Recording with the DAQ Module (1)

Initialise and configure the DAQ Module.

In [23]:
h = daq.dataAcquisitionModule()
sampling_rate = 1000  # Number of points/second
total_duration = 10   # Time in seconds
burst_duration = 0.2  # Time in seconds for each data burst/segment.
num_cols = int(np.ceil(sampling_rate * burst_duration))
num_bursts = int(np.ceil(total_duration / burst_duration))

h.set('dataAcquisitionModule/device', dev)
h.set('dataAcquisitionModule/type', 0)  # Specify continuous acquisition.
h.set('dataAcquisitionModule/count', num_bursts)
h.set('dataAcquisitionModule/duration', burst_duration)
h.set('dataAcquisitionModule/grid/mode', 2) # Linearly interpolate the data.
h.set('dataAcquisitionModule/grid/cols', num_cols)

We subscribe in the module to the data we're interested in.

In [24]:
node_path = f'/{dev}/demods/0/sample.r.avg'
h.subscribe(node_path) 

Continuous Recording with the DAQ Module (3)

We first define a simple helper routine for plotting. Here we use the module read() function to read data out from the module. It returns data in a similar format to that of poll().

In [25]:
def read_data_update_plot(timestamp0):
    prog = 100 * h.progress()[0]
    ax.set_title("Progress of data acquisition: {:.2f}%.".format(prog))
    data = h.read(True)
    if node_path in data: 
        if np.isnan(timestamp0):
            # Remember timestamp from start of acquisition.
            timestamp0 = data[node_path][0]['timestamp'][0, 0]
        for d in data[node_path]:
            t = (d['timestamp'][0, :] - timestamp0) / clockbase
            plt.plot(t, d['value'][0, :])
        fig.canvas.draw()
    return timestamp0

Continuous Recording with the DAQ Module (4)

In [27]:
h.execute()  # Start the acquisition.
ts0 = np.nan; plt.show()
while not h.finished(): 
    time.sleep(0.1)
    ts0 = read_data_update_plot(ts0)
read_data_update_plot(ts0);

Image Recording with the DAQ Module (1)

First we'll setup a UHFLI with AWG option:

  • Copy the AWG wave file static/notebook_resources/zi_logo_grey_8bit.csv to the Zurich Instruments LabOne/WebServer/awg/waves/ user subdirectory.
  • AWG Tab: Upload the sequencer program static/notebook_resources/image_generator_daqmodule.seqc
  • Config Tab: Load the device XML settings file static/notebook_resources/image_generator_daqmodule.xml

Image Recording with the DAQ Module (2)

In [28]:
h = daq.dataAcquisitionModule()
delay = -0.0001
duration = 0.0015

h.set('dataAcquisitionModule/device', dev)
h.set('dataAcquisitionModule/type', 6)        # Specify AWG Trigger type.
h.set('dataAcquisitionModule/awgcontrol', 1)  # Use AWG control.
h.set('dataAcquisitionModule/count', 1)       # 1 image.
h.set('dataAcquisitionModule/edge', 1)        # Positive edge.
h.set('dataAcquisitionModule/eventcount/mode', 1)
h.set('dataAcquisitionModule/delay', delay)
h.set('dataAcquisitionModule/grid/mode', 2)
h.set('dataAcquisitionModule/grid/cols', 256)
h.set('dataAcquisitionModule/grid/rows', 256)
h.set('dataAcquisitionModule/grid/repetitions', 1)
h.set('dataAcquisitionModule/duration', duration)
h.set('dataAcquisitionModule/holdoff/time', 0)
h.set('dataAcquisitionModule/holdoff/count', 0)

triggernode = f'/{dev}/demods/0/sample.TrigAWGTrig1'
h.set('dataAcquisitionModule/triggernode', triggernode)
h.subscribe(f'/{dev}/demods/0/sample.R.avg')

Image Recording with the DAQ Module (3)

In [31]:
R = data[f'/{dev}/demods/0/sample.r.avg'][0]['value']
im = plt.imshow(R, aspect='equal', origin='lower')
fig.colorbar(im); plt.show()

Bonus: Setting up DAQ module for acquisition with line clock

  • SPM controllers can often provide a line clock output, which has a high level for a short time (typ. ~10 µs) during each scan line.
  • Config Tab: Load the device XML settings file static/notebook_resources/settings_uhf_dig_trigger_daq.xml
  • Load AWG program
const N = 200000;
const N_pulse = 2;

wave signal = sine(N, 0, 10);
wave pulse = join(ones(N_pulse), zeros(N - N_pulse));

while (true) {
  playWave(signal, pulse);
}
  • Connect Signal Output 1 to Signal Input 1; and Signal Output 2 to Ref/Trig 1.
  • The program outputs a sine on output 1, and a ~10 us pulse on output 2.
  • The Pulse Counter is used to increment the a counter value on each pulse.
  • DAQ trigger is set to Pulse Counter value increment.

Screenshot

title