[ANSS-netops] k2ew produces bogus data from uncompressed Serial Data Streams

[Dietz, Lynn]

*k2ew Warning:*
NCSN staff have discovered that data values produced by k2ew from
uncompressed-format Altus Serial Data Streams ('sds for 0') are bogus. To
get valid data from k2ew, one must run the Altus Instrument with compressed
Serial Data Stream format ('sds for 1').

*Quick Summary:*
k2ew with 'sds for 1' GOOD : Earthworm tracebuf2 and Altus .evt data values
match
k2ew with 'sds for 0' BAD! : Earthworm tracebuf2 and Altus .evt data values
*do not* match

*Background:*
For Altus SDS streaming data, there are two format choices on the K2:
  sds for 0  (uncompressed, 24bit samples)
  sds for 1  (compressed data samples)
k2ew was written to unpack either format, in different sections of code in
k2pm_proc_streamdata() in k2pktman.c.

Since the dawn of k2ew acquisition, NCSN has always used compressed SDS
('sds for 1') to telemeter continuous data from our Altus Instruments. In a
recent transition of some K2s from triggered .evt dialup telemetry to
SDS-continuous-to-Networm, a handful of K2s were configured to use
uncompressed SDS ('sds for 0').

Then during a M4.0 earthquake, a number of our close-in stations recorded
full-scale +/- 2**23 spikes in the data output by k2ew. In addition, the
spiky channels had extremely large bias, also near 2**23. The stations
recording those spikes were those configured with 'uncompressed' data
streams. Toggling the SDS format on the K2 to 'compressed' shifted the bias
of continuous k2ew data to near-zero.

Our K2s are also configured to trigger and record an .evt file to disk. We
compared data values from .evt files with k2ew-output, and discovered that
the data values matched only if the K2 was set to compressed format 'sds
for 1'.  With uncompressed streams, both the data values and difference
between data values did *not* match (ie, the problem is more than just a DC
offset).

For a quick visual, I've attached two screenshots showing waveform data
from two stations. The first image is .evt data. The second is k2ew-output
from the same time interval, top 3 channels from a compressed data stream,
bottom 3 from an uncompressed data stream.
  1739evt-1877evt-20150721.jpg  (wiggles from .evt files)
  1739sds1-1877sds0-20150721.jpg  (wiggles from k2ew)

Bottom line: there appears to be a bug in the part of k2ew's
k2pm_proc_streamdata() that re-packages the 24bit uncompressed SDS data, or
in how the K2 packages that uncompressed data stream.

The same behavior observed with data produced by:
k2ew from EW v7.7.1 on Windows
k2ew from EW v7.7   on Linux (Ubuntu)

Given that there is a functional work-around, and that K2s are no longer
supported, we are not highly motivated to track down the bug.

*Recommendation:*
When using k2ew to acquire data, always set the Altus Instrument to
*compressed* Serial Data Streams ('sds for 1')!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://geohazards.usgs.gov/pipermail/anss-netops/attachments/20150921/3762884b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1739evt-1877evt-20150721.JPG
Type: image/jpeg
Size: 327253 bytes
Desc: not available
URL: <http://geohazards.usgs.gov/pipermail/anss-netops/attachments/20150921/3762884b/attachment-0002.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1739sds1-1877sds0-20150721.JPG
Type: image/jpeg
Size: 337845 bytes
Desc: not available
URL: <http://geohazards.usgs.gov/pipermail/anss-netops/attachments/20150921/3762884b/attachment-0003.jpe>