Tutorial on working with xwork files in Matlab
Minor formatting edits 08-13-2019 by Maxwell Gagnon

================================================================================
Reading xwork file into Matlab.
a)	Behavior data: readcxdata(); readfile_CG.m
a.	The MEX function readcxdata()is used to read xwork data files recorded by cntrlx2k into Matlab.  It was written by Scott A. Ruffner in Lisberger lab.
b.	readfile_CG.m is a script to read and convert voltage data into velocity/position/raster and stored them in a structure array.  The converting factors are based on the calibrations on the Monkey rig.
b)	Raw spike data: convertspike2.m
a.	Convert raw spike files into *.wav format.
  Transfer raw spike files onto the server; Connect to the server using Putty
  (http://www.chiark.greenend.org.uk/~sgtatham/putty/); Go to the directory where spike files are stored; Use command: spike-convert filename.
b.	<Matlab> y = wavread(FILE NAME)  voltage of recording
c.	Read .wav files into Spike2 software.
  Open Spike2; import .wav file into Spike2 software.
d.	Export sorted spike data into Matlab.
  After sorting spikes into channel #, click on channel # and export into .txt file—Check Wavemaker/event box, and leave the rest; <Matlab> converspike(FN) will give you sorted spikes time.



================================================================================
================================================================================
From http://keck.ucsf.edu/~sruffner/userguide/index.html
Minor formatting edits 08-13-2019 by Maxwell Gagnon

================================================================================
%%% Usage information -- readcxdata()

  results = readcxdata( filename [, verbose, nchans] );
 	  filename:	A Matlab string containing the pathname for the Maestro (or Cntrlx)
              data file to be processed. This can be an absolute path or a path
              relative to the current working directory.
 	  verbose:	If nonzero, detailed progress messages are written to stdout. This
              is typically used only for debugging purposes.
 	  nchans:	  For headerless Continuous-mode data files (generated by Cntrlx versions
              dated before 29 Jan 2002), the function must know how many analog data
              channels were recorded to properly parse the compressed analog data
              stored in the file. If an incorrect value is specified, then the
              decompressed analog data will be invalid.

================================================================================
%%% The table below describes the contents of the Matlab data structure that is
    returned by readcxdata(). If it encounters any serious problems while reading
    and parsing the specified data file (e.g., the file pathname is incorrect, or
    the file format is unrecognized), the function prints an error message to
    stdout and aborts. In this case, all or some of the fields in this output
    structure will be empty matrices.

trialname:
  The assigned name of the relevant trial as it appears in Maestro. Will be truncated if the original name was >40 characters. Ignore this field in Continuous-mode data files.
key:
  Contents of the data file's header record. This field is itself a data structure whose members have the same name as their counterparts in the Maestro header structure CXFILEHDR. The name field, however, is omitted since its contents are already reflected in the trialname field above. Go here for a member-by-member description of the header record.
data:
  An N-by-M Matlab matrix containing the analog data sampled from N selected channels over a recorded timeline that is M "ticks" long. One "tick" corresponds to a single scan interval on the data acquisition device. In the past and for the foreseeable future, the scan interval is "hard-coded" at 1ms in Trial mode and 2ms in Continuous mode. However, one can always verify its duration by checking the header field key.nScanIntvUS. Each row of the array corresponds to a channel trace. The rows are ordered exactly as specified in the key.chlist field, which is a vector identifying the channels that were sampled and the order of sampling (but: for headerless Continuous-mode data files, the user supplies only the number of channels sampled as an argument to readcxdata(); the list of channel IDs is not available). The data are stored as raw, 12-bit digitized samples (range -2048..2047). Typically, these may be converted to millivolts (multiply by 4.8828125), position in degrees (multiply by 0.025), or velocity in deg/second (multiply by 0.09189) -- the latter two scale factors relying upon proper calibration of Maestro's laboratory apparatus.
  [As of 10Jan2008] If there are any per-channel saccade cuts (see description of the cut field) defined on any of the three eye-velocity traces HEVEL, VEVEL or HDVEL, then the effect of those cuts is reflected in the corresponding trace data stored in this matrix. Readcxdata() essentially replaces the analog data between the saccade cut endpoints with a line connecting them -- just as would appear in XWork or JMWork. Most users prefer to mark saccades with mark1/mark2 pairs rather than with the per-channel saccade cuts; in this case, eye velocity traces are left unchanged.
spikes:
  Chronological sequence of arrival times of events detected on digital input channel DI<0>. Typically reserved to timestamp spikes detected by a window discriminator in the lab rig -- hence the field's name. All times are in milliseconds relative to the start of recording and have 10µs accuracy. If no such events were detected, this is an empty array.
events:
  Chronological sequence of arrival times of events detected on the digital input channel DI<1>. All times are in milliseconds relative to the start of recording and have 10µs accuracy. May be empty.
other:
  Chronological sequence of events detected on any of the other digital input channels DI<15..2>, stored as (channel#, arrival time)-pairs in an M-by-2 matrix, where M is the total # of events detected. Simultaneous events on two different channels are parsed as distinct (ch#, arrival time)-pairs. All times are in milliseconds relative to the start of recording and have 10µs accuracy. May be empty.
mark1:
  A 1-by-N double array containing a set of markers added to the data file in XWork and encoded by the ACTION_SETMARK1 action. Each item in the array corresponds to a marked timepoint in the recorded timeline, in milliseconds. Will be empty if no such marks have been made. NOTE: MWork and JMWork users may choose to mark saccades in the recorded timeline with mark1/mark2 pairs. In this usage, the marked saccade is not associated with a particular eye-velocity data channel. In downstream analysis, data within such marked saccades is usually ignored (ie, "NaN").
mark2:
  A 1-by-N double array containing a second set of markers added to the data file in XWork and encoded by the ACTION_SETMARK2 action. Each item in the array corresponds to a marked timepoint in the recorded timeline, in milliseconds. Will be empty if no such marks have been made.
cut:
  An N-by-3 matrix containing N "per-channel saccade cuts" executed in XWork or JMWork and encoded by the ACTION_CUTIT action. The first two columns give the start and end points of the cut (as times in milliseconds), while the third column identifies the analog channel # of the affected trace. Will be empty if no saccade cut actions have been stored in the data file.
marks:
  An N-by-2 matrix containing N "mark segments" annotated in XWork and encoded by the ACTION_MARK action. The two columns give the start and end points of each marked segment (as times in milliseconds). Will be empty if no mark segments have been stored in the data file.
marked:
  This serves as a flag indicating that the data file has been annotated by at least one "mark segment". If so, this field will have the value 1; otherwise, the field is an empty matrix. [This somewhat awkward usage is a carryover from the predecessor of readcxdata().]
targets:
  Trajectory data for targets that participated in a trial (Trial-mode data files only), as calculated from the trial codes stored in the data file. This is a Matlab structure containing the following fields:
  hpos:
    Horizontal position trajectory in degrees for all participating trial targets. This is a N-by-M matrix, where N is the number of targets and M is the length of the recorded timeline. Again, the duration of one "tick" is 1ms in Trial mode, although an analysis program could also check the header field key.nScanIntvUS. Row n gives the calculated horizontal position trace of target n in the trial's target list; the order is the same as the target IDs listed in the targets.targnums field and the target definitions found in field tgtdefns. If velocity stabilization ("opening the pursuit loop") occurs during the trial, the affected target's computed position trajectory will reflect this. See note 3.
  vpos:
    Similarly for vertical target position in degrees.
  hvel:
    Similarly for horizontal target velocity in degrees/second. (If velocity stabilization occurred, see note 3.)
  vvel:
    Similarly for vertical target velocity in degrees/second.
  patvelH:
    Similarly for horizontal target pattern velocity in degrees/second, but pattern velocity is NOT adjusted for the effects of velocity stabilization.
  patvelV:
    Similarly for vertical target pattern velocity in degrees/second.
  targnums:
    Old Cntrlx-style IDs of the targets that participated in the trial, in the order that they appear in the trial target list (data file version < 2). These IDs are not particularly informative, except for the predefined targets CHAIR (ID=0), FIBER1 (ID=2), FIBER2 (ID=3), REDLED1 (ID=1), and REDLED2 (ID=4). For XY scope and framebuffer video targets, these merely reflect the target's position in the loaded target list. While this field is prepared regardless of the data file version, it is recommended that analysis programs working with Maestro-generated data files (version >= 2) use the tgtdefns field to obtain more complete target information.
  nTrialLen:
    Total length of trial (including any initial segment(s) that were not recorded) in # of trial "ticks".
  tRecordOn:
    The trial time at which recording began. If 0, then the entire trial was recorded.
  NOTES:
    1) readcxdata() makes a concerted effort to accurately calculate the "expected" trajectories of the targets based upon the trial codes stored in the file. However, some trial operations -- such as the saccade-triggered functions -- preclude such calculations because they introduce changes in the trial at runtime. Warning messages are printed to stdout in such cases, IF the verbose flag is set.. The function does not bother to account for the effects of pixelization on the actual position and velocity of XY scope or framebuffer video targets, nor does it account for the effects of interleaving XY scope targets.
    2) It DOES, however, attempt to account for the frame update intervals of the video displays; these are both longer than a single "tick" (1ms) in Trial mode. It is assumed that the two different video displays are never used simultaneously during a trial -- not an unreasonable premise.
    3) How readcxdata() takes into account the effects of velocity stabilization. First, the function requires that eye position PE and velocity VE be recorded during the trial, since the eye trajectory is obviously required in order to calculate the effects of velocity stabilization on the expected target trajectory. If the stabilized target is Fiber1 or Fiber2, the adjustments in the planned target trajectory (PT, VT) are made on a tick-by-tick basis -- PT' = PT + PE - PE(0), VT' = VT + VE -- since these targets are in fact updated on every tick of a trial. For the video targets, adjustments can only be made once each frame period, and the adjusted trajectories reflect this. In particular, the effective target velocity is NOT computed using VE. Instead, we essentially differentiate the compensated position trajectory PT' . Since the smallest possible eye position change recordable in Maestro is 0.025deg, the minimum step size in velocity for a 2ms XYScope update frame is 0.025deg/2ms = 12.5 deg/sec -- not very accurate! So expect the fields hvel and vvel to look quite noisy during a period of velocity stabilization!
    4) For data files generated by Maestro v1.3.2 or later, readcxdata() can reproduce the effects of any perturbations on a trial target's trajectory.
    5) For data files generated by Maestro v2.1.0 or later, readcxdata() accounts for the effects of the pattern acceleration H,V components that were introduced with v2.1.0.

tgtdefns:
  A 1xN array defining parameters of all targets participating in the trial or stimulus run. Prepared for Maestro data files with version >= 2; if key.version < 2 or no stimuli were presented during the recording, this field is an empty matrix. Each element in the array is a Matlab data structure containing the following fields.
  category:
    The target's major type. This will be one of the defined constants in cxobj_ifc.h, such as CX_FIBER1, CX_CHAIR, CX_XYTARG, CX_FBTARG, CX_RMVTARG, and so on.
  name:
    The target's human-readable name as it appears in Maestro.
  params:
    The target's defining parameters. For an XY scope target (category = CX_XYTARG), this is another Matlab structure with the same format as the XYPARMS structure declared in cxobj_ifc.h. For an old VSG framebuffer video target, it will have the same format as the FBPARMS structure, and for an RMVideo target it will follow the RMVTGTDEF structure. For all other targets, which have no defining parameters, it is an empty matrix.
  dwState:
    Target state at the start of recording in Continuous mode. Its value will be some combination of the flags CXFTF_ISACVTGT, CXFTF_TGTON, etc. defined in cxfilefmt.h.
  hPos:
    Target horizontal position at the start of recording in Continuous mode, in degrees.
  vPos:
    Target vertical position at the start of recording in Continuous mode, in degrees.
  NOTES:
    The order of the N target definitions in the array reflects the order of the targets as they are defined in a Maestro trial or stimulus run. In a Continuous-mode data file, any "active" targets defined at the start of recording are listed first, followed by any XY scope targets participating in an XYseq channel (if any) in the stimulus run definition.

stimulusrun:
  Under development. This field is not yet available.
spikewave	:
  A Matlab vector containing the high-resolution spike waveform recorded on AI channel 15. The sample interval is currently "hard-wired" at 40µs (25KHz), but analysis programs should make a habit of checking the sample interval reported in key.nSpikeSampIntvUS. The data are stored in their raw, 12-bit digitized form [-2048..2047]. Will be an empty vector if spike waveform recording was not enabled, or if the data file was generated by Cntrlx (version < 2).
sortedSpikes:
  XWork-created "sorted spike trains". XWork's spike-sorting facility can cull up to 13 different spike trains from the high-resolution spike waveform recorded by Maestro or SpikesPC. The spike times representing a sorted spike train -- ideally corresponding to a distinct neural "unit" -- are appended to the relevant data file in records with an ID tag between 8 and 20. This 1x13 cell array exposes the artificial spike trains for analysis. Cell n in the array contains the spike train sorted in accordance with XWork spike-sorting channel# n+2. If no spike-sorting data was found for a given channel, the corresponding cell will hold an empty matrix; otherwise, it will hold an array of spike arrival times, formatted exactly like the spikes field described earlier.
  [As of January 2008] JMWork defines two new action codes that let the user manually add or remove individual spikes in any sorted-spike train (doing it this way makes it an easy task to "revise" these edits in JMWork at a later time, without losing the original unedited spike train). The sortedSpikes field here will include the effects of any such "manual spike edits".
tagSections:
  Description of any "tagged sections" defined on a Maestro trial presented when the data file was recorded. A "tagged section" is a named contiguous subrange of segments in the trial. It was introduced in Maestro v1.3.0 (data file version >= 4). The field is a 1xN array of structures, where N is the # of tagged sections dened on the trial. Each element in the array is a structure containing the fields described below.
  tag:
    A string holding the section's tag name. Each tagged section within a Maestro trial has a unique name tag.
  firstSeg:
    Zero-based index of first segment in the segment range spanned by the section.
  lastSeg:
    Zero-based index of last segment in the segment range spanned by the section. This will be greater than or equals to firstSeg.
  tStart:
    The start time of this tagged section relative to when recording began. Units = milliseconds. If readcxdata() is unable to compute this value, it will be set to -1.
  tLen:
    Length of this tagged section. Units = milliseconds. If readcxdata() is unable to compute this value, it will be set to -1.