proc getdata accesses or defines data for plotting. proc getdata must be invoked before doing any plotting of data. The data which are read become the "current data set" that subsequent plotting procs will access, and replace any data that were read in (or generated) previously.

Features

• Data may be specified literally within the script file, or it may be located in an external file or be the result of an externally executed command, or perhaps even an SQL command.
• Conditional selection of data records (select).
• Data filtering (filter) for purposes such as combining or rearranging fields, performing calculations, truncating outliers, etc.
• Field names may be assigned explicitly (fieldnames or fieldnamerows), or via a data file header line (fieldnameheader). These names can then be used anywhere a data field reference is expected.
• Data sets with variable number of fields per row can be accomodated by setting a maximum nfields value.
• Data may be located at the end of the script file, to get it out of the way (#intrailer).
  
  
  

Examples

Acceptable data formats

Prerequisites

Troubleshooting

Variables that are set

Handling error conditions gracefully

 #proc getdata
    ...
 
 #proc endproc
 #if @NRECORDS = 0
   #proc annotate
   location: 3 3
   text: No data found.
 
   #exit
 #endif

Setting variables from within a data file

#set varname = value

varname will be set to value. value may contain embedded spaces or any other character. value may not be a $function(), nor may it contain any other @variables. None of the other script operators, such as #if, are supported in this context.

Mandatory attributes

Attributes

data multiline-text

Literal specification of plotting data. Terminates at first blank (empty) line. Data may include a field name header. Example:

data:   "Case 1"   0   4   4.1   
        "Case 2"   1   5   4.4  
        "Case 3"   2   2   4.0 
        "Case 4"   3   9   4.8

You can also capture data from a shell command this way (this allows you to use multi-line shell commands) for example:

   #proc getdata
     data:
      #shell
         shell command
      #endshell

The #sql command can be used similarly, in builds that support it.

file filename

Shell-expandable name of a file containing plotting data. This name will be used along with cat(1) in a shell command, thus exported shell variables and metacharacters may be part of the name. A dash (-) may be used if data is to be read from the standard input, (or the standardinput attribute may be used). Example: filename: myfile.dat

pathname filename

Name of a file containing plotting data. The file will be opened directly. Shell variables and metacharacters may not be used.

command shell command line

An external shell command that will produce plot data on its standard output.
Example: command: cat mydat | uniq -c

sql sql command

In ploticus builds that support it, invoke the given SQL command, and capture the results directly. Result column names become the ploticus field names (to handle join result column names that contain embedded periods (.), you can use #control dot_in_varnames ). delim, fieldnames (etc.), select and filter have no meaning in this mode.

delim spacequoted | whitespace | comma | tab

The type of delimiting method to be used when parsing the data. spacequoted is the default (space is equivalent to spacequoted). See dataformat for details.
Example: delim: comma

fieldnames namelist

If specified, the names given in namelist may be used in any plotting proc to identify data fields. namelist is a list delimited by spaces and/or commas. Names may include any alphanumeric characters with a maximum length of 38, and are case-insensitive. Field names may not contain embedded spaces or commas.
Example: fieldnames: date group n

fieldnamerows multiline text

Same as fieldnames (see above), except that field names are given one per line. Must be terminated by a blank line. Example:

fieldnamerows:
  id
  type
  age
  sex
  

fieldnameheader yes | no

Allows field names to be embedded in the input data. If yes, the first non-comment line in the data is expected to contain field names. This line is not considered part of the data. The field name header should be delimited like the rest of the data. Field names may not contain embedded white space, commas, or quote characters.

pf_fieldnames namelist

Post-filter fieldnames, for use with filter when field names are being used, and the filter result has a different logical record format than the input.
Example: pf_fieldnames: date z sum1 sum2

nfields n

If specified, this sets the expected number of fields per record. If a data row has more than the expected number of fields, extra fields are silently ignored. If a data row has less than the expected number of fields, blank fields are silently added. This is applied after any filter processing.

standardinput yes | no

If yes, data is read from the standard input.

#intrailer

Indicates that a data attribute will be given in a #proc trailer, at the end of the script file. See EXAMPLES, below.

commentchar string

A character or group of characters that is used to signify a comment in the data file. Commented lines will be skipped. Default is //. Example: commentchar: #

showresults yes | no

If yes, the results, after any selecting and/or filtering, are written to the diagnostic file, which may be useful in debugging, etc.

rotate yes | no

Allows data to be given all in one row, even when plotting proc expects one record per instance (which most do). Only applicable if your data set has one row. (There still must be a blank line following the data attribute.) To rotate more than one row, use proc processdata.
Example: see bars1

select conditional expression

This allows data records to be selected for inclusion based upon a selection expression. Incoming data fields are referenced by number, using a double at-sign (@@) prefix. Hint: use the showresults attribute when debugging.
Example: select: @@3 = g
This would select all data records having 3rd field equal to g.

filter multiline text

An embedded script which allows flexible processing to be applied to incoming data records one at a time. Typical uses are for concatenating or splitting fields, doing on-the-fly date conversions, or generating derived fields such as the sum of several fields or the difference between two fields.

The embedded script will be applied once to every incoming data record. The script should produce some "output"; generally the last statement is a ##print. The output must use the same delimitation method as the input. The output may have a different logical record format than the input. If you are using field names, the pf_fieldnames attribute (see above) may be used to name the filter result fields when result record format differs from that of the input.

The script uses the same syntax as the greater ploticus script, except that:

• directives must begin with two pound signs (##) instead of one
• local variables begin with two at signs (@@) instead of one
• fields on the incoming data record are accessed like this: @@1 for the first field, @@2 for the second, etc. If you are using field names, these may be used as well, eg: @@score.
• the only directives that may be used are ##set, ##if, ##elseif, ##else, ##print, ##call, and ##exit

Other things worth noting:

• the filter script is terminated using a blank line.
• use the showresults attribute when debugging.
• if filter is used along with select, the select is applied first.
• filter cannot be used on data specified using the data attribute.
• filter can only access fields from one data record at a time.
• variables from the ploticus script may be referenced within the filter script (use one at-sign @). Evaluation occurs before the filter script executes.
• ploticus script #if/#else statements (single pound sign) may be used to selectively execute portions of the filter script. Interpretation occurs before the filter script executes.
• Example: This filters out data records having field 2 or field 3 equal to M. It then calulates the difference in days between two dates and puts this difference in the variable DIFF. Finally it "prints" incoming field 1 along with DIFF. Thus the result of this #proc getdata will have be data records having two fields.

   filter:
        ##if @@2 = M || @@3 = M
          ##exit
        ##endif
        ##set DIFF = $daysdiff(@@3,@@2)
        ##print @@1 @@DIFF
  

• There are several more filter examples in the FAQ
  
  
  

More examples

Data specification may be located at the end of the script file by using #intrailer and #proc trailer. This may be useful in "getting the data out of the way", or with automated building of script files. Here is how this is done:

 #proc getdata
 #intrailer

other #procs, etc.

 #proc trailer
 Data:	0.3 0.5 2.3
	3.5 9.4 1.4
	..etc..
end of file