Contents

• Introduction
• Getting Started
• Deceit Command Line Arguments
• The deceit interpreter
• Datasets
• Tables
• Columns
• Arrays
• Attributes
• History
• Comment
• Execute DAL Block
• Execute Perl BLock
• Execute Post Block
• Expressions
• Built-in Functions
• Grammer

Introduction

With deceit it is possible to create datasets without the need to do any programming.  Within the constraints of the DAL data model, datasets of any complexity can be created.  The datasets are described with a high level specification which may be thought of as a template. All the features of the DAL Data Model are supported. The template approach has the additional advantage of documenting the dataset structure.

Data contained in ASCII  files is used to populate the arrays and columns of a dataset.

Deceit is an interpreter, so there is no need for the compile-link build procedure often associated with program development.

It is assumed throughout this document that the reader has a basic understanding of the DAL data model and the FITS file format.
 

Getting started

If the file HelloWorld contains the following simple deceit template

                    deceit -infile=HelloWorld

will give rise to the dataset HelloWorld being created. It will contain no arrays, tables or attributes. The first three comment lines, which are skiped by the deceit interpreter, are not mandatory. This is essentially the simplest case of a deceit dataset specification, and gives rise to the simplest DAL dataset. The DAL supports both FITS and DAL file formats, but it is anticipated that deceit will be principally used to build FITS files.

Any number of dataset specifications may appear in a deceit template.

NB Deceit is a dataset creation tool and cannot be used to modify exisiting files.
 

Deceit Command Line Arguments

The deceit interpreter is invoked with the command

        deceit [options] [--]

where

    Options:
      --help  | -h
      --param | -p
      --trace | -t
      --noclobber | -c
      --version | -v
      --verbosity <level> | -V <level>
      --<param>=<value>
      --

The available parameters are

    Parameters:
    infile:
      type    = filename
      default = ""
    perllib: ./deceit [options] [--]

and the available options are

    Options:
      --help  | -h
      --param | -p
      --trace | -t
      --noclobber | -c
      --version | -v
      --verbosity <level> | -V <level>
      --<param>=<value>
      --
 

The deceit interpreter

Essentially white space (space, tabs, carriage-return and newline characters) and comments are skipped by the interpreter.

Comments are denoted by #, and include all text up to the end of the line. It is therefore not possible to place comments in the middle of a line.

There are some restrictions on the textual layout of a deceit template, imposed by the deceit parser (rather than by the deceit grammer). For instance, in the HelloWorld example above, the text

                dataset "HelloWorld"

must appear by itself on the line. If the template had been written as

                dataset
                    "HelloWorld"

deceit would not be able to correctly interpret the dataset name "HelloWorld". In addition, a dataset body, if present, must begin on a new line i.e.

Expressions, rather than literal values, are allowed in many of the item specifications. For example, the HelloWorld program could have been writtern as

                    # File: HelloWorld
                    #
                    # HelloWorld rewritten using an expression to construct the dataset name
                    # The built-in function param() is used to get the value of the parameter infile.
                    dataset param( "infile" ) + ".dat"

A number of built-in functions and  variables (which must never be redefined) may be used in expressions.  The syntax of expressions, together with a description of the built-in functions and variables is described later in this document.

The predefined variables are also described in each of the relevant sections.
 

Datasets

Essentially a deceit template comprises zero or more dataset specifications. The dataset specifications are placed into a file, and do not appear in the body of any other specification. There is no limit to the number of dataset specifications which may appear in a deceit template, but each dataset name must be unique (within the deceit template).

The general form of a dataset specification is as follows:

• dataset is the dataset specification designator keyword. This keyword is used to begin a new  dataset specification.
• dataset-name is the name of the dataset.  There must be no other dataset specifications with the same name in the template.  It has type string-expression (which may be a literal string).
• optional-expression is an optional expression which is evaluated after the last item in the body has been evaluated.

        dataset
        "set1"
        <
            ... dataset item specifications ...
        >

In addition, the subsequent dataset body, which is delimited by < and >, is optional but, if present, must begin on a new line.

The value of dataset-name will be the name of the created file. If a file with the name already exists, it will be overwritten.

The dataset body comprises zero or more of the following specifications:

• table
• array
• attribute
• history
• comment
• execute_dal
• execute_perl
• execute_post

The following are legal dataset specifications:

        # Examples of a dataset specifications
        dataset "ds1"        // Simplest case

       dataset "ds2"        // Empty body, which is quivalent to simplest case
        <
        >

       dataset "ds3"        // Several dataset items
       <
           attribute "att1"  int 10 "integer attribute" "m"
           attribute "att2" real 1.2 "real attribute" "m"
           array(5,4,2) "arr1"  int32 "integer array" "m"
           array(1024,1024) "arr2"  real64 "real array" "m"
           table "tab1" 10
           table "tab2" 20
       >

The variable dat is a predefined variable, the value of which is a pointer to the dataset and may be referenced in the optional expression, and also in the expressions of the dataset item specifications.  For example,

       dataset "ds1" fillDataSet( dat )
       <
       >

However, the optional expression will not usually be required  for datasets.
 

Tables

Table specifications may only appear in the body of a dataset specifcation. There is not limit to the number of table specifications which may appear in a dataset specification, but each specification should have a unique (within the dataset) name.

The general form of a table specification is as follows:

            table table-name rows optional-expression_opt
                <
                    ... table item specifications ...
                >

where,

• table is the table specification designator keyword. This keyword is used to begin a new table specification.
• table-name is the name of the table.  There must be no other table or array with the same name in the dataset.  It has type string-expression (which may be a literal string).
• rows is the number of rows in the table.  It has type integer-expression (which may be a literal integer).
• optional-expression is an optional expression which is evaluated after the last item in the body has been evaluated.

        dataset "set1"
        <
           table
                "tab1" 100
            <
            >
        >

In additon,the subsequent table body, which is delimited by < and >, is optional but, if present, must begin on a new line.

The table specification body comprises zero or more of the following specification:

• column
• attribute
• history
• comment
• execute_dal
• execute_perl
• execute_post

The following are legal table specifications:

           dataset "set1"
            <
            table "tab1" 10
            table "tab2" 20
              <
                attribute "att1" int 10 "integer attribute" "m"
                column "col1" int32 "integer column" "m"
                column "col2" int32 "integer column" "m"
              >
            table "tab3" 20
              <
                attribute "att1" real -10.12 "real attribute" "Nm"
                column "col1" real32 "real column" "Nn"
                column "col2" real64 "real column" "Nm"
              >
            >

The optional expression allows the table to be filled with data.

The variable tab is a pointer to the table and may be referenced in the expression. For example,

           dataset "ds1"
            <
               table "tab1" 100 fillTable( tab )
                <
                   column "col1" int16 "integer column" "Nm"
                   column "col2" int8 "integer column" "Nm"
                >
            >
 

Columns

Column specifications may only appear in the body of a table specification. There is no limit to the number of column specifications which may appear in the table body, but each column name must be unique (within the table).

The general form of a column specification is given by

       column column-dimensions_opt column-name data-type label units optional-expression_opt
        <
            ... column item specifications ...
        >

where,

• column is the column specification designator keyword. This keyword begins a new column specification.
• column-name is the name of the column.  There must be no other column with the same name in the table.  It has type string-expression (may be a literal string).
• column-dimensions, the dimensions of the column's cells, is optional. It takes the form of a comma-separated list of one or more integer-valued expressions, the list being delimited by parentheses i.e. the general form looks like ( integer-expression, integer-expression, ... ) e.g. (10,20) indicated that the columns cells are a two dimensional matrix with size 10 by 20.
• data-type specifies the data type of the column.  The available types are int8, int16, int32, real32, real64, bool, string
• label is a short textual description for the column.  It has type string-expression (which may be a literal string).
• units is the column's units.  When data-type is bool or string the units must be the empty string.  It has type string-expression (which may be a literal string).
• optional-expression is an optional expression which is evaluated after the last item in the body has been evaluated.

        dataset "set1"
        <
           table "tab1" 100
            <
               column
                    "col1" int32 "a column" "m"
            >
        >

In addition, the subsequent body, which is delimited by < and >, is optional but, if present, must begin on a new line.

The column specification body comprises zero or more of the following specifications:

• attribute
• execute_dal
• execute_perl
• execute_post

Scalar and multi-dimensional columns have fixed length.

Variable length columns are created by setting the column-dimensions to (0).

The optional expression allows the column to be filled with data.

The variable col is a pointer to the column and may be referenced in the expression. For example,

    dataset "ds1"
    <
       table "tab1" 100
       <
           column "col1" int32 "integer column" "m" fillColumn( col, "data.dat", 0 )
           column "col2" real64 "real column" "m" fillColumn( col, "data.dat", 1 ) )
           column(0) "col3" bool "bool column" "" fillColumn( col, "vardata.dat", 0 )        // variable size column
        >
    >
 

Arrays

Array specifications  may only appear in the body of dataset specifications. There is no limit to the number of array specifications which may appear in the array specification body, but each must have a unique name (within the dataset).

The general form of an array specification is given by

    array array-dimensions name array-data-type label units optional-expression_opt
    <
        ... array item specifications ...
    >

    where,

• array is the array specification designator keyword. This keyword is used to begin a new array specification.
• array-dimensions, the dimensions of the array, is a comma-separated list of one or more integer-valued expressions, the list being delimited by parentheses i.e. yhr general form looks like ( integer-expression, integer-expression, ... ) e.g. (10,20,30) indicates that the array has three dimensions 10, 20 and 30.
• name is the name of the array. There must be no other table or array with the same name in the dataset.  It has type string-expression (which may be a literal string).
• array-data-type specifies the data type of the array. The available types are int8, int16, int32, real32, real64, bool
• label is a short description for the array.  It has type string-expression (which may be a literal string).
• units is the array's units.  If array-data-type is bool, then the units must be the empty string.  It has type string-expression (which may be literal string).
• optional-expression is an optional expression, if present, is evaluated after the lat item in the body has been evaluated.

        dataset "set1"
        <
                array(10,20)
                "arr1" int8 "an array" "Nm"
            <
            >
        >

In addition, the subsequent body, which is delimited by < and >, is optional but, if present, must begin on a new line.

The array specification body comprises zero or more of the following specification:

• attribute
• history
• comment
• execute_dal
• execute_perl
• execute_post

The variable arr is a pointer to the array and may be referenced in the expression. For example,

    dataset "ds1"
    <
        array(10,20) "arr1" int32 "an integer array" "m" fillArray( arr )
        <
           attribute "att1" int 10 "an integer attribute" "Nm"
        >
    >
 

Attributes

Attribute specifications may appear in dataset, array, table and column body specifications.  There is no limit to the number of specifications which may appear in each of the allowed specification bodies, but each attribute specfication must have a unique name (within each of the body sections).

The general form of an attribute specification is given by

    attribute attribute-name attribute-data-type value label units optional-expression_opt
    <
        ... attribute item specifications ...
    >

where,

• attribute is the attribute specification disignator keyword. This keyword is used to begin a new attribute specification.
• attribute-name is the attribute name.  There must be no other attributes in the attribute's parent with the same name.  It has type string-expression (which may be a literal string).
• attribute-data-type is the attribute data type. The available types are int, real, bool, string
• value is the attribute value.  It is an expression (which may be a literal value) which must evaluate to a value whose type is consistent with attribute-data-type.
• label is a short description.  It has type string-expression (which may be a literal string).
• units is the attribute's units.  When attribute-data-type is bool or string, the units must be the empty string.  It has type string-expression (which may be a literal-string).
• optional-expression is an optional expression which is evaluated after the last item in the body has been evaluated.

• execute_dal
• execute_perl
• execute_post

    attribute "att1" int 10 "an integer attribute" "m"
    attribute "att2" real 10.12 "a real attribute" "m"
    attribute "att3" bool  0 "a boolean attribute" "m"
    attribute "att4" bool  true "a boolean attribute" "m"
    attribute "att5" string 10 "a string attribute" "m"
 

History

History specifications may appear in the body of dataset, table and array specifications.  There is no limit to the number of history specifications which may appear in each of the allowed specification bodies.

There are two allowed formats for the history specification.

The first is the single string form whose general format is given by

         history string

where

• history is the history designator keyword.  This is used to begin a new history specification.
• string is the text which comprises the history.  It must be a literal string.

        history < free-format-text >

where

• history is the history designator keyword.   This is used to begin a new history specification.
• free-format-text is the text which which comprises the history (delimited by < and >).  It may be spread over several lines.

For example

    dataset "ds1"
    <
        history "Updated on 31/12/99"
        history <
           Updated on 1/12/99
        >
        table "tab1" 100
        <
            history "Updated by a.n.other"
        >
    >
 

Comment

Comment specifications may appear in the body of dataset, table and array specifications.  There is no limit to the number of comment specifications which may appear in each of the allowed specification bodies.

There are two allowed formats for the comment specification.

The first is the single string form whose general format is given by

         comment string

where

• comment is the comment designator keyword.  This is used to begin a new comment specification.
• string is the text which comprises the comment.  It must be a literal string.

        comment < free-format-text >

where

• comment is the comment designator keyword.   This is used to begin a new comment specification.
• free-format-text is the text which which comprises the comment (delimited by < and >).  It may be spread over several lines.

For example

    dataset "ds1"
    <
        comment "Updated on 31/12/99"
        comment <
           Updated on 1/12/99
        >
        table "tab1" 100
        <
            comment "Updated by a.n.other"
        >
    >
 

Execute Dal Block

The execute_dal block construction allows a group of arbitrary expressions to be executed. The general form is given by

        execute_dal < list_of_ expressions >

where

• execute_dal is the designator which is used to begin a new block of dal expressions.
• list_of_expressions is the list of expressions to be executed.

It may appear in a deceit template, or the bodies of dataset, table, array, column or attribute specifications.

There is no limit to the number of expressions in the block.
 

Execute Perl Block

The text contained in an execute_perl block is paased to the Perl interpreter as is.
 

Execute Post Block

The execute_post block is the same as the execute_dal block execept that it:

• is not executed until after the last item in the containing body has been executed.
• may not appear in the top level deceit template.

Expressions
 

Literal Values

• Literal Characters
• The C character set.
• Strings

Quoted sequence of characters.
• Integers
• Reals
• Boolean

true and false.
• Identifiers

Operators
 
 

Operator	Priority	Description
+ plus	1	addition of numeric values concatenation of string values
- minus	1	subtraction of numeric values
* multilpy	2	multiplication of numeric values
/ divide	2	division of numeric values
+ unary plus	3	unary plus of numeric values
- unary minus	3	unary minus of numeric values
++ post increment	4	post increment of numeric variables
-- post-decrement	4	post decrement of numeric variables
() parenthesis	5	parenthesis

Functions

The general form of a function call is given by

    function-name ( parameter-list )

where

• function name The predefined functions are given later in this document.  The list of functions is given later in this document.
• parameter-list is a comma-separated list of expressions.

Built-in Functions
 
 

FUNCTION OR VARIABLE	DESCRIPTION
void fillColumn(     Column * column,     String       fileName,     int            columnNumber )	Copies the (ASCII) data from the column with position columnNumber (zero indexing)  in file fileName to the specified column.
void fillTable(     Table *      table      String      fileName )	Copies the (ASCII) data from file fileName to the specified column.
String getenv( string name )	Returns the value, as a string, of the specified environment variable.
String param( String parameterName )	Returns the value, as a string, of the specified parameter.
int rows( Table * table)	Returns the number of rows in the specified table.
Array * arr	The value of arr is a pointer to the current Array being constructed.
Attribute * att	The value of att is a pointer to the current Attribute being constructed.
Column * col	The value of col is a pointer to the current Column being constructed.
DataSet * dat	The value of dat is a pointer to the current DataSet being constructed.
bool false	Boolean value of false.
Table * tab	The value of tab is a pointer to the current Table being constructed.
bool true	Boolean value of true.

 

DAL Interface

The functions C interface dal_c.h are all available from deceit.
 

Grammer

deceit-unit:
    deceit-specification

deceit-specification:
    dataset-block deceit-specification
    include-statement deceit-specification
    dal-block deceit-specification
    perl-block deceit-specification

dataset-block:
    dataset string-expression expression_opt dataset-block-body_opt

dataset-body:
    < dataset-body-item >

dataset-body-item:
    table-specification dataset-body-item
    array-specification dataset-body-item
    attribute-specification dataset-body-item
    include-statement dataset-body-item
    history-specification dataset-body-item
    comment-specification dataset-body-item
    dal-block dataset-body-item
    perl-block dataset-body-item
    post-block dataset-body-item

attribute-specification:
    attribute string-expression attribute-data-type string-expression string-expresson expression_opt attribute-body_opt

attribute-body:
    < attribute-body-item >

attribute-body-item:
    dal-block attribute-body-item
    perl-block attribute-body-item
    post-block attribute-body-item

attribute-data-type:
    int
    real
    bool
    string

table-specification:
    table string-expression integer-expression string-expression expression_opt table-body_opt

table-body:
    < table-body-item >

table-body-item:
    column-specification table-body-item
    attribute-specification table-body-item
    include-statement table-body-item
    history-specification dataset-body-item
    comment-specification dataset-body-item
    dal-block table-body-item
    perl-block table-body-item
    post-block table-body-item

array-specification:
    array array-size-specification string-expression array-data-type string-expression string-expression expression_opt array-body_opt

array-body:
    < array-body-item >

array-body-item:
    attribute-specification array-body-item
    Include-statement array-body-item
    history-specification dataset-body-item
    comment-specification dataset-body-item
    dal-block array-body-item
    perl-block array-body-item
    post-block array-body-item

array-data-type:
    int8
    int16
    int32
    real32
    real64
    bool

column-specification:
    column array-size-specification_opt string-expression column-data-type expression_opt column-body_opt

column-body:
    < column-body-item >

column-body-item:
    attribute-specification column-body-item
    include-statement column-body-item
    dal-block column-body-item
    perl-block column-body-item
    post-block column-body-item

column-data-type:
    int8
    int16
    int32
    real32
    real64
    boolean
    string

array-size-specification:
    ( array-size-item )

array-size-item:
    integer-expression
    integer-expression , array-size-item

dal-block:
    execute_dal dal-block-body

dal-block-body:
    < dal-block-item >

dal-block-item:
    expression-statement dal-block-item

perl-block:
    execute_perl < free-text >

post-block:
    execute_post post-block-body

post-block-body:
    < post-block-item >

post-block-item:
    expression-statement post-block-item

include-statement:
    include string

string-expression:
    expression

integer-expression:
    expression

expression-statement:
    expression ;

expression:
    assignment-expression

assignment-expression:
    plus-expression
    plus-expression = assignment-expression
    plus-expression

plus-expression:
    plus-expression + mult-expression
    plus-expression - mult-expression

mult-expression:
    unary-expression
    mult-expression * unary-expression
    mult-expression / unary-expression

unary-expression:
    postfix-expression
    & unary-expression
    ! unary-expression
    + unary-expression
    - unary-expression
    * unary-expression

postfix-expression:
    primary-expression
    postfix-expression ( expression-list_opt )
    array-expression
    postfix-expression++
    postfix-expression--

primary-expresson:
    integer
    real
    string
    identifier
    character
    ( expression )
    compound-statement

compound-statement:
    < compound-statement-item >

compound-statement-item:
    expression-statement compound-statement-item

block-statement:
    < block-statement-item >

block-statement-item:
    expression-statement block-statement-item

history-specification:
    history text

comment-specification:
    comment text

text:
    string
    < free-format-text >