dap4.ce

Class CEConstraint

java.lang.Object

dap4.ce.CEConstraint

All Implemented Interfaces:

Constraint

Direct Known Subclasses:

Universal

public class CEConstraint
extends Object
implements Constraint

A Constraint is a structure containing a parsed representation of a constraint expression. Its purpose is define a subset of interest of a dataset.

The constraint object is defined with respect to some underlying DMR. It defines a subset view over that DMR in that it specifies a set of declarations (variables, enums, dimensions, groups) to be included. For each such variable, the constraint specifies any overriding of the dimensions of the variables. Additionally, each variable (if appropriate) may have a filter expr.

Thus, there are three 'sub' constraints within a full constraint. 1. Referencing - is a variable from the underlying dataset included in the constraint, directly (by or indirectly: e.g. fields of a structure when only the structure is referenced 2. Projection - the actual values of a variable to be included in the constraint; this is specified by a triple [start:stride:stop] for each dimension of a variable. 3. Selection (aka filters) - A predicate over the scalar fields of a row of a Sequence; if it evaluates to true, then that row matches the constraint.

There are multiple ways to effect a constraint. 1. Generate and test mode: the constraint is asked if a given element matches the constraint. E.g. a. For referencing, one might ask the constraint if a given variable or field is in the constraint b. For a projection filter, one might ask the constraint if a given set of dimension indices match the projection. c. For a selection filter, one might ask the constraint if a given sequence row matches the filter predicate 2. Iteration mode: the constraint provides an iterator that returns the elements matching the constraint. E.g. a. For referencing, the iterator would return all the variables and fields referenced in the constraint. b. For a projection filter, the iterator would return the successive sets of indices of the projection, or it could return the actual matching value. c. For a selection filter, the iterator would return either the row indices or the actual rows of a sequence that matched the filter predicate

3. Read mode: Sometimes, it may be more efficient to let the DataVariable object handle the constraint more directly. E.g. a. For example, if the data variable was backed by a netcdf file, then passing in the complete projection might be more efficient than pulling values 1 by 1. b. Similarly, if the sequence object had an associated btree, then it would be more efficient to allow the sequence object to evaluate the filter using the btree. Note that this requires analysis of the filter expression to see if the btree is usable.

Ideally, we would allow all three modes, but for now, only generate-and-test and iteration are implemented, and only a subset of those. Specifically, iteration is provided for referencing, projection, and selection (filters). Generate-and-test is provided for referencing and selection. It is not provided for projection (for now) because it essentially requires the inverse of iteration and that is fairly tricky.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	CEConstraint.Expand
protected static class	CEConstraint.FilterIterator
protected static class	CEConstraint.ReferenceIterator
protected static class	CEConstraint.Segment

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<DapNode,List<DapAttribute>>	attributes
protected List<DapDimension>	dimrefs
protected DapDataset	dmr
protected List<DapEnum>	enums
protected CEConstraint.Expand	expansion
protected boolean	finished
protected List<DapGroup>	groups
protected Map<DapDimension,DapDimension>	redef
protected Map<DapDimension,Slice>	redefslice
protected List<DapDimension>	refdims
protected List<CEConstraint.Segment>	segments "Map" of variables (at all levels) to be included Maps variables -> associated slices and is modified by computdimensions().
protected static Map<DapDataset,CEConstraint>	universals
protected List<DapVariable>	variables Also keep a raw list of variables

Constructor Summary

Constructors

Constructor and Description
CEConstraint()
CEConstraint(DapDataset dmr)

Method Summary

Methods

Modifier and Type	Method and Description
void	addAttribute(DapNode node, DapAttribute attr)
void	addRedef(DapDimension dim, Slice slice)
void	addVariable(DapVariable var, List<Slice> slices)
protected static int	compare(Object lvalue, Object rvalue)
protected void	computedimensions() Compute dimension related information using slicing and redef info.
protected void	computeenums() Walk all the included variables and accumulate the referenced enums
protected void	computegroups() Walk all the included declarations and accumulate the set of referenced groups
void	contract() Locate each Structure|Sequence and: 1.
protected boolean	contractR(DapStructure dstruct, Set<DapStructure> contracted) Recursive helper
protected void	dumpvar(CEConstraint.Segment seg, StringBuilder buf, boolean forconstraint) Recursive helper for tostring/toConstraintString
protected Object	eval(DapSequence seq, DataRecord record, CEAST expr) Evaluate a filter with respect to a Sequence record.
void	expand() Locate each unexpanded Structure|Sequence and: 1.
protected int	expansionCount(DapStructure struct) Count the number of fields of a structure that already in this view.
protected static Object	fieldValue(DapSequence seq, DataRecord record, String field)
CEConstraint.FilterIterator	filterIterator(DapSequence dapseq, DataSequence dataseq) Selection X Iterator Filter evaluation using an iterator.
protected CEConstraint.Segment	findSegment(DapVariable var)
protected int	findVariableIndex(DapVariable var)
CEConstraint	finish() Finish creating this Constraint.
List<DapDimension>	getConstrainedDimensions(DapVariable var)
List<Slice>	getConstrainedSlices(DapVariable var)
DapDataset	getDMR()
DapDimension	getRedefDim(DapDimension orig)
static CEConstraint	getUniversal(DapDataset dmr)
protected boolean	isWholeCompound(DapStructure dstruct) See if a structure is "whole", which means that none of its fields is missing from the constraint, all of fields use default (non-constrained) dimension), and all of its fields are also whole.
boolean	match(DapSequence seq, DataRecord rec) Selection X match Evaluate a filter with respect to a Sequence record.
protected boolean	matches(DapSequence seq, DataRecord rec, CEAST filter) Evaluate a filter with respect to a Sequence record.
Odometer	projectionIterator(DapVariable var) Projection X Iterator This basically returns an odometer that will iterate over the appropriate values.
CEConstraint.ReferenceIterator	referenceIterator() Reference X Iterator Iterate over the variables and return those that are referenced.
boolean	references(DapNode node) Reference X match
static void	release(DapDataset dmr)
void	setFilter(DapVariable var, CEAST filter)
String	toConstraintString() Convert the view to a constraint string suitable for use in a URL, except not URL encoded.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

universals

protected static Map<DapDataset,CEConstraint> universals

dmr

protected DapDataset dmr

segments

protected List<CEConstraint.Segment> segments

"Map" of variables (at all levels) to be included Maps variables -> associated slices and is modified by computdimensions(). Note that we keep the original insertion order

variables

protected List<DapVariable> variables

Also keep a raw list of variables

redefslice

protected Map<DapDimension,Slice> redefslice

attributes

protected Map<DapNode,List<DapAttribute>> attributes

redef

protected Map<DapDimension,DapDimension> redef

dimrefs

protected List<DapDimension> dimrefs

enums

protected List<DapEnum> enums

groups

protected List<DapGroup> groups

refdims

protected List<DapDimension> refdims

finished

protected boolean finished

expansion

protected CEConstraint.Expand expansion

Constructor Detail

CEConstraint

public CEConstraint()

CEConstraint

public CEConstraint(DapDataset dmr)

Method Detail

getUniversal

public static CEConstraint getUniversal(DapDataset dmr)

release

public static void release(DapDataset dmr)

fieldValue

protected static Object fieldValue(DapSequence seq,
                DataRecord record,
                String field)
                            throws DapException

Throws:

DapException

compare

protected static int compare(Object lvalue,
          Object rvalue)
                      throws DapException

Throws:

DapException

eval

protected Object eval(DapSequence seq,
          DataRecord record,
          CEAST expr)
               throws DapException

Evaluate a filter with respect to a Sequence record. Assumes the filter has been canonicalized so that the lhs is a variable.

Parameters:

seq - the template

record - the record to evaluate

expr - the filter

Throws:

DapException

getDMR

public DapDataset getDMR()

getRedefDim

public DapDimension getRedefDim(DapDimension orig)

addRedef

public void addRedef(DapDimension dim,
            Slice slice)

addVariable

public void addVariable(DapVariable var,
               List<Slice> slices)
                 throws DapException

Throws:

DapException

addAttribute

public void addAttribute(DapNode node,
                DapAttribute attr)

setFilter

public void setFilter(DapVariable var,
             CEAST filter)

getConstrainedSlices

public List<Slice> getConstrainedSlices(DapVariable var)

getConstrainedDimensions

public List<DapDimension> getConstrainedDimensions(DapVariable var)

toString

public String toString()

Overrides:

toString in class Object

finish

public CEConstraint finish()
                    throws DapException

Finish creating this Constraint.

Throws:

DapException

toConstraintString

public String toConstraintString()

Convert the view to a constraint string suitable for use in a URL, except not URL encoded.

Returns:

constraint string

dumpvar

protected void dumpvar(CEConstraint.Segment seg,
           StringBuilder buf,
           boolean forconstraint)

Recursive helper for tostring/toConstraintString

Parameters:

seg -

buf -

forconstraint -

references

public boolean references(DapNode node)

Reference X match

Parameters:

node - to test

Returns:

true if node is referenced by this constraint

referenceIterator

public CEConstraint.ReferenceIterator referenceIterator()
                                                 throws DapException

Reference X Iterator Iterate over the variables and return those that are referenced. The order of return is preorder. Inputs: 1. the variable whose slices are to be iterated.

Returns:

ReferenceIterator

Throws:

DapException - if could not create.

projectionIterator

public Odometer projectionIterator(DapVariable var)
                            throws DapException

Projection X Iterator This basically returns an odometer that will iterate over the appropriate values.

Parameters:

var - over whose dimensions to iterate

Throws:

DapException

match

public boolean match(DapSequence seq,
            DataRecord rec)
              throws DapException

Selection X match

Evaluate a filter with respect to a Sequence record. Assumes the filter has been canonicalized.

Parameters:

seq - the template

rec - the record to evaluate

Throws:

DapException

matches

protected boolean matches(DapSequence seq,
              DataRecord rec,
              CEAST filter)
                   throws DapException

Evaluate a filter with respect to a Sequence record.

Parameters:

seq - the template

rec - the record to evaluate

filter - the filter

Throws:

DapException

filterIterator

public CEConstraint.FilterIterator filterIterator(DapSequence dapseq,
                                         DataSequence dataseq)
                                           throws DapException

Selection X Iterator Filter evaluation using an iterator. The iterator evaluates records from a sequence one-by-one and returns the next one that matches the filter. In order to evaluate a record, we need as input: 1. the DapSequence from which the free variables in the filter are taken. 2. the DataRecord to evaluate

Parameters:

dapseq -

dataseq -

Throws:

DapException

findVariableIndex

protected int findVariableIndex(DapVariable var)

findSegment

protected CEConstraint.Segment findSegment(DapVariable var)

expand

public void expand()

Locate each unexpanded Structure|Sequence and: 1. check that none of its fields is referenced => do not expand 2. add all of its fields as leaves Note that #2 may end up adding additional leaf structs &/or seqs

contract

public void contract()

Locate each Structure|Sequence and: 1. check that all of its fields are referenced recursively and not constrained, otherwise ignore 2. contract by removing all of the fields of the Structure or Sequence. This is intended to be (not quite) the dual of expand();

contractR

protected boolean contractR(DapStructure dstruct,
                Set<DapStructure> contracted)

Recursive helper

Parameters:

dstruct - to contract

contracted - set of already contracted compounds

Returns:

true if this structure was contracted, false otherwise

expansionCount

protected int expansionCount(DapStructure struct)

Count the number of fields of a structure that already in this view.

Parameters:

struct - the dapstructure to check

Returns:

# of fields in this view

Throws:

DapException

isWholeCompound

protected boolean isWholeCompound(DapStructure dstruct)

See if a structure is "whole", which means that none of its fields is missing from the constraint, all of fields use default (non-constrained) dimension), and all of its fields are also whole. This must be done recursively.

Parameters:

dstruct - to test

Returns:

true if this structure is whole.

computedimensions

protected void computedimensions()
                          throws DapException

Compute dimension related information using slicing and redef info. In effect, this is where projection constraints are applied

Assume that the constraint compiler has given us the following info:

1. A list of the variables to include.
2. A pair (DapDimension,Slice) for each redef
3. For each variable in #1, a list of slices taken from the constraint expression

Two products will be produced.

1. The variables map will be modified so that the slices properly reflect any original or redef dimensions.
2. A set, dimrefs, of all referenced original dimensions.

The processing is as follows

1. For each redef create a new redef dimension
2. For each variable:
   1. if the variable is scalar, do nothing.
   2. if the variable has no associated slices, then make its new dimensions be the original dimensions.
   3. otherwise, walk the slices and create new dimensions from them; use redefs where indicated

Throws:

DapException

computeenums

protected void computeenums()

Walk all the included variables and accumulate the referenced enums

computegroups

protected void computegroups()

Walk all the included declarations and accumulate the set of referenced groups