adddim

Description
Add dimension to array. Prepend dimensions list with new dimension with given name and size equal to one and chunk interval equal to one.
Syntax
adddim ( input-array, new-dimension-name-1 )
Example
iquery -aq "adddim(matrix, timestamp)"

aggregate

Description
Calculate aggregates for the specified dimensions. This operator takes a list of aggregate calls and a list of dimension names by which grouping is performed and calculates the aggregates for the remaining dimensions. Result is an array with shape corresponding to the specified dimensions and one attribute for each specified aggregate.
Syntax
aggregate ( input-array, aggregate-1,...aggregate-N, dimension-1,... dimension-N )
Example
iquery -aq "aggregate(Reads, POSITION, sum(a), count(*) as num_values)"

allversions

Description
Get all versions of updatable array using version number as additional (first) dimension.
Syntax
allversions ( updatable-array-name )
Example
iquery -aq "allversions(updarr)"

apply

Description
Evaluate expression for each element of the input array. The result is represented as a separate attribute added to the input array.
Syntax
apply ( input-array, result-attribute, applied-expression )
input-array { result-attribute-1 : applied-expression-1 , result-attribute-2 : applied-expression-2 , ... }
Example
iquery -aq "apply(testarr, sum, a+b)"

attributes

Description
Get list of array attributes. Returns array with the following attributes: name, type.
Syntax
attributes ( array-reference )
Example
iquery -aq "attributes(testarr)"

average

Description
Calculate average value of the specified attribute in the array. The result is an array with single element containing average value. If input array contains only one attribute, then attribute name can be omitted.
Syntax
average ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
iquery -aq "average(testarr, A)"
iquery -aq "average(matrix)"
iquery -aq "average(Reads, Read, OFFSET)"

bernoulli

Description
Sample input array by selecting random number of cells from it. The marginal probability that a cell is included in the sample is provided in the second argument. If users desire a repeatable sample, they can provide an initializing seed value for the internal random number genertor.
Syntax
bernoulli ( input-array , probability [ , seed ] )
Example
iquery -aq "max(bernoulli(testarr, 0.01))"

between

Description
Get subarray of the input array specified by low and high boundaries preserving shape of the input array.
Syntax
between ( input-array, low-boundary-coordinate-1, ..., low-boundary-coordinate-N, high-boundary-coordinate-1, ..., high-boundary-coordinate-N )
Example
iquery -aq "between(testarr, 0, 1, 1, 2)"

build

Description
Construct single-attribute array, assigning values to its elements using a specified expression. This expression can access any of the array coordinates. First example below initializes a matrix with sequential values 1, 2, 3, ... Second example creates identity matrix.
Syntax
build ( array-reference, expression )
Example
iquery -q "create array M3x3 <val:float> [x=0:2,3,0, y=0:2,3,0]"
iquery -aq "build(M3x3, 0)"
iquery -aq "build(M3x3, y*3 + x)"
iquery -aq "build(M3x3, if x = y then 1 else 0)"

build_sparse

Description
Construct single-attribute sparse array, assigning values to its elements selected by predicate using specified expression. This expression can access any of the array coordinates. The following example initializes spars diagonal matrix:
Syntax
build_sparse ( array-reference, expression, predicate )
Example
iquery -q "create array M3x3 <val:float> [x=0:2,3,0, y=0:2,3,0]"
iquery -aq "build_sparse(M3x3, 1, x = y)"

cast

Description
Cast array type. This operator converts input array to the specified array type. The input array and target array descriptor should have the same number of attributes with the same types and also shape of the input and output arrays should be the same. So this operator just allows to rename array and its attributes and dimensions.
Syntax
cast ( input-array, output-array )
Example
iquery -aq "cast(testarr1, M4x4)"

concat

Description
Concatenate two arrays. Array should have the the name number of dimensions. Concatenation is performed by the left-most coordinate. All other dimensions should be the same. And left-most dimension of both arrays should have fixed size and same chunking schema. Also arrays should have the same attributes.
Syntax
concat ( left-array, right-array )
Example
iquery -aq "concat(testarr1, testarr2)"

count

Description
Count non-empty elements of the input array. Group-by dimensions are optional in the count operator. If one or more group-by dimensions are specified, the input array elements are first grouped by these dimensions and the count is performed on the resulting groups.
Syntax
count ( input-array [ , group-by-dimension-1, group-by-dimension-2, ... ] )
Example
iquery -aq "count(testarr)"
iquery -aq "count(Reads, OFFSET)"

create array

Description
Create an array.
Syntax
create array array-name < attribute-name : type, ... > [ dimension-name = from : till, chunk-interval, chunk-overlap , ... ]
Example
iquery -q "CREATE ARRAY test <a: int32, b: int32> [x=0:2,3,0, y=0:2,3,0]"

cross

Description
Produces cross product of two arrays
Syntax
cross ( left-array, right-array )
Example
iquery -aq "cross(matrix, vector)"

cross_join

Description
Performs cross join on specified dimensions. Pairs of dimension names (one from the left array, another from the right array) are specified as parameter of JOIN operator. Number of dimensions of result array is equals to sum of dimensions of input arrays minus number of pairs of matched coordinates.

In case the attribute name is not unique it can be qualified using array name (A.x) or operand number ($2.y). In the first case attribute "x" of input array "A" is taken. And second case refer to the attribute "y" of second input array.

Syntax
cross_join ( input-array-1, input-array-2, dim-array-1, dim-array-2, ... )
Example
iquery -aq "cross_join(testarr1, testarr2, i, j)"

deldim

Description
Delete left-most dimension from the array. Deleted dimension should have size 1.
Syntax
deldim ( input-array )
Example
iquery -aq "deldim(subarray(matrix, 1, 100, 1, 200))"

dimensions

Description
Get list of array dimensions. Returns array with the following attributes: name, start, length, chunk_interval, chunk-overlap.
Syntax
dimensions ( array-reference )
Example
iquery -aq "dimensions(testarr)"

diskinfo

Description
Get information about storage space. Returns array with the following attributes: used, available, clusterSize, nFreeClusters, nSegments.
Syntax
diskinfo ()
Example
iquery -aq "diskinfo()"

echo

Description
Print text to the output and to the log
Syntax
echo ( any-text )
Example
iquery -aq "echo('Hello world')"

explain_logical

Description
Output a textual representation of the logical plan for the given query.
Syntax
explain_logical ( query-string [ , language-string ] )
Example
iquery -aq "explain_logical('select * from foo','aql')"

explain_physical

Description
Output a textual representation of the physical plan for the given query.
Syntax
explain_physical ( query-string [ , language-string ] )
Example
iquery -aq "explain_physical('select * from foo','aql')"

input

Description
Incremental input of array from input text file. Unlike LOAD operator, input doesn't store result in the database
Syntax
input ( array-name, input-file-path [ [ [ [ , instance-id [ , input-format [ , max-errors [ , shadow-array ] ] ] ] )
instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
input-format - "opaque", "text" or binary loader template, for example: "(int32,dummy(4),string)". Default "text"
max-errors - maximal number of conversion error which can be ignored. Default 0
shadow-array - name of shadow array containing information about conversion errors. It contains the same attributes as original array + position of error in imported file. Attributes of shadow array contains text of error message or null if correspondent attribute was successfully loaded
Example
iquery -aq "store(input(testarr, 'data.dmp'), testarr2)"
iquery -aq "store(input(testarr, 'data.binary', -2, '(int32 null,float null,string null)', 10, foo_shadow), testarr2)"

filter

Description
Filter array elements not matching specified condition. Mark as empty the elements of the array for which filter-expression is not evaluated to true. If input array already has empty-tag attribute, then it is used, otherwise it will be added.
Syntax
filter ( input-array, filter-expression )
input-array ? filter-expression
Example
iquery -aq "filter(testarr, A > B)"

redimension

Description
Exchange specified dimensions and attributes of the source array. Mapping between attributes and dimensions is specified using definition of the destination array.
Syntax
redimension ( source-array, destination-array )
Example
iquery -aq "join(A, redimension(B, A))"

redimension_store

Description
Exchange specified dimensions and attributes of the source array and store result to the disk. Mapping between attributes and dimensions is specified using definition of the destination array.
Syntax
redimension ( source-array, destination-array { ',' aggregation-expression } )
Example
iquery -aq "redimension_store(B, A)"

join

Description
Perfroms natural join of two arrays with the same shape. Array should have the same dimensions and the same chunks size. Result is array with the same shape containing attributes from both arrays.
Syntax
join ( input-array-1, input-array-2 )
Example
iquery -aq "join(testarr1, testarr2)"

list

Description
Get list of operators, functions, aggregates, types, arrays names, current query IDs or loaded libraries. Returns array with single attribute name. For queries returns array with fields (query: string, id: int64). For libraries returns array with fields (name: string, major: int32, minor: int32, patch: int32, build: int32). When called without parameters returns list of arrays.
Syntax
list ( 'arrays' | 'types' | 'operators' | 'functions' | 'queries' | 'libraries')
Example
iquery -aq "list()"
iquery -aq "list('arrays')"
iquery -aq "list('operators')"
iquery -aq "list('types')"
iquery -aq "list('functions')"
iquery -aq "list('aggregates')"
iquery -aq "list('queries')"
iquery -aq "list('libraries')"

load

Description
Load an array from input text file. This is shorthand for store( input(array-name, input-file-name), array-name).
Syntax
load ( array-name, input-file-path [ [ [ [ , instance-id [ , input-format [ , max-errors [ , shadow-array ] ] ] ] )
instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
input-format - "opaque", "text" or binary loader template, for example: "(int32,dummy(4),string)". Default "text"
max-errors - maximal number of conversion error which can be ignored. Default 0
shadow-array - name of shadow array containing information about conversion errors. It contains the same attributes as original array + position of error in imported file. Attributes of shadow array contains text of error message or null if correspondent attribute was successfully loaded
Example
iquery -aq "load(testarr, 'data.dmp')"
iquery -aq "load(testarr, 'data.binary', -2, '(int32 null,float null,string null)', 10, foo_shadow)"

lookup

Description
Map elements from the second array using attributes of the first array as coordinates in the second array. Result array has the same shape as first array and the same attributes as second array.
Syntax
lookup ( pattern-array, source-array )
Example
iquery -aq "lookup(pattern, dense)"

match

Description
Match pattern array with catalog array. For each cell in pattern array find closest non-empty cell in catalog array distance to which doesn't exceed specified error. Result array has the same shape as pattern array and combined list of attributes pattern and catalog attrbiutes + dimensiones of matched catalog element. Attributes and coordinates of matched catalog entry have "match_" prefix.
Syntax
match ( pattern-array, catalog-array, error )
Example
iquery -aq "match(sky_snaphot, stars_catalog, 50)"

materialize

Description
Materialize chunks of input array, converting them to the specified format (0 - preserve format of the input chunk, 1 - RLE format, 2 - dense format)
Syntax
materialize ( input-array, format )
Example
iquery -aq "materilize(apply(A, x2, x*x), 0)"

max

Description
Calculate maximum of the specified attribute in the array. Result is an array with single element containing maximum of specified attribute. If input array contains only one attribute, then attribute name can be omitted.
Syntax
max ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
iquery -aq "max(testarr, A)"
iquery -aq "max(matrix)"
iquery -aq "max(Reads, Read, POS)"

merge

Description
Merge two or more arrays. All arrays should have the same attributes and dimensions. Merge operator combines elements from the input array in such way: if element of first array is non empty, then it is taken, otherwise correspondent element of the second array is inspected and so on.
Syntax
merge ( input-array-1, input-array-2, ... )
Example
iquery -aq "merge(main_results, supplementary_results)"

min

Description
Calculate minimum of the specified attribute in the array. Result is an array with single element containing minimum of specified attribute. If input array contains only one attribute, then attribute name can be missed.
Syntax
min ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
iquery -aq "min(testarr, A)"
iquery -aq "min(matrix)"
iquery -aq "min(Reads, Read, OFFSET)"

multiply

Description
Multiply two matrices. Matrices should have two dimensions and single attribute.
Syntax
multiply ( left-matrix, right-matrix )
Example
iquery -aq "multiply(matrix1, matrix2)"

norm

Description
Normalize vector (divide each element on length of the vector)
Syntax
norm ( input-array )
Example
iquery -aq "norm(testarr, A)"

project

Description
Project specified attributes of the input array
Syntax
project ( input-array, attribute-1, ..., attribute-N )
Example
iquery -aq "project(testarr, A)"

regrid

Description
Split the input array into grids of specified dimensions and calculate one or more aggregates for each grid. Results array has the same number of dimensions as in the input array but the length of each dimension is divided by the size of the corresponding grid dimension. Result array contains one attribute for every specified aggregate.
Syntax
regrid ( input-array, grid-interval-1, ... grid-interval-N, aggregate-1,... aggregate-N )
Example
iquery -aq "regrid(testarr, 2, 2, sum(A) as sum, avg(A), count(*) as numvalues)"

remove

Description
Remove array from the storage. Currently only metadata of array is removed, but its chunks are still present in repository. So it can be considered as a resource leak.
Syntax
remove ( array-reference )
Example
iquery -aq "remove(testarr)"

rename

Description
Rename existing array in the database.
Syntax
rename ( old-array-reference, new-array-reference )
Example
iquery -aq "rename(testarr, testarr_v1)"

repart

Description
Change partitioning (chunking) of the array. Target array should have the same number of attributes and dimensions, but chunk size may be different. Target array should be previously created using "create array" operator. Repart takes definition of attributes from the source array and definition of dimensions from the target array.
Syntax
repart ( input-array, target-array )
Example
iquery -aq "repart(testarr, testarr2)"

reshape

Description
Change array shape. Target array should have the same number of attributes and fixed size dimensions (unbounded arrays are not allowed). Size of the input and target array should be the same. So it is possible to reshape 2x2x2 array to 4x2, but not 3x3 to 2x2. Target array should be previously created using "create array" operator. Reshape takes definition of attributes from the source array and definition of dimensions from the target array.
Syntax
reshape ( input-array, target-array )
Example
iquery -aq "reshape(testarr, testarr2)"

reverse

Description
Invert order of array elements.
Syntax
reverse ( input-array )
Example
iquery -aq "reverse(testarr)"

sample

Description
Sample input array by selecting random number of chunks from it
Syntax
sample ( input-array , probability [ , seed ] )
Example
iquery -aq "max(sample(testarr, 0.01))"

save

Description
Save array in specified file in text format. This operator is the opposite of the load() operator.
Syntax
save ( array-reference , file-path [ [ , instance-id [ , store-format ] ] )
instance-id - instanceID or -1 (all instances) or -2 (coordinator). Default -2
store-format - "opaque", "store", "sparse", "dense", "csv", "csv+", "tsv", "tsv+" or binary loader template, for example: "(int32,dummy(4),string)". Default "store"
Example
iquery -aq "save(testarr, '/data/testarr.dat')"
iquery -aq "save(testarr, '/data/testarr.dat', -2, '(int32,float,string)')"

scan

Description
Read array. Array with the specified name should be previously created and loaded in the database using load operator.
Syntax
scan ( array-reference [ , timestamp ] )
Example
iquery -aq "scan(testarr)"
iquery -aq "scan(testarr, '10/28/2010')"
iquery -aq "scan(testarr, now() - 5*24*3600)"

setopt

Description
Set/get configuration option value at runtime. Option value should be specified as string. If new value is not specified, then values of this configuration option at all nodes are printed. If new value is specified, then value of option is updated at all odes and result array contains old and new values of the option at all nodes.
Syntax
setopt ( option-name [ , new-option-value ] )
Example
iquery -aq "setopt('threads', '4')"

slice

Description
Get slice of the array. Result is slice of the input array corresponding to the given coordinates. Number of dimensions of the result array is equal to the number of dimension of input array minus number of specified coordinates (which should be less than number of dimensions of the input array).
Syntax
slice ( input-array, dimension-1, coordinate-value-1 , ... , dimension-N, coordinate-value-N )
Example
iquery -aq "slice(testarr, x, 2)"

sort

Description
Sort one-dimensional array by one or more attributes. Attributes are specified either by name, either by position (1-based). To sort in descending order prepend attribute name with - or use negative value of attribute position.
Syntax
sort ( input-array, { attribute-name | [-] attribute-position } )
Example
iquery -aq "sort(testarr, A)"
iquery -aq "sort(testarr, -1)"

store

Description
Store input array in the database under specified name.
Syntax
store ( input-array, destination )
destination-array = input-array
Example
iquery -aq "store(multiply(arr1, arr2), result)"

subarray

Description
Get subarray of the input array specified by low and high boundaries.
Syntax
subarray ( input-array, low-boundary-coordinate-1, ..., low-boundary-coordinate-N, high-boundary-coordinate-1, ..., high-boundary-coordinate-N )
Example
iquery -aq "subarray(testarr, 0, 1, 1, 2)"

substitute

Description
Substitute null vakues in the input array, using thier missing reasons as index in the second (substituion) array.
Syntax
substitute ( input-array , substitute-array )
Example
iquery -aq "substitute(nullableArray, substNulls)"

sum

Description
Calculate sum of the specified attribute in the array. Result is an array with single element containing sum of specified attribute. If input array contains only one attribute, then attribute name can be omitted.
Syntax
sum ( input-array [ , attribute [ , group-by-dimension-1, group-by-dimension-2, ... ] ] )
Example
iquery -aq "sum(testarr, A)"
iquery -aq "sum(matrix)"
iquery -aq "sum(Reads, Read, OFFSET)"

thin

Description
Operator inverse to xgrid: leave only each k-th element of input array.
Syntax
thin ( input-array, from-1, step-1, from-2, step-2, ..., from-N, step-N, )
Example
iquery -aq "thin(window(A, 1, 2, 1, 2)"

transpose

Description
Transpose matrix. Input array should have two dimensions.
Syntax
transpose ( input-array )
Example
iquery -aq "transpose(testarr)"

unpack

Description
Unpack array to single dimensional array treating coordinates as attributes. Result array has single zero-based dimension and arguments combining attributes of the input array.
Syntax
unpack ( input-array, dimension )
Example
iquery -aq "unpack(testarr, no)"

versions

Description
Get list of updatable array versions. Returns array with the following attributes: version_id, time_stamp.
Syntax
versions ( array-reference )
Example
iquery -aq "versions(updarr)"

window

Description
Move specified window through the input array and calculate one or more aggregates. The shape of of result array is the same as shape of input array. Result array contains one attribute for each specified aggregate.
Syntax
window ( input-array, window-size-1, ... widow-size-N, aggregate-1,... aggregate-N )
Example
iquery -aq "window(testarr, 3, 3, sum(A), stdev(A) as standard_deviation)"

xgrid

Description
Scale input array by repeating cells of the original array specified number of times. This operator can be considered as inverse to regrid operator: regrid splits input array in "hypercubes" and calculates aggregate function for them and xgrid produces hypercube from the single element of the input array.
Syntax
xgrid ( input-array, scale-1, ..., scale-N )
Example
iquery -aq "xgrid(matrix, 2, 2)"

mstat

Description
Collect mallinfo outputs on every instance. Contains 10 attributes with parameters. Read man mallinfo for more details.
Syntax
mstat ()
Example
iquery -aq "mstat()"

scatter/gather

Description
scatter/gather operator redistributes chunks of the input array across cluster nodes according to a specified policy. sg is not a user-visible operator, but it is heavily used by the system internally. It is a physical operator for data movement within the system and users are not expected to invoke it directly. The scidb optimizer inserts an sg operator after load() to redistribute array data using round robin distribution. The optimizer may also insert sg operators to assist with aggregations, operators that don't preserve shape or operators that don't preserve distribution.
Syntax
sg ( input-array, [ partitioning schema [, node number [, result array name [, store_flag [,offset_x, offset_y,... ] ] ] ] ])
input-array - array to be redistributed. May also be the output of another operator like scan, filter, etc.
partitioning schema - integer constant specifying partitioning schema. Possible values:
0 - psReplication - replicate array on every node of cluster,
1 - psRoundRobin - distribute chunks by round robin,
2 - psLocalNode - send every local chunk to specified node,
3 - psByRow - distribute every chunk to node according to chunk coordinate of dimension 0,
4 - psByCol - distribute every chunk to node according to chunk coordinate of dimension 1
node number - a node number to send all chunks. Used only with psLocalNode. Ignored for other partitioning schemas.
result array name - a name of result array. If omitted, the result of sg is nameless and temporary with query duration. If present, and store_flag is either omitted or true, the result of sg is persisted as a new array with given name. If present, and store_flag is false, the result is temporary but may be aliased by this given name elsewhere in the query.
store_flag - a boolean indicator used to specify whether sg should store the result as a new array. By default, true if result array name is given.
offset - a distribution offset vector. For internal use.
Example
Query to distribute array by round robin onto every node:
iquery -aq "sg(seq_info, 1, -1, seq_info_sg)"
Query to gather whole array on node 0 in memory array (will not be stored on disk):
iquery -aq "sg(seq_info_sg, 2, 0)"

load library

Description
Syntax
AQL: load library library name
AFL: load_library(library name)
library name - a string constant with name of .so library that should be loaded
SciDB instance will search library in plugins folder of current directory of instance. Example of point library will be linked there authomatically if you build it from the sources.
Example
Loading library from examples folder - libpoint.so:
iquery -aq "load library 'libpoint.so'"
The use of user defined objects from libpoint.so:
iquery -aq "apply(dense, b, point2str( str2point('(1,1)') + str2point('(2,2)') ))"

cancel

Description
Syntax
AQL: Not available
AFL: cancel(query id)
query id - a int64 number with query id you want to cancel. Use list('queries') operator to know current query ids.
Example
iquery -aq "cancel(1299016933)"