Types

ODAS supports the following types.

Primitive Types

The most basic building block for dataset schemas are the primitive data types. These can be grouped in the following sub-types.

Numeric Types

For general numeric values, these are the data types available:

Type Description
TINYINT 1-byte signed integer, from -128 to 127
SMALLINT 2-byte signed integer, from -32,768 to 32,767
INT / INTEGER 4-byte signed integer, from -2,147,483,648 to 2,147,483,647
BIGINT 8-byte signed integer, from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807
FLOAT 4-byte single precision floating point number
DOUBLE / REAL 8-byte double precision floating point number
DECIMAL[(precision[,scale])] Up to 38 digits with an optional scale (if not specified, the precision is 9 digits and 0 scale)

Note: INTEGER is a synonym for INT, and REAL is the same for DOUBLE. These are available for broader SQL support.

Date/Time Types

Okera provides the following types to represent dates and/or times:

Type Description
DATE Format YYYY-MM-DD
TIMESTAMP Format YYYY-MM-DD hh:mm:ss[.fraction]

Note: ODAS does not support the DATETIME field type.

Columns of type TIMESTAMP support values in the range of 1970-01-01 00:00:01.000000 UTC to 2038-01-19 03:14:07.999999 UTC.

String Types

For text, Okera supports the following types:

Type Description
STRING Variable length set of characters
VARCHAR[(length)] Without length, same as STRING, else a length restricted set of characters. Maximum length, if given, is 65535.
CHAR(length) Similar to VARCHAR() but restricted to a maximum of 255 characters

Note: ODAS only supports Unicode character encodings.

Other Types

There are the following additional primitive data types:

Type Description
BOOLEAN Boolean true or false
BINARY Similar to CHAR() but stores binary bytes instead of characters

Complex Types

You can further combine primitive types to complex ones. These are the support complex types in Okera:

Type Description
ARRAY<type> An array of values made from the same type
MAP<key_type,value_type> A key/value pair map, where the key type must be a primitive type, while the value can be any type
STRUCT<fields> A data structure holding one or more fields that can be of any type

See Complex Types for details.

Type Conversion

The table will give you an overview of the general rules that apply when types are implicitly converted into other types.

NULL TINYINT SMALLINT INT BIGINT FLOAT DOUBLE DECIMAL DATE TIMESTAMP STRING VARCHAR CHAR BOOLEAN BINARY
NULL to No Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes No
TINYINT to No Yes Yes Yes Yes Yes Yes Yes7 Yes11 Yes11 No No No Yes5 No
SMALLINT to No Yes3 Yes Yes Yes Yes Yes Yes7 Yes11 Yes11 No No No Yes5 No
INT to No Yes3 Yes3 Yes Yes Yes1 Yes Yes7 Yes11 Yes11 No No No Yes5 No
BIGINT to No Yes3 Yes3 Yes3 Yes Yes1 Yes Yes7 Yes11 Yes11 No No No Yes5 No
FLOAT to No Yes4 Yes4 Yes4 Yes4 Yes Yes Yes7 Yes11 Yes11 No No No Yes5 No
DOUBLE to No Yes4 Yes4 Yes4 Yes4 Yes1 Yes Yes7 Yes11 Yes11 No No No Yes5 No
DECIMAL to No Yes4 Yes4 Yes4 Yes4 Yes8 Yes Yes Yes9 Yes9 Yes Yes2 No Yes No
DATE to No Yes3 Yes3 Yes3 Yes Yes Yes No Yes Yes Yes Yes2 No Yes No
TIMESTAMP to No Yes3 Yes3 Yes Yes Yes Yes No Yes Yes Yes Yes2 No Yes No
STRING to No Yes10 Yes10 Yes10 Yes10 Yes10 Yes10 Yes10 Yes11 Yes11 Yes Yes2 No Yes5 No
VARCHAR to No Yes10 Yes10 Yes10 Yes10 Yes10 Yes10 Yes10 Yes11 Yes11 Yes Yes No Yes No
CHAR to No No Yes No
BOOLEAN to No Yes6 Yes6 Yes6 Yes6 Yes6 Yes6 No Yes6 Yes6 Yes6 Yes6 No Yes No
BINARY to No No No No No No No No No No No No No No No

Notes:

  • 1 Might cause loss in precision, better use DOUBLE to fit all possible numbers
  • 2 Strings longer than the restricted target type will be silently cut off
  • 3 Integer number conversion to shorter types will use the values first n bytes of the target type
  • 4 Floating types to integers drops the fraction and only retains the number before the decimal divider3
  • 5 0 and NULL are treated as false, everything else is treated as true
  • 6 true is converted to 1 and false to 0 for numerical and character target types
  • 7 Converting to DECIMAL requires a matching precision and scale, else the value is modified to fit or may result in NULL (if fitting is impossible)
  • 8 Converting DECIMAL to floating types can lead to rounding errors
  • 9 Converting DECIMAL to date types requires values that have no fraction
  • 10 Converting strings to numerical types requires the target to be able to fit the number and the number to be valid, else will result in NULL
  • 11 Converting numerical or string types to date types require values that can be translated into dates