Types¶
Okera supports the following types.
Primitive Types¶
The most basic building block for dataset schemas are the primitive data types. These can be grouped in the following sub-types.
Numeric Types¶
For general numeric values, these are the data types available:
Type | Description |
---|---|
TINYINT |
1-byte signed integer, from -128 to 127 |
SMALLINT |
2-byte signed integer, from -32,768 to 32,767 |
INT / INTEGER |
4-byte signed integer, from -2,147,483,648 to 2,147,483,647 |
BIGINT |
8-byte signed integer, from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807 |
FLOAT |
4-byte single precision floating point number |
DOUBLE / REAL |
8-byte double precision floating point number |
DECIMAL[(precision[,scale])] |
Up to 38 digits with an optional scale (if not specified, the precision is 9 digits and 0 scale) |
Note:
INTEGER
is a synonym forINT
, andREAL
is the same forDOUBLE
. These are available for broader SQL support.
Date/Time Types¶
Okera provides the following types to represent dates and/or times:
Type | Description |
---|---|
DATE |
Format YYYY-MM-DD |
TIMESTAMP |
Format YYYY-MM-DD hh:mm:ss[.fraction] |
Note: Okera does not support the
DATETIME
field type.
Columns of type TIMESTAMP
support values in the range of 1970-01-01 00:00:01.000000 UTC
to 2038-01-19 03:14:07.999999 UTC
.
String Types¶
For text, Okera supports the following types:
Type | Description |
---|---|
STRING |
Variable length set of characters |
VARCHAR[(length)] |
Without length, same as STRING , else a length restricted set of characters. Maximum length, if given, is 65535. |
CHAR(length) |
Similar to VARCHAR() but restricted to a maximum of 255 characters |
Note: Okera only supports Unicode character encodings.
Other Types¶
There are the following additional primitive data types:
Type | Description |
---|---|
BOOLEAN |
Boolean true or false |
BINARY |
Similar to CHAR() but stores binary bytes instead of characters |
Complex Types¶
You can further combine primitive types to complex ones. These are the support complex types in Okera:
Type | Description |
---|---|
ARRAY<type> |
An array of values made from the same type |
MAP<key_type,value_type> |
A key/value pair map, where the key type must be a primitive type, while the value can be any type |
STRUCT<fields> |
A data structure holding one or more fields that can be of any type |
See Complex Data Types for details.
Type Conversion¶
The table will give you an overview of the general rules that apply when types are implicitly converted into other types.
NULL |
TINYINT |
SMALLINT |
INT |
BIGINT |
FLOAT |
DOUBLE |
DECIMAL |
DATE |
TIMESTAMP |
STRING |
VARCHAR |
CHAR |
BOOLEAN |
BINARY |
|
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
NULL to |
No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No |
TINYINT to |
No | Yes | Yes | Yes | Yes | Yes | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
SMALLINT to |
No | Yes3 | Yes | Yes | Yes | Yes | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
INT to |
No | Yes3 | Yes3 | Yes | Yes | Yes1 | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
BIGINT to |
No | Yes3 | Yes3 | Yes3 | Yes | Yes1 | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
FLOAT to |
No | Yes4 | Yes4 | Yes4 | Yes4 | Yes | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
DOUBLE to |
No | Yes4 | Yes4 | Yes4 | Yes4 | Yes1 | Yes | Yes7 | Yes11 | Yes11 | No | No | No | Yes5 | No |
DECIMAL to |
No | Yes4 | Yes4 | Yes4 | Yes4 | Yes8 | Yes | Yes | Yes9 | Yes9 | Yes | Yes2 | No | Yes | No |
DATE to |
No | Yes3 | Yes3 | Yes3 | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes2 | No | Yes | No |
TIMESTAMP to |
No | Yes3 | Yes3 | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Yes2 | No | Yes | No |
STRING to |
No | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes11 | Yes11 | Yes | Yes2 | No | Yes5 | No |
VARCHAR to |
No | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes10 | Yes11 | Yes11 | Yes | Yes | No | Yes | No |
CHAR to |
No | No | Yes | No | |||||||||||
BOOLEAN to |
No | Yes6 | Yes6 | Yes6 | Yes6 | Yes6 | Yes6 | No | Yes6 | Yes6 | Yes6 | Yes6 | No | Yes | No |
BINARY to |
No | No | No | No | No | No | No | No | No | No | No | No | No | No | No |
Notes:
- 1 Might cause loss in precision, better use
DOUBLE
to fit all possible numbers - 2 Strings longer than the restricted target type will be silently cut off
- 3 Integer number conversion to shorter types will use the values first
n
bytes of the target type - 4 Floating types to integers drops the fraction and only retains the number before the decimal divider3
- 5
0
andNULL
are treated asfalse
, everything else is treated astrue
- 6
true
is converted to1
andfalse
to0
for numerical and character target types - 7 Converting to
DECIMAL
requires a matching precision and scale, else the value is modified to fit or may result inNULL
(if fitting is impossible) - 8 Converting
DECIMAL
to floating types can lead to rounding errors - 9 Converting
DECIMAL
to date types requires values that have no fraction - 10 Converting strings to numerical types requires the target to be able to fit the number and the number to be valid, else will result in
NULL
- 11 Converting numerical or string types to date types require values that can be translated into dates