Remained things for the following weeks.
July 23, 2017
Adding more primitives
Although dozens of primitives have been added into the backend, there are a couple of important primitives are missing. Most of them are linked to efficient libraries, such as like, which requires a high-performance string matching library PCRE.
like: PCRE library, online- BLAS and CLAPACK for complex numbers
- More....
Enum
Enumeration is a mapping from a primary-key column (unique values) to a foreign-key column. In the underline implementation, it stores numeric indices other than actual value.
u <- `apple`ibm`google
v <- `google`google`apple`apple`ibm
e <- `u:v
The logic value of e is
`u:`google`google`apple`apple`ibm
while it keeps indices as follows
Note: If a foreign-key column is replaced with an enumeration, it may sacrifice data locality when a query is merely within a table without any join operations. (Meeting with Bettina on July 28)
Null and Inf
The value of an item may be null (absence of a value) or inf (infinite). It can be applied to ordinary operations as an operand without any breakdowns. We'd like to keep a HorseIR program running with maximum tolerance on these special values.
For example,
inf + 10 = inf
inf - 10 = inf
System functions
A system function is designed for connecting HorseIR and outside environment. Its tasks include
- File I/O
- Floating numbers output precision
- More...
Completing interpreter
We're working on the interpreter to execute HorseIR code with user-defined functions (UDFs). As a DB-related system, the interpreter is designed as an online system which processes requests when it accepts. We summarize our design of the interpreter as follows.
- Low-latency
- High-throughput (SIMD)
- Extensible compiler framework (for compiler optimizations)
- Sustainable for a long time (with solid memory management)
Possible back-end tricks
- A stride represents a consecutive range
- Lazy evaluation (e.g.
len(reverse(x)) and len(x) are the same)
Remained things for the following weeks.
July 23, 2017
Adding more primitives
Although dozens of primitives have been added into the backend, there are a couple of important primitives are missing. Most of them are linked to efficient libraries, such as
like, which requires a high-performance string matching libraryPCRE.like: PCRE library, onlineEnum
Enumeration is a mapping from a primary-key column (unique values) to a foreign-key column. In the underline implementation, it stores numeric indices other than actual value.
The logic value of
eiswhile it keeps indices as follows
Note: If a foreign-key column is replaced with an enumeration, it may sacrifice data locality when a query is merely within a table without any join operations. (Meeting with Bettina on July 28)
Null and Inf
The value of an item may be null (absence of a value) or inf (infinite). It can be applied to ordinary operations as an operand without any breakdowns. We'd like to keep a HorseIR program running with maximum tolerance on these special values.
For example,
System functions
A system function is designed for connecting HorseIR and outside environment. Its tasks include
Completing interpreter
We're working on the interpreter to execute HorseIR code with user-defined functions (UDFs). As a DB-related system, the interpreter is designed as an online system which processes requests when it accepts. We summarize our design of the interpreter as follows.
Possible back-end tricks
len(reverse(x))andlen(x)are the same)