1
0
mirror of https://github.com/cldellow/sqlite-parquet-vtable.git synced 2025-09-18 22:59:58 +00:00

More defensive, add caveats

This commit is contained in:
Colin Dellow
2018-03-03 20:26:41 -05:00
parent eb0b48f867
commit 18f07f4c43
2 changed files with 87 additions and 34 deletions

View File

@@ -1,7 +1,17 @@
# parquet-vtable
# sqlite-parquet-vtable
A SQLite [virtual table](https://sqlite.org/vtab.html) extension to expose Parquet files as SQL tables.
## Caveats
I'm not a professional C/C++ programmer. These are the caveats I'm aware of, but there are probably others:
* I don't use `sqlite3_malloc` and `sqlite3_free` for C++ objects
* Maybe this doesn't matter, since portability isn't a goal
* The C (SQLite API implementation) -> C++ interop (to talk to parquet-cpp) probably leaks some C++ exceptions
* Your process may crash due to my error. Sorry!
* I handle the obvious cases like file not found and unsupported Parquet types but I suspect low memory conditions aren't handled gracefully
## Building
1. Install [`parquet-cpp`](https://github.com/apache/parquet-cpp)
@@ -18,3 +28,26 @@ sqlite> create virtual table demo USING parquet('demo.parquet');
sqlite> select * from demo limit 1;
...if all goes well, you'll see data here!...
```
## Supported features
### Index
Only full table scans are supported.
### Types
These types are supported:
* INT96 timestamps (exposed as milliseconds since the epoch)
* INT8/INT16/INT32/INT64
* UTF8 strings
* BOOLEAN
* FLOAT
* DOUBLE
These are not supported:
* UINT8/UINT16/UINT32/UINT64
* Fixed length byte arrays, including JSON and BSON subtypes
* DECIMAL