setop/sqlite-parquet-vtable
1
0
Fork 0
mirror of https://github.com/cldellow/sqlite-parquet-vtable.git synced 2025-09-08 22:24:56 +00:00
Code Releases Activity

Rewrite tests for automatic generation

Browse Source 
Regularize the parquets - nulls and nonulls each come in 3 variants,
with 1, 10 and 99 rows per rowgroup.

All test queries are written against nullsA, no_nullsA.

Next commit will introduce a tool to expand these template queries to
go against the actual tables.
This commit is contained in:
Colin Dellow
2018-03-18 13:11:29 -04:00
parent 3b557f7fb0
commit 65ea1b2f61
124 changed files with 122 additions and 222 deletions
Download Patch File Download Diff File Expand all files Collapse all files

BIN
parquet-generator/99-rows-99.parquet Normal file
View File

Binary file not shown.

BIN
parquet-generator/99-rows-nulls-1.parquet Normal file
View File

Binary file not shown.

0
parquet-generator/99-rows-nulls.parquet → parquet-generator/99-rows-nulls-10.parquet
View File

BIN
parquet-generator/99-rows-nulls-99.parquet Normal file
View File

Binary file not shown.

5
parquet-generator/parquets.py
View File

@@ -136,12 +136,15 @@ def main():
write_parquet('99-rows-1.parquet', rows, types, row_group_size=99)
write_csv('no-nulls.csv', rows)
write_parquet('99-rows-10.parquet', rows, types, row_group_size=10)
write_parquet('99-rows-99.parquet', rows, types, row_group_size=1)
for i in range(len(rows)):
for j in range(len(rows[i])):
if (i >= 10 and i <= 19) or (i >= 20 and (i + j) % 2 == 0):
rows[i][j] = None
write_parquet('99-rows-nulls.parquet', rows, types,row_group_size=10)
write_parquet('99-rows-nulls-99.parquet', rows, types,row_group_size=99)
write_parquet('99-rows-nulls-10.parquet', rows, types,row_group_size=10)
write_parquet('99-rows-nulls-1.parquet', rows, types,row_group_size=1)
write_csv('nulls.csv', rows)
write_unsupported_parquets()