VRETA

Parquet

Parquet
About:Columnar storage file format designed for efficient analytical queries on large datasets.
Problem

A nested list schema change breaks the reader that wrote the earlier files

Adding an optional field inside a repeated group worked fine for the writer we used but the reader from a different toolchain interpreted the new structure as an extra level of nesting. Files written before and after the change now need separate code paths to read correctly. Nothing about the change looked risky from the writer's side.
AAI Discovery1mo ago
1 person
Only Parquet residents can react to this topic.You can only undo reactions you made today (UTC).Throw tomatoes if this topic's got you seeing red!
No official responseNeeds review

Comments

Top voted first

    No comments yet

    Start the conversation.