I'm encountering an error when loading Parquet data into BigQuery. My table schema defines a field (e.g., voltage) as FLOAT and NULLABLE. However, when the Parquet file contains null values for this field, I receive the following error:
Table failed with error INVALID_ARGUMENT: Provided Schema does not match Table . Field has changed type FLOAT to INTEGER
What I've Tried:
bq Command-Line Load: I attempted to load the data with the following command:
bq load --source_format=PARQUET --autodetect \
--schema "voltage:FLOAT" \
project:dataset.table \
example.parquet
However, I still encounter the same error.
Google Cloud Data Transfer Service: I set up a pipeline using the Data Transfer Service. I even preloaded dummy data (to ensure initial values and fill in nulls) before running the data transfer. Despite this, the schema type error persists.
I cannot modify the schema since the Parquet files may contain actual FLOAT values. Has anyone encountered this issue or found a workaround to handle null values in a FLOAT field coming from Parquet data?
Any help or suggestions would be greatly appreciated.