I've table named as logging
. Table property is like below
{
"Name": "logging",
"DatabaseName": "logging_data",
"Description": "",
"Owner": "allan",
"CreateTime": "2024-03-22T16:10:24.000Z",
"UpdateTime": "2025-02-16T08:54:25.000Z",
"Retention": 0,
"StorageDescriptor": {
"Columns": [
{
"Name": "id",
"Type": "string",
"Comment": ""
},
{
"Name": "logname",
"Type": "string",
"Comment": ""
},
{
"Name": "location",
"Type": "string",
"Comment": ""
} ],
"Location": "s3://logging-data-993562011282-us-east-1-dev/logdata/logging",
"InputFormat": ".apache.hadoop.mapred.TextInputFormat",
"OutputFormat": ".HiveIgnoreKeyTextOutputFormat",
"Compressed": false,
"NumberOfBuckets": 0,
"SerdeInfo": {
"SerializationLibrary": ".apache.hadoop.hive.serde2.lazy.LazySimpleSerDe",
"Parameters": {
"serialization.format": "\t",
"field.delim": "\t"
}
},
"SortColumns": [],
"Parameters": {},
"SkewedInfo": {
"SkewedColumnNames": [],
"SkewedColumnValues": [],
"SkewedColumnValueLocationMaps": {}
},
"StoredAsSubDirectories": false
},
"PartitionKeys": [
{
"Name": "timestamp",
"Type": "string",
"Comment": ""
}
],
"TableType": "EXTERNAL_TABLE",
"CreatedBy": "arn:aws:sts::993562011282:assumed-role/Logging-data-admin/ConduitAccountService+Dev+Default+NA+Fleet+P-allan",
"IsRegisteredWithLakeFormation": false,
"CatalogId": "993562011282",
"IsRowFilteringEnabled": false,
"VersionId": "13",
"CatalogIdentifier": "993562011282",
"DatabaseId": "1mndnsasdcdbcf692c422y2732jndjbdhfd32323jnj2",
"IsMultiDialectView": false
}
And I've data in s3 location (s3://logging-data-993562011282-us-east-1-dev/logdata/logging/timestamp=20250216001
) like below
123456 cloud \N
When I query data, I'm seeing location col value as N
in Athena
But when I remove SkewedInfo
from table properties, It shows empty
value for location
column only with new file placed into different partition. But shows N
with old partitioned data.
123456 cloud
Can anyone help me to understand why its happening ? Why its showing empty value for location col to newly generated file in new partition (same data as old partitioned
) but not with the old partitioned data, after removing SkewedInfo
from table properties ?