最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

amazon web services - How to handle special character in Aws Athena table? - Stack Overflow

programmeradmin2浏览0评论

I've table named as logging. Table property is like below

{
  "Name": "logging",
  "DatabaseName": "logging_data",
  "Description": "",
  "Owner": "allan",
  "CreateTime": "2024-03-22T16:10:24.000Z",
  "UpdateTime": "2025-02-16T08:54:25.000Z",
  "Retention": 0,
  "StorageDescriptor": {
    "Columns": [
      {
        "Name": "id",
        "Type": "string",
        "Comment": ""
      },
      {
        "Name": "logname",
        "Type": "string",
        "Comment": ""
      },
      {
        "Name": "location",
        "Type": "string",
        "Comment": ""
      }    ],
    "Location": "s3://logging-data-993562011282-us-east-1-dev/logdata/logging",
    "InputFormat": ".apache.hadoop.mapred.TextInputFormat",
    "OutputFormat": ".HiveIgnoreKeyTextOutputFormat",
    "Compressed": false,
    "NumberOfBuckets": 0,
    "SerdeInfo": {
      "SerializationLibrary": ".apache.hadoop.hive.serde2.lazy.LazySimpleSerDe",
      "Parameters": {
        "serialization.format": "\t",
        "field.delim": "\t"
      }
    },
    "SortColumns": [],
    "Parameters": {},
    "SkewedInfo": {
      "SkewedColumnNames": [],
      "SkewedColumnValues": [],
      "SkewedColumnValueLocationMaps": {}
    },
    "StoredAsSubDirectories": false
  },
  "PartitionKeys": [
    {
      "Name": "timestamp",
      "Type": "string",
      "Comment": ""
    }
  ],
  "TableType": "EXTERNAL_TABLE",
  "CreatedBy": "arn:aws:sts::993562011282:assumed-role/Logging-data-admin/ConduitAccountService+Dev+Default+NA+Fleet+P-allan",
  "IsRegisteredWithLakeFormation": false,
  "CatalogId": "993562011282",
  "IsRowFilteringEnabled": false,
  "VersionId": "13",
  "CatalogIdentifier": "993562011282",
  "DatabaseId": "1mndnsasdcdbcf692c422y2732jndjbdhfd32323jnj2",
  "IsMultiDialectView": false
}

And I've data in s3 location (s3://logging-data-993562011282-us-east-1-dev/logdata/logging/timestamp=20250216001) like below

123456  cloud   \N

When I query data, I'm seeing location col value as N in Athena

But when I remove SkewedInfo from table properties, It shows empty value for location column only with new file placed into different partition. But shows N with old partitioned data.

123456  cloud   

Can anyone help me to understand why its happening ? Why its showing empty value for location col to newly generated file in new partition (same data as old partitioned) but not with the old partitioned data, after removing SkewedInfo from table properties ?

发布评论

评论列表(0)

  1. 暂无评论