最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

amazon web services - Drophide fields from s3 based data lake consumable Athena table - Stack Overflow

programmeradmin3浏览0评论

I have a data lake implemented using AWS s3. Bronze and silver layer are implemented, with data in silver layer exposed for access via API and also via jdbc/odbc based sql client.

We have a requirement to drop/hide a field from the consumable Athena table, so far below is the option I've known and related limitations:

Option 1: Drop data for the table from silver layer and reprocess with glue script updated to drop the field

Limitation: costly, need to reprocess huge volume

Athena doesn't support dropping field, and using views every time crawler updates the silver table catalogue gets stale.

Could anyone please suggest whats a proper way handling this?

发布评论

评论列表(0)

  1. 暂无评论