Add generate_create_query to Athena for generating tables' and views' DDL query

[KhueNgocDang]

Feature or Bugfix

• Feature

Detail

• Add generate_create_query for the Athena module to generate Athena View or Athena Table DDL query.
  • The function uses the Glue client instead of Athena's to generate the DDL.
  • Athena's SHOW CREATE TABLE returns faulty queries if special characters exist in the create query. Generate the create queries through generate_create_query will keep everything intact.

Initial create query

CREATE EXTERNAL TABLE `awswrangler_test_comment`(
  `random_japanese_string` string COMMENT 'らんじゅうにほんご（特殊ブラケット）',
  `random_russian_string` string COMMENT 'случайная русская строка',
  `random_chinese_string` string COMMENT '随机中文字符串',
  `random_vietnamese_string` string COMMENT 'chuỗi tiếng việt ngẫu nhiên')
PARTITIONED BY (
  `par0` bigint,
  `par1` string)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  's3://edgar-khuedn/test_wr/'
TBLPROPERTIES (
  'classification'='parquet',
  'compressionType'='snappy',
  'projection.enabled'='false',
  'typeOfData'='file')

Results from SHOW CREATE TABLE awswrangler_test_comment;

Time in queue:
305 ms
Run time:
732 ms
Data scanned:
-
CREATE EXTERNAL TABLE `awswrangler_test_comment`(
  `random_japanese_string` string COMMENT '��X�Fk{�T�y�����
  `random_russian_string` string COMMENT 'A;CG09=0O @CAA:0O AB@>:0', 
  `random_chinese_string` string COMMENT '�:-�W&2', 
  `random_vietnamese_string` string COMMENT 'chu�i ti�ng vi�t ng�u nhi�n')
PARTITIONED BY ( 
  `par0` bigint, 
  `par1` string)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  's3://edgar-khuedn/test_wr'
TBLPROPERTIES (
  'classification'='parquet', 
  'compressionType'='snappy', 
  'projection.enabled'='false', 
  'transient_lastDdlTime'='1660516666', 
  'typeOfData'='file')

Results from generate_create_query

>>> import awswrangler as wr
>>> query: str = wr.athena.generate_create_query(table="awswrangler_test_comment",database="mart")
>>> print(query)
CREATE EXTERNAL TABLE `awswrangler_test_comment`(
  `random_japanese_string` string COMMENT 'らんじゅうにほんご（特殊ブラケット）', 
  `random_russian_string` string COMMENT 'случайная русская строка', 
  `random_chinese_string` string COMMENT '随机中文字符串', 
  `random_vietnamese_string` string COMMENT 'chuỗi tiếng việt ngẫu nhiên')
PARTITIONED BY ( 
  `par0` bigint, 
  `par1` string)
ROW FORMAT SERDE 
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION
  's3://edgar-khuedn/test_wr'
TBLPROPERTIES (
  'classification'='parquet', 
  'compressionType'='snappy', 
  'projection.enabled'='false', 
  'transient_lastDdlTime'='1660516666', 
  'typeOfData'='file')

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[malachi-constant]

AWS CodeBuild CI Report

• CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
• Commit ID: 761198e
• Result: FAILED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[malachi-constant]

AWS CodeBuild CI Report

• CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
• Commit ID: 2a6331c
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[jaidisido]

LGTM

[malachi-constant]

AWS CodeBuild CI Report

• CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
• Commit ID: 3701c36
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository