AWS Data Wrangler 2.16.0

Noteworthy

⚠️ For platforms without PyArrow 7 support (e.g. MWAA, EMR, Glue PySpark Job):

➡️ pip install pyarrow==2 awswrangler

New Functionalities

• Add support for Oracle Database 🔥 #1259 Check out the tutorial.

Enhancements

• add test infrastructure for oracle database #1274
• revisiting S3 Select performance #1287
• migrate test infra from cdk v1 to cdk v2 #1288
• to_sql() make column names quoted identifiers to allow sql keywords #1392
• throw NoFilesFound exception on 404 #1290
• fast executemany #1299
• add precombine key to upsert method for Redshift #1304
• pass precombine to redshift.copy() #1319
• use DataFrame column names in INSERT statement for UPSERT operation #1317
• add data_source param to athena.repair_table #1324
• modify athena2quicksight datatypes to allow startswith for varchar #1332
• add TagColumnOperation to quicksight.create_athena_dataset #1342
• enable list timestream databases and tables #1345
• enable s3.to_parquet to receive "zstd" compression type #1369
• create a way to perform PartiQL queries to a Dynamo DB table #1390
• s3 proxy support with data wrangler #1361

Documentation

• be more explicit about awswrangler.s3.to_parquet overwrite behavior #1300
• fix Python Version in Readme #1302

Bug Fix

• set encoding to utf-8 when no encoding is specified when reading/writing to s3 #1257
• fix Redshift Locking Behavior #1305
• specify cfn deletion policy for sqlserver and oracle instances #1378
• to_sql() make column names quoted identifiers to allow sql keywords #1392
• fix extension dtype index handling #1333
• fix issue with redshift.to_sql() method when mode set to "upsert" and schema contains a hyphen #1360
• timestream - array cols to str #1368
• read_parquet Does Not Throw Error for Missing Column #1370

Thanks

We thank the following contributors/users for their work on this release:

@bnimam, @IldarAlmakaev, @syokoysn, @IldarAlmakaev, @thomasniebler, @maxdavidson91, @takeknock, @Sleekbobby1011, @snikolakis, @willsmith28, @malachi-constant, @cnfait, @jaidisido, @kukushking

---

P.S. The AWS Lambda Layer file (.zip) and the AWS Glue file (.whl) are available below. Just upload it and run or use them from our S3 public bucket!