Run long running fault-tolerant SQL queries with Trino and Amazon EMR with checkpointing on Amazon S3 or HDFS
Today, Amazon EMR has announced support for long running fault-tolerant SQL queries on Trino engine (Project Tardigrade) with checkpointing in Amazon S3 or HDFS for fault-tolerance. Project Tardigrade aims to improve the user experience of long running, resource intensive queries on Trino, when used for ETL style workloads. Project Tardigrade uses Amazon S3 for checkpointing buffered intermediate data. With Amazon EMR 6.9 release, we are also adding checkpointing on HDFS for performance sensitive and long running SQL workloads.