Data Pump
Data Pump reads the snapshots AWS already takes of your RDS and builds a data lake you can query in Athena. Nothing connects to the production database — not even to discover which tables exist.
The problem it solves
Section titled “The problem it solves”Analyzing data from a production Postgres usually forces a bad choice: run analytical queries on the database that serves the application, maintain a replica, or install CDC. Each one charges something — contention, database configuration, an agent to maintain, a network route that becomes a security exception.
The snapshot already exists. AWS takes it every day, and the backup cost is already on the bill. Data Pump starts from there.
What you get
Section titled “What you get”A pipeline that fires on its own when a snapshot is ready, and a console to operate everything — both running in your AWS account. Your data never passes through CO2 Lab.
| Source | Automatic or manual snapshots, from your own account or another |
| Format | Parquet, cataloged in Glue |
| Query | Athena, or anything that reads the Glue Data Catalog |
| Operation | Web console, in your VPC, behind a login |
How it is installed
Section titled “How it is installed”Data Pump is licensed per installation and has no generic installer. Every installation starts with a handoff with CO2 Lab, where we define the account, the region, the VPC, the bucket and the source accounts — and you receive a package built for that scenario.
The reason is that those decisions change which resources the template declares, so they cannot be made at install time. See install for what gets defined in the handoff.
It is not open source and it is not on the AWS Marketplace. To purchase: contato@co2lab.io.
Where to start
Section titled “Where to start”- How it works — the path of the data, from snapshot to SELECT
- Requirements — what must exist in the account before the handoff
- Install — the handoff and the step by step
If the source database lives in another AWS account, see cross-account database — that has to be defined in the handoff, because it adds permissions to the template.