Install

Why the package is specific

The CloudFormation template already comes with the account, the region, the VPC, the environment and the source accounts resolved. There is nothing to choose at install time.

This is deliberate, not a limitation. Those decisions change which resources the template declares — a data lake on an existing bucket declares different resources than one that creates the bucket; a cross-account source adds policies that a single-account installation does not have. Offering those options in the installer would create choices with no effect, or worse: inconsistent resources that would only fail later.

The installer itself refuses to run in the wrong place: it compares your credentials against the account and region of the package and stops if they do not match.

The handoff

Before delivery, we define with you:

What	Why it changes the package
Account and region	Recorded in the template and in the manifest
VPC and subnets	Use an existing VPC or create one; defines where the jobs run
Data lake bucket	Reuse a bucket of yours or create a new one
Environment	`prd`, `hom` — goes into the name of every resource
Catalog database	Database name in Glue (default: `datalake`)
Source accounts	Each cross-account account adds permissions to the template
Version	You choose; earlier releases remain available

You get back a package with the template, the manifest and the scripts — specific to that scenario and to that account.

CO2 Lab never receives access to your account. You are the one running the scripts, with your own credentials.

When to ask for a new package

Some changes are not configurable after installation, because they alter the template itself:

Integrating a new source account — its permissions have to exist in the template
Changing the VPC or the data lake bucket
Moving to a new version

In those cases, talk to us and install the new package on top — CloudFormation resolves the difference.

1. Install the stack

Unpack the package and run, from inside it:

sh install-datapump.sh

The script checks that your credentials belong to the account and region of the package — applying it elsewhere would create inconsistent resources that would only fail later. If everything checks out, it creates (or updates) the stack.

To move to a new version, run the same command with the new package. CloudFormation resolves the difference.

sh post-install.sh

The console address only exists after the load balancer is created, and CloudFormation does not allow the circular dependency between it and Cognito. This script closes the loop.

Rerunning is safe — it overwrites the URLs with the current values.

3. Enable HTTPS

sh setup-https.sh

Cognito refuses callback URLs on http, except for localhost. Without HTTPS, login does not work.

Without your own domain, the script generates a self-signed certificate and imports it into ACM. The browser will warn that the certificate is not trusted — that is the cost of not having a domain. To remove the warning, use your own certificate:

CERTIFICATE_ARN=arn:aws:acm:... sh setup-https.sh

4. Create the first user

The console authenticates through Cognito. Create the first user from the AWS console, in the datapump-* user pool. After that, user management is done in Data Pump itself, under Acessos (Access).

5. Get in

The console sits behind an internal load balancer — it is not exposed to the internet. You reach it from inside the VPC, over VPN, or through a Session Manager tunnel.

The address is in the stack outputs:

aws cloudformation describe-stacks --stack-name DatapumpStack \
  --query "Stacks[0].Outputs[?OutputKey=='DatapumpConsoleUrl'].OutputValue" \
  --output text

And then

The installation creates the infrastructure, but the data lake is still empty: no database is marked for ingestion. Move on to set up ingestion.