Set up ingestion

After installing, the data lake is still empty — no database is marked. This is the minimum configuration for the first data to arrive.

The pipeline does not fail for lack of configuration

With no database marked, the snapshot is exported, the Parquet files land in S3 and the copy job ignores them silently. The run ends successfully and the data lake stays empty. The console overview lists what is still missing.

1. Choose the sources

Under Origens (Sources), turn on snapshot event reception and mark which clusters feed the data lake. You choose whether you want automatic snapshots, manual ones, or both.

2. Run the first snapshot

Here there is an order that surprises people: the first snapshot only serves to reveal the structure. The list of databases and tables comes from the export itself, not from a connection to the database — that is what makes credentials and network access to the source unnecessary. The price is that Data Pump only knows what exists after seeing an export.

Wait for the backup window, or take a manual snapshot so you do not have to wait.

3. Choose the databases

Under Bancos (Databases), the databases found in the export show up to be marked. Only what is marked gets ingested.

Rename in the data lake

Each marked database shows the name it will have in the data lake — click to change it. The database name is the prefix of the cataloged tables: with the alias vendas, the table payment becomes vendas.payment.

The same applies per table: open Escolher tabelas (Choose tables) and each table shows its destination name, also clickable. That is how tb_pgto becomes pagamentos.

Filter tables

The table list has two behaviors, and the difference shows up when a new table appears in the database:

Mode	The list holds	New table
Exclusion	what does not come in	is ingested on its own
Inclusion	what comes in	stays out until marked

Pick by whichever changes less: someone who wants nearly the whole database uses exclusion and does not have to touch anything when a table appears; someone who wants few tables uses inclusion and is protected from ingesting something unexpected.

4. Check

At the next snapshot, the pipeline runs in full. Under Execuções (Runs) you see each stage and which tables were updated; under Datalake, the cataloged tables, with columns and row counts.

If something did not arrive, see when something fails.

Reprocess without waiting

An export already in S3 can be re-ingested without taking a new snapshot. Under Extrações (Extractions), each export has a reprocess action — useful after changing aliases, filters or partitions.