Data Engineer

Data Engineer delivers the hands-on work summarized here and you can explore 20 real tasks. Profiles show time makeup and who benefits directly from each action. Each one shows where we found it, and comes with an AI prompt you can copy and use straight away.

20evidenced tasks
20ready prompts
9tools of the trade
15-1243.00O*NET-SOC code
67,140hold this job (US, BLS 2025)
$139,500median pay/yr (US)
Open Data Engineer in the interactive atlas →

What it pays

Government survey numbers — not estimates, not ads.

Half of all Database Architects in the U.S. earn more than $139,500 a year — the middle 80% land between $86,240 and $204,000. About 67,140 people in the U.S. do this work. Figures are for the U.S. occupation group “Database Architects”. (U.S. Bureau of Labor Statistics survey, published 2025.) In India, Professionals earn about ₹38,298 a month on average — around ₹4.6 lakh a year (government PLFS survey via ILOSTAT, occupation-family figure).
$139,500typical pay / year
67,140people in this work
$204,000+top 10% earn
₹4.6 lakha year in India (family avg)
Think you get this job?Six quick questions on how it really works — with a hint and the reason behind every answer.
Test yourself →

The work, task by task

These are the real jobs-to-be-done, not a wish list. Each task shows where we found it, and the prompt underneath is written for that exact task.

Keeping the record5

Develop and test database modifications

+
Apply the schema changes to the staging user_activity and orders tables, run the migration tests that…
Apply the schema changes to the staging user_activity and orders tables, run the migration tests that validate foreign keys and nullability, fix any failing queries, then schedule the same migration for the weekend maintenance window if tests pass.
The tools that do the workApache Hivejob descriptionsO*NET

Implement data backup and recovery procedures

+
Document and implement nightly snapshot and point-in-time restore procedures for the production analytics…
Document and implement nightly snapshot and point-in-time restore procedures for the production analytics cluster, run a full restore on the recent backup to a dev instance, validate row counts and critical reports, and hand over the runbook to oncall for first response.
The tools that do the workAmazon Web Services AWSjob descriptionsO*NET

Create and manage data models

+
Design canonical entity models for customer, product, and transaction, produce the physical table definitions…
Design canonical entity models for customer, product, and transaction, produce the physical table definitions with primary and surrogate keys, generate sample datasets, and run integration queries to confirm analytics and ETL jobs still produce expected totals.
The tools that do the workApache Cassandrajob descriptionsO*NET

Create data models and schemas

+
Create normalized entity-relationship models for order, inventory, and telemetry tables, produce the DDL for…
Create normalized entity-relationship models for order, inventory, and telemetry tables, produce the DDL for the canonical schemas, and publish sample datasets and field definitions to the analytics team before the architecture review on Monday.
The tools that do the workApache HiveApache AirflowESCOjob descriptionsO*NETWikipedia

Optimize database performance

+
Profile the sales and telemetry databases during peak hours, identify the top three slow queries, propose…
Profile the sales and telemetry databases during peak hours, identify the top three slow queries, propose index or partition changes with estimated impact, and prepare a rollback-safe deployment plan for Thursday maintenance window.
The tools that do the workAmazon RedshiftApache Hadoopjob descriptionsO*NET
Analysing1

Express strategic data requirements

+
Draft the strategic data requirements for the next fiscal product line: required source systems, record…
Draft the strategic data requirements for the next fiscal product line: required source systems, record retention, SLAs, key entities and ownership, and the downstream consumers, then circulate to product, BI, and compliance for sign-off by end of month.
The tools that do the workAmazon Web Services AWS CloudFormationAmazon Redshiftjob descriptionsWikipedia
The daily work14

Provide technical guidance to team members

+
Give the team a concise playbook this afternoon that defines our data exchange standards, preferred ingestion…
Give the team a concise playbook this afternoon that defines our data exchange standards, preferred ingestion patterns, code review checklist, and two example API contracts so Priya and Marcos can implement it next sprint.
The tools that do the workApache Airflowjob descriptionsO*NET

Define how the data will be stored

+
Decide the storage layout for the new customer analytics pipeline: raw event lake for incoming JSON,…
Decide the storage layout for the new customer analytics pipeline: raw event lake for incoming JSON, partitioned columnar tables for nightly aggregates, and a hot lookup store for user profiles so Martin can build the ETL by Wednesday.
The tools that do the workApache HadoopApache Hivejob descriptionsWikipedia

Review project requests describing database user needs to estimate time and cost required to accomplish project.

+
Review the three incoming project requests, list required schema changes, estimate development and testing…
Review the three incoming project requests, list required schema changes, estimate development and testing hours for each, and return a prioritized cost/time table to the product owners by Friday.
The tools that do the workApache HiveO*NET

Establish and calculate optimum values for database parameters, using manuals and calculators.

+
Calculate optimal DB parameters for the OLAP cluster using vendor manuals and our workload samples: compute…
Calculate optimal DB parameters for the OLAP cluster using vendor manuals and our workload samples: compute target memory, compaction thresholds, and parallelism settings, then document recommended values and expected impact.
The tools that do the workAmazon RedshiftO*NET

Train users and answer questions.

+
Run a one-hour training for the analysts covering the new data model, how to query the aggregates, common…
Run a one-hour training for the analysts covering the new data model, how to query the aggregates, common gotchas, and open an FAQ doc so I can answer follow-up questions during the week.
The tools that do the workAJAXO*NET

Ensure data integrity and security

+
Audit the customer data pipeline for integrity and security: validate checksums, review access controls,…
Audit the customer data pipeline for integrity and security: validate checksums, review access controls, confirm encryption in transit and at rest, and submit a remediation list to security by Monday.
The tools that do the workAmazon Web Services AWSjob descriptions

Troubleshoot database issues

+
Investigate the slow queries on the payments reporting database: gather recent error logs, check replication…
Investigate the slow queries on the payments reporting database: gather recent error logs, check replication lag and index usage, reproduce the failure on staging, and document root cause and remediation steps by end of day for Sarah in SRE.
The tools that do the workAmazon Redshiftjob descriptions

Collaborate with software developers and analysts

+
Run a one-hour sync with the backend team and analytics group to align data contracts: review the API…
Run a one-hour sync with the backend team and analytics group to align data contracts: review the API payloads, agree field types and nullability, record decisions in the shared spec, and assign follow-ups to Alex and Priya.
The tools that do the workApache Airflowjob descriptions

Create an inventory of the data needed

+
Inventory the data needed for the Q3 churn model: list source tables, owner, update frequency, cardinality,…
Inventory the data needed for the Q3 churn model: list source tables, owner, update frequency, cardinality, sample rows, and any PII concerns, then hand the spreadsheet to Maya in ML before Thursday.
The tools that do the workApache HiveWikipedia

Provide a standard common business vocabulary

+
Draft a common business vocabulary for customer records: define canonical field names, descriptions, allowed…
Draft a common business vocabulary for customer records: define canonical field names, descriptions, allowed values and examples for customer_id, account_status, and churn_flag, circulate to product and BI for review on Monday.
The tools that do the workApache CassandraWikipedia

Manage standards for data exchange

+
Create and publish the data exchange standards: specify JSON schemas, versioning rules, error codes, and…
Create and publish the data exchange standards: specify JSON schemas, versioning rules, error codes, and TLS/auth expectations for partner integrations, then notify the integration team and legal for sign-off.
The tools that do the workAJAXESCOsee the evidence ↗

Web programming

+
Implement the customer portal's backend endpoints: build the REST endpoints, validate input, connect to the…
Implement the customer portal's backend endpoints: build the REST endpoints, validate input, connect to the canonical customer dataset, add unit tests, and open a pull request for review by Noor by Wednesday.
The tools that do the workAmazon Elastic Compute Cloud EC2ESCOsee the evidence ↗

Perform data analysis

+
Analyze the last quarter's ingestion logs and the sales and customer tables to identify missing joins,…
Analyze the last quarter's ingestion logs and the sales and customer tables to identify missing joins, outlier records, and schema mismatches, produce a reproducible notebook with queries and visuals, and summarize findings for Emma and Raj by Wednesday.
The tools that do the workApache HiveApache HadoopESCOsee the evidence ↗

Create data sets

+
Assemble a clean, documented dataset of customer transactions for the marketing team: join raw events to the…
Assemble a clean, documented dataset of customer transactions for the marketing team: join raw events to the master customer table, standardise fields, apply the agreed exchange schema, remove duplicates and flag missing PII, then export parquet and notify Lina when ready.
The tools that do the workApache HiveApache HadoopESCOsee the evidence ↗

Says who?

These are the pages we read to build this. Open any of them and check us.

The logs, files & records this job keeps

Shared with other careers — the same record means something different in each.

Related careers

Same family of work — each with its own tasks and prompts.

The LLOS Work Atlas is the world's largest evidenced task library — a map of human work, with a ready prompt behind every task. 1,774 careers · every task named by the sources that witnessed it — O*NET, ESCO, real job descriptions, Wikipedia — and the deepest tasks by several at once. And it is honest about limits: where AI cannot help, the map says so.

The rest of the map

Same library, five ways in.

Copyright © LLOS.ai · 2026 — original pedagogy, voice, and design — all rights reserved.
Built on public evidence: O*NET®, ESCO, Wikipedia, U.S. Bureau of Labor Statistics, ILOSTAT. All sources & licenses