Job Description:
We’re looking for a Data Engineer with strong Python and SQL skills to build reliable data pipelines that convert semi structured data from ES URLs into clean| analytics ready datasets. You’ll work locally (Python DBeaver SQLite Postgres Dremio)| establish DB connections| flatten JSONES topics into structured tables| and enable downstream reporting in Power BI for business users.
Key Responsibilities
Data Ingestion Transformation Fetch semi structured data from ES URLs Apis (e.g.,| JSON Elasticsearch topics).Flatten| normalize| and model data into structured tables for analytics.
Build reproducible ETLELT workflows in Python (pandas| requests| SQLAlchemy).
Database EngineeringCreate and maintain schemas in SQLite| Postgres| and Dremio.
Configure and manage local DB connections via DBeaver optimize queries and indexes.
Implement data partitioning| incremental loads| and performance tuning.
Data Quality Governance
Establish validation rules| deduplication| and anomaly checks.
Version datasets| maintain data lineage| and document data contract metadata.
Ensure secure handling of credentials| tokens| and endpoints.
Use Git for version control maintain code reviews| unit tests| and CI checks.
Write clear technical documentation and runbooks support adhoc data requests.
Required Skills
Experience Python for Data Science Engineering pandas| NumPy| requests| SQL Alchemy JSON handling and API integration.
SQL advanced proficiency with SQLite| Postgres| and querying via Dremio.
Data Modeling dimensional normalized models handling nested semi structured data.
Tools DBeaver (DB connections)| Power BI (data prep for reporting).Pipelines ETLELT design| performance optimization| error handling| logging.
Collaboration strong communication with business stakeholders’ requirements to delivery
Skills: Digital : Python~PostgreSQL~Digital : Microsoft Power BI
Experience Required: 8-10
No map available.