Senior Data Engineer – AI Data Platform (OCI Lakehouse)

Job description

**Job Description**



**What You’ll Be Doing (Core Mission)**



+ **Design and implement** large-scale, fault-tolerant data pipelines on OCI, using services like **OCI Data Integration** , **OCI Data Flow (Apache Spark)** , **Object Storage** , and **Autonomous Database** .

+ Build and manage **streaming data architectures** using tools such as **OCI GoldenGate** , **Apache Kafka** , and **Spark/Flink Streaming** .

+ Enforce standards and automation across the **entire data lifecycle** , including schema evolution, dataset migration, and deprecation strategies.

+ Improve platform **resilience, data quality, and observability** with advanced monitoring, alerting, and automated data governance.

+ Serve as a **technical leader** , mentoring junior engineers, reviewing designs and code, and promoting engineering best practices.

+ Collaborate cross-functionally with ML engineers, platform teams, and data scientists to integrate data services with AI/ML workloads.

+ Partner in **AI pipeline enablement** , ensuring Lakehouse services efficiently support model training, feature engineering, and real-time inference.









**Required Technical Skills & Experience**



**Engineering & Infrastructure**



+ 5+ years building **distributed systems** or **production-grade data platforms** in the cloud.

+ Strong coding proficiency in **Python** , **Java** , or **Scala** , with an emphasis on performance and reliability.

+ Expertise in **SQL and PLSQL** , data modeling, and query optimization.

+ Proven experience with **cloud-native architectures** —especially **OCI** , AWS, Azure, or GCP.



**Lakehouse & Streaming Mastery**



+ Deep knowledge of **modern lakehouse/table formats** : **Apache Iceberg** , **Delta Lake** , or **Apache Hudi** .

+ Production experience with **big data compute engines** : **Spark** , **Flink** , or **Trino** .

+ Skilled in **real-time streaming** and event-driven architectures using **Kafka** , **Flink** , **GoldenGate** , or **Streaming** .

+ Experience managing **data lakes** , catalogs, and metadata governance in large-scale environments.



**AI/ML Integration**



+ Hands-on experience enabling **ML pipelines** : from data ingestion to model training and deployment.

+ Familiarity with **ML frameworks** (e.g., **PyTorch** , **XGBoost** , **scikit-learn** ).

+ Understanding of **modern ML architectures** : including **RAG** , **prompt chaining** , and **agent-based workflows** .

+ Awareness of **MLOps practices** , including model versioning, feature stores, and integration with AI pipelines.



**DevOps & Operational Excellence**



+ Deep understanding of **CI/CD** , infrastructure-as-code (IaC), and release automation using tools like **Terraform** , **GitHub Actions** , or **CloudFormation** .

+ Experience with **Docker** , **Kubernetes** , and **cloud-native container orchestration** .

+ Strong focus on **testing, documentation** , and system observability (Prometheus, Grafana, ELK stack).

+ Comfortable with **cost/performance tuning** , incident response, and data security standards (IAM, encryption, auditing).









**Preferred Qualifications**



+ Experience with **Oracle’s cloud-native tools** : **OCI Data Integration** , **Data Flow** , **Autonomous Database** , **GoldenGate** , **OCI Streaming** .

+ Experience with **query engines** like **Trino** or **Presto** , and tools like **dbt** or **Apache Airflow** .

+ Familiarity with **data cataloging** , **RBAC/ABAC** , and enterprise data governance frameworks.

+ Exposure to **vector databases** and **LLM tooling** (embeddings, vector search, prompt orchestration).

+ Solid understanding of **data warehouse design principles** , star/snowflake schemas, and ETL optimization.









**Minimum Qualifications**



+ Bachelor’s or Master’s degree in **Computer Science** , **Engineering** , or related technical field.

+ 4–6 year’s experience designing and building **cloud-based data pipelines** and **distributed systems** .

+ Proficiency in at least one core language: **Python** , **Java** , or **Scala** .

+ Familiar with lakehouse formats (Iceberg, Delta, Hudi), file formats (Parquet, ORC, Avro), and streaming platforms (Kafka, Kinesis).

+ Strong understanding of distributed systems fundamentals: **partitioning** , **replication** , **idempotency** , **consensus protocols** .









**Soft Skills & Team Expectations**



+ Proven ability to lead technical initiatives **independently end-to-end** .

+ Comfortable working in **cross-functional teams** and mentoring junior engineers.

+ Excellent **problem-solving skills** , design thinking, and attention to operational excellence.

+ Passion for **learning emerging data and AI technologies** and sharing knowledge across teams.









Career Level - IC3



**About Us**



As a world leader in cloud solutions, Oracle uses tomorrow’s technology to tackle today’s challenges.

We’ve partnered with industry-leaders in almost every sector—and continue to thrive after 40+ years of change by operating with integrity.



We know that true innovation starts when everyone is empowered to contribute.

That’s why we’re committed to growing an inclusive workforce that promotes opportunities for all.



Oracle careers open the door to global opportunities where work-life balance flourishes.

We offer competitive benefits based on parity and consistency and support our people with flexible medical, life insurance, and retirement options.

We also encourage employees to give back to their communities through our volunteer programs.



We’re committed to including people with disabilities at all stages of the employment process.

If you require accessibility assistance or accommodation for a disability at any point, let us know by emailing accommodation-request_mb@oracle.com or by calling +1 888 404 2494 in the United States.



Oracle is an Equal Employment Opportunity Employer.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability and protected veterans’ status, or any other characteristic protected by law.

Oracle will consider for employment qualified applicants with arrest and conviction records pursuant to applicable law.

Required Skill Profession

Other General