Find Jobs
Hire Freelancers

Create Production Ready ETL Pipeline using Python, Docker and Dagster

₹12500-37500 INR

Closed
Posted over 2 years ago

₹12500-37500 INR

Paid on delivery
We need to create a dockerizable ETL Pipeline for our data using Python and Dagster. Our requirement / data flowchart is very clear. The basic flow is read data from CSV, save to Kafka. Apply multiple transforms using functions (function code will be provided by us). Use intermediate kafka queues after each transform. Write final output to Postgres DB. Use Dagster for the data orchestration. We just need to implement it in a robust manner. We need :- 1. Docker-compose to setup the Kafka, Postgres and other services. 2. Create a DAG for orchestration of the pipeline using Dagster (using placeholders for the transform functions which our team will add later).
Project ID: 32203643

About the project

2 proposals
Remote project
Active 2 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
2 freelancers are bidding on average ₹27,500 INR for this job
User Avatar
** Can start immediately once you hire me. Hello, I am an experienced mobile app developer with an experience of 5+ years. I’m pretty confident that I will be able to create your app as required. I am an expert in making android iOS as well as web apps with remarkable reviews on my profile. I have knowledge about: Objective C, Swift, Xcode, Script, Android studio, Sqlite, cloud services, firebase, in-app purchase, payment gateways, Google API integration, third-party API integration, UI, UX, PHP, SIP, MNV, MVVC, CBD modeling, XML, Angular JS, Jquery, SQL Server, HTML 5, CSS3, JAVA, API, Backend Admin panel, Cocoa-pods, SDK, NDK, Core and many more. Available for immediate discussion, Let's proceed and work together. Please share more details about your project. Thanks,
₹20,000 INR in 5 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of INDIA
New Delhi, India
0.0
0
Payment method verified
Member since Feb 11, 2015

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759) & Freelancer Online India Private Limited (CIN U93000HR2011FTC043854)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.