Mid Data Engineer
Islamabad,PK
VisionX – Customer Products /
Full Time /
On-site
About us:
VisionX works with world-leading brands, Fortune 1000 as their innovation partner, providing product strategy and custom application development leveraging agile methodologies, technology accelerators, and by creating Intellectual Property.VisionX has been listed in the Top 10 Most Innovative Companies of 2020 by Fast Company – ranked alongside the likes of Microsoft & Snap Inc.We develop cutting-edge software products integrating computer vision, 3D modeling, AR, VR, decision sciences, and IoT addressing a wide variety of use cases across different industries.
Your role
We are looking for talented and experienced data engineers to join our growing Data & Analytics team and help us develop high-quality Data Lakehouse products and solutions for our company. As a Senior Data Engineer, you combine solid data engineering and software development skills, data modelling, integration knowledge with your business understanding, and strong analytical skills that enable you to deliver the expected benefits to the business. You will play a key role in the Data engineering team, interfacing with the Data Architect, Governance, Data Science Team, AI Team, Reporting and Business partnering teams.
Responsibilities
· Understand business requirements, identify and apply appropriate Data and Analytics tools and technologies to solve problems, and participate in the architecture and execution of technology solutions.
· You will help evolve, build, and maintain data pipelines for multiple data sources and interface with system owners.
· Design and develop data models, and ETL/ELT workflows for Data Lake House products/solutions. Manage the entire data flow, from acquisition to transformation into actionable insights using SQL, Python, and other ETL technologies AWS Kinesis, Kafka, AWS Glue, AWS EMR, etc.
· Design and implement scalable ETL solutions for structured semi structured and unstructured data, including batch and streaming processes.
· Build data pipelines, and standardize and cleanse data for AI, Data Science, and Analytics use cases to support business needs.
· Identify and apply the appropriate data integration, cleansing, quality, and master data management tools and technologies to solve problems.
· Establish and enforce best practice standards for data, such as data modeling, database design, ETL design, job scheduling, and monitoring.
· Define data engineering and governance standards, guidelines, and best practices for both business groups and technical teams.
· Drive the collection of new data and the refinement of existing data sources to continually enhance data quality by streamlining data flow and data quality processes to improve data accuracy.
· Work independently as well as collaboratively with a team and learn from colleagues. Collaborate closely with team members, providing subject matter expertise and solutions to the most challenging software development tasks.
What you need
· Minimum of bachelor’s degree in computer science, Information Technology, or a related field.
· Possess a deep passion for data and a proven ability to create high-quality data products.
· Bring at least 8+ years of data engineering experience in implementing Big Data, Data Lakehouse, and Data Warehouse solutions for banking clients.
· Have over 8+ years of hands-on experience building production data pipelines and working with structured, semi-structured, and unstructured data.
· Accumulate 6+ years of experience in engineering data solutions within both cloud and hybrid infrastructures, with strong expertise in AWS cloud.
· Extensive experience with the AWS data and AI technology stack and developing and deploying scalable tools and services to handle large data sets, ensuring the highest quality for feeding large ML models, such as LLMs.
· Experienced in identifying and evaluating new technologies to enhance the performance, maintainability, and reliability of data pipelines.
· Apply software engineering rigor and best practices to data engineering, including automation.
· Hands-on experience with data integration/ETL technologies such as Python, Spark/PySpark, Kafka/Kinesis/EMR, Lambda, API Gateway, REST API, AWS Glue, etc.
· Proficient in database technologies like AWS S3, MongoDB, DynamoDB, AWS Redshift, Glacier, and other SQL and NoSQL databases.
· Experienced with CI/CD pipelines and tools such as Jenkins, GitLab CI, or Azure DevOps.
· Deep architectural understanding of relational and non-relational systems, especially with a focus on massively parallel processing database technologies and on-premises, hybrid cloud technologies in IaaS/PaaS/DBaaS.
· Consult on optimal design of database environments, performance improvements, analyzing complex distributed production deployments, data security, and best practices for cloud adoption.
· Extensive experience in database design, ETL/ELT process development, Data Lakehouse performance tuning, and storage optimization.
· Strong understanding of modern cloud platforms (AWS/GCP/Azure) and various cloud services.
· Handson experience in different APIs and API integration, along with microservice architecture.
· Experience in integrating customer contact center data with Lakehouse is a plus.
· Strong analytical, logical, problem-solving and numerical skills.
· A self-starter and creative thinker who collaborates well and communicates confidently, driven by a passion for excellence.
· Capable of handling multiple assignments/projects simultaneously, prioritizing workloads to meet competing deadlines.
· Adept at building strong relationships that foster robust debate and efficiently resolve periodic disagreements about priorities.
· Effective in communicating with both technical and non-technical audiences.
· Proven ability to take initiative and proactively learn new technologies.
· Demonstrates a solid work ethic and provides timely, high-quality support.
· Embraces a mindset of Continuous Improvement.
Why choose us
Our global network of industry experts and mentors helps shape your growth and future. We believe in delivering client value through our work. We build products that are not good or great, but outstanding.You deliver! We will make your stay and journey with us worthwhile.
We are an equal opportunity employer, and we value diversity. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability status, or any other legally protected status.
