DataOps Engineer

A DataOps engineer helps an organization operationalize its data by creating theenvironment and processes needed to efficiently manage data and derive value from analytics. Managing backend assets ownership and accountability over configuration andspinning up of cloud data assets and pipelines are primarily responsibilities of DataOps Engineer. DataOps Engineer should be very qualified with experience in managing various flavors of Data assets (Postgres, Snowflake, GCP based DBs and pipelines). This includes various day-to-day activities, from reducing development time and improving data quality toproviding guidance and support to data engineers. Bachelor s Degree in Computer Science or equivalent work experience 5 years of Data Ops Experience Hands on experience working with Postgres, Snowflake administration and GoogleCloud Platform. Hands on experience in setting up and managing CICD pipelines on Azure DevOps. Expert in SQL including performance tuning. Excellent team skills and able to work in fast-paced environment on multiple projects/jobs concurrently. Database uptime, performance monitoring and improvement This includes various day-to-day activities, from reducing development time and improving data quality to providing guidance and support to data team members. The responsibilities of a DataOps engineer include Building and optimizing data pipelines to facilitate the extraction of data from multiplesources and load it into data warehouses. A DataOps engineer must be familiar with extract, load, transform (ELT) and extract, transform, load (ETL) tools. Using automation to streamline data processing. To reduce development time andincrease data reliability, DataOps engineers automate manual processes, such as data extraction and testing. Managing the production of data pipelines. A DataOps engineer providesorganizations with access to structured datasets and analytics they will further analyze and derive insights from. Designing data engineering assets. This involves developing frameworks to support an organization s data demands. Facilitating collaboration. DataOps engineers communicate and collaborate with other data and BI team members to enhance the quality of data products. Testing. This involves executing automated testing at every stage of a pipeline to increase productivity while reducing errors. This includes unit tests (testing separate components of a data pipeline) as well as performance tests (testing the responsiveness) and end-to-end tests (testing the whole pipeline). Adopting new solutions. This includes testing and adopting solutions and tools thatadhere to the DataOps best practices. Handling security. DataOps engineers ensure data security standards are appliedacross the data pipelines. Reducing waste and improving data flow. This involves continually striving to reducewasted effort, identify gaps and correct them, and improve data development and deployment processes.