Senior Data Engineer

Responsibilities

In this role, you will be expected to:

• Recognise and share opportunities to re-use existing data flows between their teams
• Be responsible for the build of data-streaming systems
• Coordinate teams and set best practice and standards
• Apply knowledge of systems integration to their work
• Champion data engineering across the business

Skillset Required (or demonstration of similar skills that can be transferred):

• A minimum of five years experience of effective business solution delivery using the Microsoft BI toolset
  • Azure data Platform
  • Azure Data Factory
  • Azure Synapse
  • Power BI
  • SSAS
  • SSIS
  • SSRS
• Communication skills - You can listen to the needs of technical and business stakeholders and interpret them. You know how to manage stakeholder expectations. You are flexible and capable of proactive and reactive communication. You know how to facilitate difficult discussions within the team or with diverse senior stakeholders. (Relevant skill level: practitioner)
• Data analysis and synthesis - You understand and can help teams apply a range of techniques for data profiling. You can source system analysis from a complex single source. You can bring multiple data sources together in a conformed model for analysis. (Relevant skill level: practitioner)
• Data development process - You know how to establish enterprise-scale data integration procedures across the data development life cycle and ensure that teams adhere to these. You can manage resources to ensure that data services work effectively at an enterprise level. (Relevant skill level: practitioner)
• Data innovation - You can identify areas of innovation in data tools and techniques and recognise appropriate timing for adoption. (Relevant skill level: working)
• Data integration design - You know how to establish standards and you know how to keep those standards up to date and ensure adherence to them. You keep abreast of best practice in industry. (Relevant skill level: practitioner)
• Data modelling - You understand the concepts and principles of data modelling and can produce relevant data models. You know how to work across industry, recognising opportunities for re-use and alignment between the data models in different organisations. You can design the method to categorise data model within the organisation. (Relevant skill level: practitioner)
• Metadata management - You can design an appropriate metadata repository and present changes to existing metadata repositories. You understand a range of tools for storing and working with metadata. You can provide oversight and advice to more inexperienced members of the team. (Relevant skill level: practitioner)
• Problem resolution (data) - You can ensure that the most appropriate actions are taken to resolve problems as they occur. You know how to coordinates teams to resolve problems and implement solutions and preventative measures. (Relevant skill level: working)
• Programming and build (data engineering) - You know how to use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations. You can collaborate with others to review specifications where appropriate. (Relevant skill level: practitioner)
• Technical understanding (data engineering) - You have the detailed knowledge and understanding of the technical concepts required for the role. You know how these fit into the wider technical landscape. (Relevant skill level: practitioner)
• Testing - You can review requirements, specifications and define test conditions. You can identify issues and risks associated with work while being able to analyse and report test activities and results. (Relevant skill level: working)
• Research emerging technologies and trends to ensure the best possible outcomes are achieved.
• Evaluate data management architectural options in terms of cost, benefits, risk and systems fit in order to provide recommendations. Required essential experience skills and qualifications
• Highly technical and analytical, possessing 10 or more years of Database and/or Analytics Systems development and deployment experience, IT systems and engineering experience, security and compliance experience, etc.
• Possess significant experience of software development and/or IT and implementation/consulting experience.
• Hands on experience as a database, data warehouse, big data/analytics developer or administrator.
• Implementation and tuning of Azure Data Lake, Azure SQL DB, Azure SQL DW, Cosmos DB, Oracle, Postgres or SQL Server databases at scale, including transaction processing and analytics. Experience of advanced deployment architectures such as read replicas and warm standby preferred.
• Experience of data migration techniques and process planning.
• Experience of working with complex and high-performance stored procedures.
• Experience tuning databases including query tuning, performance optimization, and diagnostics gathering. Knowledge of foundation infrastructure requirements such as Networking, Storage, and Hardware optimization.
• Experience working in both agile and more traditional delivery methodologies with respect to significant data management developments.
• Broad IT knowledge and experience within a range of project types, size and complexity.
• Strong stakeholder management skills and experience in dealing with a wide range of stakeholders with a wide range of seniority.
• Ability to build strategic roadmaps, considering a wide range of influences and constraints.
• Ability to represent complex concepts and ideas in various contexts: logical, physical, conceptual.
• To design, define and document solutions, conducting technical requirements meetings, workshops, conference calls, discussion forums and related meetings
• To estimate time allowance against prospect projects
• Managing and meeting project milestones
• Adherence to development policies, processes and internal systems update

• BS level technical degree required; Computer Science or Mathematics background preferred.

Desirable Skills

• Working knowledge of Dynamics AX
• Working knowledge of MS CRM
• Knowledge of Azure Databricks
• Snowflake
• ASP.Net development
• Python