Intro

What is data science engineering?

Data science engineering is the practice of applying the principles of software engineering to the products of data science. These principles apply end-to-end, throughout the entire data science lifecycle. This includes:

• formulating hypotheses and data collection (see Data)
• building environments and scaling compute (see Environments)
• modelling and experiments (see Models)
• deployment and integration (see Integrations)
• intrumentation, monitoring and maintenance (see Management)

How is it different to...

...data science?

Data science is the production of new knowledge from data, typically in the form of a model. Data science engineering serves to enable and accelerate data science with software engineering principles.

...DevOps?

DevOps is a philosophy and evolving practices for automating traditionally manual technology operations with further software development. Data science engineering seeks to apply the same philosophies to traditionally manual analytical operations.

When do I need it?

If you are new to data science

One of the most common mistakes that organisations make when setting up a data science capability for the first time is hiring a data scientist and expecting them to get to work delvering valuable projects from day one. Use the practices of data science engineering to set up your new hires for success.

If you have embedded data scientists

Many organisations hire data scientists directly into product or engineering teams to work directly alongside others key to the product development process. Use the practices of data science engineering to improve collaboration between product managers, data scientists and engineers, release faster and manage quality risks.

If you have a centralised data science team

Data science labs enable data scientists to mobilise quickly on new opportunities. They benefit from greater enablement but can struggle to integrate into complex engineering teams and may not always align to product strategies. Use the practices of data science engineering to position your lab as an integral part of your organisation.