Data engineer
Want to work at the cutting edge of technology and maths? In this role, you'll use your analytical skills to solve complex problems.
Also known as: big data engineer
About the job
Career insights – Data engineer
What it's like
More than 2.5 quintillion bytes of data are created every day. As a data engineer, you’ll use or develop software to collect, manage and analyse large quantities of information. Sometimes, you’ll use Artificial Intelligence (AI) to help you do this.
Businesses can use this to find out information, make decisions or predict behaviour. This helps them give customers a better experience.
For example, websites such as YouTube or Netflix show you recommendations based on things you’ve watched before. They use AI software to tap into banks of data, created by a data engineer, to learn about your preferences and habits.
If you work with traditional data you'll use structured sets of information, for example in databases. If you work with big data it's likely to be much larger amounts of data from many varied sources, such as images, social media or shopping trends.
Your tasks might include:
creating and maintaining complex computer systems that use AI to collect data
programming software to process high volumes of important information
using AI-powered tools to design mathematical algorithms that simplify data analysis
working with data architects, data scientists and analysts
testing and evaluating data solutions
making sure the data solutions you build meet business needs or requirements
You can do this job across a variety of industries, including:
finance and financial technology
IT
healthcare
retail
manufacturing
education
sales and marketing
Explore more information about this job
Here are some useful links to learn more about this career:
FinTech Scotland – hear from financial technology professionals working in Glasgow
BCS, The Chartered Institute for IT – join the global and diverse home for digital, technical and IT professionals
GOV.UK – read about the UK Government's data science community
Other careers that you might like
-
Systems analyst -
Solution architect -
IT project analyst -
Infrastructure engineer -
Systems engineer -
Data visualisation specialist -
Internet-of-things architect -
Cloud services engineer -
Cyber security architect -
Data architect
Related industries
Many jobs can be done in lots of different industries. We've highlighted the ones we think are most important for this job.
-
Finance, law and professional services -
Energy -
Digital technologies
Top skills
Skills are things you're good at. Whether you know what yours are or not, everyone has them!
It's useful to learn which ones are important in a job so you know the areas you need to brush up on. It can also help you work out if you're suited to a career.
Meta skills
Here are some of the meta skills you'll need to do this job.
- analysing
- attention to detail
- researching
- working with numbers
- problem solving
- working with technology
Your skills are important
Our unique skillsets are what make us stand out from the crowd. Learn about each skill in depth and discover what employers look for in your applications and interviews.
Getting in
Explore each section to find more information about getting into this career.
Colleges and universities will list subjects you'll need for entry to a course. Some useful subjects include:
-
Business Management
-
Economics
-
Maths (Mathematics)
-
Statistics
-
Foundation Apprenticeship: Software Development
You can get a head start in this career by doing a Foundation Apprenticeship in S5 and S6.
You'll get an SCQF level 6 qualification which is the same level as a Higher. You'll also learn new skills and gain valuable experience in a work environment.
Discover what's on offer at your school on Apprenticeships.scot.
There are many ways to become a data engineer. You could go to college, university or get work-based qualifications, like an apprenticeship.
University
A degree may be helpful for getting into this job. You could study subjects such as:
Computing Science
Data Science
Data Engineering
Software Engineering
Artificial Intelligence
Mathematics
Postgraduate data engineering courses are available. A background in a related field such as business, quantitative science or computing is helpful when applying.
Use the My World of Work course search to find degrees.
College
Getting qualifications at college, such as an HNC or HND, could help you secure a job or progress to more advanced study. Useful subjects include:
Computing: Software Development
Computing: Data Science
Computing: Technical Support
Computing: Cyber Security
Use the My World of Work course search to find college courses.
Apprenticeships
Modern Apprenticeships (MAs) mean you learn on the job. You get paid and work towards a qualification at the same time.
You might want to consider an MA in Digital Technology.
Graduate Apprenticeships (GAs) are designed for industry. You'll spend most of your time learning on the job but you'll also go to university or college. You'll have a job, get paid and work towards a qualification at the same time.
You might want to consider a GA in areas such as:
Data Science
Cyber Security
Not all employers list specific qualifications they need. They they might ask for relevant experience, usually work-based, that demonstrates a range of transferable skills.
You'll likely need to show up-to-date knowledge and experience of software and programming developments. It's also helpful to have this for many scripting languages.
