How to Build a Data Team

TL;DR To build a data team, you would need a business analyst, a data analyst, a full stack developer, a data engineer, and maybe an ML engineer.

Have been asked this question several times, I thought about writing about it. Now, this is essentials on how to hire and build an in-house data team for your business, whether you have a product, or you do projects.

Photo by Kaleidico on Unsplash

There are several types of data teams if we consider the terminology, but for this post, I am concerned about teams that build an artefact, whether it’s a product, a feature, or a dashboard or anything in between using data, and technologies like machine learning, data analytics, etc.

1. Allocate a Business Analyst

The primary step when building a product team is to agree on the high level objectives. If you are not sure about what you want, but instead want to see where it goes, the future might be a little bleak, especially if you do not have expert knowledge.

This is where a business analyst may join. Good things is, you do not have to hire a business analyst, this can be allocated from an internal product team, and does not have to be a full time assignment initially.

Responsibilities: Talk with the stakeholders and come up with a list of objectives that is needed out of the investment. This can be a revenue increase, a cost reduction , improving customer experience, etc. This has to be in context with the domain your business/your customers are operating in.

Skills: Requirement gathering, research, analytical thinking

Education: An undergraduate or similar education in the domain (e.g. marketing for a MarTech company) or business

2. Hire a Data Analyst

The first hire I would recommend externally is to hire a data analyst. Coming in different flavours, the data analyst would be able to identify the data that is available, the data that is needed additionally, checking the feasibility of the objective with the available data, etc.

Responsibilities: Analyse the existing data, and identify the data gap to achieve the objective. Then from a sample of data, carry out preliminary analysis and build up some dashboards or some other data products to communicate the findings to the stakeholders and get feedback.

Skills: Data analytics (obviously), statistical models, dashboarding, information flow mapping, optional — machine learning

Education: Undergraduate in the domain or statistics (recommended), supplementary education in domain or statistics (opposite to their undergraduate education) and knowledge in few technologies that will be important for the job:

• BI tools: Power BI or Tableau or Data Studio
• Scripting language: Python or R
• SQL
• Optional — Web development: HTML/CSS, JavaScript (with D3.js)

3. Hire a Full Stack Developer

If your data analyst does not have sufficient knowledge in web development technologies, and you want to build quality web apps, mobile apps that can be given to the customer, I would recommend you to hire a full stack developer.

Responsibilities: Building web apps, mobile apps, to be delivered to the customer.

Skills: Programming

Education: Undergraduate or similar education in web development. Should be familiar with at least one web development stack:

• Front-End: Angular or React or Vue (or similar front-end framework)
• Back-End: Node.js (with Express)
• Database: SQL/MongoDB
• Optional — Mobile: Android and iOS development or PWA

4. Hire a Data Engineer

Now, hiring a data engineer is an additional step for a data team, and is needed only if you are building up production level data products that uses high volume or continuous streams of data. If you are only doing small scale use cases, you would not need a data engineer.

Responsibilities: Building up the data pipelines from multiple sources for the analytical engine of the product. This may include ETL processes derived from the preliminary analytics conducted by the data analyst.

Skills: Programming, big data analysis

Education: Undergraduate in computer science, software engineering. Knowledge and ideally experience in big data technologies:

• Scripting language: Python
• Programming language: Java or Scala (recommended), Node.js or C++ or Golang
• Big data technologies: Hadoop or Spark, Big Query or Redshift or similar cloud data warehousing/data lake technology

After hiring a data engineer, there will be quite a lot of collaborative work between the data analyst and the data engineer. If the data analyst is competent enough, or a quick learner, machine learning models can be applied to the data to derive results. Data engineer should be able to assist the implementation of these models in the data pipelines in most cases.

Optional — 5. Hire an ML Engineer

Apart from all above, if your product needs high performance in data analysis, then you would need to hire an ML engineer. The primary job of an ML engineer would be to take the prototype model developed by data analyst and rewrite in a high performance language.

Responsibilities: Building, optimising ML models and deploying the ML models in highly scalable, production environments.

Skills: Low level programming, machine learning

Education: Undergraduate in computer science, software engineering, and supplementary knowledge in applied statistics/data science. Knowledge in:

• Programming language: C++ or Node.js or Golang
• SQL, or relevant NoSQL databases

There you have it. These should be the people you should have for your data team. Some tips:

• Composition: This can depend on your type of business. In my experience, I worked with teams having a 1 data analyst, 2 full stack developers and 1 data engineer. Never worked with an ML engineer.
• Data scientist? Mostly, you won’t need one. 9 out of 10 data scientist jobs don’t need data scientists, and 9 out of 10 data scientist applicants are not really data scientists. This may differ if you are working in industries where higher precision, care is needed, or has high regulations; like core banking (not marketing), healthcare, critical infrastructure, etc.
• Formal education? I would want data analysts, data engineers and ML engineers with formal education.
• How to integrate a data team with the rest of the company? Now that’s a topic for another day!

Your comments, feedback are welcome!