In small organizations, data scientists need to have a deep understanding of the entire machine learning workflow, from data infrastructure to model deployment, allowing them to gain hands-on experience and be involved in the end-to-end process of data science.
Working as a data scientist in a small organization provides the opportunity to try different tasks and roles, such as project management and MLOps, and have a wider scope of responsibilities compared to data scientists in large companies.
Deep dives
The Role of a Data Scientist in a Small Organization
In a small organization, the role of a data scientist is not just about training models, but about converting data into business value using data science techniques. The data scientist needs to have a working knowledge of the entire machine learning workflow, including data infrastructure, feature engineering, training models, deploying models, and monitoring their impact. The advantage of being a data scientist in a small organization is that you can have a broader and hands-on experience, working on various tasks and roles across the entire workflow. This allows you to gain a deeper understanding of different components and be involved in the end-to-end process of data science.
Challenges and Advantages of Data Science in Small Organizations
Working as a data scientist in a small organization presents its own challenges and advantages. Some challenges include the need to be a generalist and cover a wide range of tasks, such as project management, communication, and working with different stakeholders. However, the advantage lies in the ability to try different things and have a wider scope of responsibilities compared to data scientists in large companies. In small organizations, data scientists often have the opportunity to work on end-to-end projects, gain exposure to different roles like MLOps, and drive impact across the organization.
Building Trust and Communication in Small Organizations
Earning trust and effective communication are crucial for data scientists in small organizations. Building trust requires delivering results consistently and demonstrating the value of data science through tangible outcomes. Regular communication and project updates are essential to keep stakeholders informed and engaged. Using tools like Google Sheets and simple project management frameworks can help manage projects efficiently without overwhelming processes. It's also important for data scientists to understand the people and systems within the organization, establish good relationships with different teams, and showcase the value of data science in solving specific business problems.
The Future of Data Science in Small Organizations
The future of data science in small organizations looks promising. As the need for data science techniques grows, there is an opportunity to show how data science can drive significant impact in small businesses. The development of MLOps tools specifically designed for small organizations can simplify the deployment and management of data science models. The rise of large language models and their potential to increase familiarity with data science concepts may also contribute to the adoption of data science in smaller organizations. Furthermore, the shift towards measuring excellence based on impact rather than state-of-the-art performance can accelerate the adoption of data science techniques in various sectors.
Why is ML is so poorly adopted in small organizations (hint: it’s not because they don’t have enough data)? In this episode, Kirsten Lum from Storytellers shares the patterns she has seen in small orgs that lead to a successful ML practice. We discuss how the job of a ML Engineer/Data Scientist is different in that environment and how end-to-end project management is key to adoption.