Eight years after my original post on building data science teams, the landscape has transformed dramatically. From the experimental “unicorn hunting” of 2017 to today’s systematic, specialized team construction, here’s what’s changed and what works in 2025.
Back in 2017, I wrote about the challenges of building data science teams (here) based on insights from Ronert Obst’s experience at New Yorker. The field was young, companies were experimenting, and we were all essentially making it up as we went along. Fast forward to 2025, and the data science discipline has matured dramatically—transforming from isolated analytics functions into production-oriented teams deeply integrated with AI and business operations.
The numbers tell a striking story: less than 20% of companies have achieved advanced analytics at scale, not due to technical limitations, but because of leadership gaps, organizational structure challenges, and talent management difficulties. Meanwhile, entry-level data scientist salaries have jumped from $117,000 to $152,000 in 2025 alone, reflecting both the field’s maturity and continued high demand.
So what’s changed since 2017, and what does it take to build effective data science teams today?
Perhaps the most significant shift since 2017 has been the definitive end of “unicorn hunting”—the search for mythical generalists who could handle everything from data engineering to machine learning to business presentation. Harvard Business Review’s research now definitively shows that organizations need “diverse data science teams rather than searching for data scientist unicorns.”
The market has stratified into three primary role categories:
- Versatile professionals (57% of positions) combining multiple competencies
- Domain experts (38%) with deep specialization
- Full-stack generalists (5%) handling end-to-end processes
This represents a fundamental shift toward T-shaped professionals who combine deep technical expertise with broad business acumen. Salary premiums reflect this specialization: ML engineers command 15-40% higher compensation than traditional data scientists, while specialized roles average 20-35% premiums over generalist positions.
McKinsey’s current recommended structure includes data scientists for advanced analytics, data engineers for infrastructure, workflow integrators for optimization, data architects for system design, delivery managers for coordination, and critically, translators comprising approximately 10% of business unit staff to bridge technical teams with business functions.
Remember Ronert’s comments about licensing Hadoop vendors in 2017? That world has completely disappeared. The technology evolution from 2017 to 2025 represents one of the most dramatic shifts in enterprise technology, with organizations reporting 11x more AI models deployed to production year-over-year.
The modern data stack has shifted from ETL to ELT architectures, leveraging cloud warehouse compute power for transformations. dbt (Data Build Tool) experienced 206% year-over-year growth, representing the analytics engineering revolution that enables SQL-based transformations with Git workflows, testing, and documentation.
Cloud platforms have evolved dramatically: - AWS SageMaker expanded from basic model training to comprehensive MLOps with Studio, Pipelines, Model Registry, and Feature Store - Microsoft Azure integrated OpenAI models through Azure OpenAI Service - Google Cloud Platform unified offerings under Vertex AI with AutoML and specialized TPU hardware
Essential technology standards for 2025 include cloud-first architecture (AWS, GCP, or Azure), containerization with Docker and Kubernetes (Docker adoption surged 17% year-over-year), MLflow for experiment tracking, dbt for analytics engineering, and comprehensive monitoring with tools like Evidently AI.
The MLOps market exploded from concept to a $2.19 billion industry in 2024, projected to reach $16.61 billion by 2030. This isn’t optional anymore—it’s table stakes.
One area where my 2017 insights have been validated is organizational structure. Research from McKinsey, BCG, and other consulting firms definitively shows hybrid organizational structures deliver superior impact compared to purely centralized or decentralized models.
Airbnb’s evolution exemplifies best practices: starting with a fully centralized team of 30 people, they transitioned to a hybrid model with data scientists embedded in business units while maintaining central leadership for career development and standards. Netflix adopted vertical alignment with business functions, organizing data engineering and analytics by product, streaming, content, marketing, and payments verticals.
Meta’s federated model demonstrates another successful approach—individual product teams own end-to-end data/AI projects while central teams provide common infrastructure and tools. They created three career tracks: Senior Individual Contributors, Org Builder Managers, and Tech Lead Managers, enabling movement between IC and management roles.
The platform model, exemplified by Uber’s Michelangelo system, provides ML-as-a-service to all product teams through central infrastructure while enabling distributed execution. This democratizes ML capabilities while maintaining platform consistency.
Critical governance decisions for hybrid models include centralizing data governance, partnership management, and core technical capabilities while decentralizing business-specific applications, day-to-day operations, and user support.
Despite tech sector layoffs affecting 141,467+ employees across 476+ companies in 2024, data scientists comprised only 3% of layoffs versus 22% for software engineers. The Bureau of Labor Statistics projects 36% job growth from 2023-2033, while McKinsey predicts US demand will exceed supply by 50% by 2026.
Hiring channels have evolved since Ronert’s experiences: - LinkedIn remains crucial for professional networking - Specialized job boards show strong data science activity - Skills-based hiring increased 36% in job postings removing degree requirements - Technical interviews increasingly focus on SQL, algorithms, and ML demonstrations
However, educational requirements have actually intensified—data science degree requirements jumped from 47% in 2024 to 70% in 2025, with PhD requirements increasing by 10%+ and Master’s degrees required in 31.4% of roles.
Skills requirements have evolved significantly: Machine Learning appears in 77% of positions, Deep Learning mentions doubled to 20% of postings, and NLP grew from 5% in 2023 to 19% in 2024.
Data scientists in 2025 expect significantly more than their 2017 counterparts. Where Ronert’s slide showed basic expectations around salary and technology, today’s professionals demand:
Professional development has become critical for retention, with employees expecting continuous learning opportunities in AI/ML, conference attendance, certification support, and cross-functional exposure.
Retention challenges persist due to lack of structured career paths, especially at large companies, and compensation competition from tech giants and startups. The most successful companies provide multiple career tracks—technical/individual contributor, management/leadership, and technical leadership hybrid tracks.
Organizations face significant scaling obstacles, with McKinsey identifying “pilot purgatory” affecting 80% of companies struggling to move beyond small experiments. This echoes concerns from 2017 but has become more acute as expectations have risen.
The talent retention crisis represents the most critical challenge, with 62% of organizations citing talent attraction as their primary obstacle and 59% identifying lack of data science expertise as a barrier to AI adoption.
Successful scaling requires systematic approaches including parallel processing across multiple systems, distributed computing, cloud-native technologies, and automation integration. The IPTOP framework—Infrastructure, People, Tools, Organization, and Processes—provides a systematic approach to scaling challenges.
High-performing organizations demonstrate clear characteristics: they employ 4x more analytics professionals than average performers, achieve 3x better talent retention through career development, maintain 5x higher likelihood of having tools for unstructured and real-time data, and track impact on top-line revenues (54% vs. 19% of low performers).
CEO and senior leadership involvement represents the number one factor for analytics success, yet only 25% of CEOs actively lead data analytics agendas. The most successful transformations require town hall meetings communicating analytics importance, executive dashboards for real-time monitoring, and incentive alignment.
Cultural change management has become equally important as technical implementation. Trust building helps employees accept analytics over traditional intuition, while expectation setting educates organizations on analytics capabilities and limitations.
BCG’s agile sprint approach shows effectiveness with 2-3 month cycles enabling rapid iteration, cross-functional teams mixing business experts with technical specialists, and problem-first focus starting with business challenges rather than technical capabilities.
Building effective data science teams in 2025 requires embracing specialization over generalist hiring, implementing hybrid organizational structures, investing in modern cloud-native technology stacks, and providing competitive compensation with clear career progression.
The evolution from 2017’s experimental approach to 2025’s production-oriented discipline demands strategic thinking about team composition, technology investments, and organizational design. The most successful organizations focus on cultural transformation alongside technical capabilities, measure ROI systematically, and maintain strong leadership commitment.
Looking back at Ronert’s experience at New Yorker, some principles remain constant: strong leadership support accelerates implementation, hiring the right people matters more than everything else, and continuous communication of value is essential. But the execution has become more sophisticated, the stakes higher, and the opportunities greater.
Success ultimately depends on execution excellence, cultural alignment, and continuous adaptation rather than choosing any single “perfect” initial structure. The evidence from leading technology companies demonstrates that thoughtful evolution, systematic approaches to scaling, and strong business partnership create sustainable competitive advantages through data science excellence.
The field has matured, but the fundamental challenge remains the same: building teams that can translate data into business value at scale. The tools, techniques, and organizational models have evolved dramatically, but the human elements—leadership, culture, and talent—remain as critical as ever.