How AI-augmented development transformed climate data operations in weeks
Berkeley Earth is a leading independent research organisation that transforms raw temperature data into authoritative climate insights used by scientists, policymakers, and major news outlets worldwide. Their comprehensive database includes millions of measurements from over 40,000 weather stations globally, providing critical evidence for climate science and policy decisions.
Berkeley Earth approached Whitesmith to modernise a complex climate data architecture to reduce manual work in their monthly temperature updates. They also wanted to create more user-friendly interfaces for broader impact and improve efficiency without sacrificing scientific accuracy.
The scale of the data and the precision required meant that traditional development would have taken months and a whole engineering team. Berkeley Earth reached out to Whitesmith, requesting assistance to compress that timeline.
Traditionally, understanding an extensive legacy data system would have taken two to three weeks. Using advanced language models, our engineer analysed the documentation and codebase, produced a complete system map and data flow diagram, and built a prioritised roadmap, all within two days.
Data migration from legacy systems to Google Cloud Storage and BigQuery was completed in seven days instead of the usual four to six weeks. AI-assisted scripts automated boilerplate, parallelised data transfers, and implemented intelligent partitioning to reduce query costs and improve access times.
The goal for this phase was to give a diverse stakeholder audience with varying user needs an accessible and intuitive way to explore Berkeley Earth’s climate data in real time. Within the first two weeks, we delivered a functional prototype: an interactive global map that could run real-time queries against the newly migrated datasets.
The backend used FastAPI (Python) and was deployed on Cloud Run to handle variable traffic with automatic scaling. It queried climate datasets stored in BigQuery and Google Cloud Storage, leveraging table partitioning implemented during migration for faster reads and lower query costs.
On the frontend, we built a Next.js 15 app with React and TypeScript, integrating Mapbox GL for high‑performance geospatial rendering and a responsive UI. By week four, the platform supported data exports, shareable visualisations, and multiple chart types with sub‑second load targets.
Before this project, Berkeley Earth’s monthly climate anomaly reports required approximately eight hours of manual work by scientists: gathering the latest data, running analysis scripts, creating charts, and writing summaries.
We developed an intelligent report generation system that analyses the latest climate anomalies, produces narratives with customisable templates and generates dynamic charts with publication-ready visualisations which scientists can review and adjust for accuracy and clarity. The system compiles charts, text, and data tables into a consistent, ready-to-publish report format.
With this workflow, the monthly reporting process now takes roughly one hour from start to finish. Scientists spend their time verifying and refining the output instead of performing repetitive, time-consuming tasks, significantly increasing both the speed and consistency of Berkeley Earth’s public climate updates.
“AI has become a force multiplier for Berkeley Earth’s operations and impact, democratizing access to climate intelligence while allowing our team to focus on areas of our practice where human expertise matters most. Whitesmith served as true partners in the design and execution of our AI strategy, providing expert-level guidance on how to embed AI in ways that enhance operational efficiency without compromising scientific integrity. By shortening our prototyping and iteration cycles, reducing the burden of routine data processing, and enabling real-time visualization and reporting, we’ve expanded access to our science for communities on the frontlines of climate risk, all while freeing our scientists to tackle the emergent questions shaping climate adaptation and resilience.”
Kristen Sissener, Executive DirectorOur approach demonstrates optimal human-AI collaboration:
Several factors made this project successful and positioned Berkeley Earth for long-term gains:
— Precise Use of AI – AI tools were applied only where they offered the highest impact: analysing documentation and codebases, generating boilerplate, detecting patterns, and creating test scaffolding. Tasks requiring judgment, scientific rigor, and architectural planning remained in human hands.
— Continuous Human Oversight – Every AI-assisted output underwent review, testing, and adaptation by experienced engineers to meet performance, security, and scientific accuracy standards.
— Rapid Prototyping with Measured Validation – Short, iterative development cycles enabled multiple approaches to be tested and benchmarked before selecting the optimal path.
— Domain-Specific Accuracy – All workflows were designed to preserve the integrity of observational data flows and analysis, ensuring no trade-offs between speed and accuracy.
— Scalable, Sustainable Architecture – The platform was built on modern cloud services—Google Cloud Storage, BigQuery, and Cloud Run—ensuring it can handle exponential data growth, adapt to new data formats, and integrate with emerging research tools without major redesign.
Looking ahead, Berkeley Earth now has automated workflows that adapt in step with new datasets and formats, modern APIs that make it easier for researchers and collaborators worldwide to access climate insights, and infrastructure that is prepared to support advanced machine learning and predictive analytics in the future.
The approach taken reflects Whitesmith’s core principle: AI should enhance human expertise, not replace it. By combining targeted AI use with expert engineering oversight, the team delivered a solution that was faster to build, more cost-effective to maintain, and ready to support Berkeley Earth’s mission for years to come.
Project
Berkeley Earth
Categories
fCTO
AI Engineering
Product Development
Product Strategy
Schedule a call to learn whether Whitesmith are a good fit for your product.
Schedule a discovery callCookies help us deliver our Services. By using our Services or clicking "I agree", you agree to our use of cookies. Read Policy
I agree
