# From data scientists' notebooks to a weather-data product for insurers

> An agtech start-up's data scientists had the models, in MATLAB and Python notebooks, but no product. We turned them into a working back end on AWS with a front end, and shipped the MVP of a marketplace selling weather and agricultural data to insurers.

- Client: An Australian agtech start-up selling weather and farm data to insurers (Agricultural insurance data)
- Role: Business analyst and lead engineer, Nicolas Dao
- Status: Delivered. At neap.co
- Capabilities: Data science, Business analysis, Productionising models, Serverless
- Web page: https://cloudlesslabs.com/work/weather-data-for-insurers/

## Outcomes

- An MVP shipped: the data scientists' models running as a product on AWS.
- MATLAB and Python notebooks ported into tested Python and Node.js on AWS Lambda, with PostgreSQL behind them.
- A foundation the client's own engineers took over and grew.

## The situation

Insurers price agricultural risk on weather: drought, frost, rain at harvest. A start-up wanted to sell them the data and the analysis to do it better, as a marketplace for weather and agricultural data. It had data scientists, and their models worked, in MATLAB and Python notebooks. What it didn't have was a product. A notebook runs when a scientist runs it; a product runs every day, for every customer, without one.

It was one of [neap.co](https://cloudlesslabs.com/work/neap-co/index.md)'s first serious projects.

## What we did

More data science and data wrangling than app building, and a lot of business analysis first, to understand what the insurers needed and what the models produced.

- **From notebooks to code.** We ported the scientists' MATLAB and Python into production code, Python for the analysis and Node.js for the AWS Lambda functions around it, checking every result against the originals.
- **A real back end.** The data in PostgreSQL on Amazon RDS, the processing on Lambda, so the analysis ran on schedule instead of on someone's laptop.
- **A front end** for customers to use the data.

## How it ended

We shipped the MVP, and the client was happy with it. They then hired their own engineers, who took the product on from there. That's a good ending: the point of an MVP is to prove the idea and leave a foundation others can build on.

## Why it matters to you

Many companies have good data scientists and no path to production. The gap between a notebook and a product is a craft of its own: tested code, data pipelines, scheduling, and a design that customers can use. We've crossed it many times, from [weather forecasts](https://cloudlesslabs.com/work/weather-neural-nets/index.md) to [market analysis](https://cloudlesslabs.com/work/crypto-analysis/index.md).

## Built with

- [Amazon Web Services (AWS)](https://cloudlesslabs.com/stack/aws/index.md): My default in The Stack
- [AWS Lambda](https://cloudlesslabs.com/stack/aws-lambda/index.md): My default in The Stack
- [Node.js (plain JavaScript)](https://cloudlesslabs.com/stack/nodejs-javascript/index.md): My default in The Stack
- Python
- MATLAB
- PostgreSQL on Amazon RDS
- Data wrangling

## Related

- [Cutting an AWS bill from US$240k a year to under US$40k](https://cloudlesslabs.com/work/aws-bill-cut/index.md): a related project
- [Real-time ingestion for a large farm-sensor network](https://cloudlesslabs.com/work/realtime-iot/index.md): a related project
- [Deep neural networks that forecast the weather](https://cloudlesslabs.com/work/weather-neural-nets/index.md): a related project
- [Forecasting a weather-driven risk for farmers, from a research paper to agents](https://cloudlesslabs.com/work/research-to-forecasts/index.md): a related project
- [Every project](https://cloudlesslabs.com/work/index.md)
- [How we work](https://cloudlesslabs.com/about/how-we-work/index.md)
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)