How Glooko Turns 3B+ Data Points/Month into Lifesaving Diabetes Healthcare with Tiger Data

This audio is presented by Hacker Nune, where anyone can learn anything about any technology.

How Glucode turns 3B Plus data points, month into life-saving diabetes health care with Tiger Data,

by Tiger Data, Creators of TimeScaleDB. This is an installment of our Community Member Spotlight

series, in which winevite our customers to share their work, spotlight their success,

and inspire others with new ways to use technology to solve problems.

In this edition, for Grapadin, Vice President of Engineering at Glucode,

a diabetes monitoring solution used by 1M Plus patients shares how they migrated one of their

largest and most critical medical data workloads from a leading document database to Tiger Cloud,

unlocking lower storage costs, faster ingest, and a cleaner architecture for future growth.

About the company and team, I'm for Grapadin, Vice President of Engineering at Glucode,

a diabetes platform used by over 1M patients for glucose monitoring and chronic condition

tracking. We simplified diabetes care, integrate with EHR workflows, and deliver measurable health

outcomes. Glucode ingests over 100M new data points every day, primarily time stamp measurements

and patient events, to power real-time dashboards and analytics for patients and healthcare providers.

Older devices collect glucose values every 5 minutes, newer ones every minute.

To meet local data residency requirements, we utilize regional data centers in Germany,

Ireland, Canada, and the US for storage and analytics. The challenge, scaling time series workloads

with existing infrastructure, real-time time series workloads force ugly cost, performance

trade-offs. You can buy your way out of performance bottlenecks, but it gets expensive fast.

At Glucode, we hit the point where scaling our existing setup to keep up with rising demand

just wasn't worth the cost. Sluggish ingestion, 74B plus total CGM records and 100M plus new

readings per day were pushing our document database to its limits. Data bloat, existing

document database cluster had grown to 30 terabytes, largely due to index overhead.

Slow queries, 14 day and 90 day rollups were too slow for clinicians, and performance worsened

as data grew. No retention policies. Every new measurement was stored permanently, further

driving up costs and footprint. Replacing NoSQL with Tiger Data, as we evaluated alternatives

to our existing document database, Tiger Data stood out. We already trusted Postgres internally

and had considered building our OWN time series solution on it, but were uneasy about the risks

of self-hosting ATEC stack requiring HIPAA, GDPR compliance. In addition, we used a WS as the

backbone of our digital system, so Tiger Data's model running on EC2 with S3 storage made it easy

to snap into our existing architecture. With Tiger Cloud, we got the cost and performance benefits

we needed on a hosted Postgres platform, optimized for time series, managed Postgres with purpose

built time series extensions, including incremental materialized views and automatic partitioning.

Cost savings, policy-based compression and retention for automated data lifecycle

management, unified Postgres architecture resolved ingest and read bottlenecks. We unified

everything on Tiger Cloud and kept it in a single Postgres platform, with compute and storage

decoupled. That cleared our ingest, read bottlenecks without our costs exploding, and it gives

room to scale while shipping more functionality. We designed new time scale DB hyperdables around

how we actually query per patient per day. We kept only the fields we needed for analytics in the

hotpath and pushed the rest to cheaper storage. With this shift, query patterns now match the time

series partitioning relevant for clinicians and patients consuming the data, e, g, show me the last

90 days, etc. Late arriving data and continuous rights to recent time windows now occur in a normal

supported pattern rather than being managed on an ad hoc basis. In addition, we built a dedicated

medical data service, MDS, with direct access to Tiger Cloud. Legacy services and mobile clients

were updated to talk to MDS rather than with the database directly, simplifying and speeding up

the ingestion process. It also made it easier to update schema and performance characteristics

without rippling changes through the entire tech stack. Greater than the end result was excellent,

ingestion is faster than before, costs are lower greater than than before, and we ended up ahead

of our original timeline. Pergrapatin, greater than vice president of engineering at glucose

tiger data cuts costs 40%. Once the migration was done, we went back to the metrics and confirmed

that tiger data did reduce costs compared to our earlier architecture. Storage got cheaper because

we were storing less. Compute got cheaper because we could downsize instances more than we expected.

Compression ratios of approximately 95% and some compression ratios as high as 97%.

480X faster query speeds from 4 minutes to half a second. More stable ingestion for 100M

plus new readings per day across tens of billions of existing records. While we have finished

the core migration from a document database to a unified Postgres platform on AWS, we remain

focused on optimization. We plan to move our remaining medical data collections into tiger

clouds so our high volume telemetry and clinical datasets can live in one space, providing better

performance for patient and provider facing workloads. Thank you for listening to this Hackernoon

story, read by artificial intelligence. Visit Hackernoon.com to read, write, learn and publish.