At Chimnie, our mission is simple: to provide the most accurate, comprehensive, and up-to-date property data in the UK. But behind that simple mission is a sophisticated technology engine that makes it all possible.
We're pulling back the curtain to give you a glimpse into our tech stack - not to overwhelm you with jargon, but to show you how our technology directly translates into better decisions for insurers, lenders, and real estate professionals.
Building the Foundation: Data, Data, Everywhere!
Imagine trying to understand the UK property market without knowing the details of each property - its flood risk, its rebuild cost, its energy efficiency, and its history. It's impossible! That's why our data infrastructure is the heart of everything we do.
Massive Data Processing
We use powerful tools like Apache Spark on Amazon EMR to process millions of property records every day. Think of Spark as a supercharged engine for crunching huge datasets in parallel, and AWS as the reliable infrastructure that keeps it running smoothly.
We additionally augment Spark with Apache Sedona, optimised for geospatial work, and H3 hexagonal indexing. For smaller workloads that don't need the massive parallelisation of Spark, we use modern in-memory databases like DuckDB, or the blazing-fast DataFrame library Polars. This allows us to handle every kind of data challenge with speed and efficiency.
Benefit: You get the insights you need fast.
AI and Machine Learning
We use cutting-edge AI to transform raw property data into intelligent insights. Our Bayesian hierarchical models, built with NumPyro and JAX, use advanced algorithms to predict property values by learning complex regional patterns, while scikit-learn and XGBoost handle classification and gradient boosting tasks like determining property types and sale propensity.
Statsmodels provides robust statistical modeling capabilities for econometric analysis. Computer vision with OpenCV and multimodal LLMs analyse satellite imagery and property photos alongside descriptions, and segmentation models identify property boundaries from aerial imagery and façades. Time series forecasting with skforecast predicts market trends using these probabilistic foundations.
Think of all this as having an AI property expert that can instantly tell you everything about any property.
Benefit: You get comprehensive property intelligence covering every aspect.
Putting it all together
We use Bruin, a modern workflow orchestration platform, to automate and manage our complex data pipelines. Bruin allows us to define, schedule, and monitor every step of our data processing, from ingestion and transformation to machine learning and delivery. Its robust dependency management and real-time monitoring ensure that even the most intricate workflows run reliably and transparently.
Benefit: You get consistently up-to-date, high-quality data and insights, delivered with minimal delay and maximum reliability.
Organised and Accessible Data
We store our data securely and efficiently using Amazon S3, a bit like a giant, ultra-reliable hard drive in the cloud. Our preferred file format is Parquet, with Delta Lake for data that needs to be updated incrementally. We finally make our full dataset accessible as flat files via SFTP or web interface thanks to FileMage.
Benefit: You can access the data you need, when you need it, in the format you prefer.
Making Sense of it All: Web & API Architecture
Having all this data is great, but it's useless if you can't easily access and understand it. That's where our web applications and APIs come in.
Fast and Reliable APIs
We use a modern framework called NestJS to build our APIs. These APIs are the "bridges" that connect our data to your systems, allowing you to seamlessly integrate property intelligence into your workflows.
Benefit: Easy integration with your existing systems.
User-Friendly Interfaces
We build intuitive web applications using Next.js and MUI with React through the Refine meta-framework, making it easy for you to explore our data, generate reports, and get the insights you need without needing to be a tech expert.
Benefit: Get the insights you need without a steep learning curve.
Secure Authentication
We use Firebase to ensure that only authorised users can access our data and applications, as well as Amazon Cognito for enterprise SSO needs.
Benefit: Your data is safe and secure.
Finding the Right Property
We use Elasticsearch, a powerful search engine, to enable lightning-fast geographical searches, even when you're dealing with millions of records. For single-property requests, we leverage MongoDB for reliable and scalable storage and retrieval.
Benefit: Find the exact property you're looking for in milliseconds.
All Systems Nominal
To keep all our end-user systems up and running, we manage multiple Kubernetes clusters on AWS EKS. This allows us to seamlessly scale capacity as needed to always provide sub-second latency and reliable uptimes.
All our deployments leverage OpenTelemetry to send observability data to our centralised SigNoz monitoring dashboard.
Benefit: A platform you can rely on.
Keeping You Informed
We use Payload CMS to manage this very blog. This is how we create content to keep you up to speed.
The Tools That Keep Us Running Smoothly
Beyond the core data and web infrastructure, we use a range of other tools to keep Chimnie operating efficiently and effectively:
From messaging on Slack and tracking stasks with Linear, to managing payroll with Deel or keeping our accounts in order with Intuit QuickBooks, we have a host of other tools that keep us ticking over.
Always Improving: Our Future Roadmap
We're never satisfied with the status quo. We're constantly exploring new technologies and approaches to make our data even better, our systems even faster, and our services even more valuable. Some areas we're currently exploring include:
Even More Powerful Data Processing: We're looking at ways to enhance our data processing capabilities, potentially using tools like Ibis or Ray for even more advanced analytics and machine learning.
Vector Databases: We're exploring ways to leverage vector databases like Weaviate or Pinecone to enable new use cases where semantic or vector search would be useful.
Open Source and the Community We are big believers in Open Source and use a lot of open-source products, including Spark, Kubernetes, Elasticsearch, NestJS, Next.js, MUI, Zustand, and many more. We are looking at ways to help support the Open Source projects we use and potentially to open-source some of our internal tooling in the future.
The Bottom Line: Better Data, Better Decisions
Our technology stack isn't just about cool tools and cutting-edge techniques. It's about empowering you to make better decisions. Whether you're an insurer assessing risk, a lender evaluating a mortgage application or a real estate professional analysing market trends, Chimnie's technology provides the foundation for success.
Ready to see the power of Chimnie's data in action? Explore our solutions or get in touch today! ➡️ hello@chimnie.com
