Data engineering and analytics

It depends on your cloud, volumes, skills and budget more than on any single product. Snowflake, Databricks, BigQuery and Amazon Redshift all work well for most companies, and PostgreSQL is enough for many. We compare the options against your requirements and existing contracts, and design so the transformations, written in SQL with dbt, stay portable.

Yes. We use the ERP’s supported extraction methods, such as its APIs, export services or change data capture from a replica, rather than heavy queries against the live database. Loads are scheduled outside peak hours where needed. For SAP, Oracle, Microsoft Dynamics 365, NetSuite and Odoo we follow each vendor’s recommended approach.

Every load runs automated checks for completeness, duplicates, freshness and business rules, such as debits matching credits. Before a report replaces an existing one, we reconcile its figures against the source system and the old report with your finance or operations owner. A failed check stops bad data reaching dashboards and alerts the team.

Only for decisions that cannot wait, such as fraud screening, stock availability or payment status. For most reporting, loads every hour or overnight are enough and cost far less to run. We use streaming with Apache Kafka or change data capture where real time is needed, and batch where it is not, and we decide per use case with you.

Access is role-based and follows your identity provider. Personal and financial fields are masked or tokenised for users who do not need them, data is encrypted at rest and in transit, and access is logged. We keep data in the regions you specify and apply your retention rules to raw and modelled data alike.

Build cost depends on the number and complexity of sources, how clean they are, and how many subject areas and reports are in scope. Running cost depends on data volumes, refresh frequency and query patterns. We estimate both, design to control compute spend, and set up cost monitoring so usage does not grow unnoticed.

We deliver in subject areas, so the first one, such as cash or sales, reaches users well before the whole platform is finished. Timing depends on access to source systems, data quality and how much reconciliation the figures need. We agree the order of subject areas with you and plan each one after the assessment.

Either your team or ours. We hand over documentation, tests, monitoring and a runbook, and train your engineers if they will run it. If you prefer, we monitor loads and quality, fix failures, handle source system changes and add new subject areas under a support agreement.

Where you decide. We agree the location of data before work starts and design for it, whether it must stay in one country, in your own cloud account or on your premises. Where data-protection laws such as the GDPR apply, we work under a data processing agreement and limit personal data to what the work needs.