Why energy companies work with Nodus Datas AI

The energy sector is one of the most data-intensive industries in Europe — yet most utilities, suppliers, and renewable operators still struggle to turn raw operational data into reliable business decisions. Smart meters generate billions of readings, SCADA systems push telemetry from substations every second, wind turbines stream high-frequency sensor data, and regulatory reporting demands traceable lineage across all of it.

At Nodus Datas, our team has spent the last several years building production data platforms for some of the largest energy operators in Romania — including a major Distribution System Operator (DSO) and a leading national energy supplier. We have shipped what most consultancies only present in slides: working pipelines processing tens of millions of records daily, ML models in production making real revenue and grid decisions, and topology systems used by operational teams in the field.

This page walks through four real engagements that define what we do.

From Substation to Consumer, We Engineer the Data Layer

Industry: Energy distribution, retail supply, renewables

Engagement model: Architecture, data engineering, ML engineering

Cloud stack: AWS (EMR, MWAA/Airflow, SageMaker, Redshift, S3), Azure, GCP

Core technologies: PySpark, Apache Airflow, MLflow, Delta Lake, Kafka, Oracle, PostgreSQL

Strategic Energy & Storage Case Studies

Use Case 1 - Interactive Grid Topology Mapping: From Primary Substation to Consumer

The challenge

A major Romanian distribution operator needed a unified, interactive view of its electrical network — from primary substations, through medium-voltage feeders, secondary substations, low-voltage lines, all the way down to the individual consumer meter. Before our engagement, this information lived fragmented across:

Legacy GIS systems with inconsistent identifiers
Asset management databases (substation equipment, transformer ratings, feeder configurations)
Customer information systems (consumer points, contract data, meter installations)
SCADA topology snapshots updated on different cadences

Field engineers, planning teams, and outage management could not answer basic questions like *"which consumers are downstream of this transformer?"* or *"if this medium-voltage feeder trips, who loses power?"* without manually cross-referencing three or four systems.

What we built

We designed and implemented a complete topology data layer and an interactive visualization platform on top of it:

Data engineering layer:

Ingestion pipelines pulling and reconciling asset, GIS, and customer data into a unified graph model
Entity resolution logic to match consumer points to feeders despite inconsistent legacy identifiers
Daily refresh orchestration via Apache Airflow with full data quality gates and lineage tracking
A canonical topology table modeled as a directed graph (substation → feeder → transformer → service line → meter)

Interactive visualization:

A fully interactive map where every node from primary substation down to consumer meter is clickable, filterable, and traceable
Real-time downstream/upstream traversal — click any asset, see everything connected to it
Integration with operational data (load, alarms, outage status) overlaid on the topology

Impact

Operations teams reduced average outage impact assessment time from 30+ minutes of manual work to under 1 minute
Planning engineers gained a single source of truth for network analysis
The topology layer became the foundation for downstream use cases including loss analysis and fraud detection (see Use Case 2)

Technologies used: PySpark on EMR · Airflow (MWAA) · PostgreSQL/Redshift · graph data modeling · interactive web mapping (Leaflet/Mapbox stack) · S3 partitioned storage

Use Case 2 - Streaming Smart Meter Data for Energy Fraud Detection

The challenge

A national Romanian energy supplier was losing significant revenue to non-technical losses — meter tampering, illegal connections, and consumption manipulation. The existing detection process was reactive: anomalies were spotted weeks or months after the fact, by the time investigation teams arrived on site, evidence was often gone.

The supplier needed a **near-real-time fraud detection pipeline** that could:

Process smart meter readings continuously across millions of consumer points
Apply both rule-based and machine learning detection logic
Integrate predictions back into operational systems used by field investigation teams
Maintain full auditability for regulatory and legal proceedings

What we built

We architected and delivered a production-grade Energy Fraud Detection (EFD) platform spanning streaming ingestion, batch enrichment, ML inference, and bidirectional integration with operational systems:

Streaming and batch pipeline:

Smart meter readings ingested continuously, partitioned and stored on S3 in optimized Parquet format
A chained Airflow pipeline of three coordinated DAGs handling ingestion, enrichment, and fraud scoring
Heavy join workloads (44M+ rows of consumption data joined against 21M+ rows of contractual reference data) refactored from SQL warehouses into dedicated PySpark sort-merge jobs on EMR — eliminating performance bottlenecks that had caused daily SLA breaches
Schema-aligned data lake with strict typing, legacy date handling, and full backfill capability

ML inference layer:

SageMaker model endpoints scoring consumer points on rolling time windows
MLflow-managed model registry with versioning and reproducibility
Quota-aware orchestration to handle SageMaker concurrency limits without job failures

Bidirectional integration with operational systems:

Predictions written back from the data lake (Redshift) into the supplier's Oracle-based field operations system
Deduplication logic (K-day lookback window) to avoid sending the same case twice
Full traceability: every prediction logged with model version, input features, and dispatch status

Reliability engineering:

Microsoft Teams notifications on every DAG outcome (success, retry, failure) via Power Automate Workflows
Custom callbacks routing alerts to preprod and prod channels with actionable context
Validated end-to-end in production with controlled test batches before full rollout

Impact

Fraud cases reach field investigation teams **within days instead of months**
Detection coverage expanded across the full smart meter base
The pipeline runs unattended on a weekly schedule with operational team oversight, not engineering babysitting

Technologies used: PySpark on EMR (EKS) · Airflow MWAA · SageMaker · MLflow · Redshift · Oracle CROS · S3 · Spark JDBC · Microsoft Teams (Power Automate) · CloudWatch

Use Case 3 - Short-Term Load Forecasting for Energy Supply

The challenge

Energy suppliers in liberalized markets must forecast consumer demand at fine granularity (15-minute intervals, day-ahead and intra-day) to optimize their position on the wholesale market. Mispricing forecasts directly hits margins — every MWh imbalance is a real cost.

The supplier we worked with had a fragmented forecasting setup: separate codebases for different model families (LightGBM, RNN, linear regression), inconsistent feature engineering, no central experiment tracking, and significant operational pain around Daylight Saving Time transitions.

What we built

We delivered a unified, model-agnostic short-term forecasting (STF) platform:

Unified forecasting framework:

A single Airflow DAG architecture capable of training and serving any model family (LightGBM, neural networks, linear baselines) by reading configuration from MLflow
Centralized feature engineering logic — weather data, calendar features, lag windows, holiday handling — shared across all models
Time-aware column configuration system handling supplier portfolio changes (customer churn, new contracts, lifecycle events)

DST and edge case handling:

Robust handling of Fall DST transitions (which produce 100 intervals per day with duplicate 03:00–03:45 timestamps) using explicit interval sequencing
Forward-fill and gap detection for missing meter readings
Validated against multiple full DST cycles in production

Operational hardening:

SageMaker Processing jobs replacing memory-bound pandas-on-driver architectures that previously crashed on large supplier portfolios
Bulk INSERT patterns reducing database write times from hours to under one minute
Model performance monitoring and drift detection via MLflow

Meta-learning experimentation:

Ensemble experiments combining base models — LightGBM emerged as the best meta-learner, outperforming Random Forest and linear stacking

Impact

Single, maintainable codebase instead of three parallel ones
Forecasting team can deploy a new model variant by updating a config, not rewriting a pipeline
Daily runs reliable through DST, holidays, and portfolio changes

Technologies used: LightGBM · TensorFlow/Keras (RNN) · MLflow · SageMaker · Airflow · PySpark · Redshift

Use Case 4 - Wind Farm Telemetry and Generation Forecasting

The challenge

A renewable energy operator running 8 wind parks (278 turbines) across Romania needed a unified data platform for turbine telemetry. Raw SCADA data arrived as a mix of digital state tags (operational status, fault codes, mode flags) and analog measurements (active power, wind speed, nacelle position) — at high frequency, with hundreds of distinct signals per turbine, partitioned awkwardly across legacy storage.

Downstream stakeholders needed:

Clean, queryable turbine state history for availability and performance reporting
Aggregated power output time series for forecasting and grid scheduling
A foundation for predictive maintenance and generation forecasting

What we built

Turbine state platform (digital tags):

PySpark ingestion jobs processing 8 parks × 278 turbines, encoding mixed text/numeric signal families into a consistent typed schema
Event-driven state reconstruction with forward-fill strategies appropriate to each signal type
Park-level and load-date partitioning for efficient downstream queries

Active power platform (analog tags):

Aggregation of 232 turbines from raw high-frequency telemetry (79.5M raw records) down to clean 15-minute interval series (617K records)
Resolution of small-files S3 performance issues that had been crippling downstream Spark jobs
Validated binning logic confirmed with the SCADA engineering team

Wind generation forecasting:

Short-term forecasting models on top of the cleaned telemetry, integrated with weather forecast inputs
Park-level and turbine-level forecast granularity for both operational scheduling and market participation

Impact

A single, queryable data platform replacing scattered SCADA exports
Foundation in place for advanced analytics: predictive maintenance, generation forecasting, performance benchmarking across parks

Technologies used: PySpark on EMR · S3 (optimized partitioning) · Airflow · time series forecasting (LightGBM, statistical baselines) · weather data integration

Our Methodology: The NodusDataAI Team

Our Experts

Nodus Datas is a team of data architects, data engineers, ML engineers, and data analysts. Our energy practice is led by engineers with **4+ years of hands-on production experience** at major Romanian energy companies (distribution and retail supply), combining deep domain knowledge of the electricity value chain with multi-cloud engineering depth (AWS, Azure, GCP) and academic backgrounds in statistics, probability, and applied ML.

We do not subcontract. We do not parachute in junior engineers. The people who scope your project are the people who build it.

How we engage

We work in three engagement models, depending on where you are:

1. Discovery & architecture (2–4 weeks):

We assess your current data landscape, map your business questions to data products, and deliver a concrete architecture and roadmap. Fixed scope, fixed price.

2. Build & ship (8–24 weeks):

We deliver production pipelines, ML systems, or data platforms with clearly defined milestones. You own the code, the infrastructure, and the documentation at the end.

3. Embedded team (ongoing):

For organizations modernizing their data stack at scale, we embed senior engineers alongside your team — building capability, not dependency.

Let's talk

If you operate in the Romanian or wider European energy market and any of the above sounds like a problem you have, we should talk.