Big Data Strategy for Highway Toll Data: Overcoming Performance Bottlenecks in SAP

A high-traffic highway generates between 200,000 and 500,000 pass records per day. Every record contains many fields — license plate, pass time, booth number, lane, payment method, amount and status. On an annual basis, a single highway can accumulate over 100 million records. Managing this volume in an SAP S/4HANA environment leads to severe performance issues if the right architectural decisions are not made.

The Scale of the Problem

Database Bloat

Pass data typically accumulates in custom Z tables or in FI document tables (BSEG, BKPF). Over the years these tables can grow into terabytes, which:

• Pushes report runtimes from minutes into hours
• Extends batch job durations
• Widens backup windows
• Complicates system upgrades

Performance Bottlenecks

During month-end closing, pulling millions of rows into reconciliation reports, FI document searches and CO reporting causes timeout errors and user complaints in critical business processes.

Solution Architecture

1. Data Archiving

With SAP's standard archiving tools (SARA), pass data older than a certain age is moved from the active database to archive files. Archived data remains accessible when needed but no longer affects daily operational load.

When defining an archiving strategy, consider:

• Legal retention periods (typically 5-10 years)
• Operational access needs
• Audit requirements

2. Database Partitioning

With table partitioning in HANA, data is split by time periods (monthly or yearly). Since queries are routed only to the relevant partition, performance improves dramatically.

3. Analytics Layer Separation

Separating operational (OLTP) and analytical (OLAP) workloads preserves performance on both sides:

• OLTP layer: Daily pass records, posting, reconciliation
• OLAP layer: Historical reports, trend analysis, management dashboards

By creating an analytics layer with SAP BW/4HANA or Embedded Analytics, heavy reporting queries run without affecting the operational system.

4. Real-Time Dashboards

For metrics that require live data for management (current pass count, hourly revenue, booth occupancy rates), real-time dashboards are designed using SAP Fiori or external BI tools. These dashboards are fed from optimized CDS Views.

Integration Layer Design

The integration layer between pass systems (Non-SAP) and SAP is the most critical point of the data flow. In this layer:

• Queue management: Use of message queues (MQ) to prevent data loss at peak hours
• Error handling: Retry mechanisms and dead letter queues for records that fail to transfer
• Monitoring: Alarm mechanisms that monitor integration health
• Scalability: Automatic capacity adjustment based on traffic growth

Golden Rules for Sustainable Architecture

1. Clean today's data today: Don't let erroneous and duplicate records accumulate
2. Postpone archiving — but don't forget it: Start the archiving project no later than year two
3. Separate reporting: Keep heavy reports away from the operational system
4. Preserve traceability: Build a logging foundation that lets you trace the lifecycle of every pass record

At VEX-I, we provide end-to-end consulting from architecture design to implementation for the sustainable management of highway toll data in SAP.