Reading Time: 66 mins

ENGINEERED EFFICIENT ETL DATA PIPELINE FOR NEAR REAL-TIME SELF-SERVICE REPORTING

CASE STUDY

A DATA ENGINEERING & BUSINESS ANALYTICS CASE STUDY

A leading consumer finance company with over 500 branches in twenty-two states throughout the United States wanted an automated master business operations dashboards down to every single branch to improve agility and efficiency in resource allocation.

The client’s objective was to re-engineer the in-house data architecture for a futuristic solution that provides 360-degree business visibility for quick decision-making.

A DATA ENGINEERING & BUSINESS ANALYTICS CASE STUDY

A leading consumer finance company with over 500 branches in twenty-two states throughout the United States wanted an automated master business operations dashboards down to every single branch to improve agility and efficiency in resource allocation.

The client’s objective was to re-engineer the in-house data architecture for a futuristic solution that provides 360-degree business visibility for quick decision-making.

Our client is a finance company founded in 2002 to provide customers with creative, flexible, and convenient lending options. It provides personal loans, auto repair financing, credit cards, and other financial services to customers nationwide.

The company is known for its fast access to funds, transparent terms, and outstanding customer service.

ABOUT CUSTOMER

ABOUT CUSTOMER

Our client is a finance company founded in 2002 to provide customers with creative, flexible, and convenient lending options. It provides personal loans, auto repair financing, credit cards, and other financial services to customers nationwide.

The company is known for its fast access to funds, transparent terms, and outstanding customer service.

PROBLEM STATEMENT

The client wanted to improve its ability to provide customers with creative, flexible, and convenient lending options. To do that, it needed to innovate in two ways: first, by developing its digital channels, and second, by creating a holistic view of customer data from those digital channels that would enable it to develop better products and services.

But two problems were standing in the way of this goal.

The first was latency issues:
The existing process of generating reports was extremely manual-intensive and took a few hours to generate. As a result, data was updated only once in 30 to 60 days, limiting them to getting the maximum value from the campaign insight.

PROBLEM STATEMENT

The client wanted to improve its ability to provide customers with creative, flexible, and convenient lending options. To do that, it needed to innovate in two ways: first, by developing its digital channels, and second, by creating a holistic view of customer data from those digital channels that would enable it to develop better products and services.

But two problems were standing in the way of this goal.

The first was latency issues:
The existing process of generating reports was extremely manual-intensive and took a few hours to generate. As a result, data was updated only once in 30 to 60 days, limiting them to getting the maximum value from the campaign insight.

The second was performance issues:
When refreshing pages or processing real-time data on existing dashboards and reports, the client found that the system was not fast enough for the high pace requirements for competitive decision-making. This issue was caused by our client’s use of a NoSQL database (MongoDB), which uses JSON-like documents with optional schemas. A solution that is useful for storing data but difficult when it comes to build business dashboards and reporting layers on top of it.

Our client knew they needed a better solution that would allow them to meet their customers’ needs while also maintaining agility in their reporting capabilities. They required a highly effective data infrastructure for a real-time unified business analytics platform to generate business and customer insights.

To overcome these challenges, they turned to Zuci Systems, an intelligent automation solutions provider that helps businesses transform digitally through smart data engineering solutions.

PROBLEM STATEMENT

PROBLEM STATEMENT

The second was performance issues:
When refreshing pages or processing real-time data on existing dashboards and reports, the client found that the system was not fast enough for the high pace requirements for competitive decision-making. This issue was caused by our client’s use of a NoSQL database (MongoDB), which uses JSON-like documents with optional schemas. A solution that is useful for storing data but difficult when it comes to build business dashboards and reporting layers on top of it.

Our client knew they needed a better solution that would allow them to meet their customers’ needs while also maintaining agility in their reporting capabilities. They required a highly effective data infrastructure for a real-time unified business analytics platform to generate business and customer insights.

To overcome these challenges, they turned to Zuci Systems, an intelligent automation solutions provider that helps businesses transform digitally through smart data engineering solutions.

BUSINESS GOALS

BUSINESS GOALS

When the client approached Zuci Systems, they had latency and performance issues. Real-time data processing was delaying their reports. The NoSQL database they were using—MongoDB, which uses JSON-like documents with optional schemas—made it difficult to build business dashboards and reports.

Zuci Systems understood the customer’s needs and developed a bridge between MongoDB and SQL server using our in-house enterprise data hub solution ZIO. The ETL data pipelines helped to park JSON format data into a row and column format and dump it into the SQL server for high-speed dashboarding and faster query processing time. Leveraging ZIO, we automated data ingestion, integrated multiple data sources with APIs, and scaled the reporting and forecasting layers into Power BI for near real-time insights.

This allowed the client to reduce latency by 90% and quickly access accurate data about their loan performance, increasing client retention rates and improved growth rates by 11%.

SOLUTION

SOLUTION

When the client approached Zuci Systems, they had latency and performance issues. Real-time data processing was delaying their reports. The NoSQL database they were using—MongoDB, which uses JSON-like documents with optional schemas—made it difficult to build business dashboards and reports.

Zuci Systems understood the customer’s needs and developed a bridge between MongoDB and SQL server using our in-house enterprise data hub solution ZIO. The ETL data pipelines helped to park JSON format data into a row and column format and dump it into the SQL server for high-speed dashboarding and faster query processing time. Leveraging ZIO, we automated data ingestion, integrated multiple data sources with APIs, and scaled the reporting and forecasting layers into Power BI for near real-time insights.

This allowed the client to reduce latency by 90% and quickly access accurate data about their loan performance, increasing client retention rates and improved growth rates by 11%.

SOLUTION BLOCK DIAGRAM

SOLUTION BLOCK DIAGRAM

HOW ZUCI SYSTEMS HELPED

HOW ZUCI SYSTEMS HELPED

HOW ZUCI SYSTEMS HELPED

HOW ZUCI SYSTEMS HELPED

HOW ZUCI SYSTEMS HELPED

HOW ZUCI SYSTEMS HELPED

BUSINESS OUTCOME

0Hour
Data Availability SLA in Power BI
0mins
Data Ingestion (Incremental) SLA
0mins
Data Processing (Incremental) SLA
0%
Faster data collection and enrichment
0+
Tactics monitored with interactive visualizations

Near real-time visibility to
Operational Metrics and KPIs

BUSINESS OUTCOME

0Hour
Data Availability SLA in Power BI
0mins
Data Ingestion (Incremental) SLA
0mins
Data Processing (Incremental) SLA
0%
Faster data collection and enrichment
0+
Tactics monitored with interactive visualizations

Near real-time visibility to
Operational Metrics and KPIs

TECH STACK

TECH STACK

READY TO BUILD A SINGLE SOURCE OF TRUTH SYSTEM FOR REAL-TIME BUSINESS REPORTING, OPTIMIZATION, AND ANALYSIS?

DROP US YOUR EMAIL, AND OUR EXPERTS WILL GET IN TOUCH.

Download My Copy
Book a consultation