Executive Summary
A leading advertising agency in the US faced challenges in accurately linking sales to marketing efforts. Inconsistencies in the data processing pipeline caused fragmented data and incorrect sales attribution. Different advertiser names across multiple data sources made matters worse.
DataPulse Consulting designed and implemented an automated, end-to-end (E2E) data processing platform. Our goal was to standardize data, integrate it efficiently, and provide clear insights.
We used AWS technologies like AWS Glue, Amazon S3, and Amazon Athena. We also managed the process with Apache Airflow. This platform automates the normalization of advertiser names, processes user and sales data, and consolidates these streams into a final, accurate sales attribution table. As a result, our client can make confident, data-driven decisions.
Key Results
- Improved Accuracy: We significantly increased sales attribution accuracy by over 80% by standardizing advertiser names and ensuring consistent data integration.
- Operational Efficiency: Automation reduced manual work, allowing our client’s team to focus on strategic tasks.
- Scalability: The architecture scales with our client’s increasing data volumes, ensuring long-term sustainability.
- Cost Optimization: Our solution minimizes unnecessary data processing and storage costs while maintaining high performance.
Platform Architecture Overview
- Data Ingestion: Sales and user data files are uploaded to designated folders within an S3 bucket, triggering the data processing workflow orchestrated by Airflow.
- Sales Advertisers File Processing: The first AWS Glue job reads the sales_advertisers file weekly. This job separates the sales data from advertiser information and creates a master list of normalized advertiser names, consolidating any new advertiser names that appear in future files.
- User Data Processing: The second AWS Glue job, triggered every six weeks, processes the user data, cleaning it and preparing it for integration with the sales and advertiser data.
- Final Sales Attribution Table Creation: A third AWS Glue job combines the cleaned and normalized sales, advertiser, and user data from the trusted S3 bucket, creating a final, consolidated table of correctly reported sales with normalized advertiser names. This table is stored in Amazon Athena for accurate sales attribution.
Figure 1: Platform Architecture Diagram
Platform Benefits
- Real-Time Insights: The platform provides up-to-the-minute analytics, allowing the advertising agency to respond quickly to changing business conditions.
- Optimized Efficiency: By reducing lead times and enhancing inventory turnover, the platform streamlines operations and improves bottom-line performance.
- Cost-Effective Scaling: Designed to handle growing data volumes, the platform scales seamlessly while keeping costs under control, ideal for the agency’s expanding client base.
- Customizable Dashboards: The flexibility of the dashboarding system allows customers to tailor insights according to their unique business requirements, providing greater control over their data analysis.
Final Thoughts
In conclusion, the Advertising Agency Sales Attribution Platform built by DataPulse Consulting has transformed how the agency manages its data. With real-time insights, enhanced operational efficiency, and a scalable, cost-efficient architecture, the agency is now positioned to offer an even more powerful solution to its clients. Ultimately, this project demonstrates how leveraging cloud-based technologies can drive business growth and optimize performance.
