AWS QuickSight: A Complete Guide from a Data Engineer Who Automated 12 Excel Workbook Reports to Self-serviced Dashboards

Expert Insight by

Jeevan Sai Sreella

Data Analytics Engineer

Data Engineering / AnalyticsLinkedIn

Jeevan is a data engineer with 6+ years of experience in data engineering and analytics. He currently works at Rectangle Health, building cloud-scale data pipelines and analytics solutions on AWS. His work includes deploying churn prediction ML models in SageMaker that help safeguard $8.3M ARR, and building multiple AWS QuickSight dashboards used by executive and business teams. Previously, he worked at Publicis Sapient, where he led ETL migrations and delivered Power BI solutions for large user groups.

Verified Expert

What is AWS QuickSight?

When I first started at Rectangle Health, the analytics landscape was chaos. Twelve Excel workbooks, each maintained by different people, with formulas referencing other workbooks that may or may not be up to date. Sound familiar?

QuickSight solved this problem, but not just because it's a dashboarding tool. The real value is how it fits into the AWS ecosystem. If your data lives in S3, Redshift, Athena, or RDS, QuickSight connects natively without complex ETL or connector licensing.

The platform has evolved significantly since its 2016 launch. Today, it includes:

• QuickSight Q: Natural language queries (ask questions in plain English)
• ML Insights: Anomaly detection and forecasting built-in
• Embedded Analytics: Embed dashboards in your own applications
• Row-Level Security: Control data access at the user level
• SPICE: The in-memory engine that changes everything

Why I Chose QuickSight Over Power BI

I've used Power BI extensively — at Publicis Sapient, I automated marketing KPI dashboards for 600 users. Power BI is excellent, especially in Microsoft-heavy environments. But when your data stack is AWS-native, QuickSight has distinct advantages.

The AWS-Native Advantage

At Rectangle Health, our data architecture looks like this:

1. S3 stores raw data (2 TB daily ingestion)
2. AWS Glue catalogs and transforms data
3. Redshift serves as our data warehouse
4. dbt manages transformation logic
5. Airflow orchestrates everything

QuickSight slots into this architecture without friction. There's no gateway to configure (like Power BI's On-Premises Data Gateway), no ODBC drivers to maintain, no authentication headaches. You grant QuickSight an IAM role, and it reads from your data sources directly.

The Pricing Reality

Here's where QuickSight surprised me. At $3/month per Reader (compared to Power BI Pro at $10/user/month), the cost advantage is significant when scaling analytics across an organization.

At Rectangle Health, we have executives who check dashboards alongside analysts who use them daily. Everyone pays the same Reader price regardless of usage frequency, but that price is 70% less than Power BI Pro.

For a 50-person organization with 10 power users (authors) and 40 dashboard consumers (readers), the math looks like this:

• Power BI Pro: 50 × $10 = $500/month
• QuickSight: 10 authors × $24 + 40 readers × $3 = $360/month

For embedded analytics or high-volume scenarios, QuickSight offers capacity-based pricing (bulk sessions) that can reduce costs further.

Understanding SPICE: The Secret Weapon

SPICE is the feature that changed how I think about business intelligence. Before SPICE, every dashboard interaction meant a query to the source database. Heavy dashboard usage could slow down production systems. DBAs would complain about BI tools hammering their servers.

SPICE decouples dashboards from source systems. Your data is loaded into SPICE once (on a schedule you control), and all dashboard interactions hit the SPICE cache instead of your database.

How SPICE Actually Works

When you create a dataset in QuickSight, you choose between:

1. SPICE Import: Data is copied into SPICE storage
2. Direct Query: Queries hit the source database in real-time

For SPICE imports, the process is:

1. QuickSight executes your dataset query against the source
2. Results are loaded into SPICE (compressed, columnar storage)
3. Dashboard interactions query SPICE, not the source
4. You schedule refreshes to keep SPICE data current

At Rectangle Health, our SPICE datasets refresh every 15 minutes via Airflow. This means dashboards are at most 15 minutes behind reality — acceptable for most business decisions, and it protects our Redshift cluster from dashboard query load.

SPICE Optimization Techniques

After building multiple dashboards, here's what I've learned about SPICE optimization:

1. Pre-aggregate when possible

Don't import raw transaction data if you only need daily summaries. Aggregate in your dbt models or SQL before loading to SPICE. This reduces SPICE storage costs and improves query performance.

2. Use incremental refreshes

For large datasets, full refreshes are expensive. QuickSight supports incremental refresh — you define a date field, and only recent data is reloaded. We use this for our transaction data, refreshing only the last 7 days each cycle.

3. Remove unused columns

Every column in your SPICE dataset consumes storage. If a column isn't used in any visual, exclude it from the dataset. I've seen datasets cut by 40% just by removing unused fields.

4. Partition large datasets

Rather than one massive dataset, consider partitioning by time period or business unit. This enables faster refreshes and more granular access control.

Building Dashboards That Executives Actually Use

Here's a hard truth: most dashboards fail not because of technical problems, but because nobody uses them. I've seen beautifully designed dashboards that executives ignore because they don't answer the questions that matter.

Start with the Decision, Not the Data

Before building any dashboard, I ask three questions:

1. What decision will this dashboard inform?
2. Who makes that decision, and how often?
3. What's the minimum data needed to make that decision confidently?

At Rectangle Health, our merchant churn dashboard exists because the leadership team needs to decide where to focus retention efforts. The dashboard shows at-risk merchants, their revenue impact, and the factors driving churn risk. Every metric ties to a specific decision.

The 5-Second Rule

Executives are busy. If they can't understand the main story within 5 seconds of opening a dashboard, they'll close it.

This means:

• Lead with the headline: Put the most important metric at the top, largest
• Use clear labels: "Merchants at Churn Risk: 1,247" not "CRN_CNT_MTD"
• Show context: Is 1,247 good or bad? Compare to last month, last year, target
• Minimize scrolling: The key story should be visible without scrolling

Dashboard Design Patterns That Work

The Executive Summary Pattern

Top row: 3-4 KPI cards showing headline metrics with trend arrows Middle section: One or two key charts (usually time series + breakdown) Bottom section: Drill-down details for those who want to explore

The Comparison Pattern

For dashboards that answer "how are we doing vs. target/last year/competitors":

• Target vs. Actual bars or gauges
• Variance highlighting (green for positive, red for negative)
• Trend lines showing trajectory

The Investigation Pattern

For operational dashboards where users need to drill into problems:

• Filters prominently displayed
• Sortable tables with conditional formatting
• Drill-through links to detail pages

QuickSight + AWS Data Stack Integration

The real power of QuickSight emerges when you integrate it into a modern AWS data stack. Here's how we've architected our analytics pipeline at Rectangle Health.

The Architecture

Data Ingestion Layer: Sources (APIs, DBs) → S3 Lake (Raw Data) → Glue (Catalog) → Redshift (Warehouse)

Analytics Layer: Airflow (Orchestration) → dbt (Models) → SPICE (Cache) → QuickSight (Dashboards)

S3 as the Foundation

Everything starts in S3. Raw data lands in our data lake in Parquet format (compressed, columnar — better for analytics than CSV or JSON). We partition by date so Athena and Glue can scan efficiently.

Our S3 structure:

s3://our-data-lake/
├── raw/
│   ├── transactions/year=2026/month=01/day=27/
│   ├── merchants/year=2026/month=01/day=27/
│   └── events/year=2026/month=01/day=27/
├── processed/
│   └── (dbt output lands here)
└── analytics/
    └── (aggregated datasets for QuickSight)

Glue for Cataloging

AWS Glue crawlers scan our S3 buckets and maintain a metadata catalog. This catalog is shared across Athena, Redshift Spectrum, and QuickSight. When a new column appears in our data, Glue detects it automatically.

Redshift as the Query Engine

While QuickSight can query S3 directly via Athena, we route most analytics through Redshift. Why?

1. Faster joins: Redshift's distributed MPP architecture handles complex joins better
2. Materialized views: We pre-compute expensive aggregations
3. Concurrency scaling: Handles dashboard query spikes automatically
4. Integration with dbt: Our transformation layer lives in Redshift

We right-sized our Redshift cluster and compressed our Parquet partitions, trimming query time by 38% and saving $42k in annual AWS spend.

dbt for Transformation Logic

Our 40+ dbt models transform raw data into analytics-ready datasets. Each model is tested, documented, and version-controlled. When business logic changes, we update the dbt model — not individual dashboard queries.

The key insight: dbt models are the single source of truth. QuickSight datasets point to dbt-created views, not raw tables. This means metric definitions are consistent across all dashboards.

Airflow for Orchestration

Airflow ties everything together. Our DAG runs every 15 minutes:

1. Check for new data in S3
2. Run dbt models if data is fresh
3. Trigger SPICE dataset refreshes via QuickSight API
4. Alert Slack if any step fails

This is how we achieve 15-minute dashboard freshness with 99% pipeline success rate.

Automating SPICE Refreshes

QuickSight's scheduled refresh is fine for simple cases, but we needed more control. Using the QuickSight API, we trigger SPICE refreshes from Airflow after dbt models complete:

import boto3

quicksight = boto3.client('quicksight')

def refresh_dataset(dataset_id):
    response = quicksight.create_ingestion(
        DataSetId=dataset_id,
        IngestionId=f'refresh-{datetime.now().isoformat()}',
        AwsAccountId='YOUR_ACCOUNT_ID'
    )
    return response

This ensures dashboards refresh only after data is ready — no more partial refreshes or stale data.

Real Project: Replacing 12 Excel Workbooks

Let me tell you about the project that convinced me QuickSight was worth the investment.

The Problem

When I joined Rectangle Health, critical business metrics lived in 12 Excel workbooks. Each workbook was owned by a different analyst or department. They shared a few characteristics:

1. Manual updates: Someone had to download data, paste it, refresh pivots
2. Version chaos: "Revenue_Final_v3_ACTUAL.xlsx" — you know the drill
3. Formula fragility: One wrong paste and calculations broke silently
4. No single source of truth: Marketing's revenue number didn't match Finance's
5. No access control: Anyone with the file could change anything

The finance team spent roughly 10 hours per week maintaining these workbooks. More importantly, executives didn't trust the numbers — they'd seen too many "oops, that formula was wrong" corrections.

The Solution

We migrated all 12 workbooks to QuickSight over 8 weeks. Here's how:

Week 1-2: Discovery and Mapping

I met with each workbook owner to understand:

• What business questions does this workbook answer?
• What data sources feed it?
• Who consumes it, and how often?
• What are the known pain points?

The key discovery: 8 of the 12 workbooks were answering variations of the same questions. We consolidated them into 3 dashboards.

Week 3-4: Data Pipeline Construction

We built dbt models to replicate the workbook logic. Every Excel formula became a documented, tested SQL transformation. When business logic was ambiguous ("how do we define active merchant?"), we documented the decision and got sign-off.

Week 5-6: Dashboard Development

We built the QuickSight dashboards, focusing on the 5-second rule. Each dashboard opened with the key metric, with drill-down details below.

Critical feature: we added a "Data Updated" timestamp to every dashboard. Users could see exactly how fresh the data was — building trust that the old Excel workflow couldn't provide.

Week 7-8: Training and Transition

We ran training sessions with all stakeholders. We kept the old Excel workbooks running in parallel for two weeks, comparing outputs. When discrepancies appeared (they did — twice), we traced them to Excel formula errors that had been silently producing wrong numbers.

The Results

The 25% data trust improvement came from a simple internal survey. Before: "I sometimes question whether the numbers are right." After: "I trust the dashboards to be accurate."

But the real win was cultural. When executives started sharing dashboard links in meetings instead of asking analysts to pull numbers, we knew the transformation had worked.

SPICE vs Direct Query: When to Use Each

One of the most common questions I get: should I use SPICE or Direct Query for this dataset?

My Decision Framework

Use SPICE when:

• Data freshness of 15-60 minutes is acceptable
• Dashboard will have many concurrent users
• Source database performance is a concern
• Dataset is under 1 billion rows / 1 TB
• Queries involve complex calculations or aggregations

Use Direct Query when:

• Real-time data is required (trading dashboards, live ops)
• Dataset is too large for SPICE limits
• Data changes constantly throughout the day
• You're querying a high-performance source (Redshift, Athena)
• Dashboard has few users with simple queries

The Hybrid Approach

For many use cases, I use both. At Rectangle Health:

• SPICE for executive dashboards (refreshed every 15 min, need to be fast)
• Direct Query for operational dashboards (real-time merchant status)
• SPICE for historical analysis (last 2 years of data, rarely changes)
• Direct Query for ad-hoc exploration (analysts querying Redshift directly)

Common Mistakes in QuickSight Implementations

After building multiple dashboards, I've seen (and made) plenty of mistakes. Here are the ones that hurt most:

Mistake Deep Dive: Skipping Data Modeling

This is the most expensive mistake. I've seen teams build dashboards directly on raw database tables, embedding business logic in QuickSight calculated fields.

The problems compound:

1. Inconsistent metrics: Each dashboard defines "revenue" slightly differently
2. Slow dashboards: Complex calculations run on every interaction
3. Hard to maintain: Changing a business rule means editing 20 dashboards
4. No testing: QuickSight calculated fields can't be unit tested

The fix: Use dbt (or similar) to create a transformation layer. Define metrics once, test them, document them. QuickSight datasets point to dbt models, not raw tables.

Mistake Deep Dive: Ignoring SPICE Refresh Monitoring

SPICE refreshes can fail. Network issues, source database timeouts, schema changes — many things can go wrong. If you don't monitor refreshes, users see stale data without knowing it.

Our solution:

1. Airflow monitors every refresh — alerts Slack on failure
2. Dashboards show "Last Updated" timestamp — users see data freshness
3. CloudWatch alarms — trigger if refresh hasn't succeeded in expected window
4. Weekly SPICE health report — shows refresh success rates by dataset

QuickSight vs Power BI vs Tableau

People ask me this constantly. Here's my honest comparison based on production experience with all three.

When to Choose QuickSight

• Your data lives primarily in AWS (S3, Redshift, Athena)
• You have many occasional dashboard users (per-session pricing)
• You want embedded analytics without extra licensing
• You need serverless, fully managed infrastructure
• You're already investing in AWS skills

When to Choose Power BI

• Your organization is Microsoft-heavy (Azure, M365, Teams)
• You need the most advanced visualization options
• Self-service BI for business users is a priority
• You want the strongest community and training resources
• Budget allows for per-user pricing

When to Choose Tableau

• Visualization quality and flexibility are paramount
• You have complex, exploratory analytics needs
• Users need maximum self-service capabilities
• You're in a data-mature organization with analyst teams
• Budget is less constrained