> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trendteller.com.br/llms.txt
> Use this file to discover all available pages before exploring further.

# Architecture Overview

> Complete system architecture and design decisions

## System Architecture

Trendteller is designed as a modern, cloud-native multi-part application that processes e-commerce data at scale. The architecture follows industry best practices for data pipelines, API design, and frontend development.

## High-Level Architecture

```
┌──────────────────────────────────────────────────────────────┐
│                   E-Commerce Platforms (9)                   │
│        Bling, VNDA, Shoppub, Tiny, Microvix, Braavo...       │
└────────────────────────┬─────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────────────────┐
│              Airbyte Integration Layer                       │
│          Custom TypeScript Connectors (15 sources)           │
└────────────────────────┬─────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────────────────┐
│            Google BigQuery Data Warehouse                    │
│                   Project: togo-425319                       │
│                 Location: southamerica-east1                 │
└────────────────────────┬─────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────────────────┐
│                 Dataform Transformations                     │
│          Bronze → Silver → Gold (Medallion)                  │
└────────────────────────┬─────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────────────────┐
│                    Hasura GraphQL API                        │
│          BigQuery Federation + PostgreSQL                    │
└────────────────────────┬─────────────────────────────────────┘
                         │
                         ▼
┌──────────────────────────────────────────────────────────────┐
│              Nuxt 3 Frontend (Netlify)                       │
│          Auth0 + Vuetify 3 + Type-safe GraphQL               │
└──────────────────────────────────────────────────────────────┘

               ┌──────────────────────┐
               │   Kestra Workflows   │
               │   AI Insights &      │
               │   Automation         │
               └──────────────────────┘
```

## Architectural Principles

<CardGroup cols={2}>
  <Card title="Separation of Concerns" icon="layer-group">
    Each component has a single, well-defined responsibility
  </Card>

  <Card title="Cloud-Native" icon="cloud">
    Built for cloud deployment with serverless and managed services
  </Card>

  <Card title="API-First" icon="code">
    All data access through well-defined GraphQL API
  </Card>

  <Card title="Type Safety" icon="shield">
    TypeScript throughout for compile-time error prevention
  </Card>
</CardGroup>

## Component Breakdown

### Data Ingestion Layer

**Technology**: Airbyte CDK + TypeScript
**Purpose**: Extract data from e-commerce platforms

* Custom connectors for 9 platforms
* Incremental and full-refresh sync modes
* Error handling and retry logic
* Rate limiting and API quota management

[Learn more →](/architecture/connectors/architecture)

### Storage Layer

**Technology**: Google BigQuery
**Purpose**: Scalable data warehouse for all analytics data

* Serverless, auto-scaling compute
* Columnar storage for fast queries
* Partitioning and clustering for optimization
* Multi-region replication for disaster recovery

[Learn more →](/architecture/infrastructure)

### Transformation Layer

**Technology**: Dataform + SQLX
**Purpose**: Transform raw data into analytics-ready tables

* Medallion architecture (Bronze/Silver/Gold)
* Git-based version control for SQL
* Dependency management and orchestration
* Data quality assertions

[Learn more →](/architecture/dataform/architecture)

### API Layer

**Technology**: Hasura GraphQL Engine
**Purpose**: Unified API for all data access

* Auto-generated GraphQL schema from database
* Role-based access control
* Query optimization and caching
* Real-time subscriptions

[Learn more →](/architecture/integration)

### Application Layer

**Technology**: Nuxt 3 + Vue 3 + TypeScript
**Purpose**: User-facing analytics dashboard

* Server-side rendering for performance
* Type-safe GraphQL with Genql
* Component-based architecture
* Progressive web app (PWA) capabilities

[Learn more →](/architecture/frontend/architecture)

### Automation Layer

**Technology**: Kestra + Node.js
**Purpose**: AI-powered insights and automation

* Scheduled and event-driven workflows
* OpenAI and Gemini AI integration
* Web crawling with Puppeteer
* Automated reporting and alerting

[Learn more →](/architecture/workflows/architecture)

## Design Patterns

### Medallion Architecture

Three-layer data architecture for data quality and performance:

<Steps>
  <Step title="Bronze Layer">
    Raw data as ingested, no transformations
  </Step>

  <Step title="Silver Layer">
    Cleansed, standardized, and validated data
  </Step>

  <Step title="Gold Layer">
    Business-ready aggregations and analytics
  </Step>
</Steps>

### Event-Driven Architecture

Kestra workflows react to events:

* New data arrivals trigger transformations
* Data quality failures trigger alerts
* Threshold breaches trigger AI analysis

### Microservices Pattern

Loosely coupled services:

* Each component can scale independently
* Failures are isolated to individual services
* Technology choices optimized per service

## Security Architecture

<AccordionGroup>
  <Accordion title="Authentication">
    * Auth0 for user authentication
    * OAuth 2.0 / OpenID Connect
    * Social login support (Google, Microsoft)
    * Multi-factor authentication (MFA)
  </Accordion>

  <Accordion title="Authorization">
    * Role-based access control (RBAC)
    * Brand-level data isolation
    * API key management for programmatic access
    * Hasura permissions engine
  </Accordion>

  <Accordion title="Data Security">
    * Encryption at rest (BigQuery, Postgres)
    * Encryption in transit (TLS 1.3)
    * API keys stored in Google Secret Manager
    * Regular security audits and penetration testing
  </Accordion>

  <Accordion title="Compliance">
    * LGPD compliance (Brazilian data protection)
    * GDPR-ready data handling
    * Data retention policies
    * Audit logging for all data access
  </Accordion>
</AccordionGroup>

## Scalability

### Horizontal Scaling

Components scale horizontally as needed:

| Component          | Scaling Method    | Limits                |
| ------------------ | ----------------- | --------------------- |
| BigQuery           | Auto-scaling      | Virtually unlimited   |
| Hasura             | Kubernetes HPA    | 10+ instances         |
| Frontend (Netlify) | Edge distribution | Global CDN            |
| Airbyte            | Docker Compose    | Per-connector scaling |
| Kestra             | Kubernetes        | Workflow parallelism  |

### Performance Optimization

<CardGroup cols={2}>
  <Card title="Query Optimization" icon="gauge-high">
    * BigQuery clustering and partitioning
    * Hasura query caching
    * Frontend data prefetching
    * API response compression
  </Card>

  <Card title="Cost Optimization" icon="dollar-sign">
    * BigQuery query cost monitoring
    * Incremental data processing
    * Materialized views for common queries
    * API rate limiting
  </Card>
</CardGroup>

## Disaster Recovery

### Backup Strategy

* **BigQuery**: Automatic snapshots (7-day retention)
* **PostgreSQL**: Daily backups to Google Cloud Storage
* **Code**: Git repositories with protected branches
* **Configuration**: Infrastructure as Code (Terraform)

### Recovery Time Objectives (RTO)

| Component     | RTO           | RPO           |
| ------------- | ------------- | ------------- |
| Frontend      | \< 5 minutes  | 0 (stateless) |
| API (Hasura)  | \< 15 minutes | \< 5 minutes  |
| BigQuery Data | \< 1 hour     | \< 1 day      |
| PostgreSQL    | \< 30 minutes | \< 1 hour     |

## Monitoring & Observability

<Tabs>
  <Tab title="Application Monitoring">
    * **Frontend**: Netlify Analytics + Sentry
    * **API**: Hasura Cloud monitoring
    * **Workflows**: Kestra execution logs
  </Tab>

  <Tab title="Infrastructure Monitoring">
    * **BigQuery**: Cloud Monitoring metrics
    * **Postgres**: NeonDB monitoring dashboard
    * **Compute**: Resource utilization tracking
  </Tab>

  <Tab title="Business Monitoring">
    * **Data Freshness**: Pipeline execution tracking
    * **Data Quality**: Assertion failure alerts
    * **API Usage**: Request volume and error rates
  </Tab>
</Tabs>

## Technology Decisions

### Why TypeScript?

* Strong typing prevents runtime errors
* Excellent IDE support and refactoring
* Growing ecosystem and community
* Seamless JavaScript interop

### Why BigQuery?

* Serverless (no infrastructure management)
* Cost-effective for analytics workloads
* Fast query performance at scale
* Native integration with Google Cloud

### Why Hasura?

* Auto-generates GraphQL from database schema
* Real-time capabilities out of the box
* Fine-grained permissions engine
* Performance optimization built-in

### Why Nuxt 3?

* Server-side rendering for SEO and performance
* Auto-imports and file-based routing
* Strong TypeScript support
* Large ecosystem of modules

## Next Steps

<CardGroup cols={2}>
  <Card title="Data Flow" icon="arrow-right" href="/architecture/data-flow">
    Understand end-to-end data movement
  </Card>

  <Card title="Integration Architecture" icon="plug" href="/architecture/integration">
    Learn how components communicate
  </Card>

  <Card title="Infrastructure" icon="server" href="/architecture/infrastructure">
    Explore cloud infrastructure setup
  </Card>

  <Card title="Component Details" icon="cubes" href="/architecture/frontend/architecture">
    Deep dive into individual components
  </Card>
</CardGroup>