Skip to main content

​
Overview

Understanding these key concepts will help you work effectively with Trendteller’s platform and make the most of its capabilities.

​
Medallion Architecture

Trendteller implements a Medallion Architecture for data processing, organizing data into three distinct layers:
1

Bronze Layer (Raw Data)

Purpose: Store raw, unprocessed data exactly as received from source systems.
  • Data is ingested from Airbyte connectors
  • Maintains original source format and structure
  • Provides full audit trail and data lineage
  • No transformations applied
Example: Raw order data from Bling API with all original fields
2

Silver Layer (Standardized)

Purpose: Clean, standardize, and deduplicate data across sources.
  • Applies data quality rules and validations
  • Standardizes data types and formats
  • Deduplicates records across brands
  • Adds business keys and metadata
Example: Orders from all 11 brands in a unified schema
3

Gold Layer (Analytics-Ready)

Purpose: Create business-ready data models optimized for analytics.
  • Aggregated metrics and KPIs
  • Pre-joined dimension tables
  • Optimized for query performance
  • Business logic applied
Example: Daily sales metrics by brand, category, and channel
The Medallion Architecture enables both flexibility (access to raw data) and performance (optimized analytics tables).

​
Multi-Brand Consolidation

​
Brand Aggregation

Trendteller consolidates data from 11 different brands across 9 e-commerce platforms into a unified analytics system.
  • Bling - Brazilian ERP and e-commerce platform
  • VNDA - Fashion retail platform
  • Shoppub - Marketplace integration
  • Tiny - ERP and inventory management
  • Microvix - Retail management system
  • Braavo - E-commerce platform
  • JetERP - Enterprise resource planning
  • Google Shopping - Product feed integration
  • Totvs Moda - Fashion industry ERP
Each brand’s data is:
  1. Extracted via custom Airbyte connectors
  2. Loaded into Bronze layer with brand identifier
  3. Standardized in Silver layer using common schema
  4. Aggregated in Gold layer for cross-brand analytics
This enables:
  • Cross-brand performance comparisons
  • Consolidated inventory management
  • Unified customer analytics
  • Portfolio-wide forecasting

​
Brand Isolation vs. Consolidation

  • Isolated Views
  • Consolidated Views
  • Each brand’s data remains separately accessible
  • Brand-specific dashboards and reports
  • Individual brand performance tracking
  • Maintains data privacy between brands

​
Data Integration Patterns

​
Source-to-Warehouse Pattern

Source System → Airbyte Connector → BigQuery (Bronze) → Dataform (Silver/Gold)
Key Principles:
  • Full Refresh: Some sources perform complete data refresh
  • Incremental Sync: Most sources use incremental updates based on modification timestamp
  • Change Data Capture: Tracks changes at the source level
  • Error Handling: Failed syncs are logged and can be retried

​
API-First Architecture

Trendteller exposes all data through a GraphQL API powered by Hasura V2:

GraphQL Benefits

  • Type-safe queries
  • Flexible data fetching
  • Real-time subscriptions
  • Automatic schema generation

Hasura Features

  • BigQuery federation
  • PostgreSQL integration
  • Role-based access control
  • Performance optimization

​
AI-Powered Analytics

​
Insights Generation

Trendteller uses AI models to generate actionable insights:
  • Sales forecasting using historical trends
  • Inventory optimization predictions
  • Demand forecasting by product category
  • Seasonality detection and modeling
Models Used: OpenAI GPT-4, Google Gemini AI
  • Identifies emerging product trends
  • Detects anomalies in sales patterns
  • Analyzes customer behavior shifts
  • Market trend correlation
  • Daily/weekly automated insights
  • Performance summaries by brand
  • Exception reporting (low stock, unusual patterns)
  • Executive dashboards with AI commentary

​
Web Crawling

Kestra scripts use Puppeteer for competitive intelligence:
  • Competitor pricing monitoring
  • Market availability tracking
  • Product catalog comparison
  • Review and sentiment analysis

​
Type-Safe Development

​
Frontend Type Safety

The platform frontend uses Genql for type-safe GraphQL queries: 
import { createClient } from '@/generated/genql'

const client = createClient()

// Fully typed query
const orders = await client.query({
  orders: {
    id: true,
    total: true,
    customer: {
      name: true,
      email: true
    }
  }
})
Type safety prevents runtime errors and provides excellent IDE autocomplete support.

​
Backend Type Safety

Airbyte connectors are built with TypeScript:
  • Compile-time type checking
  • IDE autocomplete for API schemas
  • Reduced runtime errors
  • Better maintainability

​
Authentication & Authorization

​
Auth0 Integration

1

User Authentication

Users authenticate via Auth0 using username/password or social logins.
2

Token Generation

Auth0 issues JWT tokens with user claims and roles.
3

API Authorization

Hasura validates JWT tokens and enforces role-based access control.
4

Frontend Session

Nuxt Auth manages session state and token refresh.

​
Role-Based Access

  • Admin: Full access to all brands and data
  • Brand Manager: Access to specific brand(s) data
  • Analyst: Read-only access to analytics
  • API User: Programmatic API access

​
Data Quality & Validation

​
Quality Checks

Dataform implements multiple data quality layers:

Schema Validation

  • Required fields enforcement
  • Data type validation
  • Format standardization
  • Referential integrity

Business Rules

  • Logical value constraints
  • Cross-field validations
  • Deduplication logic
  • Historical consistency checks

​
Monitoring & Alerting

  • Data freshness monitoring
  • Pipeline failure alerts
  • Data quality score tracking
  • Anomaly detection

​
Next Steps

System Overview

Review the complete system architecture

Components

Explore individual system components

Architecture

Deep dive into architectural decisions

API Reference

Start building with the API