Market Data Ingestion
OperationalThe entry point of the pipeline: historical market-data ingestion and real-time ingestion of one-minute bars, through the Alpaca provider.
02 / MARKET DATA & TECHNICAL ANALYSIS
AutoBroker.
From market events
to structured information.
A complete C# market-data platform that ingests historical and real-time bars, persists them in TimescaleDB, aggregates higher timeframes, calculates technical indicators, and analyzes patterns and sequences of market conditions.
DOMAINMarket data & analysis
FOUNDATIONC# / .NET
ARCHITECTUREDurable persistence + lightweight messaging
THE IDEA
Bring structure
to a stream of events.
01 — THE PIPELINE
Four stages.
Clear responsibilities.
The pipeline separates collecting market data, building timeframes, deriving technical information, and interpreting that information. TimescaleDB is the durable persistence layer for market bars across all timeframes.
Market Data Aggregator
OperationalReads one-minute bars, aggregates 5m, 15m, 1h, and daily timeframes, and persists each result in TimescaleDB. Every stage works from the same durable time-series store.
Market Data Processor
OperationalCalculates technical indicators and derives support and resistance levels from aggregated bars. Coordinated processing turns stored market data into structured technical information.
Market Data Analyzer
OperationalMatches chart and indicator patterns, evaluates combinations and sequences of technical conditions, and delivers higher-level analysis over processed market data.
All four stages work together: ingestion, aggregation, technical processing, and analysis, backed by durable market-data storage.
02 — THE TOOLKIT
A purposeful
persistence boundary.
C# / .NET
Independent ingestion, aggregation, processing, and analysis services with clear responsibilities.
Alpaca
Historical market-data and real-time one-minute bar ingestion.
TimescaleDB / PostgreSQL
The primary store for time-series market bars across all timeframes, from one-minute input to higher-timeframe aggregates. Services read and write durable bars through the PostgreSQL persistence layer.
RabbitMQ
A slim messaging layer for service signaling and work distribution. Durable market state belongs in persistence; small notifications coordinate processing across services.
SQL Server
Stores asset metadata and derived key-level records alongside TimescaleDB’s dedicated time-series bar storage.
ASP.NET Core / SignalR
Dashboard endpoints and subscription-based browser updates connect the processing pipeline to the market-data interface.
JavaScript / Canvas
A browser dashboard for candle visualization and symbol/timeframe selection.
03 — SYSTEM ARCHITECTURE
Persist. Coordinate. Process.
Historical data and real-time one-minute bars enter through ingestion. TimescaleDB persists input bars and the Aggregator’s higher timeframes. The Processor derives indicators and technical structures; the Analyzer interprets that processed information. RabbitMQ coordinates work alongside the durable data path.
CONNECTED FROM INPUT TO ANALYSIS
Historical and streaming data move through a complete pipeline. TimescaleDB preserves bars across timeframes, RabbitMQ coordinates the services, and the Analyzer turns processed indicators into market insights.
04 — ENGINEERING DECISIONS
Durable state.
Slim messaging.
Persistence + lightweight messaging
As a personal project, AutoBroker was designed with infrastructure cost and operational complexity as architectural constraints. Persisting important market state and intermediate results in the database, alongside RabbitMQ for lightweight signaling and work distribution, keeps the system appropriate for its scale.
Durable state remains in the persistence layer, messages stay relatively small, and services can recover the data they need from persistent storage. RabbitMQ coordinates processing without becoming the source of truth. This intentional separation provides a clear recovery model while avoiding unnecessary infrastructure cost and complexity.
Processing before interpretation
The Processor derives indicators, support/resistance levels, and related technical information from aggregated bars. The Analyzer handles higher-level pattern matching and combinations or sequences of conditions. Distinct stages keep those responsibilities explicit.
Explicit time windows
Aggregation uses half-open time windows and distinguishes partial, finalized, and corrected bars. Aggregation components and focused tests keep those calculation boundaries consistent.
EXPLORE THE OTHER PROJECT
VaadBot