Message Queue

Message Queue (RabbitMQ)

RabbitMQ acts as the central message broker between the MongoDB storage and the formatter service, ensuring reliable data transmission and processing.

Queue Structure

// Main Queue
- main_queue    // Raw cryptocurrency data
- error_queue         // Error handling and retries

Message Flow

1. Producer (MongoDB Change Streams)
   • monitors MongoDB for new data
   • Sends raw data to RabbitMQ
2. Consumer (Formatter Service)
   • Receives messages from the queue
   • Processes data in order
   • Handles data transformation

Error Handling

• Dead letter exchanges for failed messages
• Automatic retries
• Error logging and monitoring