More Thoughts On High Level System Requirements
I've started with three simultaneous projects:
- Building a backtester that can use both end-of-day (EOD) data and 15 minute data.
- Using java for the algo components
- EOD data from http://www.ivolatility.com/
- 15-minute data from http://www.livevol.com/
- The data was loaded into a MySQL database with one table per symbol
- The 15-minute data is requiring between 10GB and 25GB per table for five years of data.
- The EOD data is requiring between 100MB and 700MB per table for two to five years of data
- Also created tables:
- containing the dates of market holidays -- will need to update this table annually -- possibly
- containing options expiration dates for indexes for the last five years
- Besides the MySQL Workbench, I am also using HeidiSQL and Navicat for MySQL.
- Building a proof of concept (POC) of the end-to-end infrastructure required to run a live trading algorithm. So far, I have built components to test
- SSL from the browser to Node.js
- Client certificates
- Browser authentication with Node.js using the Passport.js module integrating with MySQL
- Streaming data from a java process to RabbitMQ to Node.js to a Dojo table in a browser.
- Java polling for data from Yahoo
- Java streaming data via a DDE connection from ThinkOrSwim
- Building the trading system.
- Just in the design phase at this point. Will need at least the following components:
- User Interface (Node.js)
- Strategy Manager
- Position Manager
- Risk Manager
- Broker Connector (streaming and FIX)
- DB Connector (to MySQL)
- Order Manager
- Account Manager
- Communications Manager
- Message Bus - (RabbitMQ or similar)
