WIP: bootstrap and partial real Solana watcher implementation
This commit is contained in:
183
README.md
Normal file
183
README.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Solana Copy-Trading Bot (Simulation-First)
|
||||
|
||||
A highly structured, production-oriented, simulation-first Solana copy-trading bot with strong observability, persistent data collection, and a clean path toward later live trading.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture Summary
|
||||
|
||||
This bot is designed with a **Clean Architecture** (ports and adapters pattern), establishing strict boundaries between domain logic and external systems (like databases, blockchains, or APIs):
|
||||
|
||||
```
|
||||
┌───────────────────┐
|
||||
│ Wallet Watcher │
|
||||
└─────────┬─────────┘
|
||||
│ (WalletTransactionObservation)
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ Signal Processor │
|
||||
└───────────┬───────────┘
|
||||
│ (CopySignal)
|
||||
▼
|
||||
┌──────────────────┐ ┌───────────────────────┐ ┌─────────────────────┐
|
||||
│ MarketData ├────────►│ Strategy Engine │◄────────┤ Event Journal │
|
||||
│ (Interface) │ └───────────┬───────────┘ │ (Interface) │
|
||||
└──────────────────┘ │ (Approved / Reject) └─────────────────────┘
|
||||
▼
|
||||
┌───────────────────────┐ ┌─────────────────────┐
|
||||
│ Simulation Engine / │◄────────┤ MetricsProvider │
|
||||
│ Latency Scheduler │ │ (Interface) │
|
||||
└───────────┬───────────┘ └─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────┐
|
||||
│ SimulationExecutor │
|
||||
│ (TradeExecutor) │
|
||||
└───────────────────────┘
|
||||
```
|
||||
|
||||
### Key Architectural Safeguards
|
||||
1. **Simulation-First & Secure**: This version contains **no real-world trading logic**, **no private key storage**, and is completely incapable of signing or submitting live Solana transactions.
|
||||
2. **Decoupled Strategy**: The `SimpleCopyStrategy` evaluated signals purely based on input data. It doesn't query PostgreSQL, InfluxDB, or Solana RPC nodes directly.
|
||||
3. **Non-Blocking Latency Simulation**: Multiple latency scenarios (0ms, 1000ms, 3000ms, 5000ms, 10000ms) are evaluated concurrently from the same input signal. These are modeled as scheduled pending executions inside PostgreSQL, processed asynchronously when their scheduled time is reached, preventing execution blocks, sleeps, or sequential `setTimeout` delays.
|
||||
4. **Strong Observability**:
|
||||
- **PostgreSQL Event Journal**: Every lifecycle action (approvals, rejections, scheduling, entries, exits, exceptions) is recorded in a persistent central journal alongside core application metadata (`correlationId`, `appVersion`, `gitCommit`, `configurationHash`).
|
||||
- **InfluxDB Time-Series Metrics**: Custom abstraction handles metrics separately from business logic to measure latencies (detection, processing, execution), price tracking, excursions (MFE/MAE), and portfolio balances.
|
||||
5. **No Silent Database Fallbacks**: If PostgreSQL is required but unreachable, the application logs a fatal error and terminates immediately to prevent production data loss.
|
||||
|
||||
---
|
||||
|
||||
## 📂 Directory Tree
|
||||
|
||||
```
|
||||
src/
|
||||
├── application/
|
||||
│ └── index.ts # Application orchestrator, pipeline wireup & shutdown
|
||||
├── config/
|
||||
│ └── index.ts # Zod schema-validated environment configuration
|
||||
├── domain/
|
||||
│ ├── interfaces.ts # Domain core interfaces, repository contracts, and ports
|
||||
│ └── models.ts # Core type-safe data models (Signals, Trades, Events)
|
||||
├── execution/
|
||||
│ └── simulationExecutor.ts # SimulationExecutor implementing ITradeExecutor
|
||||
├── logging/
|
||||
│ └── index.ts # Pino structured JSON logging setup (pino-pretty in dev)
|
||||
├── metrics/
|
||||
│ └── index.ts # InfluxDB v2 time-series metrics provider adapter
|
||||
├── persistence/
|
||||
│ ├── db.ts # PostgreSQL pool management and health validation
|
||||
│ ├── eventJournal.ts # EventJournal business service for persistent logging
|
||||
│ ├── migrate.ts # Self-contained database migration runner
|
||||
│ └── repositories.ts # PostgreSQL repository adapter implementations
|
||||
├── signals/
|
||||
│ # Signal normalization logic
|
||||
├── simulation/
|
||||
│ ├── marketData.ts # Deterministic fake MarketDataProvider
|
||||
│ └── scheduler.ts # Multi-latency execution scheduler and loop runner
|
||||
└── solana/
|
||||
└── watcher.ts # Solana WalletWatcher simulation stub
|
||||
tests/
|
||||
├── mocks/
|
||||
│ └── InMemoryRepositories.ts # In-memory database repositories for unit tests
|
||||
└── bot.test.ts # Comprehensive suite of Vitest unit tests
|
||||
migrations/
|
||||
└── 001_initial_schema.sql # Initial database DDL schema
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Installed Dependencies
|
||||
|
||||
### Runtime Dependencies
|
||||
- **pg**: PostgreSQL client (`@types/pg` dev)
|
||||
- **@influxdata/influxdb-client**: InfluxDB v2 client
|
||||
- **pino**: High-performance structured JSON logging
|
||||
- **dotenv**: Environment variable loader
|
||||
- **zod**: Strict schema parsing and compile-time validation
|
||||
|
||||
### Dev Dependencies
|
||||
- **typescript**: Strict TypeScript compilation and type safety (v5.5.4 stable)
|
||||
- **vitest**: Lightning-fast Unit Testing framework
|
||||
- **eslint** / **typescript-eslint**: Standard code quality linting
|
||||
- **prettier**: Code formatting compliance
|
||||
- **pino-pretty**: Human-readable logging format in local development
|
||||
- **tsx**: Instant execution of TypeScript files without build step
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ Database Schema Summary
|
||||
|
||||
The database uses raw SQL migrations managed via `pnpm migrate` to establish optimal schemas, types, and indexes:
|
||||
|
||||
### Tables
|
||||
1. **`watched_wallets`**: Wallets selected for trade replication tracking.
|
||||
2. **`tokens`**: Metadata caching of copyable/discovered token mints.
|
||||
3. **`copy_signals`**: Master list of normalized wallet transaction signals with original source metadata.
|
||||
4. **`trades`**: Simulation-based trade book tracker maintaining latency parameters, PnL metrics, balances, MFE, MAE, and attribution metadata (`strategyName`, `strategyVersion`, `appVersion`, `gitCommit`, `configHash`).
|
||||
5. **`simulated_executions`**: Asynchronous task schedule recording planned execution steps ('entry'/'exit'), timestamps, prices, and status ('pending'/'success'/'failed').
|
||||
6. **`strategy_runs`**: Logging table tracking strategy accept/reject logic per correlation ID.
|
||||
7. **`event_journal`**: High-fidelity central audit trail for end-to-end replay, storing event types and JSON payloads indexed by `correlation_id` and `timestamp`.
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Configuration (.env)
|
||||
|
||||
Duplicate `.env.example` to `.env` and configure accordingly:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
### Key Parameters
|
||||
- `DATABASE_URL`: Connection string to your PostgreSQL instance (e.g., `postgres://user:pass@localhost:5432/memecoin_bot`).
|
||||
- `INFLUXDB_URL`, `INFLUXDB_TOKEN`, `INFLUXDB_ORG`, `INFLUXDB_BUCKET`: Time-series connection credentials.
|
||||
- `WATCHED_WALLETS`: Comma-separated list of Solana public keys to replicate.
|
||||
- `LATENCY_SCENARIOS`: Comma-separated latency configurations in milliseconds (e.g. `0,1000,3000,5000,10000`).
|
||||
- `SIMULATION_STARTING_BALANCE_SOL`: Accounting initial bankroll (default `1.0`).
|
||||
- `SIMULATED_POSITION_SIZE_SOL`: Fixed ticket size per trade in SOL (default `0.05`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Running the Project
|
||||
|
||||
### 1. Database Migrations
|
||||
Run migrations to set up PostgreSQL schemas:
|
||||
```bash
|
||||
pnpm migrate
|
||||
```
|
||||
|
||||
### 2. Development Mode
|
||||
Run the bot locally in watch mode (utilizes `tsx` for real-time execution):
|
||||
```bash
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
### 3. Production Build & Start
|
||||
Compile TS to standard JS and launch the production service:
|
||||
```bash
|
||||
pnpm build
|
||||
pnpm start
|
||||
```
|
||||
|
||||
### 4. Running Unit Tests
|
||||
Execute the comprehensive, deterministic test suite:
|
||||
```bash
|
||||
pnpm test
|
||||
```
|
||||
|
||||
### 5. Running Linter and Typechecks
|
||||
Validate lint compliance and strict type correctness:
|
||||
```bash
|
||||
pnpm lint
|
||||
pnpm typecheck
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ❌ Intentionally Unimplemented
|
||||
|
||||
The following operations are deliberately deferred to later development iterations:
|
||||
1. **Live Transaction Signing**: No integration with real wallets, secret key files, mnemonic storage, or Solana transaction broadcasting.
|
||||
2. **Real Solana RPC / WS Connection**: Blockchain reading is mocked via `MockWalletWatcher`. No real Solana web3.js endpoint calls are integrated to ensure total containment.
|
||||
3. **Jupiter Exchange Quote API**: Pricing utilizes the mathematical `DeterministicMarketDataProvider` wave formula.
|
||||
4. **External Services**: Web frontends, Telegram alerting channels, Grafana dashboards, and systemd Linux service configurations.
|
||||
Reference in New Issue
Block a user