Python & AI Engineering Consultant on Jonathan Franklin

Automated Trading System

Mon, 07 Sep 2026 00:00:00 +0000

Problem

Pricesearcher’s automated FX/CFD trading operation needed more than strategies. Running through MetaTrader 5 meant connecting research and backtesting to live execution, account-level risk settings, data collection and reporting. The software also needed to handle market hours and the practical constraints of a Windows-based trading terminal.

Architecture

MQL5 Expert Advisors ran inside MetaTrader 5, while a Python/FastAPI trade engine used the MT5 connector alongside the terminal on a Windows VPS. S3 held exported trade history and the OHLCV pipeline supplied bars for research and backtesting. AWS monitoring fed alerts into Slack, and an internal dashboard exposed strategy, account and commission views.

MT5 OHLCV Bars Pipeline

Fri, 04 Sep 2026 00:00:00 +0000

Problem

Trading research needed reusable histories of one-minute open, high, low, close and volume bars. MetaTrader 5 held the source data, but its Windows terminal loaded symbol history on demand. Repeated manual exports made it awkward to retrieve consistent date ranges across symbols.

Architecture

An hourly Monday-to-Friday job on a Windows EC2 instance retrieves M1 bars through the MT5 Python API. Bars are stored as compressed daily symbol/date partitions on S3, with monthly aggregates. A CLI reads selected partitions into a local cache and imports historical CSV exports into the same layout. Lambda scheduling, CloudWatch monitoring, Pulumi and PowerShell manage the runtime around the terminal.

Statistical Arbitrage Strategy

Thu, 30 Jul 2026 00:00:00 +0000

Problem

Pricesearcher wanted to research statistical arbitrage as another strategy for its FX/CFD trading operation. The approach measures a pair’s spread as a z-score, enters when it moves away from its recent range, and seeks an exit as it reverts. The difficult part is assessing whether that relationship persists, how execution and risk settings affect the result, and what happens when several setups share an account.

Architecture

Research workflows load and resample bar data, rank pairs and feed backtesting tools. Scheduled AWS services run the signal engine against the S3 bar lake, while an on-demand service uses the same engine for historical inputs. MQL5 and Python engines execute configured setups, with trade-level reporting and Slack monitoring closing the review loop.

Socketrade Trading Bot

Thu, 18 Jun 2026 00:00:00 +0000

Problem

MetaTrader 5’s Python API is request/response based. For Socketrade, I wanted the terminal to push ticks, closed bars and trade transactions to Python strategies, with commands travelling back for order execution.

The runtime also needed to manage concurrent setups—each a strategy, symbol or pair, timeframe and risk configuration. Changing a setup should not require restarting the bot, and disabling new entries should not abandon positions already open.

Architecture

An MQL5 Expert Advisor and Python asyncio hub form a newline-delimited JSON socket bridge. The hub feeds terminal events into a strategy runtime whose setups are defined by YAML configuration and model files. A backtester feeds historical bars through the same strategy runtime, with a small web UI generating forms from the configuration schema and charting results.

Quantrade Research System

Tue, 16 Jun 2026 00:00:00 +0000

Problem

I built Quantrade as a personal research system for machine-learning trading strategies. I wanted to compare ideas on consistent data and evaluation rules, rather than judge a model by prediction accuracy or a favourable chart alone. Trading fees, drawdown and the order of training and test data all change how useful a result is.

Architecture

A Python research library connects market-data providers, feature preparation, PyTorch trainers and fee-aware trade simulations. A YAML-driven multiprocessing runner executes comparable benchmark plans and writes result tables, while streaming feature and account models connect the same research ideas to an event-driven strategy interface.

Ads & Affiliate Revenue Platform

Fri, 12 May 2023 00:00:00 +0000

Problem

Pricesearcher bought traffic through Google, Bing and Facebook advertising and earned revenue through affiliate and traffic partners. Comparing advertising costs with that income required joining records from separate systems with different identifiers, timezones and reporting periods. Campaign- and keyword-level reporting helped inform bid and budget decisions.

Architecture

Airflow and Lambda jobs imported campaign, ad-group and keyword data alongside revenue and conversion data from affiliate partners. A Python reporting API joined that data for an internal Next.js/Tailwind dashboard. The surrounding platform also used S3-hosted landing-page variants and SNS-to-Slack notifications.

Pricesearcher.com Web Frontend

Mon, 27 Feb 2023 00:00:00 +0000

Problem

Pricesearcher’s public site covered product comparison and verticals such as insurance, travel, utilities, education and finance. These needed different layouts and features without becoming separate websites to maintain. Product search also had to present results from multiple providers, with market-specific relevance and affiliate tracking.

Architecture

The public frontend used React and TypeScript with server-side rendering, followed by a Next.js codebase. A configuration-driven view-mode system selected layouts, features and styles for each vertical, while product pages consumed the product search API. AWS Serverless deployments provided branch-based staging and production environments.

Data Orchestration & Platform Operations

Tue, 17 May 2022 00:00:00 +0000

Problem

Pricesearcher’s feed ingestion, product building, indexing and warehouse jobs depended on scheduled workflows. That orchestration layer needed to move from Airflow 1 to Airflow 2 while continuing to support existing jobs. Monitoring also needed to reflect changing feed schedules: a weekly feed should not trigger the same freshness warnings as a daily one.

Architecture

Airflow scheduled the data workflows, including validation tasks around the pipelines. SNS and CloudWatch events reached Slack through Lambda functions. Supporting services included query expansion, merchant and brand enrichment, Spark warehouse uploads and Terraform-managed infrastructure.

Elasticsearch Product Search Index

Sat, 11 Sep 2021 00:00:00 +0000

Problem

Pricesearcher’s search service used separate Elasticsearch product indexes for different countries. Index builds needed to carry product identity and cross-merchant links into searchable records, while accommodating market-specific merchant rules and schedules. The team also used Google Sheets to maintain smaller datasets that needed to be searchable.

Architecture

A Makefile-driven pipeline generated Avro source data through Athena, downloaded it from S3 and used a PyPy builder to populate Elasticsearch. It supported the site’s nitor index and a separate nested index type. Separate configurable builders published selected Google Sheet tabs as Elasticsearch indexes.

Multi-Source Product Search API

Fri, 16 Jul 2021 00:00:00 +0000

Problem

Pricesearcher’s website, browser extension and external integrations needed product data from both the company’s Elasticsearch index and third-party providers. Combining those sources meant dealing with different response formats, provider latency and client ranking requirements. The API also needed clearer boundaries between the interfaces used by different clients.

Architecture

The Python/Flask API combined the internal Elasticsearch index with Amazon, Yahoo and Kelkoo. Third-party requests ran asynchronously and provider failures were handled separately. Redis with RediSearch supported filtering, sorting and facets over cached results. Web, browser-extension, external-client and core APIs ran as separate deployments with their own OpenAPI specifications.

Product Web Crawler

Tue, 08 Jun 2021 00:00:00 +0000

Problem

Pricesearcher’s price comparison service used merchant websites as a source of product data. The crawl pipeline read sitemaps, found product pages and extracted structured data, with separate deployments for environments and AWS regions. Maintaining it meant handling differences between merchant sites and balancing crawl frequency against the cost of repeated requests.

Architecture

The Python crawler ran on AWS and tracked crawl state and cycles in DynamoDB. Sitemaps stored in S3 fed product-page crawling, while a management API behind Cognito and API Gateway managed merchants and ingestion sources. A companion crawler collected website metadata for the data platform.

Merchant Image Proxy

Fri, 19 Feb 2021 00:00:00 +0000

Problem

Pricesearcher displayed product images hosted by merchants, with differing URL formats and image-serving behaviour. The site needed a common endpoint for image requests, resizing and caching.

Architecture

The service was the company’s fork of willnorris/imageproxy, exposed as a signed company-domain endpoint and deployed on AWS Lambda behind API Gateway. Terraform managed separate staging and production environments.

What I built

Maintained and customised the existing Go fork.
Added options to pass through the original image or return no image when processing failed, instead of always using the default placeholder.
Fixed decoding and encoding behaviour that corrupted proxied URLs containing special characters.
Aligned the Terraform version with the deployments’ remote state.

Engineering decisions

The implementation extended the upstream proxy rather than replacing its existing resizing and caching behaviour.
Fallback behaviour was configurable between the original image, no image and the upstream default placeholder.

Technologies

Go, AWS Lambda, API Gateway, Terraform, Docker, Jenkins.

Affiliate Offers & Vouchers Pipeline

Tue, 19 Jan 2021 00:00:00 +0000

Problem

Pricesearcher’s deals content came from affiliate networks including Commission Junction and Webgains. Their authenticated feeds used different formats and conventions for promotions, dates and links. The data needed a common shape and a connection to the merchant catalogue before it could support the site’s deals search.

Architecture

Authenticated and paginated feeds were downloaded to S3, then SQS messages triggered network-specific processing. Normalised promotions were written to an Elasticsearch index for the site’s offers and vouchers search. A separate synchronisation path maintained Kelkoo pass-through URLs between Redis and Google Sheets.

Product Catalog Pipeline: Feeds to Products

Thu, 10 Dec 2020 00:00:00 +0000

Problem

Pricesearcher needed to turn merchant feeds into a catalogue suitable for price comparison. Downloaded listings had to be normalised, assigned to the correct market and currency, and combined with reference data before indexing. Expansion into additional countries also required changes to feed processing, catalogue seeding and scheduling.

Architecture

The ingestion stage processed merchant feeds into Parquet. A Spark product builder joined them with warehouse price history, categories, translated brands, linked product IDs and image-quality data, then supplied the search index pipeline. Country-specific processing and schedules ran through the same flow.

About

Mon, 01 Jan 0001 00:00:00 +0000

I’m Jonathan Franklin, a senior software engineer and technology leader with 20+ years of commercial development experience. I specialise in Python, AWS, data engineering, AI/LLM systems and algorithmic trading.

At Pricesearcher, I am Chief Technology Officer and Algorithmic Trader, responsible for the technology behind a price-comparison platform serving multiple European markets and for developing and operating the company’s automated FX/CFD trading strategies.

I work hands-on across the full lifecycle: research, architecture, implementation, deployment, monitoring and ongoing operation. My background also includes CTO, head-of-technology and technical-leadership roles in startups, contract teams and established engineering organisations.

Thank you

Mon, 01 Jan 0001 00:00:00 +0000