Summary
This essay argues that traditional EA development practices, which often rely on excessively long learning periods, can lead to overfitting and hinder performance in dynamic markets. By focusing on short-term optimization and continuous adaptation, traders can create more robust and profitable EAs. The key is to continuously refine the EA’s parameters based on recent market data, conduct rigorous out-of-sample testing, and implement robust risk management strategies. This approach allows EAs to better adapt to evolving market conditions, leading to improved performance and reduced risk.
Introduction
Expert advisors (EAs) aim to capture the inherent behavioral characteristics of trading instruments. Effective EAs rely on accurate understanding of these characteristics, which necessitates continuous learning from historical data. However, the prevailing practice in the MQL5 community emphasizes excessively long learning periods, often spanning several years. This approach, while seemingly providing a sense of security, can lead to overfitting and hinder adaptability to evolving market dynamics.
The Perils of Long-Term Learning:
Overfitting
Long learning periods increase the risk of overfitting, where the EA becomes overly attuned to past market conditions, including anomalies and noise. This can result in poor performance when market conditions change.
False Sense of Security
Presenting decades of backtest results with seemingly stable equity curves can create an illusion of safety. However, these results may not accurately reflect real-world performance, especially in volatile or rapidly changing markets.
History Reading, Not Future Forecasting
EAs trained on excessively long periods often become “history readers,” effectively memorizing past price action rather than identifying and adapting to evolving market patterns.
Big Stop-Losses High Risk of Blowing Accounts
A significant portion of MQL5 users does not t adequately test or optimize their EAs. Let’s consider an EA that exhibits a maximum drawdown of $1400 over the past five years. This should ideally represent our maximum acceptable risk.If this EA encounters significant losses, we should adhere to our stop-loss (SL) order until the maximum drawdown of $1400 is reached or exceeded. However, human psychology often tempts us to hold onto positions longer than we should, hoping for a recovery.
What if our long-term backtesting was inaccurate, and the true maximum drawdown of the EA exceeds $1400? This could lead to significant and unexpected losses, potentially jeopardizing the entire trading account. This scenario carries a substantial risk of significant account losses.
By carefully considering risk parameters and conducting thorough backtesting, we can strive to minimize these stressful situations and enhance our trading experience
The Case for Short-Term Optimization:
Adaptability to Evolving Markets
Focusing on shorter learning periods, such as 5-6 months, allows the EA to adapt more effectively to recent market trends, including short-term cycles, news-driven volatility, and shifts in market sentiment.
Reduced Risk
By focusing on recent market behavior, the EA can better assess and mitigate current risks, such as sudden market shifts or unforeseen events. This can lead to more realistic risk management and reduced drawdowns.
Improved Performance
By continuously adapting to changing market conditions, short-term optimization can lead to improved performance and potentially higher returns compared to EAs trained on static, long-term data.
Some More Considerations:
The financial markets are constantly evolving. Factors such as the behavior of market participants, advancements in trading technology, and shifts in economic conditions are constantly in flux. It’s unrealistic to expect a single trading algorithm to consistently capture the characteristics of a trading instrument over extended periods, such as five or ten years.
Even if an algorithm could achieve consistent long-term performance, it would likely require significant constraints to mitigate the risk of overfitting to historical data. This stringent approach can lead to a substantial reduction in potential returns, resulting in an unfavorable risk-reward profile.
This study proposes a novel approach to optimizing expert advisors, aiming to enhance their performance and improve risk management.
Let’s delve deeper into this concept by examining the characteristics of its short-term cycles.
A Brief Description of Short Term Cyclical Characteristics
Short-term cyclical characteristics influenced by various factors, such as macroeconomic data releases, market sentiment, geopolitical events, and central bank policy decisions. These cycles are often driven by trader psychology, market liquidity, and algorithmic trading strategies. Here’s a breakdown of the typical characteristics and durations:
1. Intraday Cycles
Duration: Hours to a single day.
Characteristics:
Typically driven by market sessions (e.g., Asian, European, and US trading hours).
Volatility spikes during key market openings and major economic data releases (e.g., nonfarm payrolls, ECB announcements, or Fed interest rate decisions).
Patterns often include range trading during low-volume hours and breakouts during high-volume sessions.
2. Multi-Day Cycles
Duration: 2–5 days.
Characteristics:
Often linked to short-term sentiment shifts, such as positioning ahead of major economic or geopolitical events.
Includes patterns like the “Monday effect” or reactionary movements following weekend news.
These cycles may reflect corrective moves after strong trends or consolidations around specific technical levels.
3. Weekly or Bi-Weekly Cycles
Duration: 1–3 weeks.
Characteristics:
May align with central bank meeting cycles, particularly for the ECB or the Federal Reserve.
Reflects market adjustments to changes in monetary policy expectations or evolving macroeconomic data.
Traders often refer to these as part of a “mini-trend” within a broader trend.
4. Seasonal Cycles
Duration: A few weeks to months.
Characteristics:
Seasonal tendencies can arise due to recurring economic factors, such as fiscal year-end flows, tax deadlines, or corporate repatriation.Mid-year and end-of-year periods often show distinct trading patterns linked to portfolio rebalancing or hedging activity.
By analyzing the short-term characteristics of price action, we can identify key cyclical patterns. If we select a sufficiently long learning period, our EAs can potentially learn from these patterns, which typically include:
Intraday cycles
Multi-day cycles
Weekly or bi-weekly cycles
Seasonal cycles
These cycles offer valuable insights into market behavior and can present potential trading opportunities. However, focusing on historical data from 8 years ago may not be relevant for current market conditions. We need to prioritize learning from the most recent price action to adapt to the evolving market dynamics.
Methodology:
1- Define Learning Period:
Determine an appropriate learning period. The study above suggests typically 5-6 months learning period should be enough. It could be shortened with respect to desired trading frequency and the instrument’s typical cycle durations.
2- Optimize:
Optimize the EA parameters within the defined learning window.
3- Out-of-Sample Testing:
Conduct rigorous out-of-sample testing, including forward and rewind tests, to assess the EA’s performance on data not used in the optimization process.
4 – Regular Re-optimization:
Re-optimize the EA periodically, ideally monthly or bi-weekly or even more frequently for high-frequency trading strategies, to ensure continued adaptation to evolving market conditions.
THE APPLICATION
If today is 21st of December, we can setup our optimization routine as follows:
When we apply this approach to a trading algorithm, we have the following equity curve. Looking at it, this set file is accepted because it performs well in and out of sample tests.
How Should You Manage Your Risk?
Significant news events or economic data releases can abruptly shift market sentiment, potentially exceeding the scope of the learning period for our EA.
Implementing a stop-loss (SL) order is crucial for risk management. The SL level should be carefully determined to avoid overly tight settings, which can lead to frequent premature exits, or excessively loose settings, which may not adequately protect capital during adverse market conditions.
Ideally, the SL should be set to limit potential losses to an amount that does not exceed a single day’s average profit. For instance, if your daily average profit is $40, the SL should not exceed this amount.
While some flexibility may be possible when trading exclusively with EAs, it’s generally advisable to limit the potential loss to no more than three days’ average profit.
Accordingly, your EA parameters and position sizing should be adjusted to align with this risk management guideline.
In our specific example, we should implement a stop-loss order when the drawdown (DD) exceeds $45, with a slight buffer for additional safety. It’s crucial to note that the long-term maximum drawdown (DD) for this expert advisor could potentially reach $700 or even $800. By shifting our focus to short-term optimization and adapting to recent market conditions, we have significantly reduced the potential for substantial drawdowns. This approach prioritizes risk management and aims to minimize the impact of unexpected market events on the trading account.
Conclusion
By embracing short-term optimization and focusing on recent market behavior, traders can enhance the adaptability, performance, and risk management of their EAs. This approach requires a more proactive and dynamic approach to EA management, but it can ultimately lead to more robust and profitable trading systems.