| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Matthew Klein (Admin) |
 |
| When: | 3/16/11 (12:14) | |
|
|
|
|
| | This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I would welcome your help and suggestions. Let's use this thread to begin presenting and kicking around ideas. Ultimately, the more detailed and actionable the results of this discussion, the better. (Meaning: if we ultimately come up with a specific algorithm that can be programmed by translating the algorithm into computer code, that would be most welcome!)
The current C2 Score is calculated in the following way:
We build a collection of "qualities" we think are important. Examples of qualities that we think are important:
Sharpe Ratio
Annualized Return
Monte-Carlo's Implied Probability of 10% account Loss
Monte-Carlo's Implied Probability of 30% account Loss
# of autotraders
# of subscribers
etc.
(These are just examples; there are more.)
Each quality is calculated and multiplied by a coefficient, which is effectively the "weight" we give to the quality.
All weighted raw numbers are added together, to get a raw score value.
All people in the C2 Universe are next ranked in percentile order by raw scores. The C2 Score is their percentile ranking. (Score of 1000 is the hundredth percentile. Score 500 is the 50th percentile).
That is the current algorithm.
I welcome suggestions, ideas, proposals, etc. for a new C2 Score Algorithm. Please let the discussion begin!
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/16/11 (12:45) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
From what I see above current C2 score is a wild mixture of:
a) system specific objective performance numbers (Sharpe, Drawdown)
b) system specific subjective numbers (# subscribers, # autotraders, reviews, analyst notes ...)
combined to get a vendor specific number.
I'd suggest to add:
c) vendor specific objective numbers (#systems, reliability of signal delivery)
d) vendor specific subjective numbers (review stars, thumbs up in forum posts)
Then I'd generate Two Scores:
1) C2 Vendor Score
2) C2 System Score
to make things more clear and useful.
I'd suggest we start with collecting a number of numbers for each group a) to d) |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/16/11 (12:53) | |
| Systems: | |
|
|
|
|
| | In response to post by Rene' Koch of 3/16/11 (12:45) From what I see above current C2 score is a wild mixture of:
a) system specific objective performance numbers (Sharpe, Drawdown)
...
See entire
Some ideas for category d) subjective vendor measures:
* add some buttons to the forum to give people a possibility to mark a posting as "useful" (some have a thumbs up button). Then calculate the usefulness of every user.
* measure the time it takers a vendor to respond to private messages or even let people judge the "value" of a PM response.
* install a Bug-Tracking / Feature Request system for C2. Give extra bonuses for every bug finally removed and every feature finally added.
* when a subscriber stops her subscription she could select a "customer satisfaction measure"
After all this is all about democratic social networking / interaction measures. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/16/11 (13:23) | |
| Systems: | |
|
|
|
|
| | In response to post by Rene' Koch of 3/16/11 (12:53) Some ideas for category d) subjective vendor measures:
* add some buttons to the forum to give people a possibility to mark a posting as "useful" (some have a thumbs up button). Then calculate the usefulness of every user....
See entire
I think, the entire C2 Scoring system needs to be changed and be a System Based only.... Feedback about the Vendor can be done in the Forum separately.
The C2 System again SHOULD NOT have the subscriber numbers as an input... that skews the result... example... a vendor can sell his system for 0.99 per month and have a huge number of subscribers and skew the C2 System, or for that matter, the current C2 Vendors who have a lot of subscribers have an unfair advantage over newbies.
I think, the length of the system as an input to C2 System Score is very good, so existing C2 Vendors who have a very stable system get the benfit of that.
More thoughts to come...... |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/16/11 (13:26) | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
I'd find a way to heavily penalize Martingale-type systems. The system that's currently at the top of the Hot Forex list is a disaster waiting to happen IMO.
2/27/11 19:14 STO 1 EUR/USD 1.37172
2/27/11 20:57 STO 1 EUR/USD 1.37329
2/27/11 23:15 STO 1 EUR/USD 1.37546
2/28/11 1:46 STO 3 EUR/USD 1.37675
2/28/11 2:15 STO 6 EUR/USD 1.37843
2/28/11 4:01 STO 11 EUR/USD 1.38017
2/28/11 4:12 STO 3 EUR/USD 1.38195
2/28/11 5:23 STO 6 EUR/USD 1.38413
2/28/11 5:59 BTC 32 EUR/USD 1.38289
I agree with Rene' that I'd like to see separate ratings for the system and the vendor.
Personally, as a potential subscriber, I'm not too interested in the vendor's social networking skills, just his trading skills. I guess you could rate the social stuff separately. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/16/11 (13:55) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/16/11 (13:26) I'd find a way to heavily penalize Martingale-type systems. The system that's currently at the top of the Hot Forex list is a disaster waiting to happen IMO....
See entire
Thank you. Exactly my point.
I think C2 has a wonderful System Performance table for each system, something that I have never seen as detailed. I think the C2 System Score should be based on the key metrics from there rather than as Dennis pointed out on "social skills" of the Vendor.
Perhaps, if possible, 2 C2 Scores are in order
1. C2 System Rank
2. C2 Vendor Rank
The latter can be based on a concept like subscriber satisfaction etc much like eBay's seller feedback, which i am citing as an example only, not that it is perfect in any sense.
Thanx
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Christina S |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 3/16/11 (14:25) | |
|
|
|
|
| | In response to post by Sharp Alpha of 3/16/11 (13:55) Thank you. Exactly my point.
I think C2 has a wonderful System Performance table for each system, something that I have never seen as detailed. I think the C2 System Score should be based on the key metrics from there rather than as Dennis pointed out on "social skills" of the Vendor....
See entire
Certainly make a distinction between a vendor score and a system score. People searching "the grid" on balance would have no idea that C2 score currently refers to vendor. I think broad assumption/expectation would be a rating pertaining to a system. Isn't a vendor's overall performance track record readily apparent, anyway, due to the inherent premise of C2?
Keep the subjective/social stuff out of a system score: far too much there for people to complain about. (NOT that that happens on C2, I know... ;) )
I think the Vendor score based on some of that stuff is a good idea in theory; however, you can't discount the reality that some of the best systems might be the lowest maintenance. A few introductory interactions with the system vendor until you're up and running may be all that's needed, and the vendor may not otherwise need/want to spend a lot of time in the forums. "Helpful" or "thumbs up" ratings on actual posts, as exist in a lot of forums, are great, but as in other forums, should go across the board (users and vendors alike).
Christina |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Mike Osborne |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 3/16/11 (14:43) | |
|
|
|
|
| | In response to post by Dennis H of 3/16/11 (13:26) I'd find a way to heavily penalize Martingale-type systems. The system that's currently at the top of the Hot Forex list is a disaster waiting to happen IMO....
See entire
I agree with this.
Idea on how to penalize these types of systems:
Find the highest percent ever risked on a single trade (position size to stop price). This would be an interesting number to have in general.
If a system doesn't use stops, give them a 0. Just kidding. Maybe still keep track of their most epic loss and use that (maybe with some multiplier?).
The two stats could not be used side by side, but they both would seem to express the ends the user will go to not take a loss. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Jay Horne |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/16/11 (17:11) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
Perhaps an approach would be to create a system scoring system and then simply make the vendor score the weighted average of all their systems. In creating the weighting, you might factor in such things as:
-- duration of the system such that the longer a system is managed, the more it counts toward the vendor score
-- If a system is killed, it is maintained in the rank, but it's duration and stats are frozen at the end, so that over a very long time its influence diminishes as other systems (still actively managed), weigh more
The system score could be a single rank such as Sharpe, or combination of several. But the vendor score would sort of be an exponential moving average of the individual system scores. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/17/11 (3:13) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/16/11 (13:26) I'd find a way to heavily penalize Martingale-type systems. The system that's currently at the top of the Hot Forex list is a disaster waiting to happen IMO....
See entire
Dennis wrote:
I'm not too interested in the vendor's social networking skills, just his trading skills.
Let me reiterate:
We have lots of objective measures about systems on C2.
From the reviews I see that for many people a responsive and helpful vendor is an importing factor when subscribing to a system.
Of course, there are "silent" subscribers, they simply use the signals and pay their fees, they never talk to the vendor.
But I can say that there are others who have a lot of questions (even if all answers are in the FAQ) and need a lot of attention.
I personally feel it important to be helpful for these people.
I think C2 should encourage good vendor behaviour (to raise overall quality and Customer Satisfaction) and a "Vendor Score" could be such a move.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/17/11 (8:05) | |
| Systems: | |
|
|
|
|
| | In response to post by Rene' Koch of 3/17/11 (3:13) Dennis wrote:
I'm not too interested in the vendor's social networking skills, just his trading skills.
Let me reiterate:
...
See entire
Rene:
I respectfully disagree. In choosing a system, from say a number of systems that a vendor offers, a subscriber should be able to quickly go thru the SYSTEM C2 score. They can then hone in the detailed system performance that C2 has.
By using the C2 VENDOR system as it stands now, a subscriber (and I am sure most subscribers would relate the current C2 system to be a measure of the SYSTEM and not VENDOR) would just be interested in C2 scores of a high magnitude. That is unfair. A vendor could have a high score and yet have broken systems in their portfolio of offering, where as a new vendor who has no subscribers yet, but has a good system in their offering never will get noticed.
I can understand where you are coming from, based on your score and you would like to keep the system as is, since it benefits you and you have certainly earned it.
But in all fairness to the subscribers, they should be paying for the System's capability rather than the Vendors ethical behavior. The ethics of a vendor should certainly be a measured metric, but as a SEPARATE score, based on several factors like his portfolio of system performances, customer feedback, etc.
I do like the fact that someone suggested that when a system is killed, it weightedly decreases its value into the VENDOR system.
Thanx |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/17/11 (8:48) | |
| Systems: | |
|
|
|
|
| | In response to post by Sharp Alpha of 3/17/11 (8:05) Rene:
I respectfully disagree. In choosing a system, from say a number of systems that a vendor offers, a subscriber should be able to quickly go thru the SYSTEM C2 score. They can then hone in the detailed system performance that C2 has....
See entire
No reason to disagree, I think we are on the same page here.
I also vote for a good, meaningful System Score.
Just as a helpful addition I'd like to see some of the soft skills added to a Vendor Score.
Of course I like my (current) C2 score of 1000, but if its based on the wrong calculations I'd rather have a lower score which is more meaningful.
(you hear the scientist speaking, he's a bad business-man ;) ) |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/17/11 (8:50) | |
| Systems: | |
|
|
|
|
| | In response to post by Rene' Koch of 3/17/11 (8:48) No reason to disagree, I think we are on the same page here.
I also vote for a good, meaningful System Score....
See entire
Well said Rene..
Best Regards |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Keith Fitschen |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 3/17/11 (11:07) | |
| Systems: | Beta Trader, BigCap Trader, BigCap Trader Longer-Term, Commodity Trader, Counter-Trend, Deviation Trader, ETF Basket Trader, ETF Basket-Trader (Closed), FX Trader, Intermediate Trader, Keith's Mini Russell Trader, Keith's Mini S&P Trader, Keith's Stock Trader, Keith's Stock-Trader (closed), Short-Term Trader |
|
|
|
|
| | In response to post by Sharp Alpha of 3/17/11 (8:50) Well said Rene..
Best Regards
I think a system score is useful if it can be wickered to provide an "apples-to-apples" comparison of all C2 systems. The best way I know how to do that is to ratio periodic return to drawdown from an equity curve that has realistic slippage/commission deductions.
On say a monthly basis you divide the dollar-return by the max dollar drawdown to score that month. The system score is the average of all scored months. For new systems, a decreasing fraction of the score can be deducted until the system reaches maturity at say 1 year.
This method will catch the traders who average down because they will have relatively large drawdowns sometime during the month. It also takes care of the problem of comparing a system that takes high risk for relatively high return with one that risks less for less return, because leverage is out of the calculation.
If you want to compare traders, just average their system scores. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | simon sung |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 3/17/11 (15:08) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
Matthew, I think the problem with the current C2 score is that it does not compare like with like (that is risk adjusted returns). Looking at headline return will no doubt favours the high leverage systems. Using the sharpe ratio should address this, however it is only meaningful if you take into account of age of the system (so technically speaking I would apply some sort of t-statistic on the sharpe). Other risk adjusted return metric could make use of drawdown to return (but again take age of system into account)
On the popularity front, it might be a good idea to measure Subscriber fees earned (say over the last 12 months) rather than number of subs. (say 1000 subs x $1/month would be no match a system that takes 10 subs x $200 and so on) . Better yet, if you could measure the duarability/attrition rate of the subscriber base (e.g. average number of years subscribes have been with the system to measure how fickle / long standing subscriber are, that should tell you how good a system really is from the users point of view)
Other measures such as kurtosis and skewness of returns, will tell you so much about the type of strategies a system deploys (trend vs mean reversion, low win rate but high R/R, vs high win rate but big tail risk etc). Your current stats on MAE, MFE are good starting points.
Would be happy to do some sort of collabration if you want more input.
emini Qsuite |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/17/11 (15:53) | |
| Systems: | |
|
|
|
|
| | In response to post by simon sung of 3/17/11 (15:08) Matthew, I think the problem with the current C2 score is that it does not compare like with like (that is risk adjuste...
See entire
Also, whatever you decide Mathew, you need to normalize the leverage factor. Perhaps, normalizing each system to start from a standard equity base and compare them at the same leverage. I am not sure how you would do that, but I have seen many systems here which use tremendous leverage using futures with small equity base... obviously, the performance would look great, but these are disasters waiting to happen. A subscriber should be able to compare two systems on a risk-adjusted basis.
Thanx
PS: When do you expect to close this thread and actually work on getting the C2 system implemented? |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Igor Smirnoff |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 3/17/11 (21:29) | |
| Systems: | |
|
|
|
|
| | In response to post by Sharp Alpha of 3/17/11 (15:53) Also, whatever you decide Mathew, you need to normalize the leverage factor. Perhaps, normalizing each system to start f...
See entire
Hi,
You wrote : "Also, whatever you decide Mathew, you need to normalize the leverage factor"
Your idea sounds logical at first glance but in reality that would be like comparing apples and oranges.
You CANNOT "normalize" the leverage for a stock trading system and a Forex trading system for example and then draw any meaningful conclusion about the profitability of each system, it simply does not work this way.
Why ?
Very simple. Take the SAME trading system and apply it simultaneously to the stock market (leverage 1 to 1) and the Forex market (leverage 100 to 1). Starting with the same capital, the same trading system will ALWAYS make 100 times more money in the Forex market than in the stock market, on average, period!
Now if you "normalize" the leverage, both systems will end up with the same profit potential of course, give or take a few dollars, but nobody truly cares about that.
Remember, people want to make more money the fastest way (while still protecting their capital), that is their first priority. If the same exact system makes $100.000 a year in the Forex (due to 100:1 leverage) but only $1.000 in the stock market , starting with the same capital, the traders would not hesitate a second, they will follow the Forex system, end of story.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/18/11 (8:17) | |
| Systems: | |
|
|
|
|
| | In response to post by Igor Smirnoff of 3/17/11 (21:29) Hi,
You wrote : "Also, whatever you decide Mathew, you need to normalize the leverage factor"
Your idea sounds logical at first glance but in reality that would be like comparing apples and oranges....
See entire
You wrote:
".....Now if you "normalize" the leverage, both systems will end up with the same profit potential of course, give or take a few dollars, but nobody truly cares about that.
Remember, people want to make more money the fastest way (while still protecting their capital), that is their first priority. If the same exact system makes $100.000 a year in the Forex (due to 100:1 leverage) but only $1.000 in the stock market , starting with the same capital, the traders would not hesitate a second, they will follow the Forex system, end of story. ....."
Does that mean nobody cares about RISK-ADJUSTED RETURN? Then why not start trading just Forex with 200 to 500:1 leverage. Or for that matter, just trade your futures account value/margin requirement contracts of Futures.
You say - "Remember, people want to make more money the fastest way (while still protecting their capital), that is their first priority.".......
That is exactly my point for having some way to disclose to the investor what the leverage usage is. Remember, it is always prudent to disclose the risk as much as you want to "focus" on the return. Not all subscribers go for the "high octane" returns. They try and find the answer to "What am I risking here to make such returns? And is that risk palatable to my portfolio?".... I personally would NEVER subscribe to a system that makes too much money too fast without understanding how it makes money..... If your approach of "....people want to make more money the fastest way....", then they should be looking for Martingale types of strategy...
All I suggest that we should brainstorm and come up with comparing 2 systems based on a common benchmark of risk and reward. It is not an easy task for C2, but I am sure with so many bright subscribers and system developers we can get to a pretty close solution.
Thanx |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/19/11 (11:05) | |
| Systems: | |
|
|
|
|
| | In response to post by Danny White of 3/18/11 (16:04) The rating should most importantly reflect if you'd have likely made or lost money trading a system. What gets me is my system is one of the few where you would have likely made money and my rating is lower then others where you'd have likely lost money....
See entire
MP,
you wrote:
my system is one of the few where you would have likely made money
Howe can you say that?
Did you ever click the "Monte Carlo" button below the chart of your system "Predictor Discretionary ETF"?
You'll notice that in the worst case you're just a tiny fraction above zero. In trading we have to expect the worst case.
I wouldn't call this "likely made money". |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Karl A |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/21/11 (4:06) | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
Hi Matthew,
Here is, for a change, a subscribers view about the developer's ranking system on C2:
Over the years I have traded many systems on C2, not once did I look at the developer's score or the systems popularity in order to come to a decision. The only things which count for me are the shape of the equity curve, the duration of the system, the statistics (including the new great feature of real brokers executions), the other systems the vendor has developed and the information he provides either on the system's description page, his web page or through message exchanges. I look at the My Analyst page and the Reviews only for entertainment purposes which does not mean that there are not some good comments occasionally.
I would suggest that you send out questionnaires to all subscribers and ask them what improvements they would like to see since very few participate on the forum here.
It baffles me why vendors are obsessed with the ratings system, they probably want to use it for marketing purposes, but I doubt that any subscriber pays much attention to it. As others have pointed out C2 provides excellent statistics and is willing to improve upon them.
I agree with others who have recommended to eliminate the C2 score.
BTW, I can't wait to see a Gen3 connection to IB, that should be a quantum leap forward.
Karl |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | John Fagan |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/23/11 (18:35) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
I think it would be wise to replace Sharpe Ratio component of your C2 Score with either UPI (Ulcer Performance Index) or Sortino Ratio since Sharpe Ratio penalizes for volatility regardless if its upside vs. downside volatility. IMO, UPI or Sortino does a much better job at penalizing for only the downside volatility which in the end, is what we really care about regarding risk of our own capital.
http://en.wikipedia.org/wiki/Ulcer_Index
http://www.investopedia.com/terms/s/sortinoratio.asp
JD |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Igor Smirnoff |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 3/23/11 (19:07) | |
| Systems: | |
|
|
|
|
| | In response to post by Sharp Alpha of 3/18/11 (8:17) You wrote:
".....Now if you "normalize" the leverage, both systems will end up with the same profit potential of course, give or take a few dollars, but nobody truly cares about that....
See entire
Hi,
You wrote : "Does that mean nobody cares about RISK-ADJUSTED RETURN? Then why not start trading just Forex with 200 to 500:1 leverage. Or for that matter, just trade your futures account value/margin requirement contracts of Futures. "
Of course not, no matter what financial instrument we are trading (stocks, futures, currencies, etc...), we should always use sound money management principles, there is no way around this.
All I am saying is that you CANNOT compare the results of a system that trades stocks (no leverage) with the results of the same exact system that trades currencies (up to 500 to1 leverage), it's like comparing a small Cesna to the Concorde! Sure, both planes will take you from point A to point B, but the (now defunct) Concorde will get you there much much faster.
You wrote : " I personally would NEVER subscribe to a system that makes too much money too fast without understanding how it makes money..... If your approach of "....people want to make more money the fastest way....", then they should be looking for Martingale types of strategy...
Of course the trader/subscriber must at least understand the logic behind any trading system (usually the system details page reveals a few clues), but that does not mean that a system that makes money "too" quickly (??) is necessarily a martingale.
Again and I will repeat that, if a system has no mathematical/statistical edge to begin with (in other words it is a losing system making random calls and the spread and/or commissions will slowly but surely empty the trading account), NO martingale will ever turn it into a winning system.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/23/11 (20:22) | |
|
|
|
|
| | In response to post by John Fagan of 3/23/11 (18:35) I think it would be wise to replace Sharpe Ratio component of your C2 Score with either UPI (Ulcer Performance Index) or Sortino Ratio since Sharpe Ratio penalizes for volatility regardless if its upside vs. downside volatility. IMO, UPI or Sortino does a much better job at penalizing for only the downside volatility which in the end, is what we really care about regarding risk of our own capital....
See entire
"I think it would be wise to replace Sharpe Ratio component of your C2 Score with either UPI (Ulcer Performance Index) or Sortino Ratio since Sharpe Ratio penalizes for volatility regardless if its upside vs. downside volatility."
I disagree. Sharpe is profit/equity-curve-volatility. Upside volatility adds to profit so there is no "penalty" for making money and it doesn't penalize winning and losing trades equally. OTOH it does penalize one-time, outlier profits, which are unlikely to be repeated, compared to slow, steady profits.
An example, I have a TS system that made a killing in the stock index futures during the flash crash. We'll probably never see another one of those. The Sharpe went up after that but not as much as the sudden spike in the equity curve might suggest. And that's how I think it should be. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | John Fagan |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/23/11 (22:24) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/23/11 (20:22) "I think it would be wise to replace Sharpe Ratio component of your C2 Score with either UPI (Ulcer Performance Index) or Sortino Ratio since Sharpe Ratio penalizes for volatility regardless if its upside vs. downside volatility."
...
See entire
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/23/11 (23:01) | |
|
|
|
|
| | In response to post by John Fagan of 3/23/11 (22:24) Further, the problem I have with Sharpe is its reliance on standard deviation:
http://www.tangotools.com/ui/ui.htm
I respect the author of the Ulcer Index but either he doesn't understand how Sharpe is calculated or he is "misspeaking."
"The calculated value of SD depends on the time period used. For most investments, the SD of annual return is roughly 7.2 times the SD of weekly return (7.2 is the square root of 52 weeks per year). Since the time period is often unstated, this creates an opportunity for misunderstandings."
"Unlike SD, the calculated value of UI is essentially the same regardless of the time interval per data point."
SD is annualized in the Sharpe calc, just like the return is annualized. Multiply the monthly SD by sqrt(12), the weekly SD by sqrt(52), etc.
That said, I do have a complaint about the way the stats are calculated at C2. They don't include commissions and the calculated Sharpe is way too high for most systems. I probably shouldn't complain because I'm a daytrader and my stuff (should I decide to publish it) would look great if I could ignore commissions. But it would give me an unfair advantage when comparing to a system that holds longer and for which commissions aren't such a big deal. Commissions should be included but they should be realistic, not the ridiculously high "typical commissions." Even the MBT commissions for futures are way more than what most of us pay. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | John Fagan |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/24/11 (0:00) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/23/11 (23:01) I respect the author of the Ulcer Index but either he doesn't understand how Sharpe is calculated or he is "misspeaking."
...
See entire
I think a more important critique from Martin is below:
"The calculated value of SD is not affected by the sequences in which gains and losses occur. Thus, SD does not recognize the strings of losses that result in significant drawdowns in value. The three hypothetical investments in the chart below have the same annualized return and the same SD, but no rational investor would consider them as having the same risk."
UI solves this and thus UPI which makes use of UI is superior IMO. UPI = (Return - RiskFreeReturn) / UI. Whereas Sharpe = (Return - RiskFreeReturn) / SD.
Anyways, regardless of our debate above, I agree on your point that C2 should include commissions within its stats calculations which does skew the high frequency systems as being better then the others. Ideally C2 would calculate the stats for the different sets of commissions (e.g., Typical, OEC, etc.). Then when user selects his commissions option which affects the charts, the proper calculated stats is chosen as well (using the commissions value from that commissions option he selected).
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/24/11 (2:46) | |
|
|
|
|
| | In response to post by John Fagan of 3/24/11 (0:00) I think a more important critique from Martin is below:
"The calculated value of SD is not affected by the sequences in which gains and losses occur. Thus, SD does not recognize the strings of losses that result in significant drawdowns in value. The three hypothetical investments in the chart below have the same annualized return and the same SD, but no rational investor would consider them as having the same risk."
...
See entire
The premise is faulty and so are the graphs on the UI page. He shows a graph that steps slowly up and says it has the same SD as one that goes way down and then goes back up to end at the same profit. I typed some numbers into Excel and had it calculate the SD (the last number in the list). Judge for yourself.
Edit: the forum formatting doesn't make it clear but there are two columns of numbers with the SD at the bottom.
0 0
-1 1
-2 1
-3 1
-4 2
-5 2
-6 2
-7 3
-8 3
-7 3
-6 4
-5 4
-4 4
-3 5
-2 5
-1 5
0 6
1 6
2 6
3 7
4 7
5 7
6 8
7 8
8 9
9 9
10 10
--------------
5.28 2.80
The other thing is the order in which the trades happened in one historical run is only one possibility of many. The idea behind Bootstrap and Monte Carlo sims is to shuffle the trades into a different order, recalculate the MaxDD and repeat several thousand times. Sort those drawdowns and you get a better idea of the probability of having a drawdown of X% than a single sample can give you. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | John Fagan |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/24/11 (18:33) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/24/11 (2:46) The premise is faulty and so are the graphs on the UI page. He shows a graph that steps slowly up and says it has the same SD as one that goes way down and then goes back up to end at the same profit. I typed some numbers into Excel and had it calculate the SD (the last number in the list). Judge for yourself....
See entire
According to Peter Martin who I emailed about this issue of SD, his response to me regarding your last comment:
"If I understand him correctly, he's calculating the SD of the values, not the SD of the returns!"
So perhaps you are not calculating that example of his correctly according to Peter. I'll leave it at that as I'm not the expert with Peter's own work here. Maybe he'll jump in this thread to give more input since I pointed the URL out to him.
Cheers,
JD |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/24/11 (19:09) | |
|
|
|
|
| | In response to post by John Fagan of 3/24/11 (18:33) According to Peter Martin who I emailed about this issue of SD, his response to me regarding your last comment:
"If I understand him correctly, he's calculating the SD of the values, not the SD of the returns!"
...
See entire
We're getting really OT here but here's the same thing on the returns instead of the raw equity curve. It's way simplified but the point is the same.... the down-up curve has a worse SD than the one that just steps up.
This is like deja-vu all over again for me. :) I was fighting the misconceptions about Sharpe on the old Omega List (Tradestation) back in the 90's. I have TS code for calculating Sharpe the "right" way. PM me if you want it. My friend, the author, consulted with Dr. Sharpe when writing it. It's way more complicated than the standard Sharpe calc but it's true to Dr. Sharpe's original intent and it covers all the possible ways you can scale a system.
-1 1
-1 0
-1 0
-1 1
-1 0
-1 0
-1 1
-1 0
1 0
1 1
1 0
1 0
1 1
1 0
1 0
1 1
1 0
1 0
1 1
1 0
1 0
1 1
1 0
1 1
1 0
1 1
---------
0.94 0.50
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/24/11 (20:09) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/24/11 (19:09) We're getting really OT here but here's the same thing on the returns instead of the raw equity curve. It's way simplified but the point is the same.... the down-up curve has a worse SD than the one that just steps up....
See entire
Going a bit further, the reality is that people would like to get a superior return to just Buy and Hold with minimum return volatility. The three simplistic measures that I think would solve this and can become an input to the C2 System Score are
1. % of Buy/Hold since inception.
2. Straightness of the equity curve (closer to a 45 degree)
3. % of Perfect that was extracted by the system. % Perfect being the theoretical max money you can make, knowing the signals well in advance. It is a look-back calc.
I have seen software that do calc these so the math is out there.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/25/11 (2:45) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/24/11 (19:09) We're getting really OT here but here's the same thing on the returns instead of the raw equity curve. It's way simplified but the point is the same.... the down-up curve has a worse SD than the one that just steps up....
See entire
HI Dennis,
I'd love to see some "good" calculation of the Sharpe Ratio, because the standard method has some flaws.
In your example you should use the correct returns to compare the SD of dn-up vs. jiggle:
-1 -1
-1 1
-1 -1
1 1
1 -1
1 1
will result in exactly the same SD.
BTW, this one 1, 1, 1, -1, -1, -1 also has the same SD, despite most (naive) traders would describe it as "green, good, happy" |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Rene' Koch |
|
| | Ignored by 9% of those who use ignore feature. | | |
|
| When: | 3/25/11 (2:49) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/24/11 (19:09) We're getting really OT here but here's the same thing on the returns instead of the raw equity curve. It's way simplified but the point is the same.... the down-up curve has a worse SD than the one that just steps up....
See entire
> PM me if you want it
There is no way to PM you on C2 as long as you don't have an active system... |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Silly Ross |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 3/25/11 (6:42) | |
| Systems: | |
|
|
|
|
| | In response to post by Rene' Koch of 3/25/11 (2:45) HI Dennis,
I'd love to see some "good" calculation of the Sharpe Ratio, because the standard method has some flaws....
See entire
Funny.
How you guys want to measure systems or even vendors, if you are not even able to agree on one indicator/parameter?
My advice: Start measure your ego and use it as a top level parameter.
Everything will be much easier then. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Dennis H |
|
| | Ignored by 6% of those who use ignore feature. | | |
|
| When: | 3/25/11 (10:35) | |
|
|
|
|
| | In response to post by Rene' Koch of 3/25/11 (2:49) > PM me if you want it
There is no way to PM you on C2 as long as you don't have an active system...
Hi Rene',
Oops, my noobness is showing. :) I emailed it to your info address. The main difference from the standard method is it uses log returns so you'll get about the same Sharpe whether you are trading constant size, constant risk or fixed fractional. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | John Fagan |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/26/11 (14:02) | |
| Systems: | |
|
|
|
|
| | In response to post by Dennis H of 3/25/11 (10:35) Hi Rene',
Oops, my noobness is showing. :) I emailed it to your info address. The main difference from the stan...
See entire
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Igor Smirnoff |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 3/26/11 (17:50) | |
| Systems: | |
|
|
|
|
| | In response to post by John Fagan of 3/26/11 (14:02) I'd like to point out another stat that may be of interest for C2 score called R-Cubed by Curtis Faith. See towards bottom of this page here:
...
See entire
Hi John,
You wrote : "I also recommend his book "Way of the Turtle" which has lots of discussion on how to measure robust trading systems."
No a bad book but remember that the Turtles had to change their trading system because it is much less profitable now. Basically they traded off a 20 day simple Donchian system but over the years profits became smaller and smaller and they had to change the lookup period, from 20 days to 40 days.
For system developers or traders who want to build and/or evaluate robut trading systems I recommend The Evaluation and Optimization of Trading Strategies by Robert Pardo and of course the famous New Trading Systems and Methods by legendary futures expert and author Perry J. Kaufman.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | z- trader |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 3/27/11 (14:08) | |
| Systems: | |
|
|
|
|
| | In response to post by Igor Smirnoff of 3/26/11 (17:50) Hi John,
You wrote : "I also recommend his book "Way of the Turtle" which has lots of discussion on how to measure robust trading systems."
...
See entire
Some great comments here. My wish list would include:
All Risk-Adjusted Performance; there is often too much focus on nominal returns, without normalizing for risk. (Even the "Gainers of the Week" emphasizes nominal performance...)
For risk, can use SD, down-side measures of risk (for instance, semi-deviation captures kurtosis, etc.), as well as drawdown measures.
Perhaps "average drawdown" and/or "worst drawdown" over various time frames (and/or drawdown that is normalized for age of system).
Some measures/monte carlo techniques can be applied to trade data (especially worst risk/trade that C2 already has) -- to help weed out/penalize Martingale approaches.
Age of system is important, but should be balanced with meaningful risk-adjusted measures.
I like splitting out a trader score as well as a vendor "customer service" score.
zFutures Diversified
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Sharp Alpha |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 4/13/11 (9:47) | |
| Systems: | |
|
|
|
|
| | In response to post by z- trader of 3/27/11 (14:08) Some great comments here. My wish list would include:
All Risk-Adjusted Performance; there is often too much focus on nominal returns, without normalizing for risk. (Even the "Gainers of the Week" emphasizes nominal performance...)
...
See entire
Mathew:
Any plans or agreement on the C2 Score algo development? |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | GLEN BROWN |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 4/27/11 (20:01) | |
| Systems: | |
|
|
|
|
| | In response to post by Sharp Alpha of 3/16/11 (13:23) I think, the entire C2 Scoring system needs to be changed and be a System Based only.... Feedback about the Vendor can be done in the Forum separately....
See entire
I agree with the following point:
"I think, the entire C2 Scoring system needs to be changed and be a System Based only"
Also you could introduce a "Subscriber Retention Score". This look at the age for your subscribers.
Forex Automated Trading Systems(FATS) |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Michael Cook |
|
| | Ignored by 4% of those who use ignore feature. | | |
|
| When: | 6/07/11 (16:00) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
I think my first question to you Matt would be why is the calculation methodology secret? It is hardly a particularly valuable statistic from the viewpoint of intellectual property. Can you just give the exact formula, inputs and weightings to aid the improvement discussion.
I honestly think that the statistic as it stands is counterproductive as anything that repeatedly gives extremely high scores to systems that have immense (e.g. +75%) drawdowns is harmful to the reputation of C2. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Frank Alden |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 6/07/11 (18:50) | |
|
|
|
|
| | In response to post by Michael Cook of 6/07/11 (16:00) I think my first question to you Matt would be why is the calculation methodology secret? It is hardly a particularly valuable statistic from the viewpoint of intellectual property. Can you just give the exact formula, inputs and weightings to aid the improvement discussion....
See entire
I would like to be able to personalize the current score formula by adjusting the weighting factors to suit my own needs. Adding more components (UI, UPI, etc.) as suggested in this thread is also a good idea. It would be a cafeteria score. Each component should be explained in an informative help page listing its derivation and examples of pros and cons.
I am also shocked to learn that the performance statistics table does not include commissions. That needs to be corrected.
It would also be nice to be able to export a list of strategies statistics into an excel spreadsheet for side by side comparison. The user can do whatever with the data. Copy and paste currently available is too time consuming. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | ECM, LP |
|
| | Ignored by 1% of those who use ignore feature. | | |
|
| When: | 6/08/11 (13:51) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
I think the c2 score should remain completely quantitative and a reflection of the returns and volatility. Qualitative inferences are a slippery slope in that this is supposed to happen socially, rather than through a c2 assessment.
With that said, the big thing missing from my perspective is tracking skewness / kurtosis. This would call out the martingale problem as well as cases where several losses are covered by large occasional profits or any other form of return distribution. Not knocking any, although I prefer normally distributed with positive skewness or better myself, but at least a subscriber gets a sense for how much turbulence he's in for. Similarly, one can avoid negative skew if he/she wishes.
Just my 0.02.
CCF & Co. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Tony Robinson |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 6/08/11 (16:13) | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
I'd give this a go from the statistical pattern recognition viewpoint if you'd make your historic data available.
Tony
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Iris kelly |
|
| | Ignored by 4% of those who use ignore feature. | | |
|
| When: | 6/08/11 (18:36) | |
| Systems: | |
|
|
|
|
| | In response to post by Tony Robinson of 6/08/11 (16:13) I'd give this a go from the statistical pattern recognition viewpoint if you'd make your historic data available.
Tony
I see no value in the C2 score; in fact, it is worthless to subscribers. I think it should be eliminated immediately.
Let's rely on the mathematical statistics that speak for themselves.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | User Removed |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 10/31/11 (2:01) | |
| Systems: | |
|
|
|
|
| | In response to post by Iris kelly of 6/08/11 (18:36) I see no value in the C2 score; in fact, it is worthless to subscribers. I think it should be eliminated immediately.
...
See entire
Wow, what a can of worms this is!
I've been trading for over 10 years and I my conclusion is that you can never define a trading system by one single number. Of course younger traders will not accept this and probably demand to know what the Sortino ratio is, or the Modified-var Sharpe or the Hamburgler Coefficient Factor or the Rubbishbin Statagizer Ratio ....
The point is that everyone has their own view of what is 'right' to them. But the truth is there is no 'right'. No one single number will ever define such complex phenomena.
That is why I suggest you provide all the major performance metrics and let people chose the one they like best. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Eli Jones |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 10/31/11 (16:41) | |
| Systems: | |
|
|
|
|
| | In response to post by User Removed of 10/31/11 (2:01) Wow, what a can of worms this is!
I've been trading for over 10 years and I my conclusion is that you can never define a trading system by one single number. Of course younger traders will not accept this and probably demand to know what the Sortino ratio is, or the Modified-var Sharpe or the Hamburgler Coefficient Factor or the Rubbishbin Statagizer Ratio .......
See entire
Agree. see my post in recent C2 Score thread. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Eli Jones |
|
| | Ignored by 5% of those who use ignore feature. | | |
|
| When: | 10/31/11 (16:51) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
heavier weight on the Sortino Ratio and VAR downside measures.
less consideration to time duration, more consideration to skill, trade attribution, favorable excursions, adverse excursions, entry, exit efficiency.
the CFA institute has valuable cross reference in the area of risk management, and performance measurement. In fact, look into using the GIPS reporting methods which helps create excellent comparison performance analysis.
The market can only be perfect, no one should be higher than 999.
The score is too liberal if too many have it. ERA and sports averages are hardly above 50%.
no score should go beyond 500.
It should fall faster than it can rise and should attribute skill relative performance.
the C2 score should be removed until the matter is remedied, as imo, Matt is risking his entire system more than he realizes, all in the respected effort to protect it.
no score may be the best answer.
hope all this makes sense.
|
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Steve Auger |
|
| | Ignored by 3% of those who use ignore feature. | | |
|
| When: | 11/09/11 (14:40) | |
| Systems: | |
|
|
|
|
| | In response to post by Eli Jones of 10/31/11 (16:51) heavier weight on the Sortino Ratio and VAR downside measures.
less consideration to time duration, more consideration to skill, trade attribution, favorable excursions, adverse excursions, entry, exit efficiency....
See entire
The problem as I see it is that the C2 score is reflective of past performance. But subscribers want comfort that it will somehow be reflective of the system providers future performance.
I would like to suggest that there are enough statistics gathered at C2 and enough collective brains that a scoring system that provides some level of future prediction could be developed.
In other words, instead of attempting to determine what past "performance" parameters are of interest, let's focus on likely future "performance". This wouldn't be an easy task but one needs to start by choosing factors and determining if there is any correlation between the factor and future performance. Once some factors are identified then they could be combined to generate a C2 score.
Science fiction? Perhaps but otherwise I don't see the value in the C2 score. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Greg Trainor |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 12/04/11 (19:34) | |
|
|
|
|
| | In response to post by Steve Auger of 11/09/11 (14:40) The problem as I see it is that the C2 score is reflective of past performance. But subscribers want comfort that it will somehow be reflective of the system providers future performance....
See entire
Steve, all a system can measure is past performance. Developers use a period of time, say four (4) years to develop a system. This is called "in sample" development. Once a system is developed to the point a programmer thinks it is profitable, he tests the system on an "out of sample" period. If the system continues to be profitable, then and only then should it be traded with real money. Most of the equity curves I have seen on this website shouldn't be traded. One of the matrix scores I like to see for judging a system is RAR/MAXDD (Annual Rate of return / maximum draw down). |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Steve Auger |
|
| | Ignored by 3% of those who use ignore feature. | | |
|
| When: | 12/04/11 (22:31) | |
| Systems: | |
|
|
|
|
| | In response to post by Greg Trainor of 12/04/11 (19:34) Steve, all a system can measure is past performance. Developers use a period of time, say four (4) years to develop a sy...
See entire
Greg - you might not understand what I am saying. There is plenty of historical data to work with on this site.
How about somebody do a simple backtest - let's call it "in sample" if that suits you.
Start with the parameter: RAR/MAXDD. Gather statistics for RAR/MAXDD for each system historical points in time; every month for example. With each data point gathered also capture the forward profitability of the system. Process each active system, historically month by month. At the end of this exercise you will know beyond a shadow of a doubt whether RAR/MAXDD had any predictive power for the system's future results.
Then then try another parameter, and so forth. In the end C2 would have some parameters that would have some predictive power. If they want a separate "out of sample" data set then that's cool too.
The reason why I suggest this is because it is absolutely meaningless to generate a C2 score based on what individuals feel are good factors. Base it on statistics, not gut feel.
Steve |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Greg Trainor |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 12/07/11 (20:06) | |
|
|
|
|
| | In response to post by Steve Auger of 12/04/11 (22:31) Greg - you might not understand what I am saying. There is plenty of historical data to work with on this site.
How about somebody do a simple backtest - let's call it "in sample" if that suits you....
See entire
Steve,
Well now that you explained it it does make more sense. It is a good idea and I do think it would add value in the selection process. Having said that, it still wouldn't guarantee future success of a trading system. The truth is that all trading systems eventually fail. Trading is always done on out of sample data. You can look at the old indicators from the past such as the Donchian system and realize eventually the success of a system brings its demise. In a lot of the systems I trade now only show stellar gains since 2008. Markets change and systems have to too. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Gnosis Daydream |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 12/07/11 (22:23) | |
|
|
|
|
| | In response to post by Greg Trainor of 12/07/11 (20:06) Steve,
Well now that you explained it it does make more sense. It is a good idea and I do think it would add value i...
See entire
Wowsers. Keep It Simple, Smarties!
Ratios galore:
It is my observation that a lot of systems have an excellent record until subscribers come aboard; so a performance ratio of the system with subs vs. without subs is important. (Perhaps C2 can completely recalculate stats for subscribers as a "separate system." And perhaps not a System Score and a Vendor Score, but a System Score and a *Subscriber* Score (how "well" do the subs do?) Or all three!?!?)
What %-age of cumulative is coming from what %-age of trades? If profits are evenly distributed, the system is credited - if most of the cumulative profit comes from 1 or 2 trades over many more trades it is penalized (basically means they hit the jackpot once or twice.)
What is the relation (ratio) of cumulative profit (pos or neg) to average dd? (I saw someone mention something like this in the "gainers" column - would like to see the best *ratio* in there, not just a bunch of systems recouping from a large dd.) I think we can all agree that a 50% gainer with an avg. dd of 8% is a "better system" than a 100% gainer with a 50% dd (50/8 is > 100/50.) Gnosis |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Alex Magnus |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 12/12/11 (20:57) | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
Why not consider using the Sortino ratio instead of the Sharpe?
Most investors/traders here likely do not consider upside volatility as a problem. That is what they are looking for. No?
As an aside: I know you are working on the score, but I would love to see the Sortino ratio in the grid if you can squeeze it in there. Thanks for your time and consideration. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Mr. Eydelman |
|
| | Ignored by 7% of those who use ignore feature. | | |
|
| When: | 12/13/11 (1:16) | |
| Systems: | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
Just from my observations I think the scroe should be based on performance, there are strategies on here with a 999, and 1000 perfect score and over the last 6 months have lost 20%-33% , its hard to explain to a subscriber why they lost a third of their account and the strategy is rated 1000. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Aaron Bompus |
|
| | Ignored by 2% of those who use ignore feature. | | |
|
| When: | 4/17/12 (12:58) | |
| Systems: | |
|
|
|
|
| | In response to post by Mr. Eydelman of 12/13/11 (1:16) Just from my observations I think the scroe should be based on performance, there are strategies on here with a 999, and...
See entire
I would like to propose that the following could also be used to help calculate the score:
1. Compounded annual return / average of 25% largest draw downs (higher is better)
2. Standard Trade Deviation (lower is better)
3. Standard Deviation by Month (lower is better)
4. Standard Deviation by Year (lower is better)
5. Kurtosis (0 is best)
6. Positive Skew (0 is best)
7. Negative Skew (0 is best)
8. Z-Ratio (higher is better)
1. This seems like a great factor to include as it will not severely impact good systems that have experienced larger drawdowns, and it still rewards those that manage to keep them to a minimum.
2-8. This will reward system creators that generate consistent and statistically significant returns, which can help create a smoother equity curve. |
|
| | |
|
| |
|
| |
| Subject: | Call for C2 Score Algorithm |
| Posted by: | Matthew Clinger |
|
| | Ignored by 0% of those who use ignore feature. | | |
|
| When: | 4/25/12 (17:09) | |
|
|
|
|
| | In response to post by Matthew Klein of 3/16/11 (12:14) This is an open invitation to all Collective2 Members -
C2 is looking to revamp the C2 Score algorithm, and I w...
See entire
While you could just have a single number, why not make available the numbers that are felt to be most important? This doesn't mean you won't have a C2 Score, but it could show people what the C2 score is based on and at the same time give each person a much broader view of what is contributing to the score. Personally, I would find such tools as the sharpe ratio, sortino ratio, and others to be more useful than 1 ratio that has a prioritized different things than I might.
In fact, why not let people make up their own custom ratio based on what they want? You could supply the standard information and a standard C2 Score, but letting people customize their own C2 Score using different formulas just might be better as not only does this allow for more personal judgment (and thus people can't get mad at being stuck with just 1 standard C2 Score), but encourages innovation in coming up with formulas to find consistently successful systems, presumably making for smarter investors. |
|
| | |
|
| |
|
|
