demodocus i've been reading quite a bit on creating systems/models for NBA recently and what you've posted relates to what i've been reading.

The guys on there would use 1 set of data to create the system. They would then use a second set of data to test and make any slight adjustments that may smooth the results to be more aligned etc providing it still shows a profit on this data. If its not showing a profit on this data after it did on the original data you used to create the system, then go straight back to the drawing board.

Once the second lot of data even with some tweaks has validated the system, run a final test on a 3rd set of data without any tweaks at all. If you continue to be profitable on this data without much change to the results (SR, POT etc) then your on a winner, if it falls over at this step then back to the drawing board.

Creating systems, ratings, models is not an easy process, if it was everyone would do it and everyone would be millionaires. I've been lucky enough to have some success with my ratings for Australian horse racing, but trying to crack it with a model for the NBA has been near on impossible for me so far.

I think finding a niche where you have some form of an edge is vital. American sports is very difficult to find an edge because the market is so huge and there are so many people betting on it, identifying the sharp lines and therefore creating lines that are very much a 50/50 spread across the season.

This is where Australian horse racing is different, you can find an edge a lot easier IMO. The public are prone to betting on a 100 rater or the top weight or the TAB's tips etc (sometimes all 3 align for a lot of public money). These horses may very well be the favourites for good reason in a race, but it means the odds available are way under the horses actually chance/ability to win the race. This provides opportunity at the other end of the scale providing you can select the right horses at the right time.

I'll post some stats tonight that i've found over the weekend using my ratings that are particularly interesting to horse numbers 10 and below (i.e. lower weighted horses). It's basically shown that whenever my ratings have had a top rater with a TAB number 10 or higher (11,12,13 etc) the POT is huge as its simply an underbet and underrated part of the market.

Anyway thats a bit off track but hopefully that adds to the discussion.