(statistics). I'd like to find the ones that show a large and abrupt change at time T. This may be a simple type of data mining, but as I can't find a data mining newsgroup, I thought I'd post to some related groups. There is a fair amount of noise in the data, and in addition to the step change I am looking for, the variables tend to contain other slower moving changes (so far I have been avoiding the last issue by looking at a relatively short time windows). What I have done so far is this: - Correlate each varaible with a step function at time T. This does a good job of finding variables with the abrupt change, and I can use the correlation coefficient (R^2) to rank them. - Calculate an average value before/after time T, and rank based on the percent change. My questions are: - Is this a known problem that has some clever, well thought out solution? - Any ideas on other ways to do my ranking? - Any ideas on good ways to combine various rankings to produce a composite ranking? The motivation is I'm most interested in varaibles that show an abrupt AND significant change in value. Gene Other posts:
• How is MSE = Variance + bais^2
• What a photon really is: • Math Researcher: Increase Your Research Productivity with the Leading Web 2.0 Research Portal • Negative r, positive B? regression • For a fixed entropy, which distirbution is closest (in KL sense) to any possible distribution • Ranking random variables that show step change • how do i know that data lie in linear subspace or not? • Standard error of the ARIMA constant • check for Gaussianity in high dimensional data sets • Testing the difference between betas within a regression model • Entropy of a function of a random variable |