This week, we focused on the midterm project, which covered everything we’ve done up until this point. The objective of this assignment was to take a large data set of property values and leverage other variables to build an optimal regression model. Our first task was all about structuring, exploring and transforming our data. The next step was to perform clusters and identify outliers. Lastly, we used correlation and data step techniques to build our first model, Then we used a more advanced approach to proc reg to include all the variables that had correlation of > .10, and stripped out the remaining variables with p values of > .05 from there.
The challenging part of this assignment was that it encompassed everything we’ve learned from soup to nuts, and remembering the syntax of how to write everything was taxing at times – going back to separate assignments and lectures, etc. The part of that project that perplexed me a little bit was how to appropriately identify and treat outliers to increase the efficacy of the model. e.g. normalizing through logs, then identifying large variances, box plots (IQRs), etc. Lastly, proc GLM Select I found to be a little confusing, mainly because I was used to using proc Reg. I think the lengthy options threw me off.