Mini Project 3: Deep Dive
DSC 365: Introduction to Data Science
Think about how to improve your model to better answer the research question, with a focus on prediction
In Mini Project 2, you have performed a preliminary analysis on your data with a focus on interpretation. Please review the feedback before you move to the next step. If you want to change the data, please let the instructor know.
Now you have a data set and a basic understanding of the data, so the next step is to try more models to see if this is an improvement. You need to modify your current report about your findings. You will need to submit two things for MP3.
- Submit your report in HTML or PDF to Blueline before class on Tuesdsay, November 17. The report should not exceed 10 pages, including the plots. This will only be graded based on completion and will be used during class that day for peer review.
- Second, submit a reflection on your peer review in Word/PDF to Blueline by the end of the next day.
Report
There’s only one explicit requirement for MP3: Include more than 6 variables into a random forest model and report the ones that are important. For the important variables, fit a linear model and a decision tree model. Report the RMSE for both of them. Be sure to address these questions in your report:
- Which is better?
- Interpret the better model and use it to help answer your research question.
- Any concerns about your final model?
You do not need to keep your linear model from MP2. That is, you could start the analysis portion of your report with the random forest described above.
Here are things you could consider doing/adding to your report:
- Check your response variable. Is it normally distributed? If not, try transforming it and re-run your modeling with the transformed variable.
- Check the missing values. Are there any missing values in your data? Is there are reason for the missing values? What have you done to account for them?
- Adjust your categorical variables. Prediction often gets worse the more levels a categorical variable has. Can you find sensible ways to combine levels that brings you down to 5 or less?
We’re hitting the open-ended portion of things: you’ve got a toolbox (data wrangling skills, modeling skills) and a question to answer with data. Have fun with it!
Reflection
We are going to conduct a peer review on your report and everyone is expected to submit reflection on the review they get from their peers.
- Modify your MP2 report (by adding and/or replacing things, as described above), and upload before the start of class on November 17th.
- During class that day, in groups of 3-4, read your other group members’ projects and score them according to the provided rubric (to be shared later). Each group member should receive their finished rubrics at the end of the class period.
- Discuss pieces of your review with your group.
- Submit a reflection on the peer review to the BlueLine Assignment.
Write a few paragraphs (no more than a page) reflecting on the review and the review process. Be sure to address the following:
- After reading all the projects you chose and reading your feedback, is there anything you have not considered/included in your project, but you would want to consider/include now?
- Was there any part of your peers’ feedback that was surprising to you? Why do you think there might be a mismatch between their assessment and yours?
- List some changes you want to make and why (or, that you don’t want to make changes, and why).
Your reflection should not exceed 1 page. Any format (PDF/Word) is acceptable.
Grading
This will be graded out of 50, broken down something like this:
| 5 points | Turning the report in on time |
| 5 points | Including the required component (random forest, see above) |
| 5 points | Completing the rubrics for your peers |
| 35 points | Reflection |
| 50 points | Total |