AD699: Data Mining for Business Analytics

Spring 2019

02APR

Quiz #2

Version:  GOLF

You have one hour to complete this quiz.  You may use a calculator, along with your book and/or notes, but may not use a smartphone or anything else with Internet connectivity.

For any multiple choice question, you are not being asked to choose the “best” from among four  possibilities;  instead,  there  are  three  wrong answers, and one right answer. Any multiple choice question must be answered with one completely clear answer choice.

For  any  free  response  questions, show your work.   Rounding  is  completely okay (and showing your work helps me to see what you did).

Free  response  questions  that ask for multiple pieces of info will be scored in a binary fashion (1 or 0 points).

There are three versions of this quiz, but all contain the same content.

1.   In  a  recent  survey, 855 BU students were asked about their preferred choice of beverage in the morning.   345 of the students stated that they never drink coffee. 250 of the students said that they never drink orange juice.  145 said that they never drink  coffee  or  orange juice.   What  is  the  probability that a  randomly-selected student from the survey drinks orange juice, but never drinks coffee?

2.   A social media marketer recently concluded a study of the social media habits of 680 BU students.  500 of the students indicated that they use Instagram.  320 of the students indicated that they use Facebook.  Among the students who use Facebook, 60 indicated that they never use Instagram.   Given  that a student in the survey indicated  that  she does not use Facebook, what is the probability that she uses Instagram?

3.   Is it possible to create a classification tree when your outcome variable has more than two classes?

a.   Yes, a classification tree maybe built for a situation in which there are more than two outcome classes.

b.   Yes, but only if the outcome classes are first separated in two groups -- there can be one “important class” and the others must be lumped together as “other.”

c.   No  --  the  limitation  to  just  two  outcome  classes  is  one  of  the drawbacks of classification trees.

d.   No, but if the person building the tree model uses a random forest, then it will be possibly to create meaningful tree models with more than two outcome classes.

4.   You are building a linear regression model, and you are engaged in the important task of variable selection.   One of your potential input variables is not correlated with any other input variables, but is highly correlated with your outcome variable.   What likely impact would you expect to see if you remove this input variable?

a.   The removal of this particular input variable is likely to increase the average error of your predictions.

b.   The removal of this particular input variable is likely to eliminate your risk of multicollinearity.

c.   By removing this particular variable, you are likely to increase your risk of overfitting the model to your training data.

d.   If this variable is removed, your overall r-squared can be expected to increase, due to the correlation between this input and the outcome.

5.   You are about to build a multiple linear regression model.  There are nine possible input variables that you might use to build the model.   For each of the nine potential inputs, you must decide to either keep it (use it in the model) or discard it (throw it out). How many total models are possible in this situation?

6.   Of all the variables  shown below in this correlation matrix, which pair of unique variables have the smallest correlation distance?   (Place two different variables in the blanks below -- the order does not matter).

AND

7.    If a tree model is 100% accurate in classifying the training data from which it was built, what can be concluded about the model?

a.   Its performance will be equally strong when the accuracy is checked against the inputs from the validation set.

b.   Its terminal nodes are completely pure (i.e. completely homogenous).

c.   It  is  robust  against  most  forms  of  cross-validation,  but  could  be unstable.

d.   Its number of terminal nodes will be less than the square root of its number of decision nodes.

8.  A person who lives in New Haven, Connecticut (a city with an almost-even number of Red Sox fans and Yankees fans) is building a classification tree in order to place people into the correct classification (Yankees or Red Sox).  She finds that if she splits the variable “AGE” at 42 years old, two nodes will be created:

(1)  The node for “GREATER THAN 42” will contain 105 records, with 55 Yankees fans and 50 Red Sox fans.

(2) The node for “LESS THAN OR EQUAL TO 42” will contain 85 records, with 60 Yankees fans and 25 Red Sox fans.

Answer both parts of the question below:

Gini impurity level for the node created with AGE > 42:   _____________

Gini impurity level for the node created with AGE <= 42: ______________

9.   When  building  a  classification  tree  model,  what result can you expect after pruning the tree?

a.   Pruning will make your tree model robust to outliers.

b.   Pruning the tree will eliminate the need to consider using a more advanced model, such as a boosted tree or random forest.

c.   Pruning will make your tree model less accurate for the data on which the tree was built, but will make it more reliable when used with new   data in the future.

d.   Pruning the tree will completely eliminate any risk of overfitting or model inaccuracy when new data is run through the model.

10.   Linear regression is typically performed with one of two main purposes in mind: explaining the relationship of data that you have already captured, or predicting the outcomes of new records.

For  which of these two methods is data partitioning more typically performed, and why?

a.   Predictive, because partitioning helps the person building the model to see how the model will perform. against completely new data.

b.   Predictive, because a data partition is the best way to ensure that the variation in the y-variable (outcome) can be explained by the inputs used in the model.

c.   Explanatory, a data partition will guarantee that the y-variable follows a linear pattern.

d.   Explanatory, because by partitioning the data, the analyst can then be sure that there is no risk of heteroskedasticity.

11.   Of the  input variables in the model shown below, which is the least statistically significant?

12.   We are building a model that attempts to predict what neighborhood in New York City a person lives in.  Our training set contains 7043 records, and includes people from the following four neighborhoods:  Williamsburg, Bushwick, Harlem, and the Upper East Side.  If we build the model with k = 7043, can we know how it will classify a new record, without even seeing the numerical predictors used to build the model?

Answer this question in two parts:  First, write either “YES” or “NO” in the first blank; in the second blank, write either the classification that you’d expect to see (if you wrote “YES” in the first part) or write N/A (if you wrote “NO” in the first part)

AND

Yes/No                                   Expected Classification (or N/A, if no)

13.  After class, you get onto the Green Line and begin to daydream about your plans for the summer.  As you start to doze off a little bit, you are awoken to the sound of a person shouting very loudly into his cellphone.  You take off your headphones in order to listen in, and you hear this:

“Yes, yes...I know we’re a detective agency, but our resources are limited!  The best way for us to handle the fraud investigations is this -- we’ll build a naive   Bayes model to check for the likelihood that a company has committed fraud, but we’ll slide the cutoff to something much bigger than .50. Let’s use a cutof of.85, and we’ll only investigate those cases.”

Why would the speaker advocate for this approach?

a.   By sliding the cutoff to .85, rather than use a default value such as .50, the firm will be able to guarantee that it has not missed any actual cases of fraud that need to be investigated.

b.   By deciding to only investigate cases that a naive Bayes model would predict to be at least 85% likely to be fraudulent, the firm can focus its energy more narrowly on the cases that have the highest likelihood of fraud.

c.   By moving the cutoff to .85, the firm can be certain that only cases that are truly fraudulent at a confidence level of 15% or higher will be investigated.

d.   With a cutoff of .85, the model’s accuracy will be at least 35% greater than a model using a standard default cutoff of .50.

14.  Why is cross-validation used when building classification tree models?

a.   If the classification tree has been built with faulty or incomplete data, cross-validation  will  solve  this  problem  by  eliminating  rows  that contain outliers or questionable values.

b.   Without cross validation, it is impossible to know whether the tree will be able to make predictions for new records.

c.   Cross-validation ensures that the tree only splits on numerical values, and not on categorical data -- this guarantees precision that a tree model would not otherwise have.

d.   Since  the initial split in a model impacts all of the resulting splits, cross-validation becomes an important way to examine the level of errors that occur when slight variations are made on the data used to train and score the model.

15.  Which of the following is NOT considered an advantage of using tree models for data mining?

a.   Tree  models  are  visually intuitive, and easy to explain to someone unfamiliar with data mining.

b.   Tree models are considered highly reliable across time periods -- for instance, if a tree-based model is built with data that was captured in a particular year, the model will still reliably work many years later.

c.   An  analyst  building  a  tree  model  can  generate  the  model without needing to remove missing values from the data first.

d.   Trees  are  flexible  --  tree  models  can  take  numeric  or categorical inputs,  and  tree  models  can  be  built  for  either  classification  or regression purposes.