COMP809: Data Mining & Machine
Learning
Assignment 1 (weight: 40%)
Semester 2 2022
Data Mining Data Exploration and Analysis
This is an individual assignment.
Submission: A soft copy needs to be submitted through Canvas. Include your actual clean code (no
screenshot) in an Appendix with appropriate comments for each task.
Due date: Sunday 28 August 2022 at midnight NZ time.
Late penalty: maximum late submission time is 24 hours after the due date. In this case, a 5% late
penalty will be applied.
Page | 2
AIMS
This assignment provides an opportunity to solve two real-world data mining problems
using the machine learning workbench. In the two questions given below justification
of your answers carries a high proportion of the marks awarded. You are required to
conduct experiments for both case studies and report them according to the specified
requirements. Your answers below need to be supported by suitable evidence,
wherever appropriate. Some examples of suitable evidence are the Confusion
Matrices, Model Visualizations and Summary Statistics.
Study Area I ( bank.csv use the Bank.zip)
This application is concerned with predicting the outcome of direct bank marketing
campaigns (phone calls) of Portuguese banking. The dataset contains 17 attributes for
which outcomes of subscribing to a term deposit (yes/no) on a term deposit are known.
You are required to build a model using the Decision Tree Classifier and answer the
following questions based on the model built. Use the data segment on the subscriptions
whose outcomes are known. In building the model, use the 10-fold cross-validation
option for testing.
a) Describe the pre-processing steps you have performed to prepare your data and
perform initial data exploration by analysing the summary statistics of the dataset
attributes. [5 marks]
b) Using an appropriate method to identify the most influential features in
classifying this dataset. Explain the process of the chosen feature selection
method and use the top four features for building your model. Use a
‘breakdown’ analysis for selected features by the class and describe their
distribution using appropriate plot(s). [10 marks]
c) Now build a model using the Decision Tree algorithm. By adjusting two suitable
parameters (one at a time) reduce the size of the tree to not more than 15 nodes
to improve the interpretability of the model generated. Analyse your findings
and discuss the results. Visualise the final generated decision tree and describe it.
[10 marks]
d) Describe the role of the two parameters in the model building that you used in
c) above. Do you expect that manipulating the parameter, in the same way, will
improve accuracy for other types of datasets? Justify your answer. [5 marks]
e) Provide and carefully examine the confusion matrix. Generate and provide a
classification report, showing precision, recall, F1 and overall accuracy, to
evaluate your model performance. Describe your findings, is there any
significant finding? [5 marks]
Page | 3
Study Area II (Autism-Child-Data)
This application is from the medical domain and is concerned with the diagnosis of
childhood Autistic Spectrum Disorder Screening (ASDS) for a collection of individuals
from whom relevant medical data has been obtained. The dataset contains 10
behavioural features (AQ-10-Child), 10 individual characteristics, and the outcome
(effectiveness of detection). The objective is to predict whether the given individual
characteristics are effective in detecting ASD cases. The effectiveness of ASDS
detection is labelled as ‘Yes’ or ‘No’ in this dataset.
For this dataset, you are required to use both the Na?ve Bayes (NB) and Decision Tree
classifier algorithms to build a predictive model for the ASDS.
a) Describe what is an autism spectrum disorder (ASD) and discuss the significance
of the early diagnosis of ASD. Briefly describe the Autism-Spectrum Quotient
(AQ) and include two recent references to support your answer (no more than
one page). [5 Marks]
b) Describe the pre-processing steps and perform initial data exploration. Use an
appropriate method of feature selection to identify the top five significant
features. State the method used and list the features produced and explain why
this feature selection method was used. Use a ‘breakdown’ analysis for selected
features by the class and describe their distribution using appropriate plot(s).
[15 marks]
c) Discuss the independence assumption between the features in Na?ve Bayes (NB)
algorithm and support your answer concerning the selected features.
[5 marks]
d) Run the Na?ve Bayes algorithm with the GaussianNB implementation for the
selected features. Provide the evaluation metrics including the confusion matrix
showing the performance of the NB model. Discuss the results. [10 marks]
e) Run the Decision Tree Classifier algorithm and compare the top five features
produced by the Decision Tree model with the list selected in part (b). Identify
similarities and differences. Discuss any differences. [10 marks]
f) Provide the evaluation metrics including the confusion matrix showing the
performance of the Decision Tree model. Compare the performance of your
models (NB and Decision Tree) and discuss your findings. [5 marks]
There will be 5 marks for the presentation of the assignment including spelling and
grammar, layout, formatting, and readability of the figures.