• Home

Machine learning

CSI436/536 homework 5 Machine Learning

Requirement:

Β· You need to submit your homework as a .py or an .ipynb file.

Β· You need to test your code to make sure it runs with no bugs. Code that cannot run will lose all points.

Logistic Regression

Consider the objective function in logistic regression problem 𝑙(πœƒ) =

π‘š

βˆ‘ (𝑦𝑖 log𝜎(πœƒπ‘‡π‘₯𝑖) + (1 βˆ’ 𝑦𝑖 )log(1 βˆ’ 𝜎(πœƒπ‘‡π‘₯𝑖))),

𝑖=1

where 𝜎(𝑧) = (1 + π‘’βˆ’π‘§)βˆ’1 is the logistic function.

The given hwX.txt and hwY.txt contain the inputs π‘₯𝑖 ∈ 𝑅2 and outputs 𝑦𝑖 ∈ {0,1} respectively for a binary classification problem, with one training example per row.

1. (30 points) Implement the gradient descent method for optimizing 𝑙(πœƒ), and apply it to fit a logistic regression model to the data. Initialize gradient descent method with πœƒ = 0 (the vector of all zeros). What are the coefficients πœƒ resulting from your fit? (Remember to include the intercept term.)

2. (40 points) Implement Newton’s method to maximize 𝑙(πœƒ), and compare the overall running time and number of iterations needed to converge to the same precision.

3. (10 points) Plot the training data (your axes should correspond to the two coordinates of the inputs, and you should use a different symbol for each point plotted to indicate whether that example had label 1 or 0).

4. (20 points) Also plot on the same figure the decision boundary fit by logistic regression. (i.e., this should be a straight line showing the boundary separating the region where 𝜎(πœƒπ‘‡π‘₯) > 0.5 from where𝜎(πœƒπ‘‡π‘₯) < 0.5.)

Machine learning

0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00

1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00

Machine learning

1.3432504e+00 -1.3311479e+00
1.8205529e+00 -6.3466810e-01
9.8632067e-01 -1.8885762e+00
1.9443734e+00 -1.6354520e+00
9.7673352e-01 -1.3533151e+00
1.9458584e+00 -2.0443278e+00
2.1075153e+00 -2.1256684e+00
2.0703730e+00 -2.4634101e+00
8.6864964e-01 -2.4119348e+00
1.8006594e+00 -2.7739689e+00
3.1283787e+00 -3.4452432e+00
3.0947429e+00 -3.6446145e+00
2.9086652e+00 -4.0065037e+00
2.6770338e+00 -3.0198592e+00
2.7458671e+00 -2.7100561e+00
4.1714647e+00 -3.4622482e+00
3.9313220e+00 -2.1099044e+00
4.3786870e+00 -2.3804743e+00
4.8016565e+00 -3.3803344e+00
4.1661050e+00 -2.8138844e+00
2.4670141e+00 -1.6108444e+00
3.4826743e+00 -1.5533872e+00
3.3652482e+00 -1.8164936e+00
2.8772788e+00 -1.8511689e+00
3.1090444e+00 -1.6384946e+00
2.2183701e+00 7.4279558e-02
1.9949873e+00 1.6268659e-01
2.9500308e+00 1.6873016e-02
2.0216009e+00 1.7227387e-01
2.0486921e+00 -6.3581041e-01
8.7548563e-01 -5.4586168e-01
5.7079941e-01 -3.3278660e-02
1.4266468e+00 -7.5288337e-01
7.2265633e-01 -8.6691930e-01
9.5346198e-01 -1.4896956e+00
4.8333333e+00 7.0175439e-02
4.3070175e+00 1.4152047e+00
6.0321637e+00 4.5029240e-01
5.4181287e+00 -2.7076023e+00
3.4590643e+00 -2.8245614e+00
2.7280702e+00 -9.2397661e-01
1.0029240e+00 7.7192982e-01
3.6637427e+00 -7.7777778e-01
4.3070175e+00 -1.0409357e+00
3.6929825e+00 -1.0526316e-01
5.7397661e+00 -1.6257310e+00
4.9795322e+00 -1.5087719e+00
6.5000000e+00 -2.9122807e+00
5.2426901e+00 9.1812865e-01
1.6754386e+00 5.6725146e-01
5.1708997e+00 1.2103667e+00
4.8795188e+00 1.6081848e+00
4.6649870e+00 1.0695532e+00
4.4934321e+00 1.2351592e+00
4.1512967e+00 8.6721260e-01
3.7177080e+00 1.1517200e+00
3.6224477e+00 1.3106769e+00
3.0606943e+00 1.4857163e+00
7.0718465e+00 -3.4961651e-01
6.0391832e+00 -2.4756832e-01
6.6747480e+00 -1.2484766e-01
6.8461291e+00 2.5977167e-01
6.4270724e+00 -1.4713863e-01
6.8456065e+00 1.4754967e+00

7.7054006e+00 1.6045555e+00
6.2870658e+00 2.4156427e+00
6.9810956e+00 1.2599865e+00
7.0990172e+00 2.2155151e+00
5.5275479e+00 2.9968421e-01
5.8303489e+00 -2.1974408e-01
6.3594527e+00 2.3944217e-01
6.1004524e+00 -4.0957414e-02
5.6237412e+00 3.7135914e-01
5.8836969e+00 2.7768186e+00
5.5781611e+00 3.0682889e+00
7.0050662e+00 -2.5781727e-01
4.4538114e+00 8.3941831e-01
5.6495924e+00 1.3053929e+00
4.6337489e+00 1.9467546e+00
3.6986847e+00 2.2594084e+00
4.1193005e+00 2.5474510e+00
4.7665558e+00 2.7531209e+00
3.0812098e+00 2.7985255e+00
4.0730994e+00 -3.0292398e+00
3.4883041e+00 -1.8888889e+00
7.6900585e-01 1.2105263e+00
1.5000000e+00 3.8128655e+00
5.7982456e+00 -2.0935673e+00
6.8114529e+00 -8.3456730e-01
7.1106096e+00 -1.0201158e+00
7.4941520e+00 -1.7426901e+00
3.1374269e+00 4.2105263e-01
1.6754386e+00 5.0877193e-01
2.4941520e+00 -8.6549708e-01
4.7748538e+00 9.9415205e-02
5.8274854e+00 -6.9005848e-01
2.2894737e+00 1.9707602e+00
2.4941520e+00 1.4152047e+00
2.0847953e+00 1.3567251e+00

Machine Learning

 Attached Files:

1. Please download the attached Jupiter notebook  HW2.ipynb.  Please complete all 10 steps in the Jupyter notebook file according to the instructions. 2. Please download the attached dataset  ufo_sightings_large.csv to complete HW2 

    • 20

    Machine Learning

     

    Introduction to Machine learning

    You are expected to be able to program in R prior to taking this class. Use Titanic dataset and perform EDA on various columns. Without using any modeling algorithms, and only using basic methods such as frequency distribution, describe the most important predictors of survival of Titanic passengers, e.g. were males or females more likely to survive, were young and rich females more likely to survive than old poor males etc?

    Please submit the assignment in a word or pdf file