Ask Ghassem - Recent questions tagged ele888-midterm

How to calculate feed-forward (forward-propagation) in neural network for classification?

Wed, 02 Oct 2024 14:47:26 +0000

For the following neural network, calculate accuracy of classification, given these settings

How to calculate the residual errors, (MSE),(MAE), and (RMSE)?

Fri, 27 Jan 2023 04:09:28 +0000

Given the following sample dataset with 5 samples and 2 features:

Sample	Feature 1	Feature 2	Actual Value	Predicted Value
1	2	3	4	6
2	3	4	5	6
3	4	5	6	7
4	5	6	7	8
5	6	7	8	9

Calculate the residual errors, mean squared error (MSE), mean absolute error (MAE), and root mean squared error (RMSE) using a sample model.

How to calculate the probability and accuracy of a Logistic Regression classifier?

Mon, 03 Feb 2020 20:31:49 +0000

How to solve this problem?

https://i.imgur.com/8urywpf.jpg

Q1) Complete the ? sections

Q2) Accuracy of system if threshold = 0.5?

Q3) Accuracy of system if threshold = 0.95?

How to perform a classification or regression using k-NN?

Thu, 27 Jun 2019 02:54:42 +0000

Suppose, you have given the following dataset where x and y are the 2 features and color Red or Blue is the target variable.

a) A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN, what you predict as the color for this data point?

Dataset
x	y	Color
-1	1	Red
0	1	Blue
0	2	Red
1	-1	Red
1	0	Blue
1	2	Blue
2	2	Red
2	3	Blue

b) Now assume we have the following dataset and the target value is the price. A new data point $x=1$ and $y=1$ is given. Using Euclidean distance in 3-NN. What would be the estimated price?

Dataset
x	y	Price
-1	1	$100
0	1	$50
0	2	$20
1	-1	$40
1	0	$30
1	2	$40
2	2	$70
2	3	$30

How to calculate k-means clustering with a numerical example?

Thu, 27 Jun 2019 02:16:32 +0000

Use the k-means algorithm and Euclidean distance to cluster the following 8 examples into 3 clusters:

$A1=(2,10), A2=(2,5), A3=(8,4), A4=(5,8), A5=(7,5), A6=(6,4), A7=(1,2), A8=(4,9)$.

Suppose that the initial seeds (centers of each cluster) are $A1$, $A4$ and $A7$. Run the k-means algorithm for 1 epoch only. At the end of this epoch show:

a) The new clusters (i.e. the examples belonging to each cluster)

b) The centers of the new clusters

c) Draw a 10 by 10 space with all the 8 points and show the clusters after the first epoch and the new centroids.

d) How many more iterations are needed to converge? Draw the result for each epoch

How to optimize weights in Logistic Regression?

Wed, 05 Jun 2019 17:38:50 +0000

The hypothesis (model) of Logistic Regression which is a binary classifier ( $y =\{0,1\} $) is given in the equation below:

Hypothesis

$S(z)=P(y=1 | x)=h_{\theta}(x)=\frac{1}{1+\exp \left(-\theta^{\top} x\right)}$

Which calculates probability of Class 1, and by setting a threshold (such as $h_{\theta}(x) > 0.5 $) we can classify to 1, or 0.

Cost function

The cost function for Logistic Regression is defined as below. It is called binary cross entropy loss function:

$J(\theta)=-\frac{1}{m} \sum_{i}^{m}\left(y^{(i)} \log \left(h_{\theta}\left(x^{(i)}\right)\right)+\left(1-y^{(i)}\right) \log \left(1-h_{\theta}\left(x^{(i)}\right)\right)\right)$

Iterative updates

Assume we start all the model parameters with a random number (in this case the only model parameters we have are $\theta_j$ and assume we initialized all of them with 1: for all $\theta_j = 1$ for $j=\{0,1,...,n\}$ and $n$ is the number of features we have)

$\theta_{j_{n e w}} \leftarrow \theta_{j_{o l d}}+\alpha \times \frac{1}{m} \sum_{i=1}^{m}\left[y^{(i)}-\sigma\left(\theta_{j_{o l d}}^{\top}\left(x^{(i)}\right)\right)\right] x_{j}^{(i)}$

Where:
$m =$ number of rows in the training batch
$x^{(i)} = $ the feature vector for sample $i$
$\theta_j = $ the coefficient vector corresponding the features
$y^{(i)} = $ actual class label for sample $i$ in the training batch
$x_{j}^{(i)} = $ the element (column) $j$ in the feature vector for sample $i$
$\alpha =$ the learning rate

Dataset

The training dataset of pass/fail in an exam for 5 students is given in the table below:

If we initialize all the model parameters with 1 (all $\theta_j = 1$), and the learning rate is $\alpha = 0.1$, and if we use batch gradient descent, what will be the:

$a)$ Accuracy of the model at initialization of the train set ($\text{accuracy} = \frac{\text{number of correct classifications}}{\text{all classifications}}$)?
$b)$ Cost at initialization?
$c)$ Cost after 1 epoch?
$d)$ Repeat all $a,b,c$ steps if we use mini-batch gradient descent and $\text{batch size} = 2$

(Hint: For $x_{j}^{(i)}$ when $j=0$ we have $x_{0}^{(i)} = 1$ for all $i$ )

How to calculate Softmax Regression probabilities in this example?

Thu, 04 Apr 2019 18:20:53 +0000

1) What will be the probability of an iris with petal length = 4.6 and petal width = 1.7 to be classified as Virginica?

2) What will be the probability of Virginica, if we use all features petal length = 4.6 and petal width = 1.7, sepal length = 5.5 and sepal width = 3.0 with the same weight initialization?

How to calculate feed-forward (forward-propagation) in neural network?

Thu, 04 Apr 2019 15:54:17 +0000

In the figure below, a neural network is shown. Calculate the following:

1) How many neurons do we have in the input layer and the output layer?

2) How many hidden layers do we have?

3) If all the weights initialized with 1 ($w1=w2=w3=...=w19=1$), what is the output of this network after feed-forward for the sample shown in the figure (X = (x1,x2,x3) = (2,5,3) and y=10)? What is the error of the network ($\text { Error }=\frac{1}{2}(\hat{y}-y)^{2}$)? Assume activation functions for all neurons except the output neuron is $f(z)=z$.

4) If we change the activation function of all the neurons in the second hidden layer to Sigmoid ($S(x)=\frac{1}{1+e^{-x}}=\frac{e^{x}}{e^{x}+1}$), what would be the output of the network after this change? Calculate the error as well.

https://i.imgur.com/rtqPiRa.jpg

How to calculate Softmax Regression probabilities?

Thu, 21 Mar 2019 16:11:09 +0000

The scatter plot of Iris Dataset is shown in the figure below. Assume Softmax Regression is used to classify Iris to Setosa, Versicolor, or Viriginica using just petal length and petal width. If all the weights required for Softmax Regression initialized to 0.5 and the network includes bias nodes:

1) Write the weight vectors and equations for calculating the class probabilities.

2) We have a new iris and we have measured petal length = 4.5 and petal width = 1.6. Using the above initial model, what would be the result of classification?

3) If we change all the weights related to the class blue to 1 and keep all other weights 0.5, what will be the predicted class?

How to calculate LogLoss in logistic regression?

Mon, 18 Mar 2019 20:34:40 +0000

The dataset of pass/fail in an exam for 5 students is given in the table below. If we use Logistic Regression as the classifier and assume the model suggested by the optimizer will become the following for Odds of passing a course:

$\log_e(Odds) = -64 + 2 \times hours$

1) How to calculate the loss of model for the student who studied 33 hours?

2) What is the total loss of the model given in equation below?

$Logloss = -\frac{1}{N} \sum_{i=1}^N(y_i\log_e(p_i) + (1 - y_i)\log_e(1 - p_i))$