Ask Ghassem - Recent activity

Answered: How to analyse imbalanced categorical colum in dataset

Thu, 19 Feb 2026 18:40:15 +0000

For imbalanced categorical data, you shouldn't drop the column. Instead, you can try techniques like oversampling the minority class or using models that handle imbalanced data well, like XGBoost. This way, you can still extract useful insights without losing important information.

Answered: Step-by-Step Hidden State Calculation in a Recurrent Neural Network

Mon, 01 Dec 2025 18:33:19 +0000

We compute each hidden state step-by-step using

$$ h_t = \text{ReLU}(W_{ih} \cdot x_t + W_{hh} \cdot h_{t-1}). $$

$ h_1 = \text{ReLU}(0.4 \cdot 3 + 0.6 \cdot 0) = 1.2 $

$ h_2 = \text{ReLU}(0.4 \cdot 3 + 0.6 \cdot 1.2) = 1.92 $

$ h_3 = \text{ReLU}(0.4 \cdot 3 + 0.6 \cdot 1.92) = 2.352 $

$ h_4 = \text{ReLU}(0.4 \cdot 3 + 0.6 \cdot 2.352) = 2.6112 $

Final Answer: $ h_4 = 2.6112 $

Answered: How to calculate feed-forward (forward-propagation) in neural network for classification?

Wed, 09 Oct 2024 13:00:11 +0000

The answer is provided below. Please comment if you have question of find mistakes

https://i.imgur.com/Qchg1sl.jpeg

Commented: How to update weights using gradient decent algorithm?

Wed, 17 Apr 2024 22:58:56 +0000

Isn't the derivative in 3) wrong? I got 2w-5 instead of 4w-10? I think person in solution forgot to incorporate the 1/2

Commented: How to update weights in backpropagation algorithm (a numerical example)?

Fri, 05 Apr 2024 21:48:20 +0000

It does make a difference, because after taking the derivative it should be (target - output) but in the solution it's (output - target)

When to use one hot encode a category and when to segment by category?

Wed, 22 Feb 2023 20:30:38 +0000

When pre processing data for machine learning. Is there any difference in using one hot encoding to turn categoric variables into numeric variables or to segment the data and the model being used along the category. So say you run a multivariate regression model on data covering 5 cities. Would a single model with one variable for each city be more better or worse than having 5 models specific for each city? Or is there no difference? Or does it depend on certain factors and intuition?

Answered: How to calculate the residual errors, (MSE),(MAE), and (RMSE)?

Fri, 27 Jan 2023 04:16:33 +0000

1. First, we need to calculate the residual errors. Residual errors are the difference between the actual values and predicted values.

Sample	Feature 1	Feature 2	Actual Value	Predicted Value	Residual Error (Actual - Predicted)
1	2	3	4	6	-2
2	3	4	5	6	-1
3	4	5	6	7	-1
4	5	6	7	8	-1
5	6	7	8	9	-1

Next, we can calculate the MSE by taking the average of the squared residual errors.

$MSE = ((-2)^2 + (-1)^2 + (-1)^2 + (-1)^2 + (-1)^2) / 5 = 10 / 5 = 2$

To calculate the MAE, we take the average of the absolute residual errors.

$MAE = (|-2| + |-1| + |-1| + |-1| + |-1|) / 5 = 6 / 5 = 1.2$

Finally, to calculate the RMSE, we take the square root of the MSE.

$RMSE = sqrt(2) = 1.41$

Therefore, the residual errors are [-2, -1, -1, -1, -1], the MSE is 2, the MAE is 1.2, and the RMSE is 1.41.

Can you verify the validity of this chart comparing the review scores for Marvel Phase 4?

Mon, 09 Jan 2023 16:29:14 +0000

I have some skepticism about the validity of the charts below comparing the critic and audience reviews for Phase 4 of the MCU to the previous 3 phases. There are over 18 movies and tv shows in Phase 4 compared to the 6 movies in Phases 1 & 2 and the 11 movies in Phase 3. Also, there are far fewer critic reviews for the Phase 4 tv shows than the Phase 4 movies. For example, on Rotten Tomatoes there are only 40 critic reviews for The Falcon and the Winter Soldier and 452 critic reviews for Black Widow. Could this uneven and inconsistent number of reviews between tv shows and movies in Phase 4 be inaccurately making the overall averages higher than they should be? Or do you agree with the conclusions presented in the charts?

https://cdn.discordapp.com/attachments/997145183172964435/1059948060194652230/image.png

https://cdn.discordapp.com/attachments/997145183172964435/1049356020469739520/image.png

Edited: How to use Genetic Algorithm to optimize a function?

Wed, 23 Nov 2022 10:19:30 +0000

Assume the function is defined as $f(x,y)=x^2+y^2-4xy$, and $1\leq x \leq 4,1\leq y \leq 4$. The Genetic Algorithm is selected to maximize the function. If the first population for pairs of $(x,y)$ is defined as $S=\{A=(1,2), B=(2,1), C=(2,2), D=(2,3), E=(3,1) \}$.

a) Calculate the fitness of each of individuals (A,B,C,D,E) in population if: $\text{fitness function}=f(x,y)$
b) Calculate the probability of each individual and sort them in descending order. Which individual has the maximum fitness (probability)? $p_{i}=\frac{f_{i}}{\sum_{j=1}^{N} f_{j}}$
c) Draw the roulette wheel and calculate the boundaries for each individual
d) If we use two individuals and their arithmetic mean for crossover each time, and for mutation, we add 0.1 to x and subtract 0.1 from y for each individual created after crossover, what will be the next population with five members?
For part (d), use the following random numbers in order whenever you need them in the selection process:
$\text{random numbers} = \{0.780,0.220,0.776,0.507,0.822,0.765,0.288,0.881,0.895,0.421\}$

Answered: List of free Qwiklabs labs

Fri, 28 Oct 2022 23:51:28 +0000

Retagged: What is the difference between a batch and an epoch in a Neural Network?

Wed, 28 Sep 2022 10:40:08 +0000

Both of the batch size and number of epochs are integer values and seem to do the same thing in Stochastic gradient descent. What are these two hyper-parameters of this learning algorithm?

Which code has best runtime and why?(the one commented or the other one)

Fri, 02 Sep 2022 14:39:49 +0000

# for key, value in dict.items():
            #     if value >= long:
            #         long = value
            #         long_name = key
            #     if value < short:
            #         short = value
            #         short_name = key
        long = max(dict.values())
        long_name = max(dict, key=dict.get)
        short = min(dict.values())
        short_name = min(dict, key=dict.get)

Creating tables from unstructured texts about stock market

Tue, 02 Aug 2022 00:47:49 +0000

I am trying to extract information such as profits, revenues and others along with their corresponding dates and quarters from an unstructured text about stock market and convert it into a report in the table form but as there is not format of the input text, it is hard to know which entity belong to what date and quarters and which value belong to which entity. Chunking works on few documents but not enough. Is there any unsupervised way to linking entities with their corresponding dates, values and quarters?

How do I compare the count of a value in each year while having a different sanple size each year.

Wed, 08 Jun 2022 10:32:33 +0000

How do I accurately compare between the number of something a survey measure from my employees each year with a varying umber of survey engagement and employee size?

If I was measuring the satisfaction of my employees over the years by collecting a survey from my them each year by asking them wether they are satisfied or not, and then comparing yes’s over the years but the number of employees who answer is not the same each year and the number of employees increases every year. How do I correctly compare this throughout each year?

In other words, how do I remove the effect of the survey engagement rate when calculating the results?

Is it possible to make a forecast of a future value of Air Temperature using Fast Fourier Transform?

Thu, 02 Jun 2022 16:10:26 +0000

Is it possible to make a forecast of a future value of Air Temperature using Fast Fourier Transform, if yes, what should be the process or how you'll be able to do it. Thank you!

forecast log transformed fitted values for 2 years using ARMA model

Wed, 04 May 2022 20:31:44 +0000

Input is a stock price in exponential transformation. We are asked to forecast using ARMA results for 2 years.

Kmeans clustering in python - Giving original labels to predicted clusters

Wed, 27 Apr 2022 05:32:54 +0000

I have a dataset with 7 labels in the target variable.

X = data.drop('target', axis=1)
Y = data['target']
Y.unique()

array(['Normal_Weight', 'Overweight_Level_I', 'Overweight_Level_II',
'Obesity_Type_I', 'Insufficient_Weight', 'Obesity_Type_II',
'Obesity_Type_III'], dtype=object)

km = KMeans(n_clusters=7, init="k-means++", random_state=300)
km.fit_predict(X)
np.unique(km.labels_)

array([0, 1, 2, 3, 4, 5, 6])

After performing KMean clustering algorithm with number of clusters as 7, the resulted clusters are labeled as 0,1,2,3,4,5,6. But how to know which real label matches with the predicted label.

In other words, I want to know how to give original label names to new predicted labels, so that they can be compared like how many values are clustered correctly (Accuracy).

Bankruptcy prediction and credit card

Sun, 10 Apr 2022 05:50:14 +0000

Hello everyone newbie data scientist here.
I'm working on a project to predict companies (probability of default) bankruptcy probability and to assign them a credit rating/score based on that :
For example below 50 probability is good and above is bad ( just for the example)
I have a dataset contains financial ratios and a class refers if the company is bankrupted or not (0 and one).
I'm planning to use this models:
Logistic regression linear discrimination analysis, decision trees, random forest, ANN, adaboost, Svm.

The question is and i know it is a dumb question:
Does those models return a probability? Which i can transform to labels, I saw that in a thesis and I'm not sure about it.

Otherwise, any guidance,tips anything will be appreciated.

Answered: how to output f1-score instead of accuracy

Sat, 02 Apr 2022 14:32:22 +0000

from sklearn.metrics import f1_score
f1_score(data_test,target_test)

I cannot get this code to work. please help.

Mon, 21 Mar 2022 05:59:53 +0000

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from sklearn.model_selection import train_test_split

model = Sequential()
model.add(LSTM( 10, input_shape=(1, 1)))
model.add(Dense(1, activation="linear"))
model.compile(loss="mse", optimizer="adam")

X, y = get_data()

X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=1)
X_train_2, X_val, y_train_2, y_val = train_test_split(X_train, y_train, test_size=0.25, random_state=1)

model.fit(X_train, y_train, epochs=800, validation_data=(X_val, y_val), shuffle=False)

html, body, table, thead, input, textarea, select {color: #bab5ab!important; background: #35393b;} input[type="text"], textarea, select {color: #bab5ab!important; background: #35393b;} [data-darksite-inline-background-image-gradient] {background: linear-gradient(rgba(0, 0, 0, 0.5), rgba(0, 0, 0, 0.5))!important; -webkit-background-size: cover!important; -moz-background-size: cover!important; -o-background-size: cover!important; background-size: cover!important;} [data-darksite-force-inline-background] * {background-color: rgba(0,0,0,0.7)!important;} [data-darksite-inline-background] {background-color: rgba(0,0,0,0.7)!important;} [data-darksite-inline-color] {color: #fff!important;} [data-darksite-inline-background-image] {background-image: linear-gradient(rgba(0,0,0,0.3), rgba(0,0,0,0.3))!important}

Recategorized: Battery data projects

Thu, 03 Mar 2022 09:42:53 +0000

Where can I find projects related to battery data?

Answer selected: How to activate hive interpreter for Apache Zeppelin on Dataproc on GCP?

Sun, 13 Feb 2022 14:14:24 +0000

We can activate Hive interpreter by creating a new interpreter with some changes in the properties and dependencies

1. Set hive.url as
jdbc:hive2://localhost:10000

2. Remove username and password (keep them empty)

3. Set hive.driver property as:
org.apache.hive.jdbc.HiveDriver

4. Adding two artifact under dependencies tab:
org.apache.hive:hive-jdbc:0.14.0
org.apache.hadoop:hadoop-common:2.6.0

You can watch this video for more explanations and code is here

https://www.youtube.com/watch?v=0nqhoLWVdwU

How can you build dynamic pricing model with data only from rigid pricing?

Fri, 21 Jan 2022 06:44:31 +0000

I want to build a dynamic pricing model which means if product is too expansive for a client and there is a risk that we might loose a client we lower the price for them but if client doesn't care that much about the price we might increase price a little.

All the articles I've seen describe some kind of A/B testing for the pricing and then create a model.

I want to build a model only on the existing rigid pricing data. So I have prices offered to customers and I know who bought the product and who went to other company.

How can I do the increasing price part?

What analytical software would be good for a company to use?

Fri, 14 Jan 2022 16:46:38 +0000

This would be for a company that is just now looking into using a software to track data for wine making.

Do you usually collect you own data or there is always a resource available for you? Or it depends on the company?

Sun, 09 Jan 2022 22:13:34 +0000

Answered: When dealing with categorical values, should the 'year' column be encoded using OHE or OrdinalEncoder?

Mon, 20 Dec 2021 18:10:13 +0000

You should ask yourself if the order of years has an effect in predicting the price? It seems it is important. Therefore, OrdinalEncoder seems to be a better choice. If you use OneHotEncoder, you consider the years with equal weights in predicting the price.

Answer selected: How to create a Decision Tree using the ID3 algorithm?

Wed, 01 Dec 2021 11:56:37 +0000

a) See the following figure for the ID3 decision tree:

https://i.imgur.com/kizNjoc.png

b) Only the disjunction of conjunctions for Martians was required:

$\begin{aligned}
&(\text { Legs }=3) \vee \\
&(\text { Legs }=2 \wedge \text { Green }=\text { Yes } \wedge \text { Height }=\text { Tall }) \vee \\
&(\text { Legs }=2 \wedge \text { Green }=\text { No } \wedge \text { Height }=\text { Short } \wedge \text { Smelly }=\text { Yes })
\end{aligned}$

Python Code

Step 1: Organize the Dataset

Our data has the following features and values:

Species: Target variable (M or H)
Features:
- Green: $ N $ or $ Y $
- Legs: $ 2 $ or $ 3 $
- Height: $ S $ or $ T $
- Smelly: $ N $ or $ Y $

Index	Species	Green	Legs	Height	Smelly
1	M	N	3	S	Y
2	M	Y	2	T	N
3	M	Y	3	T	N
4	M	N	2	S	Y
5	M	Y	3	T	N
6	H	N	2	T	Y
7	H	N	2	S	N
8	H	N	2	T	N
9	H	Y	2	S	N
10	H	N	2	T	Y

Step 2: Calculate the Initial Entropy for the Target Variable (Species)

We start by calculating the entropy of the target variable, Species, which has two classes: M (Martian) and H (Human).

Total Counts

Martians (M): 5
Humans (H): 5
Total: 10

Entropy Formula

The entropy $ E $ for a binary classification is calculated as:

$$ E = -p_+ \log_2(p_+) - p_- \log_2(p_-) $$

Where:

$ p_+ $: Probability of positive class (M)
$ p_- $: Probability of negative class (H)

Calculation

$$ p(M) = \frac{5}{10} = 0.5 $$

$$ p(H) = \frac{5}{10} = 0.5 $$

$$ E(Species) = -0.5 \cdot \log_2(0.5) - 0.5 \cdot \log_2(0.5) $$

$$ = -0.5 \cdot (-1) - 0.5 \cdot (-1) $$

$$ = 1.0 $$

Step 3: Calculate Entropy and Information Gain for Each Feature

We’ll calculate the entropy for each feature split and determine the information gain.

Feature: Green

Green can be either Y or N.

For Green = Y:

Martians (M): 3
Humans (H): 1
Total: 4

Entropy:

$$ E(Green = Y) = -\left(\frac{3}{4}\right) \log_2\left(\frac{3}{4}\right) - \left(\frac{1}{4}\right) \log_2\left(\frac{1}{4}\right) $$

$$ = -0.75 \cdot \log_2(0.75) - 0.25 \cdot \log_2(0.25) $$

$$ = -0.75 \cdot (-0.415) - 0.25 \cdot (-2) $$

$$ = 0.311 + 0.5 = 0.811 $$

For Green = N:

Martians (M): 2
Humans (H): 4
Total: 6

Entropy:

$$ E(Green = N) = -\left(\frac{2}{6}\right) \log_2\left(\frac{2}{6}\right) - \left(\frac{4}{6}\right) \log_2\left(\frac{4}{6}\right) $$

$$ = -0.333 \cdot \log_2(0.333) - 0.667 \cdot \log_2(0.667) $$

$$ = -0.333 \cdot (-1.585) - 0.667 \cdot (-0.585) $$

$$ = 0.528 + 0.389 = 0.917 $$

Weighted Entropy for Green

$$ E(Green) = \frac{4}{10} \cdot 0.811 + \frac{6}{10} \cdot 0.917 $$

$$ = 0.3244 + 0.5502 = 0.8746 $$

Information Gain for Green

$$ IG(Species, Green) = E(Species) - E(Green) $$

$$ = 1.0 - 0.8746 = 0.1254 $$

Continue this process to calculate the entropy and information gain for each feature (Legs, Height, and Smelly) similarly.

Commented: How to filter a dataframe?

Mon, 29 Nov 2021 04:14:50 +0000

Since it's the first row you can also do df.head(1).

How do I know which encoder to use to convert from categorical variables to numerical?

Mon, 29 Nov 2021 04:09:06 +0000

So say I have a column with categorical data like different styles of temperature: 'Lukewarm', 'Hot', 'Scalding', 'Cold', 'Frostbite',... etc.

I know that we can use pd.get_dummies to convert the column to numerical data within the dataframe, but I also know that there are other 'converters' (not sure if that's the correct terminology) that we can use, i.e. OneHotEncoder from Sk-learn (like I could use the pipeline module to make a nice pipeline and feed my dataframe through the pipeline to also get my categorical data encoded to numerical).

How do I know which to use? Does it matter? If it does matter, when does it matter the most (i.e. what types of problems? When there are lots of categorical variables, or few?) If anyone can give me any pointers on this type of stuff I'd greatly appreciate it.

ValueError: Length mismatch: Expected axis has 60 elements, new values have 2935849 elements

Fri, 26 Nov 2021 06:09:16 +0000

I'm creating a new data frame with the most used items grouped together. But I got the following error when grouping through ID and items. ValueError: Length mismatch: Expected axis has 60 elements, new values have 2935849 elements.

df = sales_df[sales_df['shop_id'].duplicated(keep=False)]
df['Grouped'] = sales_df.groupby('shop_id')['item_name'].transform(lambda x: ','.join(x))
df2 = df[['shop_id', 'Grouped']].drop_duplicates()

In the aforementioned code, I'm making a data frame with respect to shop id and then grouping through shop items. My objective here is to group items with similar ID.

Text Mining, Artificial Neural Networks, Speech Processing, Cloud Computing in DS? Essential for a good Data Scientist ?

Wed, 27 Oct 2021 19:15:16 +0000

Classification of data object might be incorrect

Mon, 25 Oct 2021 15:26:46 +0000

I am learning a new Salesforce product (Evergage) for the company I work for. In the program's documentation they have listed a set of data objects as an example. It appears to me that the classification might be incorrect. Their system makes a division between 'catalog objects' and 'profile objects' and the example they have given is a banking institution. They classified Customer Credit Card as a profile object and Credit Card Level as a catalog object. Seems to me that it should be the other way i.e Customer Credit Card = catalog object and Credit Card Level = profile object. Maybe I am not reading the context correctly?

here is a link to an image with the complete classification: https://drive.google.com/file/d/1nG4aX4Ty_NoHxm04AQo1Ow61m3MZ3pXm/view?usp=sharing

Can Data Science solve this problem?

Sun, 24 Oct 2021 15:43:11 +0000

So, I live in Brazil, and I have a task for college that I don't know what data science method to use, if at all, to solve it. My idea is the following: We Brazilians have Real (BRL) as currency, and we of course have the dollar quotation value to see "how many Reais a dollar is worth". What I wanted to do was to make a research and see whether the Country News have any influence over this price. So for example, if Bolsonaro, our president, says some dumb stuff, the dollar got up in price, and vice versa. What I wanted to do was collect all dollar values and variance over a set time interval, and try and get webscraping to get the news over some economy sites. Here's my question then: How can I correlate the news with the dollar variance over a set time? Can data science do that? How do I preprocess this, if at all? Do I need to use bag-of-words? At least I heard so... Please help and thank you for reading.

Answer selected: How to calculate LogLoss in logistic regression?

Sun, 17 Oct 2021 16:48:29 +0000

Answer#2: Total Loss of the model

first we have to find all the probability of the student passing the course

lets i is representing the sampling index of the student

P1:

Z=-64+(2*29)=-6

P=1/(1+e^6)=0.0024

P2:

Z=-64+(2*15)=-34

P=1/(1+e^34)=0 (THE VALUE IS SO SMALL)

P3: ALREADY KNOW = 0.88

P4:

Z=-64+(2*28)=-8

P=1/(1+e^8)=0.00033

P5:

Z=-64+(2*39)=14

P=1/(1+e^-14)=0.999

THE TOTAL LOSS OF THE MODEL IS CALCULATED BELOW, BY USING THE FORMULA

Log-loss= -(yi*ln(P1)+(1-yi)ln(1-P1))

LOG-LOSS 1= -2.4E-3

LOG-LOSS 2= 0

LOG-LOSS 3= - 0.128

LOG-LOSS 4= -8.0164

LOG-LOSS 5= -0.001

TOTAL LOSS OF THE MODEL= LOG-LOSS= - (1/5)(-2.4E-3+0- 0.128-8.0164-0.001) = 1.6296

Answer 1:the loss of model for the student who studied 33 hours

Step 1: we have to find the probability to passing the course

P=1/(1+e^-z)

where z= odd= -64+(2*33)=2

after putting the values... P=1/(1+e^-2)=0.88

Now, lets calculate the log-loss of the model for that particular student, has sample number 3 which is "i" the sampling index

Log-loss= (yi*ln(P1)+(1-yi)ln(1-P1))

Log-loss=[1*ln(0.88)+(1-1)ln(1-0.88)]

Answer#1: Log-loss= - 0.128 loss of model for the student

Which algorithm is best to detect anomalies within a data set of 5k+ user-login events?

Tue, 05 Oct 2021 17:45:38 +0000

I am trying to build an unsupervised ML model to detect anomalies within 5000+ users' login data. I selected 5 features contained within each of the user-login events (e.g. IP, hour of day, day of week, device_id, OS). I am looking for the best algorithm to use. I am considering using density function to determine probabilities of the feature values and whether an event is an outlier. The problem is that feature values are only relevant to the specific user. For example, you cannot compare login IP across users, login IP is only applicable to the user.
Ultimately, I want to detect events that are changes in a user login behavior, like different IP, day, hour, device_id, or OS, where the more features that have changed increase the probability of an outlier.
At this point, I am not sure how to build a model with data that contains multiple users, because I don't know how to separate the user data so the model is trained per user and finding anomalies within the individual user's features.

I also don't have any labeled data to use for testing, should I fabricate some?

Any advice greatly appreciated.

Thank you!

How we incorporate the polyline in machine learnning tools

Wed, 29 Sep 2021 06:16:30 +0000

Suppose I have to predict the traffic of a road segment based on available data such as number of houses and business along the road segment. Which machine learning tool would be the option to use that can incorporate the road segment (polylines) through coordinates in the attributes.

Commented: How to calculate Softmax Regression probabilities?

Mon, 05 Jul 2021 18:14:35 +0000

The question doesn't make any mention of a bias, so we just assume it is 1?

Commented: How to calculate convolutions on a CONV layer for a Convolutional Neural Network?

Sat, 03 Jul 2021 17:34:53 +0000

for part a I am getting below results fo rconcolved features:
R: 0 2 0 , 2 2 0, 3 3 2
G: -1 2 2 , -3 0 4 , 0 0 1
B: -2 -2 0, 1 2 2 , -1 1 3
How do we consider the bias for this section?

Commented: How to update the weights in backpropagation algorithm when activation function in not linear?

Sat, 03 Jul 2021 14:44:08 +0000

While the question says activation function for hidden layer, the solution applies the same activation function to output layer as well. Do we also need to apply activation function to output layer?

Commented: What are the main branches of Machine Learning?

Mon, 28 Jun 2021 11:17:30 +0000

Hi, can I use this picture as reference in my master thesis? Thank you

should i start as a data analyst then data science?

Mon, 21 Jun 2021 20:31:04 +0000

should I start as a data analyst then data science?

I am a second-year Bachelor's in Computer Science and wanted to pursue to be a Data Scientist.

However, when I am trying to apply for internships/jobs, most of it requires a Masters's/Ph.D.

But, a Data Analyst has fewer requirements.

Do you recommend starting off as a Data Analyst and then change to Data Science?

how many samples do we need to test image segmentation using synthetic data ?

Mon, 21 Jun 2021 12:26:32 +0000

Hello,

I trained a CNN using synthetic data to perform a segmentation task on human faces. During the test and to evaluate the prediction of this network, I used 200 examples from the database to compute precision and recall.

Is this number sufficient, knowing that I control myself the data generator and that I build the database by randomly drawing the elements using centered Gaussian distributions.

Thank you,

Answered: How best to ensure data quality?

Fri, 11 Jun 2021 18:09:27 +0000

This is really a broad question. The best quality usually comes from a good quality source of data generation.

Answered: Can we have multiple target values in a ML problem dataset for supervised learning?

Fri, 11 Jun 2021 16:48:41 +0000

Yes, not only it is possible it is also possible with Neural Networks with output layers with more than one neuron, but also with traditional machine learning algorithms.

Searching for movie dataset containing movie synopses/plots?

Thu, 27 May 2021 09:57:31 +0000

Hello
To build a hybrid recommendation system, I used the movielens 1M dataset, for the collaborative filtering part. Now, I'm looking for a database/dataset that contains descriptions/summaries/details/synopses/plots of movies for the content-based recommendation.
Is there someone who could help me and tell me where I can find a such dataset?
thank you in advance.

Intermittent Mathematics (Logarim)

Wed, 05 May 2021 12:16:20 +0000

The old keypad of the telephone, it has 10 numbers (10 keys) , this keypad allows the user to enter a text by successively pressing certain key many times in a small period of time. you need to draw a graph of entering a text input using this keypad. after that you need to have a certain algorithm of finding the length of a path to enter certain text
example
aaa --> 6
aba --> 5

the link below shows the phone keypad

https://commons.wikimedia.org/wiki/File:Telephone-keypad.png

Answered: How to calculate average with deviating sensors?

Wed, 05 May 2021 11:49:02 +0000

What seems to work is simple: create 4 new columns:
x=average(3 values) ; y=stdev.p(3 values) ; low threshold = x-y ; high threshold = x+y
, then repeat the numbers if these are within the boundaries and make a 'n/a' if outside of stdev.
With the values repeated (as within thresholds), the average can be calculated neglecting the extreme values.
Example:
10 ; 11 ; 20 : avg=13.67: stdev.p = 4.49; Low=9.17; high=18.16, so
10 ; 11; n/a (as 20 > 18.16)
this gives an average of 10.5 :-)

Agree?

The old keypad of the telephone, it has 10 sume. yypad. after that yout

Tue, 04 May 2021 14:39:49 +0000

هاد سؤال رياضيات متقطعة

The old keypad of the telephone, it has 10 numbers (10 keys) , this keypad allows the user to enter a text by successively pressing certain key many times in a small period of time. you need to draw a graph of entering a text input using this keypad. after that you need to have a certain algorithm of finding the length of a path to enter certain text

design a computer-based system that will encourage autistic children to communicate and express themselves better.

Thu, 01 Apr 2021 07:04:59 +0000

a) A company has been asked to design a computer-based system that will encourage autistic children to communicate and express themselves better.

b) What type of interaction would be appropriate to use at the interface for this particular user group?

Answer selected: Terminology clarification in Spark

Wed, 17 Feb 2021 16:04:05 +0000

The fact is the engine is still the same, regardless of which interface language you use. For some tasks, such as special cleaning we probably do not have SQL commands, and we have to use Scala or Python. Using Zeppelin, you can switch back and forth among languages the engine supports, however it is not a common practice. For some specific tasks, you can use pure Spark SQL or if you want to use the SQL in pyspark or scala, there are functions that can help you achieve the goal.

I believe observing more examples will help you understand when you can use what.

Index	Species	Green	Legs	Height	Smelly
1	M	N	3	S	Y
2	M	Y	2	T	N
3	M	Y	3	T	N
4	M	N	2	S	Y
5	M	Y	3	T	N
6	H	N	2	T	Y
7	H	N	2	S	N
8	H	N	2	T	N
9	H	Y	2	S	N
10	H	N	2	T	Y

Index	Species	Green	Legs	Height	Smelly
1	M	N	3	S	Y
2	M	Y	2	T	N
3	M	Y	3	T	N
4	M	N	2	S	Y
5	M	Y	3	T	N
6	H	N	2	T	Y
7	H	N	2	S	N
8	H	N	2	T	N
9	H	Y	2	S	N
10	H	N	2	T	Y

Index	Species	Green	Legs	Height	Smelly
1	M	N	3	S	Y
2	M	Y	2	T	N
3	M	Y	3	T	N
4	M	N	2	S	Y
5	M	Y	3	T	N
6	H	N	2	T	Y
7	H	N	2	S	N
8	H	N	2	T	N
9	H	Y	2	S	N
10	H	N	2	T	Y