Questions tagged [data-science]

Use for questions related to Data Science aspects of AI. Generally speaking, only basic Data Science questions should be asked on this Stack, ideally involving the fundamental concepts. For more advanced questions related to Data Science, please use the Data Science stack: https://datascience.stackexchange.com/

Filter by
Sorted by
Tagged with
2
votes
1answer
33 views

How to make a distinction between item feature and environment feature?

My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' ...
12
votes
5answers
18k views

Why does C++ seem less widely used in AI?

I just want to know why do Machine Learning engineers and AI programmers use languages like python to perform AI task and not C++ even though C++ is technically a more powerful language than python.
-1
votes
1answer
36 views

How to calculate the false positives and negatives?

I have a huge amount of data and I want to calculate my false positive and false negative. Is there a software that can help me determine it?
5
votes
3answers
54 views

What should we do when we have equal observations with different labels?

Suppose we have a labeled data set with columns $A$, $B$, and $C$ and a binary outcome variable $X$. Suppose we have rows as follows: ...
0
votes
1answer
68 views

multi vs one prediction using Regression

I was trying to build a prediction system where I have the input data arranged in multiple columns. The input data would be of the type where I have weather, service type (bronze, silver, gold), size(...
0
votes
2answers
90 views

Ensemble models - XGboost

I am building 2 models using XGboost, one with x number of parameters and the other with y number of parameters of the data set. It is a classification problem. A yes-yes, no-no case is easy, but ...
0
votes
1answer
22 views

How to rescale data to its original range after MinMaxScaler?

I'm using sklearn's MinMaxScaler in order to scale my data down. However, it would be nice to be able to rescale it back to its original range. Is there any way I ...
0
votes
0answers
11 views

Scikit-Learn: monotoneous quantile estimation

I would like to implement various AI-estimators for quantile estimation for a regression problem. It would be necessary to have non-crossing quantiles, that is larger quantiles would correspond to ...
4
votes
1answer
64 views

How to deal with small amount of labeled samples?

I'm trying to develop skill to deal with very small amount of labeled samples (250 labeled/20000 total, 200 features) practicing on Kaggle "Don't Overfit" dataset (Traget_Practice have provided all 20,...
4
votes
1answer
60 views

Psychological models at Facebook et al

As a layman in AI I want to get an idea how big data players like Facebook model individuals (of which they have so many data). There are two scenarios I can imagine: Neural networks build clusters ...
1
vote
2answers
52 views

What is the impact of scaling the features on the performance of the model?

I am trying to generate a model that uses several physicochemical properties of a molecule (including number of atoms, number of rings, volume, etc.) to predict a numeric value $Y$. I would like to ...
2
votes
1answer
44 views

How do I know if my dataset is ready for a machine learning model?

I am new in this area of Machine Learning and Neural Networks. Currently, I'm taking some courses on Udemy and reading a book about it, but I still have one big question regarding data pre-processing. ...
0
votes
1answer
35 views

Analyze the pattern of merchants account transactions in a bank [closed]

How can I analyze the pattern of merchants account transactions in a bank and identify the merchants that are becoming inactive? Which methods should I use to solve this problem?
0
votes
0answers
8 views

Are hadoop ecosystem tools main goal is to break up large data sets into fast readable files?

I am new to big data theory, and during the past 3 days, I took an official big data course with some of the best instructors available in my country in this domain. The things was little bit obscure ...
1
vote
1answer
25 views

What data formats/pipelining are best to store and wrangle data which contains both text and float vectors?

Often in NLP project the data points contain both text and float embeddings, and it's very tricky to deal with. CSVs take up a ton of memory and are slow to load. But most the other data formats seem ...
1
vote
1answer
38 views

What types of machine learning model would fit?

I am working on a supervised machine learning problem where I have more than 10 probably 50 or 100 predicting label categories. Which type of model can be used to work on this type of problem in ...
1
vote
1answer
20 views

Loss/accuracy on Synthetic data

I am trying to understand if there is any difference in the the interpretation of accuracy and loss on synthetic data vs real data.
0
votes
0answers
2 views

How to improve the AUC - test of LightBoost based on BayesSearchCv

I need to use LightGBM for improve the auc score of my binary classification algorithm. I have used BayesSearchCv with score:"roc-auc" but i dont know how can i make it able to search the best ...
0
votes
0answers
31 views

Mimicking certain human functions

Is it a common practice to do 3-dimensional imagary data acquisition and implementing the dataset to certain high resolution motor driven humanoid candidates to mimic the human micro expressions or ...
0
votes
1answer
163 views

Parameters to calculate affluence in localities of Metro city

I have to calculate the affluence in localities of Metro city. To calculate affluence, I am considering a parameter per capita income. Where I can get a dataset of it? What are other parameters I ...
0
votes
1answer
55 views

Which field to study to learn & create a.i generated simulations?

I wasn't sure how to title this question so pardon me please. You may have seen at least one video of those "INSANE A.I created simulation of {X} doing {Y & Z} like the following ones: A.I ...
1
vote
0answers
67 views

An idea I had to create the first Humanoid using Deep Learning

I've come up with an idea on how we could use a combination of Deep Learning and body sensors to create a walking talking living humanoid. Here goes: First, we will recruit 1 billion people and have ...
0
votes
1answer
42 views

Help with Novelty Recognition and Binary Classification for Emotion Recognition

I’m looking for advice regarding my ML project. Using a special wristband, I am able to collect a bunch of physiological data from human subjects. I want to develop an application to recognize when ...
-1
votes
1answer
25 views

is it possible to train several Neural Networks on different types of data and combine them?

I want to create a NHL game predictor and have already trained one neural network on game data. What I would like to do is train another model on player seasonal/game data and combine the two models ...
3
votes
1answer
144 views

Text analytics for text analysis

I have a huge text where a paragraph is written about a project execution. I need to parse that data and make sense out of it. For example the text states, "Est. 200k USD has been committed on the ...
1
vote
1answer
140 views

What does a hybrid Bayesian network contain?

The field of artificial intelligence is so vast. So many methodologies of handling continuous data and just now I have read about Hybrid Bayesian network. I just want to know that what a hybrid ...
1
vote
1answer
18 views

How can I evaluate the performance of a system that generates text?

I am preparing to perform research comparing the performance of two different systems that probabilistically generate the next word of an input sentence. For example, given the word 'the', a system ...
1
vote
1answer
72 views

What are the most challenging tasks aiming to achieve the lowest error rate?

For example there is the MNIST database which is used to test artificial neural network (ANN), however it's not so challenging, because some hierarchical systems of convolutional neural networks ...
4
votes
2answers
113 views

Decision making systems applications

Machine learning and data science are mainly made for processing large amounts of data nowadays, for example - a multitude of pictures. But do these fields have some applications in the decision ...
0
votes
1answer
75 views

Automatic prediction of whether a customer will come into the shop or not

So as my university project I am planning to make a prediction system as described in the title. My current idea is to use the age/gender classifier and run it on a video(taken in front of a shop) ...
3
votes
1answer
103 views

Neural Network training beginner question

I have a question about the training sequence regarding Neural Network recognition. Let's say an image has 28*28 pixels, which leads to 784 Input Nodes with various greyscale values and 10 output ...
3
votes
1answer
77 views

Find anomalies from records of categorical data

I have a data-set with m observations and p categorical variables (nominal), each variable ...
2
votes
0answers
168 views

How do to mitigate or design out hidden feedback loops when designing ML systems?

Two months ago, I've found myself working on a churn detection problem which can be briefly described as follows: Assume the current date is N Use customer behavior for N-1,..N-x dates to develop ...
2
votes
0answers
95 views

Performance Evaluation Metrics used in Training, Validation and Testing

Which specific performance evaluation metrics are used in training, validation and testing and why? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide ...
1
vote
0answers
156 views

Deep NN architecture for predicting a matrix from two matrices

Recently my friend asked me a question: having two input matrices X and Y (each size NxD) where D >> N, and ground truth matrix Z of size DxD, what deep architecture shall I use to learn a deep model ...
5
votes
2answers
2k views

How to implement exploration function and learning rate in Q Learning

I'm trying to implement Q-learning (state-based representation and no neural / deep stuff) but I'm having a hard time getting it to learn anything. I believe my issue is with the exploration function ...
1
vote
0answers
622 views

Forecasting and predict using matlab Artificial Neural Network [closed]

I selected the below data set for forecast and predict using artificial neural network as my final year project. https://archive.ics.uci.edu/ml/datasets/Bank+Marketing. I normalized the data set and ...