# Tag Info

5

This is a good question. I tend to think the answer is yes it is necessary to know the details, because a person without mathematical understanding of these algorithms cannot consistently make a model as good as someone who does have that understanding. The reason is right at the core of computer science: abstractions are useful, but usually obscure details....

5

Yes. Here are some of the most prominent ones and their respective state-of-the-art errors: CIFAR-10: ~3.5% error CIFAR-100: ~24% error STL-10: ~26% error SVHN: ~1.7% error ImageNet tasks: the best 2012 classification task solution got 15% top-5 error, better results are currently available You can check an updated list of solutions here. Also, a more ...

4

The problem you are portraying looks like a modified XOR problem. You can't throw away the lines with a label of 1 because a the model won't be able to learn this class.

3

scikit-learn has a small data sets API http://scikit-learn.org/stable/datasets/index.html I imagine one can add more data sets locally. Some data sets are for classification, other for regressions. This is the only one I know about.

3

Your main problem is that you need to separate out what is driving the behaviour policy from the Q-table. Q Learning is an off-policy algorithm. The Q-table that it eventually learns is for an optimal policy (also called the target policy). In order to be able to learn that policy, the agent needs to explore. The usual way to do this is to make the agent ...

2

Before jumping to modeling, there are a few tasks a data scientist (or ML/AI practitioner) must do: Ideation (or hypothesizing): Before applying any modeling approach, we need to ask the right questions. We must clearly mention our assumptions and declare how we want to measure the effectiveness of the pipeline. Note that, some tools/algorithms might not ...

2

Is your question about storing, writing, or reading/processing huge data? I'm not an expert in this topic, but I know a couple of possible ways to handle huge datasets: If the data is too big to be fully uploaded to RAM, you can iterate over it in Pandas. You can find a brief explanation in the article Why and How to Use Pandas with Large Data, section 1. ...

2

If you don't have a lot of data, I suggest you using time series models, such as ARMA, ARIMA, etc. If you have enough data, I suggest you use LSTMs. In both cases, I would aggregate data to some periods, say amount of transactions for month, and use them (and data from all previous months) to predict transactions in the next month.

2

The extraction of features from data and categorizing media are not business decision making. However, determining what patterns in real world information are distinctive and placing an item in one bin rather than another are of a simple kind of decision. That large amounts of data are required to perform these tasks does not discount these limited decision ...

2

One problem with clustering algorithms is that they will typically find you a solution, ie they will split your data set into clusters, but it will find you a structure even if there isn't one. Your data looks like it could consist of about 5 to 7 clusters, but it could equally well just be 2 or only 1. What you need to do after the clustering is to assess ...

1

Just for clarification: your description (1 sample per minute) does not match the example data (far fewer data points which is understandable, but also two data points in one minute which contradicts the initial assertion.) If your actual measurements are like that you should first work on the sampling process to get reliable data. For creating predictions, ...

1

Thinking about this more, the answer is in fact yes, but not for the application you mention. You cannot use alpha-beta pruning to learn a model to predict customer outcomes, because it is only useful for domains where you are concerned about an adversary. In finding a customer model, there is no reason to worry about someone coming in and forcing you to ...

1

Yes, you can use sklearn's confusion_matrix. To explicitly extract the false positives and negatives, you can do from sklearn.metrics import confusion_matrix y_true = [0, 1, 0, 1] y_pred = [1, 1, 1, 0] tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()

1

This is perfectly acceptable in a stochastic environment. Generally your loss is to minimize $-log\ p(Y|X)$ or equivalently $-\sum_i log\ p(y_i|x_i)$. This optimization is equivalent to $-\mathbb{E}\log\ p(y_i|x_i)$. In other words you are minimizing in this case:  \begin{align*} L &= -log\ p(1|x_0) - log\ p(0|x_0) \\ &= -log [p(1|x_0) * p(0|...

1

You can use the function inverse_transform of the created MinMaxScaler object. See also this Stack Overflow question for other answers and examples.

1

TensorFlow has a nice datasets package, which meets the requirements stated above. documentation link PyPI link

1

There are multiple python packages with inbuilt toy-datasets for testing purposes: sklearn.datasets seaborn.load_dataset() statsmodels.api.datasets rpy2 (requires R and pandas) PyDataset

1

This is a tricky issue. I assume you are using transition probabilities to pick the next suitable word, so you could use some other corpus data, derive probabilities from it, and compare those to your system. Not very satisfactory, though, as you might end up evaluating the system in a circular way, deriving your test data in the same way as you generate ...

1

Without any additional information, lean towards the vote of the best performing classifier when it comes to ties. However, as others have stated already, it is best to analyze the performances in more detail (e.g. confusion matrices). For instance, it could be that model B almost always classifies class X correctly (hardly any false positives). In that ...

1

Welcome to AI.SE @Par! What you have might be either a multi-label or a multi-class classification problem. If the classes are disjoint (each example belongs to just 1 of the 50 classes), it's a multi-class problem. If not (so each example can belong to several classes at once), it's a multilabel problem. Multi-label classification is usually handled by ...

1

No, there is no difference. Of course, you are likely not able to extrapolate results obtained from synthetic data to expect identical or similar results in real life to unless you have very compelling reasons to do so. Without a more specific question, I'm afraid a more specific answer is not possible.

1

Affluence could encompass several parameters: Income; Wealth (property ownership); Life expectancy; Access to services such as education and health; Access to clean natural resources; Low levels of criminality. Property prices in each locality might be easy to obtain from real estate agent sources Ratings for schools or medical facilities in each area might ...

1

You need to define "simulation" more specific. Playing Mario, Swapping face on image/video, or generating simulation of objects that are orbiting use different techniques. Playing Mario or "AI that playing game": the AI agent trained on available environment (Mario game, so the environment is not generated) and learn the best sequential actions to achieve ...

1

The Project It appears from the question that emotional detection and response is the longer term goal of the project and that recognizing potential emotional manifestations in easily detectable physiological metrics is an initial R&D objective. Mobile device applications are already available to do this, but biometric monitoring via a wristband, ...

1

In general, algorithms that exploit distances or similarities (e.g. in the form of scalar product) between data samples, such as k-NN and SVM, are sensitive to feature transformations. We do feature scaling to make our model robust to outliers and make an initial impact of every feature on the model will be roughly similar Graphical-model based classifiers, ...

1

The term you need is “model ensembles”, that’s the way models are combined. Pretty hard to be more specific since you don’t give a language or any other details.

1

This should be possible, but it's not completely clear what you are trying to do. If you're trying to predict customer age and gender from a video, then you've got a computer vision problem. Deep learning methods are the state of the art for this, and probably some sort of convolutional deep neural network is your friend. If you're trying to predict ...

1

However, doesn't a new training picture destroy the trained and balanced weights and nodes values? Because the weights and hidden nodes have been calibrated to recognize the former training picture? This can happen, and happen to various degrees depending on how the neural network is set up, but it is usually something you want to avoid. Provided a neural ...

Only top voted, non community-wiki answers of a minimum length are eligible