New answers tagged

0

The OP's (i.e. my) assumption is wrong. It is not true that for almost all random graphs under no permutation a pattern is visible, as can be seen in Tiago P. Peixoto's paper Bayesian stochastic blockmodeling, p. 4: The opposite seems to be correct: For almost all random graphs there are permutations such that patterns are visible.


0

The basic idea of most state of the art handwritten text recognition (HTR) systems is as follows: 1. Feature extraction Have a method of compressing your input into an abstract representation which retains some concept of sequence. So, you might take your CNN and apply it to this: Then your feature map might look something like 512 x 4 x 24 (channels, ...


0

Check out this answer, and other answers too. They explained the what the ideal order is and why so. Also, you can just add layer of activation function. from tensorflow.keras import layers from tensorflow.keras import activations model.add(layers.Activation(activations.relu)) They are actually first things when you search on google, so I kindly recommend ...


1

Q-Learning and Genetic algorithm are both good algorithms to create an IA that plays Snake. The one you use depends mostly on how you understand and model your IA environment. Q-Learning algorithm is an algorithm that needs a State (give by the environment), Actions it can take, and Rewards to give him according to how it performs. Genetic algorithm needs ...


3

Short answer: Generally, you don't need to do softmax if you don't need probabilities. And using raw logits leads to more numerically stable code. Long answer: First of all, the inputs of the softmax layer are called logits. During evaluation, if you are only interested in the highest-probability class, then you can do argmax(vec) on the logits. If you want ...


0

GPT-3 (and the likes) don't really have any understanding of the semantics nor pragmatics involved in the language. However, they are good at constructing text content similar to the contents created by a person (when the texts and the concepts are not too "complicated").


0

The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer and $o'$ is the derivative of output layer. I found this formula in the book Artificial Intelligence: A Guide to Intelligent Systems by Michael Negnevitsky.


2

Interesting question, I can come with 2 explanations why we don't initialize weights with 1 mean value : It may be easier for the network to learn identity function, but we may have a similar issue about not being able to learn comparison, comparison is quite an important reasoning in my opinion, this is why having negative weight values is important, and ...


1

It sounds like you have structured/tabular data. So, a fully-connected feedforward network should do the job.


1

It depends on what your outputs are. For example, if both outputs are similar then you can use one output branch. However, what if the two outputs are different? With two output branches you can used two different loss functions. Now your model will optimize the two branches separately. Imagine if you have a model that has to output a class label for the ...


3

Estimating from an observation is a function, but "really counting" is a process. Feed-forward neural networks can learn arbitrary functions from training examples, but they cannot represent (and therefore cannot learn) processes. They can attempt to estimate the results of completing a process as a function, but that is not the same thing as ...


0

Firstly, I would suggest you do not use softmax for exploration, because it does not imply the model's uncertainty. Training with softmax and cross-entropy, your model may be very confident, but wrongly, because of overfitting. Another reason why you should not use softmax to measure uncertainty is that your estimate of the variance (optimal estimate of the ...


0

If I understand your query correctly, you want to create a latent space that groups similar objects. You should then probably look for Siamese networks. However, your loss function will need another term to increase dissimilarity between different labels. Otherwise, as pointed out by Mike NZ, the net would collapse(yes, it is possible). Perhaps this will ...


0

As explained here only if larger function classes contain the smaller ones are we guaranteed that increasing them strictly increases the expressive power of the network. For deep neural networks, if we can train the newly-added layer into an identity function $f(x)=x$, the new model will be as effective as the original model. As the new model may get a ...


3

In the early days of neural networks the theorists and practitioners were educated in mathematics, psychology, neurophysiology, electrical engineering, and neurobiology. Computer science was still in its infancy. The first neural networks were modeled as electrical circuits. There is evidence of this in the 1943 paper by Warren McCulloch and Walter Pitts [1],...


2

Well, the goal of any paper is to allow the reader to understand what the author is trying to describe. A lot of people have a lot of experience looking at circuit diagrams and figuring out those circuits will do. For these people, a circuit diagram may be the clearest and easiest way for them to understand how a particular thing works. So, it makes sense ...


0

If your data is well balanced but small, I would recommend using a simpler algorithm to classify your data.


0

I presume you are attempting to solve a classification problem. IMO, there's no decision-making template you could follow to know whether to use over sampling or not.I would typically compare results (ROC AUC, PRC curves) across datasets (Original vs Undersampled vs Oversampled) to decide. You can consider some additional variants of SMOTE like SMOTE NC (...


0

Gretel is a good tool for processing data. Facets is good for the visualizations. Is it worth it? most learners will exhibit bias towards the majority class, and in extreme cases, may ignore the minority class altogether. It really depends on the goal and requirements of your project. Not because it's desirable it's better for your particular project, if ...


0

Random over sampling creates duplicates of existing examples, so applying this to your training data would be the same as increasing the weight of the oversampled examples. If it's done to all of the examples uniformly then the effects will probably cancel out. SMOTE, on the other hand, creates synthetic examples that are linear combinations of existing ...


0

I would like to highlight an import step for face recognition which is features extraction. Based on my experience, you can evaluated robust feature extraction methods like, Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) using several matching approaches such as Brute Force Matcher, K-Nearest Neighbor (KNN), Best-Bin-First (...


0

It seems Alex has just used the Matlab function mat2gray, as described here: https://www.mathworks.com/help/vision/ug/image-category-classification-using-deep-learning.html The visual outcome of the features is very similar. mat2gray will simply scale the weights between 0 and 1 (no clipping). Leaving the (slightly adapted) example code of Mathworks here for ...


1

For the ANN, it should be the average of the error per instance from testing (prediction) when each instance is left out of training. ANNs can unfortunately learn based on the order of instances used for training, so it helps to train/test and then shuffle (permute, or randomly re-order) and then assign to k-folds, then train/test again in order to prevent ...


0

There are multiple standard ways of feature selection, for example ranking features by information gain, that you could use, and then you can train the neural network on just those features. However, let's assume you have trained a neural network on all of the features and now want to estimate their importance. One approach you could take is to perform a ...


0

There is a single neural network that guides self-plays in the Monte Carlo Tree Search algorithm. The neural network gets the current state of the board $s$ as an input and outputs current policy $\pi(a|s)$ and value $v(s)$. The action probabilities are encoded in a (8,8,73) tensor. First two dimensions encode the coordinates of the figure to "pick"...


2

I am not an expert in RL. I have been playing Go for some years. Let's quote from AlphaZero's paper first: Aside from komi, the rules of Go are also invariant to colour transposition; this knowledge is exploited by representing the board from the perspective of the current player (see Neural network architecture). In the Game of Go, the difference between ...


1

Rosenblatt was probably discussing a specific architecture, for which there are many. However, for general purpose feed-forward back-propagation ANNs used for function aproximation and classification analysis, you can use whatever activation functions you want on the input-side, hidden layers, and output-side. Examples are identity, logistic, tanh, ...


0

For the multiclass SVM, there will be an ensembling effect since you are learning 5*4=20 1vs1 classifiers. It could be an interesting experiment to try the same thing with simple neural networks. Also, since you are standardizing the inputs you could try tanh activations on the first layer after the input. I presume you are using softmax on the output layer.


1

The meaning of invertible here is the standard definition of invertibility for a mathematical function $f \colon X \to Y$. Invertible simply means "the function has an inverse map $f^{-1} \colon Y \to X$". Equivalently the function $f$ is bijective, which means the following two conditions hold: $f$ is injective: for any two distinct $x_1, x_2 \in ...


1

Your fitness function has two objectives that are added together, but they are not necessarily on the same scale. The component cos(drone_angle) must have a value from 0..1. The component 1/distToTarget will have a range that depends on how you measure distToTarget; e.g. if distToTarget has a range 0..1000, then this part of the fitness function will always ...


3

The Attention is All you Need has this footnote at the passage motivating the introduction of the $1/\sqrt{d_k}$ factor: To illustrate why the dot products get large, assume that the components of $q$ and $k$ are independent random variables with mean 0 and variance 1. Then their dot product, $q \cdot k = \sum^{d_k}_{i=1}q_ik_i$ has mean 0 and variance $d_k$...


-2

To some extend, it get rid of low intensity numerical noise. Condition properties of the optimization problem is always an issue, i suspect BatchNorm alleviate this instability.


1

I no longer really use validation that much, but rather only training and testing. Why? Because I mostly follow Ron Kohavi's (Stanford Univ) approach to CV. I have done a lot of validation but it seemed to be overkill, essentially causing me to ask why I have this very small-sampled parameter watch on the side from which I am supposed to learn from. You ...


1

I commonly use softmax for all 2-class or k-class problems, basically, because I always like to have an output node for each class. For sigmoid, i.e., logistic, you cannot estimate MSE for each sample using the relationship $E_i = \sum_c^C (y_c - \hat{y}_c)^2$, where $C$ is the number of classes, $y_c$ is 0 or 1 for true class membership, and $\hat{y}_c$ is ...


1

Sigmoid is used for binary cases and softmax is its generalized version for multiple classes. But, essentially what they do is over exaggerate the distances between the various values. If you have values on a unit sphere, apply sigmoid or softmax on those values would lead to the points going to the poles of the sphere.


-1

The networks with one-dimensional convolution over the temporal dimensions are called Temporal Convolutional Networks. The authors of this study claim that TCNs are comparable to RNNs in performance on common tasks associated with RNNs.


4

Yes, it is not unusual to omit the bias by adding a neuron which always outputs a constant 1, which will then be multiplied by an appropriate weight to give the same formula as you would get using an explicit bias. One notable text using this convention is Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David. ...


0

From your question it sounds like your only training data is {𝑥1,…,𝑥𝑛} and the network has to magically come up with values {y1,…,y𝑛} such that an unknown function is minimised. How do you plan to give feedback to the network during training? Your situation appears to be something like this: X-->Model-->Y-->f(X,Y) where X is being copied from ...


4

According to wikipedia of backpropagation: In fitting a neural network, backpropagation computes the gradient of the loss function during supervised learning with respect to the weights of the network for a single input–output example, and does so efficiently, unlike a naive direct computation of the gradient with respect to each weight individually. ...


0

I don't think you need a recurrent neural network for this. Why not just train a feedforward model with angle of attack etc as input and translation velocities as output? The size of the output will depend on how frequently you want updates to the velocities. e.g. if the update rate is 0.5 seconds and the network is predicting 2 seconds in advance it could ...


1

Check the documentation for Dense layer: Note: If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), ...


1

two hidden layers each comprising two neurons From your description it looks like that you only have 6 parameters for your inner layer (2x2 weight matrix + 2 biases). The whole network should be easy to interpret: you've got two 13-dimensional weight vectors $\vec{w}_1,\vec{w}_2$ that are dot-multiplied with the inputs, plus two biases $b$ and activation $\...


Top 50 recent answers are included