0
$\begingroup$

I'm new to machine learning and I've been working through a dataset of ~3000 records with ~100 features. I've been hand rolling Python and R scripts to analyse the data. For example, plotting the distribution of each feature to see how normal it is, identify outliers, etc. Another example is plotting heatmaps of the features against themselves to identify strong correlations.

Whilst this has been a useful learning exercise, going forward I suspect there are tools that automate a lot of this data analysis for you, and produce the useful plots and possibly give recommendations on transforms, etc? I've had a search around but don't appear to be finding anything, I guess I'm not using the right terminology to find what I'm looking for.

If any useful open source tools for this kind of thing spring to mind that would be very helpful.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Hello. Welcome to Artificial Intelligence SE. It seems to me that this question is off-topic here and would be better suited for softwarerecs.stackexchange.com and datascience.stackexchange.com. Asking for tools or software libraries is generally off-topic here. You can read more about our scope on our on-topic page (basically, we focus on the theoretical, philosophical and social aspects of AI, so programming issues, including asking for tools, are generally excluded). $\endgroup$
    – nbro
    Commented Aug 4, 2021 at 10:25

2 Answers 2

Reset to default
1
$\begingroup$

This is a great question because indeed there are many tools out there to make this part of the process faster. I have used as I usually stick to the following two:

You can also search for alternatives Hopefully, the community can help complement my answer.

Improve this answer
$\endgroup$
0
$\begingroup$

all you mentioned belongs to any statistical analysis - it is suited for Normal Data & is done with least-squares methods, but Maximum-Likelihood-Estimations can be applied for non-normal data (e.g. for rare-events analysis)... ML, in general, do not need Normality of data distribution, but some preprocessing is needed from the developer by his desire - see opportunities for preprocessing in sklearn's preprocessing class

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .